Curl: Difference between revisions
→One-Liners
m (→Switches: m) |
|||
| (5 intermediate revisions by the same user not shown) | |||
Line 61:
</div>
= One-Liners =
* Save the output of the URL to a file
* Save with name same as remote file
* Download files securely via SSH
curl -u username:password https://domain.com
* Get HTTP header information
* Access an FTP server
curl ftp://ftp.domain.com --user username:password
* Download a file via FTP
curl ftp://ftp.domain.com/file.zip --user username:password
curl -u ftpuser:password -O ftp://ftp_pub/public_html/index.html
* Upload a file to the FTP server
curl -T file.zip ftp://ftp.domain.com/ --user username:password
curl -u ftpuser:password -T linuxtechi.txt ftp://ftp_pub/public_html/
* To upload multiple files to FTP server
curl -u ftpuser:password -T "(linuxtechi1.txt linuxtechi2.txt)" ftp://ftp_pub/public_html/
* Deleting files from ftp server
curl ftp://ftp_pub/public_html -X 'DELE linuxtechi.zip' --user ftpuser:password
* Handle URL redirects
curl -L http://domain.com
* Debug level details
curl -v http://domain.com
* Using proxy to download a file
curl -x proxy.server.com:3128 https://domain.com
* Limit data transfer rate
curl --limit-rate 1024B -O https://domain.com
* Download file modified after a given date
curl -z 1-Jan-17 https://domain.com
* Download file modified before a given date
curl -z -1-Jan-17 https://domain.com
* Resume a download
curl -C https://domain.com
* Verifying SSL certificate
curl --cacert new-ca.crt https://domain.com
* Ignoring the ssl certificate warning
curl -k https://domain.com
* Getting information about supported methods
curl -i -X OPTIONS http://10.107.88.68:8082
= Scripts =
* Testing Response Times:
while true; do curl -s -w 'Testing Response Time for :%{url_effective}\n\nLookup Time:\t\t%{time_namelookup}\nConnectTime:\t\t%{time_connect}\nPre-transfer Time:\t%{time_pretransfer}\nStart-transfer Time:\t%{time_starttransfer}\n\nTotal Time:\t\t%{time_total}\n' -o /dev/null https://google.com ; sleep 10; done
=URL syntax=
Line 254 ⟶ 298:
|}
= Header Modifications =
Basic syntax for spoofing user agent:
curl -A "UserAgentString" https://aman.info.tm
Basic syntax for User Agent Spoofing along with other headers:
curl -A [user-agent] -H [headers] "https://aman.info.tm"
Two methods to spoof USer-Agent:
curl -L -A "Mozilla/5.0" https://aman.info.tm
curl -L -H "user-agent: Mozilla/5.0" https://aman.info.tm
One of the most common situations of different source HTML and CSS are for websites with stripped down mobile versions, you could retrieve iPhone-specific source code with:
curl -A "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5" https://aman.info.tm
Some sites do this with other browsers too. This would be Chrome 12 in Mac OS X 10.6.8:
curl -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.112 Safari/534.30" https://aman.info.tm
Other Examples
curl -L -H "Host: aman.info.tm" -H "Cache-Control: max-age=0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36" -H "HTTPS: 1" -H "DNT: 1" -H "Referer: https://www.google.com/" -H "Accept-Language: en-US,en;q=0.8,en-GB;q=0.6,es;q=0.4" -H "If-Modified-Since: Thu, 23 Jul 2015 20:31:28 GMT" --compressed https://aman.info.tm
curl -L -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36" https://aman.info.tm
<br />
| |||