Curl

From Network Security Wiki


Curl is a tool to transfer data from or to a server, using one of the supported protocols

DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP. 

Curl command has the following functionality:

   Multiple URLs
   Usernames and Passwords support
   IPv6 support
   Retry failed download
   URL globbing/sequences
   Win32 support
   Large file support
   GnuTLS support
   DarwinSSL support
   Schannel support
   Cyassl support
   PolarSSL support
   AxTLS support
   SSL Session ID 
   SSL Private Certificate
   netrc support
   Metalink support
   IDN support
   Bandwidth limiting
   Happy eyeballs
   SOCKS
   TFTP
   SCP upload/download
   SFTP upload/download
   HTTP Proxy
   HTTP Resume
   HTTP Ranges
   Follow HTTP Redirects
   HTTP Post
   HTTP Post Chunked
   HTTP Put
   Cookie support
   HTTP 1.1
   HTTP 2 (plain text upgrade)
   HTTP 2 (TLS ALPN)
   HTTP 2 (TLS NPN)
   HTTP persistent connections
   HTTPS
   HTTP Digest Auth
   HTTP NTLM Auth
   HTTP Negotiate Auth
   HTTP Multipost Part
   HTTP Deflate gzip
   FTP resume
   FTP ranges
   FTP active mode
   FTP SSL
   FTP upload
   FTP Kerberos
   FTP Connection re-use
   GOPHER

One-Liners

Save the output of the URL to a file		curl -o website.html https://domain.com
               					curl -o archive.zip https://domain.com/file.zip
						curl https://domain.com > website.html
Save with name same as remote file		curl -O https://domain.com/file.zip
       	        				curl -O https://domain.com/file.zip -O https://domain.com/file2.zip
Download files securely via SSH 		curl -u user sftp://server.domain.com/path/to/file
                                       		curl -u username:password https://domain.com
Get HTTP header information			curl -I http://domain.com
Access an FTP server				curl ftp://ftp.domain.com --user username:password
Download a file via FTP				curl ftp://ftp.domain.com/file.zip --user username:password
                                   		curl -u ftpuser:password -O ftp://ftp_pub/public_html/index.html
Upload a file to the FTP server			curl -T file.zip ftp://ftp.domain.com/ --user username:password
                                      		curl -u ftpuser:password -T linuxtechi.txt ftp://ftp_pub/public_html/
To upload multiple files to FTP server 		curl -u ftpuser:password -T "(linuxtechi1.txt linuxtechi2.txt)"  ftp://ftp_pub/public_html/ 
Deleting files from ftp server         		curl ftp://ftp_pub/public_html -X 'DELE linuxtechi.zip' --user ftpuser:password
Handle URL redirects 				curl -L http://domain.com
Debug level details 		 		curl -v http://domain.com
Using proxy to download a file  		curl -x proxy.server.com:3128 https://domain.com
Limit data transfer rate   			curl --limit-rate 1024B -O https://domain.com
Download file modified after a given date  	curl -z 1-Jan-17 https://domain.com
Download file modified before a given date   	curl -z -1-Jan-17 https://domain.com
Resume a download     				curl -C https://domain.com
Verifying SSL certificate   			curl --cacert new-ca.crt https://domain.com
Ignoring the ssl certificate warning   		curl -k https://domain.com

URL syntax

  • You can specify multiple URLs or parts of URLs by writing part sets within braces as in:
http://site.{one,two,three}.com
  • You can get sequences of alphanumeric series by using [] as in:
 ftp://ftp.numericals.com/file[1-100].txt
 ftp://ftp.numericals.com/file[001-100].txt
 ftp://ftp.letters.com/file[a-z].txt
  • Nested sequences are not supported, but you can use several ones next to each other:
http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html
  • You can specify a step counter for the ranges to get every Nth number or letter:
 http://www.numericals.com/file[1-100:10].txt
 http://www.letters.com/file[a-z:2].txt
  • If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want.
  • It will then default to HTTP but try other protocols based on often-used host name prefixes.
  • For example, for host names starting with "ftp." curl will assume you want to speak FTP.
  • Curl will attempt to re-use connections for multiple file transfers, so that getting many files from the same server will not do multiple connects / handshakes.
  • This improves speed.
  • Of course this is only done on files specified on a single command line and cannot be used between separate curl invokes.

Switches

Progress Meter							-# 
Append in FTP/SFTP						-a
User-agent							-A "Mozilla/4.0"
Tells curl to figure out auth method				--anyauth
Cookie								-b
Cookie-jar(file curl should use to save all cookies)		-c
        This section is under construction.



References





{{#widget:DISQUS |id=networkm |uniqid=Curl |url=https://aman.awiki.org/wiki/Curl }}