shell终端curl模拟浏览器请求页面 这里用的是post数据


curl -A "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" -e "http://xxx.html"  -d "17263[]=105&pid=17263&p=最佳&count=1&receipt=1&poll=投票" http://xxx/test.php
 

解说:

-A “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)” 的意思就是申明客户端是windows2000下的ie6的浏览器; -e “ http://post.html"的意思就是refer是这个页面;来伪装来路 ,一般 用–referer 更形象。 -d 后面的参数都是将用post方式提交到服务器去的 最后面的http://xxx/test.php就是将要post到的目标url地址 这里提到了-A、-e和-d参数, 其实用的比较多的还有 -x(后面用ip:port方式接代理服务器)等等, 这里就不一一列举了,可以参考man curl

也可以是 这样 curl $URL -d “17263[]=105&pid=17263&p=最佳&count=1&receipt=1&poll=投票” 输出:


Array
(
    [17263] => Array
        (
            [0] => 105
        )

    [pid] => 17263
    [p] => 最佳
    [count] => 1
    [receipt] => 1
    [poll] => 投票
)
 

其它方法: “-k"或”–insecure” 允许忽略证书访问 请求https网站一般会证书报错,可以用k 忽略 :

curl -k https://www.sohu.com

“-b” 发送cookie,从服务器响应set-cookie得到值返回给服务器

curl -b “domain=.53kf.com” http://www.iruance.com

“-X"或”–request" 发送指定请求

“-d” 指定发送的请求参数(提交用户名、密码登录网页)

1、curl url 作用:获取页面内容或接口响应 示例:以视频页领取优惠券为例:

curl https://act.vip.iqiyi.com/api/process.action?cb=cb_1540200657317&cid=afbe8fd3d73448c9&interfaceCode=b5018a28d5f8609f&pid=92b01a8207f5c404

(注意:上述curl的路径中使用到了\转义符号,当url中包含多个参数时候,需要使用\转义符) 返回值:

curl  https://act.vip.iqiyi.com/api/process.action?cb=cb_1540200657317\&cid=afbe8fd3d73448c9\&interfaceCode=b5018a28d5f8609f\&pid=92b01a8207f5c404

var cb_1540200657317={"code":"Q00330","data":{},"msg":"活动已结束"}

2、curl -I url 作用:获取请求页面或接口的请求头信息 示例:

curl -I https://act.vip.iqiyi.com/api/process.action?cb=cb_1540200657317&cid=afbe8fd3d73448c9&interfaceCode=b5018a28d5f8609f&pid=92b01a8207f5c404
返回值:
 
[root@simonsfan code]# curl -I  https://act.vip.iqiyi.com/api/process.action?cb=cb_1540200657317\&cid=afbe8fd3d73448c9\&interfaceCode=b5018a28d5f8609f\&pid=92b01a8207f5c404
HTTP/1.1 200
Server: nginx
Date: Mon, 22 Oct 2018 09:45:52 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 72
Connection: keep-alive
Accept-Charset: big5, big5-hkscs, cesu-8, euc-jp, euc-kr, gb18030, gb2312, gbk, ibm-thai, ibm00858, ibm01140, ibm01141, ibm01142, ibm01143, ibm01144, ibm01145, ibm01146, ibm01147, ibm01148, ibm01149, ibm037, ibm1026, ibm1047, ibm273, ibm277, ibm278, ibm280, ibm284, ibm285, ibm290, ibm297, ibm420, ibm424, ibm437, ibm500, ibm775, ibm850, ibm852, ibm855, ibm857, ibm860, ibm861, ibm862, ibm863, ibm864, ibm865, ibm866, ibm868, ibm869, ibm870, ibm871, ibm918, iso-2022-cn, iso-2022-jp, iso-2022-jp-2, iso-2022-kr, iso-8859-1, iso-8859-13, iso-8859-15, iso-8859-2, iso-8859-3, iso-8859-4, iso-8859-5, iso-8859-6, iso-8859-7, iso-8859-8, iso-8859-9, jis_x0201, jis_x0212-1990, koi8-r, koi8-u, shift_jis, tis-620, us-ascii, utf-16, utf-16be, utf-16le, utf-32, utf-32be, utf-32le, utf-8, windows-1250, windows-1251, windows-1252, windows-1253, windows-1254, windows-1255, windows-1256, windows-1257, windows-1258, windows-31j, x-big5-hkscs-2001, x-big5-solaris, x-compound_text, x-euc-jp-linux, x-euc-tw, x-eucjp-open, x-ibm1006, x-ibm1025, x-ibm1046, x-ibm1097, x-ibm1098, x-ibm1112, x-ibm1122, x-ibm1123, x-ibm1124, x-ibm1166, x-ibm1364, x-ibm1381, x-ibm1383, x-ibm300, x-ibm33722, x-ibm737, x-ibm833, x-ibm834, x-ibm856, x-ibm874, x-ibm875, x-ibm921, x-ibm922, x-ibm930, x-ibm933, x-ibm935, x-ibm937, x-ibm939, x-ibm942, x-ibm942c, x-ibm943, x-ibm943c, x-ibm948, x-ibm949, x-ibm949c, x-ibm950, x-ibm964, x-ibm970, x-iscii91, x-iso-2022-cn-cns, x-iso-2022-cn-gb, x-iso-8859-11, x-jis0208, x-jisautodetect, x-johab, x-macarabic, x-maccentraleurope, x-maccroatian, x-maccyrillic, x-macdingbat, x-macgreek, x-machebrew, x-maciceland, x-macroman, x-macromania, x-macsymbol, x-macthai, x-macturkish, x-macukraine, x-ms932_0213, x-ms950-hkscs, x-ms950-hkscs-xp, x-mswin-936, x-pck, x-sjis_0213, x-utf-16le-bom, x-utf-32be-bom, x-utf-32le-bom, x-windows-50220, x-windows-50221, x-windows-874, x-windows-949, x-windows-950, x-windows-iso2022jp
Access-Control-Allow-Credentials: true
X-Frame-Options: SAMEORIGIN

上述命令只会返回请求头信息,而不包含响应内容,可以使用 -l 参数来返回请求头和响应信息:

[root@simonsfan code]# curl -i  https://act.vip.iqiyi.com/api/process.action?cb=cb_1540200657317\&cid=afbe8fd3d73448c9\&interfaceCode=b5018a28d5f8609f\&pid=92b01a8207f5c404
HTTP/1.1 200
Server: nginx
Date: Mon, 22 Oct 2018 09:47:35 GMT
Content-Type: text/plain;charset=UTF-8
Content-Length: 72
Connection: keep-alive
Accept-Charset: big5, big5-hkscs, cesu-8, euc-jp, euc-kr, gb18030, gb2312, gbk, ibm-thai, ibm00858, ibm01140, ibm01141, ibm01142, ibm01143, ibm01144, ibm01145, ibm01146, ibm01147, ibm01148, ibm01149, ibm037, ibm1026, ibm1047, ibm273, ibm277, ibm278, ibm280, ibm284, ibm285, ibm290, ibm297, ibm420, ibm424, ibm437, ibm500, ibm775, ibm850, ibm852, ibm855, ibm857, ibm860, ibm861, ibm862, ibm863, ibm864, ibm865, ibm866, ibm868, ibm869, ibm870, ibm871, ibm918, iso-2022-cn, iso-2022-jp, iso-2022-jp-2, iso-2022-kr, iso-8859-1, iso-8859-13, iso-8859-15, iso-8859-2, iso-8859-3, iso-8859-4, iso-8859-5, iso-8859-6, iso-8859-7, iso-8859-8, iso-8859-9, jis_x0201, jis_x0212-1990, koi8-r, koi8-u, shift_jis, tis-620, us-ascii, utf-16, utf-16be, utf-16le, utf-32, utf-32be, utf-32le, utf-8, windows-1250, windows-1251, windows-1252, windows-1253, windows-1254, windows-1255, windows-1256, windows-1257, windows-1258, windows-31j, x-big5-hkscs-2001, x-big5-solaris, x-compound_text, x-euc-jp-linux, x-euc-tw, x-eucjp-open, x-ibm1006, x-ibm1025, x-ibm1046, x-ibm1097, x-ibm1098, x-ibm1112, x-ibm1122, x-ibm1123, x-ibm1124, x-ibm1166, x-ibm1364, x-ibm1381, x-ibm1383, x-ibm300, x-ibm33722, x-ibm737, x-ibm833, x-ibm834, x-ibm856, x-ibm874, x-ibm875, x-ibm921, x-ibm922, x-ibm930, x-ibm933, x-ibm935, x-ibm937, x-ibm939, x-ibm942, x-ibm942c, x-ibm943, x-ibm943c, x-ibm948, x-ibm949, x-ibm949c, x-ibm950, x-ibm964, x-ibm970, x-iscii91, x-iso-2022-cn-cns, x-iso-2022-cn-gb, x-iso-8859-11, x-jis0208, x-jisautodetect, x-johab, x-macarabic, x-maccentraleurope, x-maccroatian, x-maccyrillic, x-macdingbat, x-macgreek, x-machebrew, x-maciceland, x-macroman, x-macromania, x-macsymbol, x-macthai, x-macturkish, x-macukraine, x-ms932_0213, x-ms950-hkscs, x-ms950-hkscs-xp, x-mswin-936, x-pck, x-sjis_0213, x-utf-16le-bom, x-utf-32be-bom, x-utf-32le-bom, x-windows-50220, x-windows-50221, x-windows-874, x-windows-949, x-windows-950, x-windows-iso2022jp
Access-Control-Allow-Credentials: true
X-Frame-Options: SAMEORIGIN
 
var cb_1540200657317={"code":"Q00304","data":{},"msg":"用户未登录"}

3、curl -d “params” url 作用:使用-d发送带参数的请求(默认是post方式提交) 示例:

curl -d “cb=cb_1540200657317&cid=afbe8fd3d73448c9&interfaceCode=b5018a28d5f8609f&pid=92b01a8207f5c404” https://act.vip.iqiyi.com/api/process.action
返回:

curl -d "cb=cb_1540200657317&cid=afbe8fd3d73448c9&interfaceCode=b5018a28d5f8609f&pid=92b01a8207f5c404" https://act.vip.iqiyi.com/api/process.action 

var cb_1540200657317={"code":"Q00301","data":{},"msg":"接口代码不存在"}

作用和上面说的第一点一致,你也可以使用 -X GET 参数来指定GET方式提交请求:

curl -d "cb=cb_1540200657317&cid=afbe8fd3d73448c9&interfaceCode=b5018a28d5f8609f&pid=92b01a8207f5c404" -X  GET https://act.vip.iqiyi.com/api/process.action 

{"code":"Q00301","data":{},"msg":"接口代码不存在"}

4、curl -H Head_infos 作用:自定义Header头信息

示例:curl -H “User-Agent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36” -H “Referer:http://www.iqiyi.com” http://vip.iqiyi.com

curl  -H  "User-Agent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"  -H  "Referer:http://www.iqiyi.com"  http://vip.iqiyi.com

<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>QWS</center>
</body>
</html>

5、curl -L url 作用:跟踪链接url重定向,有些页面或接口被重定向,直接使用curl url会返回

<html>
    <head><title>301 Moved Permanently</title></head>
    <body bgcolor="white"> <center><h1>301 Moved Permanently</h1></center> <hr> 
    <center>nginx/1.14.0</center> </body>
</html>

但有时候使用-L会报如下错

curl: (60) Issuer certificate is invalid.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

这时候就要结合-k选项来使用

curl -L -k nginx.baidu.com

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

6、curl -O url…… 作用:下载指定资源文件到当前目录中 示例:

curl -O http://sr4.pplive.cn/cms/15/70/0dbb8ec002f1353e487d13fd949727a3.jpg -O http://sr4.pplive.cn/cms/39/88/4441be4257c21285c504509e479a43ea.jpg

curl -O http://sr4.pplive.cn/cms/15/70/0dbb8ec002f1353e487d13fd949727a3.jpg   -O   http://sr4.pplive.cn/cms/39/88/4441be4257c21285c504509e479a43ea.jpg 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41503  100 41503    0     0   102k      0 --:--:-- --:--:-- --:--:--  103k
100 23242  100 23242    0     0   217k      0 --:--:-- --:--:-- --:--:--  217k

7、curl –user 作用:指定用户名和密码访问资源

curl 172.16.0.13:9200 --user elastic:123456