一些常见的搜索引擎网络爬虫IP整理


一些常见的搜索引擎网络爬虫IP整理:

整理了一下各搜索引擎平台爬虫的信息,记录一下(有异议的地方请底下留言指正)。

以下整理的爬虫IP都是根据对应的User-Agent排序的,所以可能统一号段的IP分属不同UA(当然UA也是可以随意伪造的),加上网上搜集的一些但是UA不确定所以都放到每个分组的最后边。

百度爬虫IP列表:

爬虫Agent:Mozilla/5.0 (Windows NT 6.1; WOW64; Baidu Transcoder) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11
61.135.169.75
61.135.169.55
61.135.169.22
220.181.51.40
220.181.51.39
111.13.13.74
111.13.13.73
111.13.13.72
111.13.13.71
111.13.13.6
111.13.13.5
111.13.13.4
111.13.13.3
101.254.184.206
1.95.9.244
爬虫Agent:Mozilla/5.0 (compatible; heritrix/3.1.1 +http://www.baidu.com)
59.50.71.83
211.89.227.50
211.89.227.16
211.89.227.15
183.224.87.36
124.119.30.90
117.25.173.119
116.113.28.179
爬虫Agent:Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
61.191.191.87
61.184.241.206
61.183.41.95
59.49.46.165
58.222.20.226
58.218.204.136
45.113.253.53
27.151.30.176
27.151.28.24
222.216.28.45
222.186.3.229
220.181.51.81
220.181.51.76
220.181.51.75
220.181.51.74
220.181.51.110
220.181.51.109
220.181.51.107
220.181.108.97
220.181.108.96
220.181.108.87
220.181.108.86
220.181.108.84
220.181.108.83
220.181.108.81
220.181.108.80
220.181.108.79
220.181.108.78
220.181.108.77
220.181.108.76
220.181.108.75
220.181.108.186
220.181.108.184
220.181.108.181
220.181.108.179
220.181.108.178
220.181.108.177
220.181.108.175
220.181.108.174
220.181.108.173
220.181.108.172
220.181.108.171
220.181.108.170
220.181.108.168
220.181.108.166
220.181.108.165
220.181.108.161
220.181.108.158
220.181.108.157
220.181.108.156
220.181.108.155
220.181.108.153
220.181.108.152
220.181.108.151
220.181.108.150
220.181.108.149
220.181.108.147
220.181.108.146
220.181.108.145
220.181.108.144
220.181.108.143
220.181.108.142
220.181.108.141
220.181.108.140
220.181.108.139
220.181.108.123
220.181.108.122
220.181.108.120
220.181.108.119
220.181.108.117
220.181.108.116
220.181.108.115
220.181.108.114
220.181.108.113
220.181.108.112
220.181.108.111
220.181.108.110
220.181.108.109
220.181.108.108
220.181.108.107
220.181.108.106
220.181.108.105
220.181.108.104
220.181.108.103
220.181.108.101
220.181.108.100
219.153.68.137
219.138.135.236
202.46.52.201
202.46.51.156
202.46.51.132
183.61.171.152
183.61.171.137
183.2.242.150
180.97.35.165
180.76.15.9
180.76.15.8
180.76.15.7
180.76.15.6
180.76.15.5
180.76.15.34
180.76.15.33
180.76.15.32
180.76.15.31
180.76.15.30
180.76.15.29
180.76.15.28
180.76.15.26
180.76.15.24
180.76.15.23
180.76.15.22
180.76.15.21
180.76.15.19
180.76.15.18
180.76.15.17
180.76.15.163
180.76.15.162
180.76.15.161
180.76.15.158
180.76.15.157
180.76.15.156
180.76.15.155
180.76.15.154
180.76.15.153
180.76.15.152
180.76.15.151
180.76.15.150
180.76.15.15
180.76.15.149
180.76.15.148
180.76.15.147
180.76.15.145
180.76.15.144
180.76.15.143
180.76.15.142
180.76.15.141
180.76.15.14
180.76.15.138
180.76.15.137
180.76.15.136
180.76.15.13
180.76.15.12
180.76.15.11
180.76.15.10
14.29.49.206
124.237.78.217
124.232.151.230
124.228.91.179
123.125.71.98
123.125.71.94
123.125.71.92
123.125.71.91
123.125.71.90
123.125.71.89
123.125.71.87
123.125.71.86
123.125.71.85
123.125.71.84
123.125.71.83
123.125.71.82
123.125.71.81
123.125.71.78
123.125.71.74
123.125.71.73
123.125.71.70
123.125.71.69
123.125.71.60
123.125.71.58
123.125.71.57
123.125.71.56
123.125.71.55
123.125.71.53
123.125.71.52
123.125.71.50
123.125.71.49
123.125.71.47
123.125.71.45
123.125.71.44
123.125.71.43
123.125.71.42
123.125.71.41
123.125.71.39
123.125.71.38
123.125.71.34
123.125.71.33
123.125.71.32
123.125.71.31
123.125.71.30
123.125.71.29
123.125.71.28
123.125.71.27
123.125.71.26
123.125.71.23
123.125.71.22
123.125.71.20
123.125.71.19
123.125.71.18
123.125.71.17
123.125.71.15
123.125.71.14
123.125.71.13
123.125.71.12
123.125.71.116
123.125.71.115
123.125.71.114
123.125.71.113
123.125.71.111
123.125.71.110
123.125.71.109
123.125.71.108
123.125.71.107
123.125.71.106
123.125.71.105
123.125.71.103
123.125.71.102
123.125.71.100
123.125.67.152
123.125.67.148
122.10.89.22
120.39.244.111
117.34.115.181
117.25.139.66
117.25.139.42
117.25.139.35
117.25.139.34
116.9.156.37
116.55.241.24
115.47.26.143
115.231.26.31
113.113.120.28
112.124.57.11
爬虫Agent:Baiduspider+(+http://www.baidu.com/search/spider.htm)
61.146.178.41
61.146.178.186
爬虫Agent:Baidu-YunGuance-VSBot(ce.baidu.com)
111.206.36.17

#网络搜集
180.149.130.*
180.76.5.66

google爬虫列表:

爬虫Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/27.0.1453 Safari/537.36
45.79.76.220
爬虫Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Search Console) Chrome/27.0.1453 Safari/537.36
66.249.84.239
66.249.84.233
爬虫Agent:Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.79.26
66.249.79.142
66.249.79.138
66.249.75.7
66.249.75.23
66.249.75.15
66.249.69.48
66.249.69.15
66.249.65.55
66.249.65.162
203.208.60.158
203.208.60.152
爬虫Agent:Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.79.69
66.249.79.64
66.249.79.25
66.249.79.24
66.249.79.231
66.249.79.210
66.249.79.194
66.249.79.187
66.249.79.180
66.249.79.174
66.249.79.171
66.249.79.168
66.249.79.146
66.249.79.127
66.249.75.8
66.249.75.24
66.249.75.16
66.249.69.254
66.249.69.213
66.249.69.209
66.249.69.206
66.249.69.197
66.249.69.192
66.249.69.19
66.249.69.188
66.249.69.176
66.249.69.140
66.249.69.137
66.249.69.134
66.249.69.128
66.249.69.11
66.249.66.67
66.249.66.64
66.249.66.254
66.249.66.197
66.249.66.186
66.249.66.183
66.249.66.180
66.249.66.168
66.249.66.166
66.249.66.162
66.249.66.158
66.249.66.154
66.249.66.150
66.249.66.142
66.249.66.139
66.249.66.138
66.249.66.135
66.249.66.134
66.249.66.126
66.249.65.61
66.249.65.60
66.249.65.59
66.249.65.57
66.249.65.56
66.249.65.186
66.249.65.183
66.249.65.180
66.249.65.168
66.249.65.166
66.249.64.50
66.249.64.140
66.249.64.137
66.249.64.134
54.159.17.214
203.208.60.157
203.208.60.156
203.208.60.155
203.208.60.154
203.208.60.153
203.208.60.151
203.208.60.150
203.208.60.149
122.155.168.168
爬虫Agent:Mozilla/5.0 (compatible; Google-Site-Verification/1.0)
66.249.82.245
66.249.82.242
66.249.82.239
66.249.82.208
66.249.82.205
爬虫Agent:DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
66.249.66.192

#网上搜集
67.221.235.*
66.249.68.*
66.249.67.*
203.208.60.*
66.249.72.*
66.249.71.*

360爬虫列表:

爬虫Agent:Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
42.236.49.171
42.236.10.*
爬虫Agent:Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
182.118.25.*
182.118.22.*
182.118.21.*
182.118.20.*
爬虫Agent:Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
180.153.187.*
180.153.186.*
180.153.185.*
爬虫Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
101.226.169.*
101.226.168.*
101.226.167.*
101.226.166.*

必应爬虫列表:

爬虫Agent:Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
40.77.167.95
40.77.167.39
40.77.167.19
207.46.13.183
207.46.13.151
207.46.13.64
207.46.13.13
157.55.39.212
157.55.39.203
157.55.39.188
157.55.39.169
157.55.39.131

#网上搜集
65.52.108.142
74.125.75.3
74.125.64.81
74.125.44.82
74.125.158.86
74.125.156.82
65.52.108.146

sogou爬虫列表:

#网上搜集
220.181.94.231
220.181.94.229
220.181.94.223
220.181.125.71
220.181.125.69
220.181.125.45
123.126.50.76
123.126.50.70
220.181.94.237
220.181.94.235
220.181.94.233
220.181.94.225
220.181.94.224
220.181.94.213
220.181.125.43
220.181.125.162
220.181.125.108
202.85.214.159
123.126.50.81
123.126.50.78
123.126.50.77
123.126.50.75
123.126.50.74
123.126.50.72
123.126.50.71
123.126.50.69
123.126.50.68
123.126.50.66

有道爬虫列表:

#网络搜集
61.135.217.27
61.135.249.9
61.135.249.89
61.135.249.78
61.135.249.75
61.135.249.62
61.135.249.6
61.135.249.59
61.135.249.37
61.135.249.32
61.135.249.249
61.135.249.239
61.135.249.236
61.135.249.232
61.135.249.231
61.135.249.229
61.135.249.220
61.135.249.216
61.135.249.213
61.135.249.210
61.135.249.191
61.135.249.187
61.135.249.175
61.135.249.171
61.135.249.146
61.135.249.144
61.135.249.143
61.135.249.137
61.135.249.122
61.135.249.112
61.135.248.228
61.135.248.226
61.135.248.219
61.135.248.210
61.135.248.203
61.135.220.9
61.135.220.3

msn爬虫列表:

#网络搜集
207.46.204.38
207.46.204.37
207.46.204.35
207.46.204.128
207.46.199.244
207.46.199.242
207.46.199.213
207.46.194.95
207.46.194.91
207.46.194.88
207.46.194.85
207.46.194.78
207.46.194.67
207.46.194.55
207.46.194.140
207.46.194.130
207.46.194.129
207.46.204.44
207.46.204.43
207.46.204.42
207.46.204.40
207.46.204.39
207.46.204.34
207.46.204.31
207.46.204.30
207.46.204.138
207.46.204.137
207.46.204.133
207.46.204.129
207.46.199.249
207.46.199.246
207.46.199.240
207.46.199.238
207.46.199.229
207.46.199.218
207.46.199.216
207.46.199.215
207.46.195.167
207.46.194.96
207.46.194.94
207.46.194.90
207.46.194.89
207.46.194.87
207.46.194.83
207.46.194.82
207.46.194.81
207.46.194.77
207.46.194.75
207.46.194.74
207.46.194.73

头条搜索

#官网提供
110.249.201.0/24
110.249.202.0/24
111.225.148.0/24
111.225.149.0/24
220.243.135.0/24
220.243.136.0/24
60.8.123.0/24
60.8.151.0/24

相关阅读 >>

seo一般要做哪些事情?

写网站文章的内容需要注意些什么?

百度或谷歌为什么不收录我的网站?

一些常见的搜索引擎网络爬虫ip整理

几个新手seoer最常见问题解答

怎么写出搜索引擎喜欢的标题,获得高得分的网站标题写法

网站权重是指什么?

网站怎么不被收录?如何判断网站是否被k?

关键字也有权重吗?

更多相关阅读请进入《搜索引擎》频道 >>



打赏

取消

感谢您的支持,我会继续努力的!

扫码支持
扫码打赏,您说多少就多少

打开支付宝扫一扫,即可进行扫码打赏哦

分享从这里开始,精彩与您同在

评论

管理员已关闭评论功能...