一直传闻google和python有很紧密的联系,google adsense上的faq就是用的python,据说google爬虫程序最初也使用python做原型。不过,分析热视网上apache的日志
208.223.208.181 - - [24/Jan/2007:04:26:56 +0800] "GET /music HTTP/1.0" 200 7388 "-" "Python-urllib/1.16"208.223.208.181 - - [24/Jan/2007:04:26:58 +0800] "GET /movie/17537 HTTP/1.0" 200 8069 "-" "Python-urllib/1.16"208.223.208.181 - - [24/Jan/2007:04:26:56 +0800] "GET / HTTP/1.0" 200 38389 "-" "Python-urllib/1.16"208.223.208.181 - - [24/Jan/2007:04:26:59 +0800] "GET /movie HTTP/1.0" 200 9994 "-" "Python-urllib/1.16"66.249.65.201 - - [24/Jan/2007:04:52:48 +0800] "GET / HTTP/1.1" 200 38389 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
由于热视网是新站上线,所以还没有被google等收录,但是我的网站先是被一个叫做Python-urllib/1.16的agent访问,气候googlebot才来访问,这不由不让我觉得这个Python-urllib/1.16的agent就是google的bot程序。
