We should ban Baiduspider!

For log, Baiduspider does not consider robots.txt enough! It updates robots.txt at  a very low frequency that your new robots.txt forbidden pages will be crawled within a month or longer time. Which is considered as a bad spider! Ban it!

从log来看，百度蜘蛛（Baiduspider）不理会robots.txt的存在！或者更新robots.txt的频率实在太低！以至于你新添了应用，同时更改了robots.txt，但百度蜘蛛依旧会在一个月内或更长时间内爬行你禁止了的页面！

基于此行为，建议禁止Baiduspider的访问！