I was wrong about robots.txt

A good lesson about unintended consequences that may come out of blocking all UserAgents in your robots.txt file.