#Robots.txt file from http://www.51haojob.com #All robots will spider the domain User-agent:* Disallow:/