robots.txt
Posted: 7/12/1999 1:43:23 PMBy: Comfortably Anonymous
Times Read: 2,146
Likes: 0 Dislikes: 0
Topic: Test-Only
Make sure and add /books to robots.txt so that crawlbots do not suck down all the books in the library as seems to be happening right now!
Rating: (You must be logged in to vote)