Robots on the Web
So I just discovered that you could actually block spiders from crawling your site. I wanted that for the Church's site, but I didn't know how (I actually didn't bother to read up on it). But anyway, I know now that in order to do that put a robots.txt file in the root directory.
so now, i need to protect the forums site by putting a robots.txt into the root directory
so now, i need to protect the forums site by putting a robots.txt into the root directory