- Is it not 'robots.txt' that can be configured to tell those 'bots (at = least the well-behaved ones) where and where not they can go on any given = site? Maybe you don't want to limit their probing. I'm just not sure I'd want = them crawling through every nook & crannie. -=3D Russ =3D- (Bot Fee) > -----Original Message----- > From: geocaching-bounce@xxxxxxxxxxxxx=20 > [mailto:geocaching-bounce@xxxxxxxxxxxxx] On Behalf Of Glenn > Sent: Monday, August 23, 2004 18:11 > To: geocaching@xxxxxxxxxxxxx > Subject: [GeoStL] interesting observation. Non-slaga related >=20 >=20 > - > I have been playing with the forum system for mogeo. When I=20 > am logged in,=20 > I can see when the google "spiders" and "web crawling bots"=20 > hit the site ,=20 > jump from page to page indexing the various pages. It is=20 > really weird, dont=20 > know how they figure out what to do and were to go or how the=20 > site was even=20 > there. The bots showed up just hours after turning the thing=20 > on and they=20 > visit often. >=20 > It has been up just a week and now googling just on "mogeo=20 > forums" will=20 > get you > mogeo.forumflash.com/index.php?act=3Didx that goes right to=20 > the site and it=20 > is 3rd on the list behind 2 other mogeo pages! Pretty neat=20 > and not bad. I=20 > thought you had to submit something to have the google bots=20 > come visit. >=20 > Bots and spiders.... sounds like something that would come=20 > out from under a=20 > log.. >=20 > --------------------- > Glenn >=20 >=20 >=20 > **************************************** > Our WebPage! Http://WWW.GeoStL.com =20 > Mail List Info.=20 > //www.freelists.org/cgi-bin/list?list_id=3Dgeocaching > Mail List FAQ's: //www.freelists.org/help/questions.html=20 > **************************************** > To unsubscribe from this list: > send an email to geocaching-request@xxxxxxxxxxxxx with=20 > 'unsubscribe' in the Subject field >=20 >=20 >=20 >=20 >=20 >=20 **************************************** Our WebPage! Http://WWW.GeoStL.com Mail List Info. //www.freelists.org/cgi-bin/list?list_id=geocaching Mail List FAQ's: //www.freelists.org/help/questions.html **************************************** To unsubscribe from this list: send an email to geocaching-request@xxxxxxxxxxxxx with 'unsubscribe' in the Subject field