[GeoStL] Re: interesting observation. Non-slaga related
- From: "tklnhl" <sydstyr@xxxxxxxxxx>
- To: <geocaching@xxxxxxxxxxxxx>
- Date: Mon, 23 Aug 2004 19:25:36 -0500
-
Note: Glenn does not want bots or spiders crawling through his private
areas...
Just an observation ...
From: "Glenn" <Glenn@xxxxxxxxxx>
> -
> i think that they are limited to main pages and none of the private areas.
>
> At 06:19 PM 8/23/2004, Russ Wilner wrote:
>
> >-
> >Is it not 'robots.txt' that can be configured to tell those 'bots (at =
> >least
> >the well-behaved ones) where and where not they can go on any given =
> >site?
> >
> >Maybe you don't want to limit their probing. I'm just not sure I'd want =
> >them
> >crawling through every nook & crannie.
> >
> >
> > -=3D Russ =3D- (Bot Fee)
> >
> > > -----Original Message-----
> > > From: geocaching-bounce@xxxxxxxxxxxxx=20
> > > [mailto:geocaching-bounce@xxxxxxxxxxxxx] On Behalf Of Glenn
> > > Sent: Monday, August 23, 2004 18:11
> > > To: geocaching@xxxxxxxxxxxxx
> > > Subject: [GeoStL] interesting observation. Non-slaga related
> > >=20
> > >=20
> > > -
> > > I have been playing with the forum system for mogeo. When I=20
> > > am logged in,=20
> > > I can see when the google "spiders" and "web crawling bots"=20
> > > hit the site ,=20
> > > jump from page to page indexing the various pages. It is=20
> > > really weird, dont=20
> > > know how they figure out what to do and were to go or how the=20
> > > site was even=20
> > > there. The bots showed up just hours after turning the thing=20
> > > on and they=20
> > > visit often.
> > >=20
> > > It has been up just a week and now googling just on "mogeo=20
> > > forums" will=20
> > > get you
> > > mogeo.forumflash.com/index.php?act=3Didx that goes right to=20
> > > the site and it=20
> > > is 3rd on the list behind 2 other mogeo pages! Pretty neat=20
> > > and not bad. I=20
> > > thought you had to submit something to have the google bots=20
> > > come visit.
> > >=20
> > > Bots and spiders.... sounds like something that would come=20
> > > out from under a=20
> > > log..
> > >=20
> > > ---------------------
> > > Glenn
> > >=20
> > >=20
> > >=20
> > > ****************************************
> > > Our WebPage! Http://WWW.GeoStL.com =20
> > > Mail List Info.=20
> > > //www.freelists.org/cgi-bin/list?list_id=3Dgeocaching
> > > Mail List FAQ's: //www.freelists.org/help/questions.html=20
> > > ****************************************
> > > To unsubscribe from this list:
> > > send an email to geocaching-request@xxxxxxxxxxxxx with=20
> > > 'unsubscribe' in the Subject field
> > >=20
> > >=20
> > >=20
> > >=20
> > >=20
> > >=20
> >
> > ****************************************
> > Our WebPage! Http://WWW.GeoStL.com
> > Mail List Info.
//www.freelists.org/cgi-bin/list?list_id=geocaching
> > Mail List FAQ's: //www.freelists.org/help/questions.html
> > ****************************************
> >To unsubscribe from this list:
> > send an email to geocaching-request@xxxxxxxxxxxxx with 'unsubscribe' in
> > the Subject field
>
> ****************************************
> Our WebPage! Http://WWW.GeoStL.com
> Mail List Info. //www.freelists.org/cgi-bin/list?list_id=geocaching
> Mail List FAQ's: //www.freelists.org/help/questions.html
> ****************************************
> To unsubscribe from this list:
> send an email to geocaching-request@xxxxxxxxxxxxx with 'unsubscribe' in
the Subject field
>
>
>
>
****************************************
Our WebPage! Http://WWW.GeoStL.com
Mail List Info. //www.freelists.org/cgi-bin/list?list_id=geocaching
Mail List FAQ's: //www.freelists.org/help/questions.html
****************************************
To unsubscribe from this list:
send an email to geocaching-request@xxxxxxxxxxxxx with 'unsubscribe' in the
Subject field
Other related posts:
