ToothyWikiInternals/CrawlerSightings

18-97-9-172.crawl.commoncrawl.org | ToothyWiki | ToothyWikiInternals | RecentChanges | Login

Things wot have crawled over toothycat.net recently

Googlebot/2.1 - data used by [Google] for the obvious purpose

http://www.almaden.ibm.com/cs/crawler - no idea what they're going to use the data for

ia_archiver - The [Internet Archive] is preserving toothycat.net for posterity (although we don't seem to have been added to the database yet).

NPBot - AIUI, [these] people crawl your site looking for trademarked stuff; they then notify the trademark owner so the owner can sue you if you don't have permission to use the trademarked stuff, and NP presumably gets a cut. MoonShadow is wondering about blocking this one.