Methodology

How do we compare AI bot traffic trackers?

Every report on Bot Trackers follows the same process, so that prices, features and crawler details can be checked by anyone.

Which sources do we use?

  1. Vendor and operator pages first. Pricing, plan limits and features come from each vendor's pricing, product and documentation pages. User agents, purposes and IP ranges come from the crawler operators' own documentation.
  2. Independent sources second. Studies, standards documents and reputable reporting are used for context and cited with a numbered source.
  3. Dated checks. Every table states the date on which it was checked. When sources disagree, we follow the vendor or operator page and say so.

Which criteria matter for a bot tracker?

CriterionWhat we look at
Data sourceCDN integrations, log upload, log API or a JavaScript tag; supported hosts and log formats
Bot coverageWhich AI user agents are recognised, and whether new ones are added
Purpose classificationTraining, search index and user-triggered fetches reported separately
VerificationIP-range or reverse-DNS checks to separate real crawlers from spoofed user agents
Page and status detailFetched URLs, status codes, blocked requests, crawl frequency
Connection to outcomesWhether crawler data is linked to AI citations and referral visits
Price to startWhich plan includes bot analytics, and what it costs

What do we not do?

  • We do not publish star ratings, invented scores or anonymous reviews.
  • We do not present illustrative examples as real traffic data; samples are labelled as such.
  • We correct errors when a vendor or reader points them out and note the new check date.

How often are reports updated?

Crawler names, vendor features and prices change often. We re-check every report at least monthly and whenever an operator documents a new crawler or a vendor changes its pricing.