Skip to content
InternetTimes.
  • Home
  • Audit
  • Service Status
  • DNS ▼
    • DNS Checker
    • DNS Lookup
    • DNS Propagation
    • Reverse DNS
    • WHOIS Lookup
    • Domain Age
  • Network ▼
    • What is My IP
    • IP Lookup
    • Speed Test
    • Cloud Region Detector
  • Email ▼
    • MX Record Lookup
    • IP Blacklist Check
    • DMARC Analyzer
    • DKIM Lookup
    • Email Validator
  • Security ▼
    • SSL Certificate Checker
    • HTTP Header Inspector
    • Subdomain Enumerator
    • robots.txt Validator
    • Sitemap.xml Validator
    • HTTP/2 + HTTP/3 Test
    • Tech Detector
    • AEO Checker
    • AI Crawler Validator
  • Pricing
  • Blog
Sign in

Quick find — tools, articles, and pages

esc
↑ ↓ navigate ↵ open esc close
  1. Home
  2. Glossary
  3. AI bot opt-out

AI bot opt-out

Robots.txt directive blocking AI training crawlers.

Blocking GPTBot, ClaudeBot, CCBot etc. opts your site out of model training. Use our AI Crawler Validator to see who's allowed.

Try the Ai Crawler Validator tool →

Get one fundamentals article a week

DNS, IPs, email deliverability, TLS — explained for sysadmins and curious developers. No spam, unsubscribe in one click.

Internet Times

Audit any domain in one paste — DNS, email deliverability, TLS, network, crawlability and on-page SEO in a single scored report — then keep watching it for changes. Free to run, no account needed.

DNS & domain

DNS Checker DNS Lookup Reverse DNS WHOIS Lookup MX Record Lookup

IP & network

What is My IP IP Lookup IP Blacklist Check Speed Test SSL Checker HTTP Headers

Resources

Blog API docs About Contact Privacy Terms Disclaimer

© 2026 Internet Times. All rights reserved.

Press ⌘K to search anywhere

We respect your privacy

We use two essential cookies: one keeps you signed in, the other protects forms against cross-site requests. No analytics, no advertising, no third-party cookies. Privacy policy.