Skip to main content
LLM Metrix

The Technical Playbook for AI Visibility

llms.txt, structured data, entity disambiguation, and the crawl signals that determine whether AI engines understand your site. A code-first guide for engineers implementing AI visibility.

~25 minTechnicalCode examples
  1. 1
    Step 1: The Crawlers

    Which AI Crawlers Hit Your Site

    Every major AI engine runs its own web crawler to build and refresh training data and retrieval indices. Here are the ones you need to know about and their current user agent strings.

    AI crawler user-agent strings (for robots.txt rules)
    OpenAI / ChatGPTGPTBot
    Anthropic / ClaudeClaudeBot
    PerplexityPerplexityBot
    Google GeminiGoogle-Extended
    Meta AImeta-externalagent
    Common CrawlCCBot
    Check your robots.txt firstIf your robots.txt has User-agent: * followed by Disallow: /, you're blocking all AI crawlers. Add explicit Allow rules for each crawler listed above.
  2. 2
    Step 2: llms.txt

    Implementing llms.txt

    llms.txt is a Markdown file served at /llms.txt that gives AI crawlers a structured summary of your brand, your canonical content, and which pages to prioritize. Think of it as robots.txt for brand context.

    Example llms.txt structure
    # LLM Metrix
> AI visibility tracking platform for brands and agencies

## About
LLM Metrix monitors brand presence across 7 AI search engines:
ChatGPT, Perplexity, Gemini, Claude, Grok, Meta AI, and DeepSeek.
It tracks citation frequency, share-of-voice, and provides GEO
recommendations.

## Key Pages
- [Homepage](https://llmmetrix.com/)
- [Features](https://llmmetrix.com/features/)
- [Pricing](https://llmmetrix.com/pricing/)
- [Getting Started](https://llmmetrix.com/start-here/)

## Documentation
- [Knowledge Base](https://llmmetrix.com/knowledge-base/)
- [Tutorials](https://llmmetrix.com/tutorials/)

    Deployment checklist

    • Serve at yourdomain.com/llms.txt with Content-Type: text/plain; llms.txt (plural) is the llmstxt.org filename, and the path our free auditor fetches
    • Reference it in your sitemap.xml alongside sitemap entries
    • Keep it under 100KB (AI crawlers have fetch size limits)
    • Update it when you launch major new products or pages
    • Use the free generator to build a first version in minutes
    Generate your llms.txt free
  3. 3
    Step 3: Schema

    Structured Data for AI Engines

    AI engines use structured data to build knowledge graph entries for brands and entities. The more complete your schema, the lower the chance of hallucination, and the higher your citation rate.

    OrganizationCritical| Homepage, About page
    nameurllogodescriptionsameAs (LinkedIn, Crunchbase, Wikipedia)foundingDatenumberOfEmployees
    Product / SoftwareApplicationHigh| Product and feature pages
    namedescriptionapplicationCategoryoffers (price, priceCurrency)aggregateRatingscreenshot
    FAQPageHigh| All pages with Q&A content
    mainEntity (Question > acceptedAnswer)Each answer self-contained (no external refs)Max 150 words per answer
    Article / BlogPostingMedium| Blog posts, knowledge base articles
    headlineauthor (Person with sameAs)datePublisheddateModifiedpublisher (Organization)image
    sameAs is the most important fieldThe sameAs array on your Organization schema links your website entity to your presence in external knowledge bases (LinkedIn, Wikipedia, Crunchbase, Wikidata). AI engines use these to resolve entity disambiguation. It's the single field most likely to reduce hallucinations.
  4. 4
    Step 4: Verification

    Testing Your Implementation

    After deploying llms.txt and schema, verify things are working before waiting weeks for citation data.

    Google Rich Results Testsearch.google.com/test/rich-results

    Validate FAQPage and Article schema. If Google can parse it, AI engines can too

    Schema.org Validatorvalidator.schema.org

    Full JSON-LD validation with error messages for missing required fields

    LLM Metrix Hallucination Checker/free-ai-seo-tools/hallucination-checker

    Confirm AI engines are producing accurate brand facts after schema deployment

    LLM Metrix Schema Validator/free-ai-seo-tools/schema-validator

    Fetch the deployed page and validate the JSON-LD actually served (the LLM Metrix tool that reads your markup)

    LLM Metrix Content Grader/free-ai-seo-tools/content-grader

    Score any URL for content structure and AI-readability (it reads page text, not markup)

    Allow 2–4 weeksAfter deploying, AI engines re-crawl at their own cadence, typically 2–4 weeks for content changes to propagate into citation behavior. LLM Metrix re-scans daily on paid plans, so you'll see movement as it starts.
  5. 5
    Step 5: Next Steps

    Monitoring with LLM Metrix

    Technical implementation is the foundation. LLM Metrix is the feedback loop that tells you whether it's working.

    Citation Intelligence

    Verify which sources are cited after your technical changes go live, at page level where engines return links (Perplexity always; others when grounding is on). That is the ground truth for whether crawlers picked up your updates.

    Hallucination Checker

    Detect when AI engines are producing factually wrong information about your brand, often a sign of missing or conflicting schema.

    GEO Recommendations

    Technical and content recommendations tagged critical through low, each with the scan finding behind it, useful for building a sprint backlog.

    Content Grader

    Score any URL for AI-readability: structure, entity clarity, and direct-answer patterns.

    Your implementation checklist

    1. 1

      Generate and publish llms.txt

      Create a machine-readable brand context file at yourdomain.com/llms.txt using our free generator. This is the first thing AI crawlers look for.

    2. 2

      Audit your robots.txt

      Ensure you're not accidentally blocking AI crawlers. GPTBot, ClaudeBot, PerplexityBot, and Google-Extended all need explicit allow rules if your robots.txt has a wildcard block.

    3. 3

      Add FAQPage schema to key pages

      Every product and feature page should have FAQ structured data wrapping your Q&A content. This is the highest-ROI schema type for AI citation.

    4. 4

      Implement Organization and Product schema

      Entity schema helps AI engines build an accurate knowledge graph entry for your brand, the foundation of non-hallucinated AI mentions.

    5. 5

      Run LLM Metrix to verify indexing

      After deploying, use LLM Metrix to confirm AI engines are seeing your updates and citation rates are moving in the right direction.

Ready to see your score?

Run your first AI visibility scan in under 2 minutes. No credit card required.

Start my free scanBrowse the tutorials first

Continue your learning journey

New to AI Search

Start From Zero

A complete beginner's guide to understanding AI search, why your brand needs to be visible in it, and exactly what to do first.

SEO Professional

Migrate Your Skills

Everything you know about search optimization still matters, but AI engines play by different rules.