llms.txt

What It Is

llms.txt is a root Markdown file offering a human-curated shortlist of a site's most important content for LLMs to use during live retrieval, without indexing the whole site.

Honest 2026 Status

  • No major LLM vendor (OpenAI, Anthropic, Perplexity) commits to fetching it yet.
  • Google states no Search system reads or acts on it ("the next keywords meta tag").
  • Citation studies show no measurable improvement.

Verdict: ship it anyway

  1. Negligible cost — ~20 minutes, zero maintenance once auto-generated from the content pipeline.
  2. Real optionality — the moment any vendor honors the spec, you're already compliant (first-mover on a tiny file).
  3. The .md-suffix convention is independently valuable today — publishing each page as both .html and .html.md gives RAG pipelines a tokenizer-friendly extraction target (the .md is converted to text first; HTML is parsed last).

How It Applies to Marketing Factory

A perfect low-cost, high-optionality item for the factory's llm-search technical baseline: auto-emit llms.txt and per-page .md variants from the build pipeline. Treat it as cheap insurance, not a citation lever — the measurable wins come from content-authority, schema-markup, and answer-first content structure. Declare crawler access in robots-txt alongside it.

  • llm-search — llms.txt is part of the GEO technical baseline
  • robots-txt — the companion crawler-policy file
  • schema-markup — the structured-data layer that does move citations

Referenced from: llm-search-visibility-and-content-metrics