llms.txt
llms. txt is a proposed plain-text file placed at the root of a website (e. g. /llms. txt) that summarizes the site's purpose, lists its most important canonical URLs, and provides AI crawlers with a compact, structured map of what the site is authoritative on. It is the AI-engine analog to robots.txt and sitemap.xml, designed specifically to help large language models index, ground, and cite the right pages. Why it matters: As ChatGPT, Perplexity, Claude, Google AI Overviews, and Bing Copilot increasingly drive discovery, llms.txt is becoming a meaningful AEO and GEO infrastructure layer. A well-crafted llms.txt tells AI engines exactly which pillar guides, services, and authoritative resources to cite when answering questions in the brand's domain — reducing the risk of being misrepresented or omitted. Sites without llms.txt are not penalized, but sites with a clean, accurate llms.txt give themselves a structural advantage in AI citation outcomes. Smart Money Media's own llms.txt is publicly available at /llms.txt, and any site can generate a spec-compliant file in 30 seconds with our free llms.txt generator at /tools/llms-txt-generator.
Why llms.txt matters
This file prevents AI models from hallucinating incorrect facts or citing outdated landing pages by providing a definitive single source of truth for your site’s hierarchy. It optimizes the limited token window of a crawler, ensuring the most relevant brand data is ingested before the model reaches its context limit.
In practice
A digital editor at a publication like The Verge or Smart Money Media creates a Markdown-formatted file at their root domain to prioritize specific diagnostic guides over archive pages for Perplexity crawlers.
Common mistake
Treating the file as a comprehensive sitemap for humans rather than a condensed Markdown summary intended for LLM context windows and token efficiency.
How it connects
This architectural file directly feeds into Answer Engine Optimization (AEO) and provides the grounding data used by Generative Engine Optimization (GEO) strategies.
Learn more:
→ llms.txt Pillar GuideArticles About llms.txt
Deep-dive guides and tactical breakdowns from our editorial team.
How to Create llms.txt for AI SEO
Building a machine-readable index gives AI agents direct access to your most important context without relying entirely on traditional search crawlers.
AI Overview Optimization: A Complete Brand Guide
The shift to zero-click search means traditional rankings are no longer enough. Discover why ai overview optimization is your key to sustained brand visibility.
llms.txt File: Get Cited by ChatGPT, Claude & Perplexity
Discover how an llms.txt file maps your website for AI agents and RAG systems, helping you build verifiable brand authority in an era of zero-click search.
llms.txt Generator: The Complete Guide for AI Visibility
Are generative AI engines actually reading your content? Learn how an llms.txt generator structures data and whether it improves your brand's visibility.
Frequently Asked Questions
What is llms.txt?
In short: llms.txt is llms. See the full definition above for context.
How does this file differ from a standard robots.txt or XML sitemap?
While robots.txt dictates crawling permissions and sitemaps list every URL for indexing, llms.txt provides a high-level narrative and curated list of canonical resources. It functions more as an executive summary for AI models to understand a site's core value proposition quickly.
Where should this file be placed on a web server for maximum visibility?
The file should reside in the root directory of your server, typically accessible via the /llms.txt path. This standardized location allows crawlers from OpenAI, Anthropic, and other labs to discover your structured data automatically without custom instructions.
What is the best formatting style to use within the text file?
Markdown is the preferred format because modern large language models are natively trained to parse its hierarchical structure. Using simple headers and bulleted lists helps the AI identify the relationship between your primary topics and their corresponding URLs.
Related Terms
ChatGPT is the conversational AI assistant developed by OpenAI, launched in November…
Answer Engine Optimization (AEO)Answer Engine Optimization (AEO) is the discipline of structuring web content so that…
AI Visibility AuditAn AI visibility audit is a structured assessment of how often and how favorably a brand…
AI SitemapAn AI sitemap is a machine-readable file — most commonly an llms.txt or llms-full.txt —…
ClaudeClaude is Anthropic's family of large language models, used in the Claude.ai consumer…
ClaudeBotClaudeBot is Anthropic's web crawler, used both for collecting training data and for…