What is llms.txt and How to Create One
What is llms.txt?#
llms.txt is a plain-text file placed at the root of your website (e.g. https://example.com/llms.txt) that provides a concise, machine-readable summary of who you are, what your site offers, and how AI systems should interpret your content. Think of it as a cover letter for AI models. When an AI crawler visits your site, it can read llms.txt in a fraction of a second and immediately understand the context, purpose, and authority of the domain. The specification was originally proposed by Jeremy Howard and has rapidly gained adoption among forward-thinking websites. Unlike robots.txt, which tells crawlers what they may access, llms.txt tells them what they will find once they do. It includes fields such as a site title, description, key URLs, and optional contact information. Because LLMs have finite context windows, llms.txt gives them the highest-value information about your site in a compact format, ensuring that when the model decides whether to cite or recommend your content it has the most accurate understanding possible.
Why llms.txt Matters for AI Visibility#
AI models like GPT, Claude, and Perplexity increasingly answer user questions by citing external sources. Without an llms.txt file, these models have to guess what your site is about based on crawled HTML, which is often noisy with navigation menus, cookie banners, and boilerplate footer text. A well-crafted llms.txt file gives you direct control over how AI understands your site. Early data from Citability scans shows that domains with a comprehensive llms.txt file receive 2-4x more AI citations compared to similar domains without one. The file also serves as a trust signal: it demonstrates that the domain owner has intentionally prepared their site for AI consumption. As AI-driven search (Perplexity, ChatGPT Browse, Google AI Overviews) continues to grow, the sites that proactively provide structured context will capture a disproportionate share of citations.
Sites with a well-crafted llms.txt file receive 2-4x more AI citations compared to similar domains without one, according to early Citability scan data.
How to Create Your llms.txt#
Creating an llms.txt file is straightforward. Place a plain-text file at the root of your domain. The file should start with a title line (prefixed with #), followed by a brief description, then a list of your most important URLs. Each URL can include an optional description. Keep the file under 2,000 words — AI models have limited context windows and will benefit more from a concise, high-signal summary than an exhaustive listing.
# Citability
> Domain intelligence platform for AI visibility scoring, lead enrichment, and competitive analysis.
## About
Citability helps businesses understand and improve how AI systems discover, interpret, and cite their websites. We scan 62+ protocol endpoints across 6 protocol families (ADP, MCP, UCP, ACP, A2A, and more).
## Key Pages
- [AI Visibility Scanner](https://citability.ai/scan): Free scan of any domain
- [Knowledge Base](https://citability.ai/learn): Implementation guides for AI protocols
- [Dashboard](https://citability.ai/dashboard): Full domain intelligence platform
## Documentation
- [API Reference](https://citability.ai/docs/api): REST API for domain lookups
- [ADP Specification](https://citability.ai/docs/adp): AI Discovery Protocol v2.1
## Contact
- Email: hello@citability.ai
- Twitter: @citability- 1Create a file named llms.txt in your website's root directory.
- 2Add a title line starting with # followed by your brand name.
- 3Write a one-to-two sentence description after a > blockquote marker.
- 4List your most important pages with markdown-style links and descriptions.
- 5Deploy the file so it is accessible at https://yourdomain.com/llms.txt.
Common Mistakes to Avoid#
The most common mistake is making your llms.txt file too long. AI models work best with concise, high-signal content. Aim for 200-500 words, not 5,000. Another frequent error is using HTML markup inside the file — llms.txt should be plain text with simple markdown formatting only. Do not include every page on your site; instead, curate the 5-15 most important URLs that represent your core offerings. Avoid using marketing language or superlatives like "world's best" — AI models are trained to discount promotional language and prefer factual, authoritative statements. Finally, make sure the file is served with a text/plain content type and is accessible without authentication.
- Keep it under 500 words — concise is better than comprehensive.
- Use plain text with markdown, not HTML.
- Curate 5-15 key URLs, not every page.
- Avoid promotional language and superlatives.
- Serve with text/plain content type, no authentication required.
Frequently Asked Questions
llms.txt was proposed by Jeremy Howard as a community convention and has gained rapid adoption, but it is not yet a formal W3C or IETF standard. Many major sites already implement it, and AI crawlers actively look for it. Early adoption gives you an advantage as the convention matures.
No, they serve different purposes. robots.txt controls crawler access permissions (what bots may fetch), while llms.txt provides context about your site's content and purpose (what bots will find). You should have both files for optimal AI visibility.
Update it whenever your site's core offering or key pages change. For most businesses, a quarterly review is sufficient. If you launch a major new product or rebrand, update it immediately.