Google says its search ignores llms.txt, and no other AI engine has said it reads one
Resources · Your website
Google says its search ignores llms.txt, and no other AI engine has said it reads one
15% of the apartment community websites we checked publish an llms.txt, and none of the AI answers we have recorded linked one. Google says its search ignores the file, no other AI company has said it reads one, and the largest log study found almost no AI search crawler asking for it. Here is what the file is, what the evidence says, and how to add one in a few minutes if you choose to.
By HaloScore TeamUpdated October 2026 · 8 min readBallast Commons welcomes dogs and cats, with a limit of two pets per home.
Its website lists the pet fees and a fenced dog run beside building B.
# Ballast Commons> Apartments in Maple Heights, WA## Key pages- [Floor plans](/floor-plans)- [Pet policy](/pets)Key findings
- Few apartment websites have one. 15% of the community websites we checked on October 15, 2026 publish an llms.txt file.
- No AI answer we recorded linked one. 0% of the AI answers we have recorded cited an llms.txt. That shows it is not shown as a source; it cannot show whether a crawler read it.
- Google says its search ignores it. Google's guide for its AI features says an llms.txt will "neither harm nor help" a site in Google Search. OpenAI, Anthropic, Microsoft and Perplexity have not said they read one.
- Optional, and quick. If your platform makes one for you, or you can add one in a few minutes, it does no harm. It is not a priority.
HaloScore data
15% of the apartment websites we checked publish an llms.txt
HaloScore puts the questions renters ask to the major AI engines every month and records every source each answer cites. For this article we also requested /llms.txt from the website of every apartment community we measure, on October 15, 2026, the way any automated reader would.
Few apartment websites have one. 15% of the community websites we could read served a real llms.txt file. Another 2% answered that address with an ordinary web page, which an automated check can mistake for the file, and 84% had nothing there.
The files we found were of three kinds. Some were a short summary written for the community: its name, address, leasing phone, homes and amenities. Some were a template filled in by the management company's website for each of its communities. And some were a list of the site's recent blog posts, made by a website plugin and refreshed every day.
No AI answer we have recorded linked one. Across every AI answer HaloScore has recorded, 0% cited an llms.txt file. An AI engine shows the pages it took its answer from, and a summary file for machines would rarely be one of them, so this tells us only that the file is not shown as a source. It cannot tell us whether a crawler read it.
Figure 1
Most apartment websites have no llms.txt
What community websites returned when we requested /llms.txt, as a share of the websites we could read.
Source: HaloScore check of measured apartment community websites, October 15, 2026.
| What the website returned | Share of community websites |
|---|---|
| A real llms.txt file | 15% |
| An ordinary web page instead | 2% |
| Nothing | 84% |
Key takeaway
On apartment websites an llms.txt is uncommon, and it has not appeared as a source in any AI answer we have recorded.
The basics
An llms.txt is a short summary of a website, written for AI tools
The idea. An llms.txt was proposed in 2024 by Jeremy Howard of Answer.AI: "We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content." The file "offers brief background information, guidance, and links to detailed markdown files."[1] The only required line is a heading with the name of the site; most files add a one-line summary and a list of the site's key pages.
What it was built for. The proposal's own page says that "llms.txt files are used most heavily for software documentation, where coding agents follow them to find API references and tutorials."[1] That is why OpenAI, Anthropic and Perplexity each publish one for their developer documentation, while saying nothing about reading other websites' files.
What it is not. It does not control which AI crawlers visit your site or what they may use. That is the job of robots.txt. Common Crawl, which runs one of the largest open web crawls, puts it plainly: "llms.txt is not an access-control mechanism, and nothing obliges any crawler to read it."[2]
The research
Google says its search ignores the file, and log studies find almost nothing reads it
Google: "neither harm nor help." Google's guide to its AI features in Search, updated in July 2026, says: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." On llms.txt by name, it adds that creating one for other services "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."[3] That covers AI Overviews and AI Mode, and Google's own Chrome browser treats the file as optional too: its Lighthouse check marks a site without one as "Not Applicable", "as providing the file is optional at the moment."[4]
The other AI companies have not said they read one. OpenAI, Anthropic and Perplexity each document their crawlers, and each describes robots.txt as the way a site owner controls them. None of the three pages says the company's crawlers read a website's llms.txt.[5][6][7] We found no statement from Microsoft either way.
The largest log study found almost no AI search crawler asking for it. Ahrefs examined the server logs of 137,210 websites that use its analytics. It found that "28% of the 137K domains using Ahrefs Web Analytics publish an llms.txt file" and that "97% of those files received zero traffic in May 2026." AI search crawlers, the ones that fetch pages to answer a question, made "just 1.1% of AI bot requests" to those files.[8] Ahrefs notes that its customers "skew more technical and SEO-aware than the web at large, so treat the 28% adoption figure as an upper bound."
Across the web, most files are made by a platform. Common Crawl's analysis of its July 2026 crawl found that "for /llms.txt, 11.72% of sampled URLs returned a text body", that "68.27% of the corpus is templated", and that one website builder, Wix, accounts for "41.34% of the corpus."[2] The HTTP Archive's Web Almanac, which covers a much broader sample of the web, found the file on 2.13% of desktop sites.[9]
Figure 2
How common llms.txt is depends on which websites you count
Four different samples, measured different ways; read each row on its own terms.
| Study | Which websites | Share with an llms.txt |
|---|---|---|
| HTTP Archive Web Almanac, 2025 | A broad crawl of the whole web | 2.13% of desktop sites |
| Common Crawl, July 2026 crawl | Hosts Common Crawl can fetch | 11.72% of sampled hosts |
| Ahrefs, May 2026 | Websites using Ahrefs' analytics | 28% (an upper bound, by Ahrefs' account) |
| HaloScore, October 15, 2026 | Apartment community websites we measure | 15% |
Source: Web Almanac 2025;[9] Common Crawl, August 31, 2026;[2] Ahrefs, June 15, 2026;[8] HaloScore check, October 15, 2026.
Key takeaway
No AI engine has said it reads a website's llms.txt, Google says its search ignores the file, and the largest log study found almost no AI search crawler requesting one.
The takeaway
Treat an llms.txt as optional, and put the time into what AI engines cite
An llms.txt is cheap to publish and unproven. Three points to plan around:
- 01Nothing shows it helps an apartment community today.Google says it neither harms nor helps in Google Search, and no other AI company has said it reads one.
- 02It does no harm.If your website platform makes one for you, or you can add one in a few minutes, keep it accurate and leave it there.
- 03The time is better spent elsewhere.AI engines cite your website, your listing sites and your Google Business Profile, and those are where your facts need to be right first.
How to do it
A short file, in a few minutes, if you choose to add one
- Check whether your platform already makes one. Open yoursite.com/llms.txt in a browser. Wix generates and maintains the file automatically,[10] and on WordPress the Yoast SEO plugin can create one when you switch it on under its AI tools settings.[11] If your website is run by your property management software provider, ask them whether they offer one.
- If you write it yourself, keep it short. Create a plain-text file called llms.txt at the root of your website, next to robots.txt. Start with a heading line with the community's name, then a one-line summary, then the essentials: address, leasing phone, the kinds of homes, the main amenities and the pet policy, followed by links to your key pages (floor plans, amenities, pets, neighborhood, contact).
- Leave prices out. Link your live availability page instead, so the file never needs a hand update when rents change.
- Match your website and listings word for word. The same community name, address and phone as everywhere else.
- Check that it loads as text. Open yoursite.com/llms.txt and make sure you see your plain text, not your homepage. A website that answers every address with its homepage looks as if it has the file when it does not.
- Publish nothing in it you would not publish on your website. Anyone can read it, and it does not keep any crawler out: robots.txt does that.
Questions
Common questions
Will an llms.txt get us named more often in ChatGPT or Google?
No one has shown that it does. Google says the file "will neither harm nor help" a site in Google Search, including its AI Overviews and AI Mode,[3] and OpenAI, Anthropic and Perplexity have not said their crawlers read it. HaloScore does not count an llms.txt toward your score, and its recommendation to add one carries no expected lift.
Can an llms.txt stop AI engines from using our content?
No. It is a summary, not a set of rules, and "nothing obliges any crawler to read it."[2] To control which AI crawlers visit your website, use robots.txt, which OpenAI, Anthropic and Perplexity all say their crawlers follow.
Our website provider offers an llms.txt as an extra. Is it worth paying for?
On present evidence, no. The file takes minutes to write by hand, several platforms make one at no extra cost, and nothing shows it changes how often an apartment community is named.
Is an llms.txt the same as robots.txt or a sitemap?
No. Robots.txt tells crawlers where they may go, and a sitemap lists every page for search engines to find. An llms.txt is a short, readable summary with links to a handful of key pages.
Get started
See where your community stands in AI search
HaloScore checks what the AI engines cite for your community each month, so you can spend your time on the sources that shape what renters are told.
Get started with HaloScoreUseful? Share it with your team or your management company.
Sources
- Jeremy Howard, "The /llms.txt file, v2", llmstxt.org (first published September 3, 2024; version 2, August 2026). llmstxt.org
- Malte Ostendorff, Common Crawl, "A Content Analysis of llms.txt Files from the July 2026 Crawl Archive" (August 31, 2026). commoncrawl.org
- Google Search Central, "Google's Guide to Optimizing for Generative AI Features on Google Search" (last updated July 10, 2026; the llms.txt note added June 15, 2026). developers.google.com
- Chrome for Developers, "llms.txt", Lighthouse (last updated May 5, 2026). developer.chrome.com
- OpenAI, "Overview of OpenAI Crawlers" (read October 15, 2026). developers.openai.com
- Claude Help Center, "Does Anthropic crawl data from the web, and how can site owners block the crawler?" (read October 15, 2026). support.claude.com
- Perplexity, "Perplexity Crawlers" (read October 15, 2026). docs.perplexity.ai
- Louise Linehan, Ahrefs, "We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read" (June 15, 2026, updated August 27, 2026). Ahrefs sells search and AI visibility tools. ahrefs.com
- HTTP Archive, "SEO", The Web Almanac 2025 (January 15, 2026, updated June 9, 2026). almanac.httparchive.org
- Wix Help Center, "Understanding Your Site's llms.txt File" (read October 15, 2026). support.wix.com
- Yoast, "How to enable llms.txt with Yoast SEO" (read October 15, 2026). yoast.com