Testing Your llms.txt Implementation Beyond Simple Conformance

Your file returns HTTP 200, a free checker says pass, and none of that tells you whether the file earns its place. Testing your llms.txt implementation means answering four questions no validator asks: are the thirty listed pages the right thirty, were the descriptions written or lifted from meta tags, how many tokens does the file cost the model that reads it, and has a crawler ever requested it. llms.txt is the Markdown file at your domain root that tells an AI system which pages to read, proposed by Jeremy Howard of Answer.AI in September 2024. Ahrefs checked 137,210 domains in May 2026 and found that 97% of published files received zero traffic that month. What follows is the missing instrument: a rubric scored out of 50, a diff routine, and one log query.

Key takeaways

• Testing your llms.txt implementation starts where the validator stops. Ahrefs analysed 137,210 domains in May 2026: 28% publish a file, 97% of those received zero traffic that month.
• Free checkers test syntax against the specification published in September 2024. None grades whether the listed URLs are the ones worth exposing.
• The rubric below scores five dimensions at 10 points each, maximum 50: positioning line, link ordering, description quality, token weight, buyer question coverage.
• A 3,850 character file costs roughly 960 tokens at four characters per token. A file inlining full descriptions for 30 links runs three times that.
• Six crawler tokens matter in your logs: OAI-SearchBot, GPTBot and ChatGPT-User from OpenAI, ClaudeBot, Claude-User and Claude-SearchBot from Anthropic. Confirm each against a published IP list, never the user agent string alone.

Passing a validator is the easy half of testing your llms.txt implementation

A conformance check answers one question: does this file parse. Every free tool returns that binary in under two seconds. The harder half of testing your llms.txt implementation starts after the pass, because nothing in the specification says the file has to be useful.

Chrome makes the asymmetry visible. The Lighthouse agentic browsing audit "flags the pages if a server error occurs when attempting to retrieve the llms.txt file", and when the file is absent it is "marked as Not Applicable (N/A), as providing the file is optional at the moment", per the Chrome for Developers documentation. A 404 costs nothing. A 500 costs the check. Reachability is the entire scoring surface, so treat the Chrome audit as an external check on serving rather than a quality grade.

Adoption points the same way. Rankability's dataset of 17 July 2026, built on the Tranco list of June 2026, found 8.7% of the top 1,000 websites publish an llms.txt, rising to 15.8% among the 549 sites that were reachable rather than CDN infrastructure. Four failure modes survive conformance: wrong page selection, scraped descriptions, excess token weight, and zero observed demand. Each gets a number below.

What the free checkers actually test, and the rules they never explain

The specification is short enough to hold in your head. It requires an H1 with the project name and states this "is the only required section". It then allows "a blockquote with a short summary of the project", "zero or more markdown sections... of any type except headings", and "zero or more markdown sections delimited by H2 headers, containing 'file lists' of URLs". Each list item is "a required markdown hyperlink name, then optionally a : and notes about the file". That is the whole llms.txt specification Jeremy Howard published in September 2024.

The checkers enforce subsets of it. MRS Digital fetches by domain and renders a Markdown preview beside the raw file, framing the goal as content "structured and instantly understandable by AI crawlers", without publishing the rule list it applies. Radarkit names the hand-editing mistakes it looks for. Rankray goes deepest on structure, inspecting H1 titles, blockquote summary lines and link integrity. llmstxtchecker.net returns a verdict from a single field with no explanation behind it.

None explains why a rule exists. The H1 is required because a model reading the file cold has no other signal for what the site is. The blockquote is where positioning lives, and a model that skips it reads your URL list without knowing what you sell. The note after the colon is the only place a link states what decision it settles. Testing your llms.txt implementation past this point means grading the file on what it says, not on whether it parses. Confirm the serving checks that come first before any of it counts: status code, content type, encoding.

A five-dimension usefulness rubric you can score in ten minutes

Score each dimension from 0 to 10, for a maximum of 50. Below 30 means the file passes validation and fails its job. Testing your llms.txt implementation against a shared rubric turns an argument about taste into an argument about a number.

1. Positioning line. The blockquote states what the business does, who it serves, and what a buyer can act on.
2. Link ordering by commercial value. The first section carries the pages that produce revenue, not the navigation order.
3. Description quality. Each note after the colon states what the page settles for the reader.
4. Token weight. The file fits the budget a model will spend on it.
5. Buyer question coverage. A fixed set of 15 to 20 questions your buyers actually ask maps onto a listed page.

A worked example on a live file. The La Boétie file at the domain root carries one H1, one blockquote, nine H2 sections and 23 links across roughly 3,850 characters. Score the three dimensions readable from the file's shape alone: positioning line 7, because the blockquote names the category and the sovereignty thesis but never the buyer; link ordering 6, because the paid self serve tool sits in the sixth section rather than the first; token weight 8, because 3,850 characters is roughly 960 tokens. That is 21 out of 30 on the structural half. The remaining 20 points require reading every note and running the query set.

Token weight: what your file costs the model that reads it

Every token in your llms.txt is a token the reading model does not spend on your actual pages. A token is the unit a language model bills and reasons in, running at roughly four characters of English, so 100 tokens carry about 75 words.

Run the arithmetic on your own file: divide the byte count by four. A 3,850 character file lands near 960 tokens. A file inlining a 400 character description for each of 30 links reaches 12,000 characters, roughly 3,000 tokens, three times the weight for the same 30 destinations. llms-full.txt, the companion file that carries expanded page content rather than a link index, is heavier by an order of magnitude and belongs to a separate decision with its own budget.

The fix is ordering, not deletion. Cut the notes to one line each, and move the twenty links that serve nobody into a second H2 section below the commercial ones. Testing your llms.txt implementation on weight takes one command and produces a number you track month over month. Generating the file in the first place sets the ceiling; testing tells you whether you held it.

Drift: diffing the regenerated file against the live one

Your site changes and your file does not. Drift is the gap between what a regenerated file would contain today and what is actually being served. Catch it by regenerating into a temporary path on a schedule and diffing against the live file.

Not every diff deserves a response. Testing your llms.txt implementation for drift means sorting three classes.

A link returning 404 is an alert. The file points a model at a page that no longer exists, and a model that follows it learns your site is unreliable. Fix it the same day.

A new page in a commercial section is an alert. A pricing page or a product launch that exists on the site and not in the file is revenue the agent layer cannot see. Fix it within the week.

A reworded note or a reordered link inside the same section is a log line. Record it, review it at the monthly checkpoint, wake nobody up.

Set the cadence to your publishing rate. A site shipping weekly regenerates weekly; a site shipping twice a year regenerates quarterly and still catches the 404s, because link rot runs on someone else's schedule. Store each version rather than overwriting it: testing your llms.txt implementation over time is mostly the ability to say when something broke.

The log query, and reading the user agent honestly

Grep your access log for the file path and group by user agent. That single query settles what the rubric cannot: has anything ever fetched this file.

OpenAI documents its current agents as OAI-SearchBot/1.4, GPTBot/1.4 and ChatGPT-User/1.0 in its bots and crawlers reference, each with its own IP range file. Anthropic publishes one list and states that "if a crawler has a source IP address on this list, it indicates that the crawler is coming from Anthropic".

Read the user agent honestly. A user agent string is a header the client chose to send, so a line reading GPTBot is a claim rather than an identity. Reverse DNS the source IP, forward resolve the result, and check the address against the vendor's published range. A request that fails forward confirmation is spoofed whatever the header says.

Expect a discouraging answer. In the Ahrefs May 2026 data, 96% of requests reaching llms.txt files came from bots, yet only 19.5% of fetches came from named AI tools, and retrieval bots accounted for 1.1%. SEO audit tools were the largest single requester at 21.7%. Cloudflare's network analysis of 1 July 2025 put GPTBot at 7.7% of bot requests against ClaudeBot at 5.4%, shares that have inverted since, so read any single month as a snapshot. Testing your llms.txt implementation against logs means reporting the number you find, including zero. For the wider picture of the crawlers that fetch your file, the six tokens above are the starting set.

Turning testing your llms.txt implementation into a monthly one-page report

Four blocks, one page, sent on the same day each month.

Requests by crawler. The six tokens above with a request count and the share that passed IP verification. Zero is a finding; report it with the date range.

Diffs applied. Alerts and log lines from the drift routine, each alert named and dated. A month with two 404 fixes reads better than a month with none, because it proves the routine ran.

Rubric score. The number out of 50, the movement since last month, and the dimension that moved. A score stuck at 31 for four months is a decision, not a plateau.

Questions tested. The frozen set of 15 to 20 buyer questions run against ChatGPT, Claude and Perplexity, with the count that returned your domain.

That page is what testing your llms.txt implementation produces for people who never open a log. For the revenue side of the same measurement, mesurer ce que votre fichier llms.txt rapporte covers attribution from server logs through to citations.

How La Boétie ships and measures the agent-facing layer

La Boétie is a venture studio, digital agency and technical consultancy running a flexible team of five to six engineers, and it treats the agent-facing surface as infrastructure the client owns.

Generation. The llms.txt Generator on the studio's Tools surface produces the file end to end for $25 or 20 € a run. No account, no call, no subscription.

Serving and installation. The same file has to survive WordPress, Shopify, Next.js, Webflow and Squarespace, each with a different path from source of truth to domain root. The studio has shipped client builds across finance, legal, insurance, auctions and eco transition.

Measurement. Cortex, the studio's own SEO and GEO growth engine, runs the insights archive you are reading. It is whitelist only, by application, and capped at three brands per industry.

Run the file through the llms.txt Generator, score it against the rubric, then keep testing your llms.txt implementation on a monthly schedule.

FAQ: testing your llms.txt implementation

How often should you repeat testing your llms.txt implementation?

Match the cadence to your publishing rate. Testing your llms.txt implementation splits into three loops: regenerate and diff weekly if you ship weekly, quarterly if you ship twice a year; review the rubric monthly; re-run the buyer question set monthly. The log query runs monthly in every case, because crawler behaviour moves independently of your site. Cloudflare's analysis of 1 July 2025 put GPTBot at 7.7% of bot requests against ClaudeBot at 5.4%, and those shares have inverted since.

Does a passing validator mean AI systems will read the file?

No. Ahrefs found that 97% of the llms.txt files it identified across 137,210 domains received zero traffic in May 2026, and that zero requests came from AI bots for files that did not exist. Conformance changes nothing about demand. The only evidence that a system read your file is a verified request in your access log, matched to a documented crawler token and confirmed against the vendor's published IP list.

What token weight should an llms.txt target?

Under 1,500 tokens for a commercial site, which is roughly 6,000 characters at the standard four characters per token conversion. Divide your file's byte count by four for the estimate. Above 5,000 tokens the file competes with the pages it exists to introduce. llms-full.txt is a separate artefact with a separate budget and should never be conflated with the index file.

Can you trust the user agent string in your logs?

No. The user agent is a header the client chose to send, so any scraper writes GPTBot into it. Verify every claimed crawler by reverse DNS on the source IP, forward resolving the result, and checking the address against the published range. OpenAI publishes a separate JSON range file per agent. Anthropic publishes one list covering ClaudeBot, Claude-User and Claude-SearchBot.

Which diffs should trigger an alert rather than a log line?

Two. A link that returns 404 sends a model to a dead destination and is fixed the same day. A new page in a commercial section that is missing from the file hides revenue from the agent layer and is fixed within the week. Rewordings and reorderings inside an existing section are recorded and reviewed at the monthly checkpoint instead of paging anyone.

Conclusion

A file that passes every published checker can still list the wrong thirty pages, carry notes no human wrote, cost a model 3,000 tokens for 30 destinations, and sit untouched by a single crawler for a year. Four defects, four measurements. The rubric handles the first three in ten minutes and returns a number out of 50. The diff routine catches the site moving underneath the file. The log query answers the demand question with a date attached, and answers it honestly when the answer is zero.

Testing your llms.txt implementation is not a one time gate at publication. It is a monthly number, and the number is the only thing that survives an argument about taste.

Sources

Also read:

• Generating an llms.txt that earns its place
• Serving checks: status, content type and encoding
• Chrome now scores your site for agents
• The crawlers that fetch your llms.txt
• Mesurer ce que votre fichier llms.txt rapporte

External sources:

• The /llms.txt file : llmstxt.org, Jeremy Howard, Answer.AI, 2024
• Bots and crawlers : OpenAI developer documentation, 2026
• Does Anthropic crawl data from the web, and how can site owners block the crawler? : Anthropic support, 2026
• llms.txt audit, agentic browsing : Chrome for Developers, 2026
• We analyzed 137K sites: 97% of llms.txt files never get read : Ahrefs, Louise Linehan and Xibeijia Guan, 2026
• LLMS.txt adoption data : Rankability, 2026
• From Googlebot to GPTBot: who's crawling your site in 2025 : Cloudflare, 2025
• LLMs.txt checker and validator : MRS Digital, 2025