Skip to content

Site-level indexes

The markdown route gives an agent one page at a time. It has no way to find out what pages exist, and no way to take the whole site in one request.

llmsTxt adds the files that answer both questions:

starlightLlmActions({
llmsTxt: true,
})
File Contents
/llms.txt The llmstxt.org index: your site title and description, a link to each bundle, and a link to every page’s own Markdown route.
/llms-full.txt Every page concatenated into one Markdown document.
/llms-{subset}.txt One file per named subset.

llms.txt on this site looks like:

# starlight-llm-actions plugin
> Every page of the starlight-llm-actions plugin documentation, as Markdown.
## Documentation Sets
- [Complete documentation](https://holdenhewett.github.io/starlight-llm-actions/llms-full.txt): every page of the starlight-llm-actions plugin documentation as Markdown
- [Configuration](https://holdenhewett.github.io/starlight-llm-actions/llms-configuration.txt): every configuration option
## Documentation
- [starlight-llm-actions](https://holdenhewett.github.io/starlight-llm-actions/index.md): Page Actions dropdown for Starlight — copy, view, save as PDF, and open in any LLM.
- [Copy as Markdown](https://holdenhewett.github.io/starlight-llm-actions/actions/copy.md): Copy the current page's markdown source to the clipboard.

The H1 and the blockquote default to Starlight’s title and description. This site overrides both, because its Starlight title is a bare package name and an agent that fetched the file needs to know whose docs it is holding — see llmsTxt.title and llmsTxt.description.

The two sections serve different readers. Documentation Sets is for an agent that wants the corpus in one request. Documentation is for one that would rather fetch the three pages it needs than a megabyte it mostly discards — worth listing here only because this plugin publishes a route per page.

Links are absolute, so site has to be set in your Astro config. An agent that fetched llms.txt over HTTP has a base to resolve relative links against, but one handed the file as a blob does not — and that second case is what the format exists to serve.

The bundles are built by running the same renderMarkdown pipeline the per-page route runs, on the same pages, and concatenating the results. The Markdown in llms-full.txt is byte-identical to the Markdown at each page’s own .md URL.

That is the reason this plugin generates its own indexes rather than leaving them to a separate package. Two generators on one site means two qualities of Markdown: a <Card> title that is a real heading on one surface and a bare paragraph on the other, an aside that keeps its label in one file and loses it in the next. Whatever you configure once applies everywhere.

Each page is rendered once per build regardless of how many files it appears in.

Every page of every collection named in collections, which defaults to Starlight’s docs alone. A site that keeps a changelog, a blog, or an API reference in its own collection lists it there to bring those pages into the indexes:

starlightLlmActions({
collections: [
'docs',
{ name: 'changelog', path: 'changelog/entry/{id}' },
],
})

The path template maps an entry id onto the URL the site serves it at. It is worth getting right beyond the indexes: the same path is what the page’s own .md route is built from, and what the glob options below match.

Indexes are ordered alphabetically by site path, with two escape hatches.

promote lifts patterns to the top; demote pushes them to the bottom. Earlier patterns outrank later ones within each list.

starlightLlmActions({
llmsTxt: {
promote: ['index*', 'getting-started/**'],
demote: ['reference/**', 'changelog*'],
},
})

That reads top-down the way a person would: the home page, then the introduction, then everything else, then the reference material an agent should consult rather than read.

promote: ['index*'] is the default. Pass [] to switch it off.

Patterns match a page’s site path — guides/example, not /guides/example/ — through picomatch. A single * stays inside one path segment; ** crosses them.

exclude drops pages from llms.txt and llms-full.txt, the two indexes that speak for the site as a whole:

starlightLlmActions({
llmsTxt: {
exclude: ['internal/**', 'sandbox/**'],
},
})

This is about the indexes, not about hiding a page. An excluded page still serves its own Markdown route and still carries its linkAlternate tag. It just stops being advertised in bulk.

Drafts are already excluded, the same way they are already skipped by the Markdown route.

A named subset overrides exclude. See Subsets beat exclude.

On a large site, llms-full.txt can be big enough that fetching it to answer a question about one section is wasteful. subsets publishes that section as its own bundle:

starlightLlmActions({
llmsTxt: {
subsets: [
{
label: 'REST API',
description: 'the complete REST API reference',
paths: ['api/**'],
},
{
label: 'Guides',
description: 'task-oriented walkthroughs',
paths: ['guides/**', 'tutorials/**'],
},
],
},
})

Each entry emits one file — /llms-rest-api.txt, /llms-guides.txt — and adds one line to the Documentation Sets section of llms.txt. The file name comes from label, lowercased and hyphenated.

A subset inherits the promote/demote order, so it lists its pages in the same relative order llms-full.txt puts them in. Nothing stops two subsets from overlapping.

A subset whose paths match no page is a build error, not an empty file. The usual cause is a leading slash or a file extension: paths are globs over site paths, so guides/example, never /guides/example.md.

paths wins over exclude. A page you excluded still appears in any subset that names it:

starlightLlmActions({
llmsTxt: {
// Too big for the everything-bundle, and the full reference already
// repeats what the endpoint and schema pages say.
exclude: ['api/generated/**'],
subsets: [
{
label: 'REST API',
description: 'the complete REST API reference',
paths: ['api/generated/**'],
},
],
},
})

llms-full.txt skips those pages and /llms-rest-api.txt carries them. That pairing is the main reason to reach for a subset: a section too large or too duplicative for the everything-bundle can still ship on its own. The narrower statement wins, so listing a page by paths is a considered request that a corpus-wide glob does not override.

The pattern dialect and the ordering rules are ported from starlight-llms-txt, so an existing promote/demote list produces the order it already produced. Its customSets entries move over to subsets unchanged — same label, paths, and description fields.

Three differences to know before you cut over:

  • exclude reaches further here. There, it filters llms-small.txt only, which means the exclusions you wrote for it never shaped llms-full.txt. Here it filters llms.txt and llms-full.txt, so those pages finally drop out of both. Your customSets keep working either way, because subsets beat exclude.
  • There is no llms-small.txt. A named subset is the way to publish a smaller file, and it slices by section rather than by stripping elements out of every page.
  • projectName and description are llmsTxt.title and llmsTxt.description. Both default to Starlight’s own values here, so a site whose header already reads as a corpus name can drop them.

These options have no equivalent, on purpose:

Option Why not
minify It only shapes llms-small.txt, which this plugin does not generate. The flattening is already lossy in the ways that matter; a second pass that drops asides and <details> blocks is a debugging problem waiting to happen.
customSelectors, rawContent These exist to patch a flattener that cannot handle a site’s markup. The fix is a better flattener, which is what renderMarkdown: 'simple' is; for markup it still cannot handle, the { module } escape hatch hands you the whole pipeline.
details, optionalLinks Free-form prose and a link list appended to the header. llms.txt already links every page and every bundle; a third hand-maintained list drifts.
pageSeparator Bundles always join documents with a blank line.

Run both plugins for one build and diff the output before you cut over.