llms.txt is the convention for telling AI crawlers what lives on your site. Think of it as a markdown sitemap written for models instead of browsers. My first instinct was the obvious one: drop a file in public/ and move on.

That instinct was wrong, and I want to save you the rebuild it cost me.

A file in public/ is frozen the moment you write it. Every post I published after that point left the index a little more out of date, and I never once remembered to go back and hand-edit a text file. Two weeks in, my llms.txt described a site that no longer existed.

The fix on Astro is to stop treating it as a file and make it an endpoint that reads from your content collections at build time:

// src/pages/llms.txt.ts
import { getCollection } from "astro:content";
import type { APIRoute } from "astro";

export const GET: APIRoute = async () => {
  const posts = await getCollection("blog", ({ data }) => !data.draft);
  const lines = posts.map(
    (p) => `- [${p.data.title}](https://example.com/blog/${p.id}/): ${p.data.description}`
  );
  return new Response(`# Site\n\n${lines.join("\n")}`, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};

On a static build, Astro renders this to dist/llms.txt like any other route. Every deploy now ships a complete, current index with zero maintenance. Add a post, run the build, the index already knows about it.

Here is the gotcha that actually cost me the rebuild: if public/llms.txt still exists, it shadows the endpoint. Astro copies everything in public/ over the build output, so the stale file wins and your shiny new endpoint silently never serves. Delete the old static file first, then verify with curl yoursite.com/llms.txt that you are getting the generated list and not the frozen one.

The same pattern scales straight to llms-full.txt. Return post.body for each entry instead of the description, and a model can ingest your entire site in a single fetch. That is the version I actually ship now, because if I am going to make the site legible to LLMs, I would rather hand them the whole text than a table of contents.