# docspack — Version-locked documentation for AI agents > Local, version-locked documentation packages for AI agents, indexed in SQLite and served over MCP. > docspack is a CLI, not a service: it reads `node_modules` and a local SQLite index, and > `sync`, `ask`, `search` and `list` make no network requests at all. Version 1.2.0, > MIT, requires Node >=22.5.0. ## Install ```sh npm i -D docspack # or: pnpm add -D docspack npx docspack sync # index the docs packages this project depends on npx docspack ask "how do I verify a webhook signature" ``` A documentation package is an ordinary npm package named `@vendor/docspack`, `@vendor/-docspack` or `@docspack-community/`. Add it to `package.json`, run `docspack sync`, and its chunks are indexed into a shared SQLite database with FTS5. ## Giving an agent access Any agent with a shell can run `docspack ask`, so one line in AGENTS.md or CLAUDE.md is the whole setup — no server, no per-client configuration, nothing resident when nobody is asking: ``` Run `docspack ask ""` for documentation on this project's dependencies. It answers from the installed versions. ``` `docspack agent install` writes that block into whichever of AGENTS.md and CLAUDE.md the project already has, plus a Claude Code skill when there is a `.claude` directory, and rewrites it in place on the next run. `docspack agent check` writes nothing and exits non-zero when the wiring is missing or out of date, which is the form CI can run. ## Indexing sources nobody published The same index also serves a project's own material — notes, decision records, an export, rows out of a query: ``` npx docspack index --from ./notes npx docspack index --from-json ./rows.json # or - for stdin npx docspack recall "what did we decide about retries" ``` A JSON record is `{ "title", "text" }` with optional `id`, `tags` and `entities`. A record carrying an `id` becomes exactly one chunk under that id. Two rules matter here: - **`recall` is not `ask`, and they cannot reach each other.** `ask` answers only from installed package versions; `recall` answers only from indexed local sources. Neither `--all` nor a shared `--store` changes that. Use `ask` for a dependency, `recall` for the project's own material. - **A local corpus goes stale and says so.** `index` records each source's size, mtime and hash. If an answer from `recall` begins `NOTE: the corpus is out of date`, the passages may be superseded: run `docspack index` again before relying on them. ## Closed sets A name that is not in these lists does not exist. Do not invent one — a plausible-looking command or kind fails at the CLI, and an invented manifest field is accepted and ignored. ### Commands (17) - `docspack sync` — Index the docs packages this project depends on - `docspack ask` — Answer from the local index — the command to give an agent - `docspack index` — Index this project's own sources, so an agent can ask them instead of reading them - `docspack recall` — Answer from this project's indexed corpus, not from its dependencies - `docspack search` — Same index, formatted for a human reading the terminal - `docspack list` — Show this project's docs packages and their index state - `docspack agent` — Wire docspack into the agent tooling this project already uses - `docspack changed` — What a library's exports gained and lost between two versions - `docspack verify` — Check the docs still describe the code you installed - `docspack feedback` — Record documentation problems: add, list, submit, remove - `docspack mcp` — Serve the index over MCP instead, as a long-lived process - `docspack sources` — List curated sources that `docspack build` can fetch - `docspack init` — Scaffold a documentation package, then build and check it - `docspack build` — Generate the .llms/ payload for publishing - `docspack doctor` — Check a package the way the indexer and a reviewer would - `docspack preview` — Answer a query from the local package, as an agent would - `docspack eval` — Measure retrieval against a set of questions ### Feedback kinds (3) - `drift` — a name the docs use that the library does not declare. Must name the identifier. - `incorrect` — documented behaviour that differs from actual. Must carry --expected, --actual and --repro. - `missing` — behaviour the docs do not cover. Must carry --expected, --actual and --repro. There is deliberately no kind for "this page is confusing": an unfalsifiable claim is infinitely generatable and of no use to a maintainer. ### Response limits - Chunks per answer: 3 by default (`--limit`) - Tokens per answer: 3000 by default (`--max-tokens`), counted from the manifest before content is returned ### Manifest schema The `.llms/manifest.json` of a docs package validates against https://docspack.dev/schema/v1.json. The package format — naming, layout, every manifest field and the rules a consumer enforces — is specified at https://docspack.dev/spec, and served as Markdown at https://docspack.dev/spec.md. ## Published packages - [docspack](https://www.npmjs.com/package/docspack) — The CLI and MCP server: build, sync, search and ask. - [@docspack/registry](https://www.npmjs.com/package/@docspack/registry) — Curated llms.txt sources for bootstrapping docs packages. - [@docspack/docspack](https://www.npmjs.com/package/@docspack/docspack) — docspack's own documentation, shipped as a docs package. - [@docspack/sheaf](https://www.npmjs.com/package/@docspack/sheaf) — Sheaf's content graph: a directory of Markdown as one serializable object. - [@docspack/sheaf-astro](https://www.npmjs.com/package/@docspack/sheaf-astro) — Renders a Sheaf graph with Astro's own Markdown pipeline. - [@docspack/sheaf-react](https://www.npmjs.com/package/@docspack/sheaf-react) — The Sheaf docs shell — sidebar, page, contents and pager, built on cascivo. - [@docspack/sheaf-emit](https://www.npmjs.com/package/@docspack/sheaf-emit) — Turns a Sheaf graph into a docspack package, so one source serves both audiences. ## Rules to install in a host project Copy-paste blocks for an adopter's AGENTS.md / CLAUDE.md / .cursor/rules, including which source wins when a retrieved chunk disagrees with recalled knowledge: https://docspack.dev/ai-rules.md ## Curated sources (27) Projects whose published llms.txt `docspack build ` can turn into a docs package. The id is the first column. Any llms.txt URL also works without being listed. - `anthropic` — Anthropic / Claude: Claude API, Agent SDK and Claude Code documentation. (publishes llms-full.txt) - `better-auth` — Better Auth: Framework-agnostic authentication library for TypeScript. - `bun` — Bun: JavaScript runtime, bundler, test runner and package manager. (publishes llms-full.txt) - `clerk` — Clerk: User management, authentication and session handling. (publishes llms-full.txt) - `cloudflare` — Cloudflare Developers: Workers, Pages, R2, D1, KV and the rest of the developer platform. (publishes llms-full.txt) - `convex` — Convex: Reactive backend-as-a-service with a TypeScript function model. (publishes llms-full.txt) - `drizzle` — Drizzle ORM: TypeScript ORM with a SQL-like query builder. (publishes llms-full.txt) - `elysia` — Elysia: Ergonomic TypeScript web framework for Bun. (publishes llms-full.txt) - `expo` — Expo: Framework and platform for universal React Native apps. (publishes llms-full.txt) - `hono` — Hono: Small, fast web framework built on Web Standards. (publishes llms-full.txt) - `nextjs` — Next.js: React framework with file-system routing and server rendering. (publishes llms-full.txt) - `nuxt` — Nuxt: Vue framework for server-rendered and static applications. (publishes llms-full.txt) - `prisma` — Prisma: Type-safe database toolkit, schema and migrations. (publishes llms-full.txt) - `pydantic-ai` — Pydantic AI: Agent framework for building applications with generative models. (publishes llms-full.txt) - `resend` — Resend: Email API for developers. (publishes llms-full.txt) - `sentry` — Sentry: Error monitoring, tracing and performance tooling. - `shadcn-ui` — shadcn/ui: Copy-and-paste React component collection built on Radix and Tailwind. - `stripe` — Stripe: Payments, billing, checkout and financial APIs. - `supabase` — Supabase: Postgres platform with auth, storage, realtime and edge functions. (publishes llms-full.txt) - `svelte` — Svelte: Svelte and SvelteKit documentation. (publishes llms-full.txt) - `tanstack` — TanStack: Query, Router, Table, Form and the rest of the TanStack libraries. (publishes llms-full.txt) - `trigger-dev` — Trigger.dev: Background jobs and durable workflows for TypeScript. (publishes llms-full.txt) - `turborepo` — Turborepo: Build system for JavaScript and TypeScript monorepos. (publishes llms-full.txt) - `vercel` — Vercel: Deployment platform, edge network and build system. (publishes llms-full.txt) - `vite` — Vite: Build tool and dev server for modern web projects. (publishes llms-full.txt) - `vitest` — Vitest: Vite-native test runner. (publishes llms-full.txt) - `zod` — Zod: TypeScript-first schema validation with static type inference. (publishes llms-full.txt) ## Documentation - [docspack overview](https://docspack.dev/docs/overview.md) — docspack gives AI coding agents the documentation of the dependencies a project actually installed. Documentation is distributed as npm packages, indexed locally into SQLite, and served to agents over the Model Context Protocol. - [Installing and syncing](https://docspack.dev/docs/install-and-sync.md) — docspack is a CLI. Run it with `npx docspack `, or install it as a dev dependency to pin the version your team uses. - [Connecting an agent](https://docspack.dev/docs/agents-and-mcp.md) — There are two ways to give an agent the documentation. The command line is the simpler one and works everywhere; MCP is there for clients that prefer a declared tool. - [Searching from the command line](https://docspack.dev/docs/searching.md) — `docspack search` runs the same query an agent would, and prints the results. Use it to check that a package is indexed, to see what an agent will receive, or to find something yourself. - [Authoring a documentation package](https://docspack.dev/docs/authoring-a-package.md) — A documentation package is a normal npm package whose payload is Markdown rather than code. `docspack init` scaffolds one, `docspack build` regenerates it, `docspack doctor` checks it and `docspack preview` shows what an agent would get back. - [Documenting an HTTP API](https://docspack.dev/docs/documenting-an-http-api.md) — An HTTP API is the one documentation topic most projects already describe in a machine-readable file. `docspack build --openapi` reads that file and writes one chunk per operation: - [Verifying documentation against code](https://docspack.dev/docs/verify.md) — Documentation drifts when a library is renamed and its prose is not. `docspack verify` catches that class of error before a reader does: it compares the identifiers a package's chunks name against what the libraries it documents actually declare. - [Trust and safety](https://docspack.dev/docs/trust-and-safety.md) — Documentation packages are third-party input that ends up in a model's context. docspack treats them accordingly. - [How docspack compares](https://docspack.dev/docs/how-it-compares.md) — docspack overlaps with several things an agent can already do. The differences are worth being precise about, because most of them are not about capability but about which version of the truth reaches the model, and how much of it. - [Writing documentation for agents](https://docspack.dev/docs/writing-for-agents.md) — A documentation package is read by a model, not a person browsing a website. That changes what belongs in it. `docspack doctor` enforces most of this, so the rules below are what the warnings mean. - [Answers from the installed build](https://docspack.dev/docs/answers-from-the-installed-build.md) — Half of a well-documented library's exported names are mentioned nowhere in its own documentation. Measured against two projects that document themselves well, hono's site covers 51% of what hono exports and zod's covers 27%. Every one of those uncovered names is declared in the package sitting in `node_modules`. - [One source, two audiences](https://docspack.dev/docs/one-source-two-audiences.md) — Documentation has two readers with incompatible needs. A person wants a page: navigation, prose, a table of contents. An agent wants a passage: small, self-contained, and matched to a question. Most projects serve them from two pipelines, and the two drift. - [Indexing your own sources](https://docspack.dev/docs/your-own-corpus.md) — `docspack sync` indexes documentation somebody published. `docspack index` indexes material this project already has — notes, decision records, an export, rows out of a query — and `docspack recall` answers from it. ## This site's own HTTP API Every route above is served by an API this site describes in OpenAPI, and renders three ways. Fetch the third one: it is the same facts in about a tenth of the tokens. - Whole API, dense: https://docspack.dev/api.lapis — LAPIS notation, one signature per endpoint - One endpoint, everything needed to call it: https://docspack.dev/api/%3CoperationId%3E.md - The OpenAPI document itself: https://docspack.dev/openapi.json - For a person: https://docspack.dev/api ## Everything in one file - [llms-full.txt](https://docspack.dev/llms-full.txt) — this file with every page above inlined, for a single fetch - Offline, no web access: `npm i -D @docspack/docspack` installs the same documentation as a docs package ## Links - Site: https://docspack.dev - Source (MIT): https://github.com/docspack/docspack - npm: https://www.npmjs.com/package/docspack - Releases: https://github.com/docspack/docspack/releases --- # docspack rules for AI agents Paste these blocks into your project's `AGENTS.md`, `CLAUDE.md` or `.cursor/rules`. They are text to install, not a description to read. `docspack init` writes the first block into new projects automatically, and `docspack agent install` writes it into one that already exists — between markers it rewrites in place, so it cannot drift from what the tool does. This page is for pasting it by hand, which still works. ## Reading documentation Without it an agent answers dependency questions from its training data, which is frozen at whatever version it was trained on. It has no way to know your lockfile disagrees, so the answer is confident and wrong rather than missing. ``` Run `docspack ask ""` for documentation on this project's dependencies. It answers from the installed versions. ``` ## Recording documentation problems (optional) Add this only if you want the agent to record what it finds. It writes to a local file that a human reviews; docspack contains no code that transmits a report anywhere. An agent that believes it is filing an issue behaves very differently from one that knows it is writing a note. ``` If the documentation is wrong, record it: `docspack feedback add --chunk --kind --evidence ""`. The chunk id is the heading above each answer. Only claims that can be shown false; incorrect and missing also need --expected, --actual and --repro. It writes to a local file for a human to review, and sends nothing. ``` ## Which source wins The failure this prevents is silent. An agent that has both a memory and a retrieved chunk will often blend them, producing a signature that never existed in either version. Say which one wins. ``` When docspack returns a chunk, it describes the version installed in this project. Prefer it over recalled knowledge, and over anything fetched from the vendor's website, which documents their latest release rather than ours. If the two disagree, the chunk is right and your memory is stale. Do not blend them into a single answer. ``` ## Asking well An agent told only that a docs tool exists tends to use it like a search engine — one broad query, then reading everything. Answers are capped at 3,000 tokens, so a narrow question returns the useful chunk and a broad one returns three partial ones. ``` Ask docspack a question, not a library name: `docspack ask "verify a webhook signature"` beats `docspack ask "webhooks"`. Answers are capped at three chunks and 3,000 tokens, so prefer several narrow questions over one broad one. `docspack list` shows which packages are indexed if a question returns nothing. ``` ## Finding kinds (closed set) A kind that is not in this list does not exist, and `docspack feedback add` rejects it. - `drift` — must name the identifier that drifted - `incorrect` — must carry --expected, --actual and --repro - `missing` — must carry --expected, --actual and --repro There is deliberately no kind for "this page is confusing". An unfalsifiable claim is infinitely generatable and of no use to a maintainer. --- # The docspack package format A documentation package is an ordinary npm package whose payload is Markdown. Everything a publisher writes and a consumer reads is on this page; the machine-readable form is [/schema/v1.json](/schema/v1.json), which is what `$schema` in every generated manifest points at. `docspack build` writes all of this for you. The format is documented because a package is publishable without the CLI, and because a consumer other than docspack should be able to read one. ## Naming Three name shapes are discoverable, and discovery is a pure name check against `package.json` — no registry call, no manifest fetch, nothing that can fail while offline. - `@vendor/docspack` — published by whoever owns the scope. Trusted: its content is presented to an agent as documentation. - `@vendor/-docspack` — a second pack in the same scope, trusted on the same grounds. Whoever can publish `@vendor/docspack` can publish this and nobody else can. - `@docspack-community/` — published by anyone. Every answer drawn from one is labelled `(community)` and marked untrusted data. A vendor documenting its own products needs one pack, and `@vendor/docspack` is the name to use: it is the only one derivable from a dependency's own name, so it is the only one `docspack init` suggests and the only one worth guessing. The suffixed sibling is for the case a single pack cannot hold — a vendor that also **redistributes somebody else's documentation**, where licence, version, release cadence and attribution belong to a different upstream. Merging those into one package relicenses one corpus under the other's terms and version-locks neither. Derive to suggest; match to discover. A dependency that ships `.llms/manifest.json` under any other name is not indexed — the name check is what decides whether a package may put content in front of a model, and a `docspack` field or a manifest alone would let any transitive dependency opt itself in. It is reported instead: `docspack sync` and `docspack list` name it, say how many chunks its manifest declares, and say why it was skipped, and `docspack doctor` fails on it. ## Layout `.llms/` is the payload and must be listed in the package's `files`. Omitting it publishes a package that installs cleanly and indexes nothing — `docspack doctor` treats that as an error. ```text @vendor/docspack/ ├── package.json name, version, and the docspack build settings ├── llms.txt human-readable table of contents └── .llms/ ├── manifest.json every chunk, its file, size, tags and entities └── chunks/ ├── overview.md └── webhooks-signing.md ``` `llms.txt` at the package root is the table of contents, in the [llmstxt.org](https://llmstxt.org) shape: an `H1` naming the package, an optional `>` summary, then one or more `##` sections of Markdown links. Nothing in the index reads it — `manifest.json` is the machine copy — so it is what a person or an agent browsing `node_modules` opens first. ```md # @acme/docspack > Documentation package for @acme/docspack. Generated by docspack; the > machine-readable payload is in .llms/. ## Chunks - [webhooks-signing](.llms/chunks/webhooks-signing.md): webhooks, signature, security - [api-auth](.llms/chunks/api-auth.md): authentication, bearer, api key ``` `docspack build` writes one link per chunk, annotated with its first six tags. A hand-written file may say more; it may not say less than where the chunks are. ## manifest.json The one file a consumer must read. It is the whole index of the package: a reader budgets a response from `tokens` before opening a single chunk file. | Field | Type | | Meaning | | --- | --- | --- | --- | | `name` | string | required | npm package name, e.g. @stripe/docspack. | | `version` | string | required | Must match the version in package.json. | | `documents` | array | | Libraries this package documents, each "name" or "name@version". An array even for one. | | `chunks` | array | required | Every retrievable unit of documentation in the package. | ### Each entry in `chunks` | Field | Type | | Meaning | | --- | --- | --- | --- | | `id` | string | required | Unique within the package; used to build the chunk_id in the index. | | `file` | string | required | Path relative to .llms/. Must not escape that directory. | | `tokens` | integer | | Approximate token count, used to budget a response. Omit it to have it estimated; 0 is not a value. | | `tags` | array | | Search terms indexed alongside the content. | | `entities` | array | | Identifiers the chunk documents, e.g. Stripe.setApiKey. | | `documents` | array | | Libraries this chunk describes, overriding the package's documents. For a monorepo documenting many libraries at many versions from one surface. | ```json { "$schema": "https://docspack.dev/schema/v1.json", "name": "@acme/docspack", "version": "1.4.0", "documents": ["acme@1.4.0"], "chunks": [ { "id": "webhooks-signing", "file": "chunks/webhooks-signing.md", "tokens": 412, "tags": ["webhooks", "signature", "security"], "entities": ["client.verifySignature", "WebhookEvent"] } ] } ``` `documents` is what lets an answer say which release it describes. A package version cannot always imply it: a repository publishing eighteen libraries at four versions from one documentation surface has no single version to mirror. `docspack ask` prints it under each hit, and `docspack verify` checks the chunks' entities against every library named there. A chunk may carry its own `documents`, and then that is what it describes. On the package the field is the union — true of the pack and useless about any one chunk — so a monorepo names the library per chunk, and an answer, and `verify`, narrow to it. Absent on a chunk, the package's list stands. ## Rules a consumer enforces - A chunk `file` resolves inside `.llms/` or it is refused. It must not escape that directory. A manifest is third-party input, so this is a security boundary rather than a convenience check. - The installed `package.json` version wins over the manifest's when the two disagree, and the mismatch is reported. - Chunk ids are unique within a package and match `^[A-Za-z0-9][A-Za-z0-9._-]*$`. The id an answer is headed with is `@/`. - `tokens` is omitted or at least 1. It is never 0: absent means "estimate it", and a chunk that is genuinely empty is a different error. - Unknown fields are accepted and ignored — `additionalProperties` is true at every level. An invented field is not an error, so it is also not a feature: nothing reads it. `docspack doctor` warns about keys outside the set and `--strict` fails on them, because a publisher is the last party who can still tell a typo from an extension. ## Chunk files A chunk file is Markdown, and it is the one part of a package that reaches a model unaltered: a consumer returns its text verbatim. Nothing strips a heading, resolves a relative link or rewrites a code fence. ````md # Verifying a webhook signature Call `client.verifySignature(payload, header)` before trusting a delivery. It throws `SignatureError` when the header is absent. ```ts client.verifySignature(await request.text(), request.headers.get("acme-signature")); ``` ```` - It SHOULD open with one `#` heading naming the section. It is the line an agent reads first, and the only context the chunk has once it is lifted out of the document it came from. - It SHOULD then carry its provenance as ``. An HTML comment renders as nothing, survives being pasted into a context window, and is what lets a reader say where a claim came from. - It carries no front matter. Front matter is an authoring control on a source document and is consumed when the package is built. - The rest is ordinary Markdown. `tokens` in the manifest measures this file, heading and comment included. One idea per file is the whole design. A chunk that answers two questions is returned for both and spends the budget twice; `docspack doctor` warns above ~1,500 tokens and below ~30. ## Chunk ids An id matches `^[A-Za-z0-9][A-Za-z0-9._-]*$` and is unique within its package. It is a public name, not an implementation detail: an answer is headed with `@/`, and `docspack feedback add --chunk @acme/docspack@1.4.0/api-auth` pins one. - Ids SHOULD be stable across versions. Renaming a chunk is a breaking change to the package: it orphans every finding filed against the old id and every link that quoted it. `docspack build` warns when a rebuild drops an id the previous payload published. - The derivation `docspack build` uses is the document title and the section heading, slugified and joined — lowercased, every run of non-alphanumerics collapsed to `-`, trimmed to 60 characters — or the title alone when the heading is the title. - That derivation collides, and uniqueness does not fall out of it. A generator must detect a collision and resolve it deterministically. `docspack build` appends `-2`, `-3` and warns, because which of the two takes the bare id depends on the order the files were read in. ```text docs/ ├── cli.md ## Casen → cli-casen └── cli/ └── casen.md (intro) → cli-casen ← the same id ``` Deriving an id from a heading is what makes editing a heading rename a chunk. Where that matters, write the id rather than deriving it: the manifest is the authority, and nothing requires an id to resemble the text it points at. ## Ranking and the response budget Two conforming implementations should return the same three passages for the same query, so the retrieval is specified rather than left to the reader. - The index is SQLite FTS5 with `tokenize="porter unicode61"`, over two searchable columns: `content`, the chunk file's text, and `tags`, the chunk's `tags` and `entities` joined by a space. - Results are ordered by `bm25(chunks_fts, 1.0, 3.0)` — content weighted 1.0, tags and entities 3.0 — using FTS5's own bm25 with its fixed `k1 = 1.2` and `b = 0.75`. bm25 normalizes for length, so a short chunk outranks a long one on the same match. - A query is lowercased and split on letters and digits. Terms of one or two characters are dropped, then closed-class function words; each surviving term is quoted and joined with `OR`. Each narrowing falls back to the one before it, so a query made entirely of short or common words still searches for something. - Up to `--limit` chunks (3) are taken in rank order, then kept while the running total of `tokens` stays within `--max-tokens` (3,000). The first hit is always returned, even alone over budget. The budget is spent from the manifest, before any chunk file is opened. `tokens` may be omitted to have it estimated, so a publisher's count and a consumer's estimate have to agree on what they are counting. The reference estimate is the trimmed file contents in characters, divided by four, rounded up, and never below 1: ```ts tokens = Math.max(1, Math.ceil(contents.trim().length / 4)) ``` It is deliberately not a tokenizer. Its only job is to be the same number on both sides of a publish, so the 3,000-token cap means one thing everywhere; a real tokenizer would make it mean one thing per model. ## Authoring controls Front matter carries `title`, `tags` and `documents` for a whole document. A single section takes its own with an HTML comment under the heading, which is what to reach for when one page holds eighty of them: ```md ## Two-column layout ``` Tags are weighted above prose in the ranking, so they are the lever that aims one chunk at one question. A directive's tags are indexed as written; words taken from a heading have function words filtered out of them first, because a section titled "How to use it" otherwise outscores the corpus on every question beginning "how do I". A directive may also narrow what one section documents, which is the per-chunk `documents` above. Both other levers are in the ranking: tags aim a chunk, and length decides how far it carries. ## Next - [/schema/v1.json](/schema/v1.json) — the JSON Schema, versioned with the format - [/spec.md](/spec.md) — this page as Markdown, for an agent - [Authoring a package](/docs/authoring-a-package.md) — writing one with the CLI - [Verifying documentation against code](/docs/verify.md) — what `docspack verify` checks, and what it will not - [/llms.txt](/llms.txt) — the agent-facing summary of all of it --- # docspack overview docspack gives AI coding agents the documentation of the dependencies a project actually installed. Documentation is distributed as npm packages, indexed locally into SQLite, and served to agents over the Model Context Protocol. ## What problem docspack solves An agent answering a question about a dependency has three bad options. It can recall the API from training data, which is frozen at some past date. It can scrape the vendor's website, which is slow and returns whichever version the vendor publishes today. Or it can read the library's source, which is expensive and rarely explains intent. All three fail the same way: the answer does not correspond to the version in `package.json`. docspack removes that gap by treating documentation as a dependency. The docs for `acme@1.4.0` live in a package pinned next to `acme@1.4.0`, so an agent reading them is reading about the code that is actually installed. ## How the pieces fit together Four moving parts: - A **docs package** is an npm package named `@vendor/docspack`, `@vendor/-docspack` or `@docspack-community/`. It contains Markdown split into chunks and a manifest describing them. - The **indexer** (`docspack sync`) reads every docs package the project depends on and writes its chunks into a shared SQLite database. - The **index** is one database per machine, at `~/.docspack/store.db`. It uses SQLite's FTS5 extension for full-text search. - The **query interface** is `docspack ask`, a command any agent with a shell can run. `docspack mcp` serves the same index over the Model Context Protocol for clients that prefer a declared tool; both return identical text. Nothing in that path touches the network. After `docspack sync`, everything works offline. ## Why chunks instead of files A library's documentation is much larger than a context window. Pointing an agent at a folder of Markdown means it either reads the wrong file or fills its context with prose that does not answer the question. docspack stores documentation pre-split into chunks, each with a token count. A query returns the few passages that match, ranked, and stops once a token budget is reached. The default is three chunks and 3,000 tokens. The agent gets an answer, not a library. ## Why the index is global The index lives in one database per machine rather than one per project, the same way pnpm keeps one content-addressed store. If five projects depend on `@stripe/docspack@2025.4.1`, the chunks are read and indexed once. Different versions coexist. `@stripe/docspack@1.0.0` and `@stripe/docspack@2025.4.1` are separate rows with separate chunks, and a query from a project is scoped to the versions that project installed. A project never sees documentation for a version it does not use, even though the database holds it for a neighbour. --- # Installing and syncing docspack is a CLI. Run it with `npx docspack `, or install it as a dev dependency to pin the version your team uses. ## Requirements Node 22.5 or newer. The index uses `node:sqlite` from the Node standard library, so there is no native module to compile and `npx docspack` starts without a build step. ## Adding a documentation package Documentation packages are ordinary npm dependencies. Add one the way you add anything else: ```bash pnpm add -D @acme/docspack npx docspack sync ``` `docspack sync` reads `dependencies` and `devDependencies` from the project's `package.json`, keeps the entries named `@vendor/docspack`, `@vendor/-docspack` or `@docspack-community/*`, resolves each one in `node_modules`, and indexes its chunks. A dependency that ships a `.llms/manifest.json` under any other name is not indexed, and `sync` says so rather than passing over it: it names the package, the number of chunks its manifest declares, and the shapes it would have matched. A pack nobody can read is nearly always a publishing mistake, and the alternative to reporting it is a question answered confidently from the wrong corpus. It then reads the other dependencies — the ordinary libraries — and indexes one entry per name their type declarations export, so a question about something nobody documented can still be answered from the installed build. See [answers from the installed build](10-answers-from-the-installed-build.md). Both halves of that walk upwards from the working directory: the declaration is read from the nearest `package.json` and from every ancestor, and the install is resolved through every `node_modules` above. So a monorepo that declares its documentation once in the repository root is found from any workspace member, which is where an agent usually is. ## What sync does on the second run Indexing is keyed by name and version. A package already present in the index is reported as `cached` and its files are not read again: ``` + @acme/docspack@1.4.0 128 chunks indexed = @other/docspack@2.0.0 44 chunks cached ``` Bump a dependency and the new version is indexed as a new entry; the old one stays in the database for any project still using it. To re-read a package whose contents changed without a version bump, which happens while authoring, pass `--force`. ## Inspecting a project `docspack list` shows the documentation packages this project depends on and whether each is in the index: ``` @acme/docspack@1.4.0 128 chunks indexed ``` Problems are reported rather than hidden. A package that is declared but not installed, or installed but missing its manifest, is named on stderr so it does not silently disappear from search results. ## Choosing where the index lives The default is `~/.docspack/store.db`. Override it with `--store ` on any command, or by setting the `DOCSPACK_STORE` environment variable. A per-project index is occasionally useful in CI, where a clean database makes runs reproducible: ```bash DOCSPACK_STORE=.cache/docspack.db npx docspack sync ``` The database is safe to delete. Everything in it can be rebuilt from `node_modules` with `docspack sync`. --- # Connecting an agent There are two ways to give an agent the documentation. The command line is the simpler one and works everywhere; MCP is there for clients that prefer a declared tool. ## The command line, which is the recommended way Any agent that can run a shell command can use docspack: ```bash npx docspack ask "how do I verify a webhook signature" ``` Tell the agent the command exists, with one line in `AGENTS.md` or `CLAUDE.md`. One command writes it for you, into whichever of those files the project already has: ```bash npx docspack agent install ``` ```md Run `docspack ask ""` for documentation on this project's dependencies. It answers from the installed versions. ``` That is the whole setup. There is no server to configure, nothing per-agent, and nothing resident when nobody is asking. It works in Claude Code, Cursor, Codex, Aider, a shell script, or a CI job, including agents that do not speak MCP at all. `ask` prints exactly what the MCP tool returns: the best-ranked chunks with their package, version and file, bounded by the same token budget. The two interfaces are the same code path, so neither is second class. ## Writing the wiring, and keeping it current `docspack agent install` writes a block delimited by `` and ``. Everything outside the markers is left exactly as it was, and re-running rewrites the block in place rather than appending a second copy — so a repository wired a year ago gets the current text by running the command again. It writes more than the block when the project has somewhere to put it: | Flag | What it writes | | --- | --- | | *(default)* | the block in `AGENTS.md` or `CLAUDE.md`, whichever exists | | *(default, with `.claude/`)* | `.claude/skills/docspack/SKILL.md`, a skill whose body loads only when it is used | | `--feedback` | the reporting block below, as well | | `--hooks` | a `SessionStart` hook that runs `docspack sync`, so the index follows the lockfile | | `--mcp` | the MCP server in `.mcp.json` | | `--dry-run` | nothing; prints the plan | ```bash npx docspack agent check ``` `check` writes nothing and exits non-zero when the wiring is missing or out of date. It needs no flags: it reads back which of those surfaces the project chose and compares them against what this version would write, so it belongs in CI. A pasted instruction that has drifted from what the tool does is otherwise something nobody ever notices. ## Letting an agent report a problem An agent reading documentation is unusually well placed to notice it is wrong. Add a second block to `AGENTS.md` if you want it to write those observations down: ```md If the documentation is wrong, record it: `docspack feedback add --chunk --kind --evidence ""`. The chunk id is the heading above each answer. Only claims that can be shown false; incorrect and missing also need --expected, --actual and --repro. It writes to a local file for a human to review, and sends nothing. ``` The chunk id is the `## @acme/docspack@1.4.0/webhooks` heading printed above every answer, so an agent already has what `--chunk` needs without another query. The last sentence is the one that matters. An agent that thinks it is filing an issue behaves differently from one that knows it is appending to a file a human will read, and only the second is true. See the chunk on trust and safety for what `add` refuses and why. This block is deliberately separate. Reading documentation and reporting on it are two decisions, and a team that only wants the first pastes only the first. ## What a query costs A query spawns a process, which sounds expensive and is not: about 101 milliseconds, of which roughly 41 is Node itself starting. That is invisible next to the seconds a model spends thinking, and it is only paid when a question is actually asked. Commands that need heavier machinery load it when they run, not at startup. Importing the whole library costs about 263 milliseconds, against about 16 for the query path alone, so `ask` imports only what it needs. ## MCP, for clients that prefer a tool ```bash claude mcp add docspack -- npx -y docspack mcp ``` Or, for Cursor and other clients that read a JSON configuration: ```json { "mcpServers": { "docspack": { "command": "npx", "args": ["-y", "docspack", "mcp"] } } } ``` The server exposes two tools. `query_local_docs` takes a free-text `query` and an optional `packageFilter`, and returns text identical to `docspack ask`. `record_docs_problem` is the MCP equivalent of `docspack feedback add`, taking `chunkId`, `kind`, `evidence` and — for anything but `drift` — `expected`, `actual` and `repro`. The server speaks the protocol on stdout and writes diagnostics to stderr. `record_docs_problem` is the only tool that writes anything, and what it writes is a line in a local file. It cannot reach a maintainer: docspack has no code that can send a report, over MCP or otherwise. Its description says so, because a model that believes it is filing an issue behaves differently from one that knows a human reads the file first. Every rule is enforced in one place, so MCP is not a looser way in than the command line. A `kind` outside `drift`, `incorrect` and `missing` is refused by the schema; an `incorrect` finding without a reproduction is refused by the same check the CLI uses; a chunk that is not installed is refused before anything is written. When a call is refused, the reason comes back naming the field that was missing, so an agent can correct itself instead of guessing. The trade is real in both directions. A declared tool advertises itself: the agent sees the name, the schema and the description, and knows when to reach for it without being told. In exchange, the tool definition occupies the context window for the whole session whether or not it is used, the server process stays resident, and every client configures MCP differently. Choose MCP when you want the agent to discover the capability by itself. Choose the command line when you want zero setup and nothing running in the background — which is most of the time. ## How results are scoped Both interfaces read the `package.json` of the directory they run in, and restrict results to the package versions that project installed. This matters because the index is shared. Another project on the same machine may have indexed a different major version of the same library. That version is in the database, but it is invisible to this project's queries. An agent cannot accidentally answer from documentation for a version you are not running. ## How much context a query returns Agents over-fetch. Left alone, a retrieval tool will happily return more than the model can use and crowd out the rest of the conversation. docspack bounds every response, over MCP and on the command line alike. It returns the three best-ranked chunks by default and stops adding chunks once the response would exceed 3,000 tokens, using the token count recorded for each chunk rather than measuring after the fact. A single chunk larger than the budget is still returned, so a query never comes back empty because of the limit. --- # Searching from the command line `docspack search` runs the same query an agent would, and prints the results. Use it to check that a package is indexed, to see what an agent will receive, or to find something yourself. ```bash npx docspack search "webhook signature" ``` An answer can also carry a declaration read from an installed library, ahead of the prose, when the question names something no documentation mentions. That path does not go through the ranking described below — see [answers from the installed build](10-answers-from-the-installed-build.md). ## How matching works The index is a SQLite FTS5 table tokenized with `porter unicode61`. Porter stemming means a search for `authentication` matches a chunk that only says `authenticate`, and `retries` matches `retry`. Both the chunk's text and its tags are indexed, so a manifest that tags a chunk with `idempotency` makes that chunk findable even when the word never appears in the prose. Results are ranked by BM25, which favours chunks where the query terms are frequent and distinctive. Tags weigh three times as much as prose, so the chunk an author tagged for a subject beats one that merely repeats the words. Query text is tokenized and each term is quoted before it reaches SQLite. Punctuation in a query is searched for literally instead of being interpreted as FTS syntax, so a query like `auth AND (NOT "x")` returns results rather than a syntax error. Terms of one or two characters are dropped, because a question is mostly `how`, `do`, `i` and `in`, and every chunk containing one of those becomes a candidate. A query made only of short terms keeps them all, so `docspack search "id db"` still searches. ## Narrowing and widening results - `--package ` restricts results to packages whose name contains that text, for example `--package stripe`. - `--limit ` changes how many chunks are returned. The default is 3. It works on `ask` and `preview` too, so an agent that gets three near-misses can ask again for more. - `--max-tokens ` changes the token ceiling for the whole result set. The default is 3,000. Also honoured by `ask` and `preview`. - `--all` searches every version in the index instead of only the versions this project installed. Useful when comparing releases; not what an agent should see. ## Machine-readable output Every command accepts `--json`, which prints the full result including chunk ids, token counts, file paths and trust flags: ```bash npx docspack search "retry policy" --json ``` ## Exit codes `ask` and `search` distinguish the two ways an answer can be empty, so a script does not have to read the prose to tell them apart: | Code | Meaning | | ---- | ------- | | 0 | Chunks were returned | | 3 | Nothing matched, and a docs package is installed but not indexed — run `docspack sync` | | 4 | Nothing matched, and everything installed is already indexed | | 1 | The command failed | | 2 | The command was used wrongly | Code 3 is the first-run case. The answer says which package it is, because "you have not indexed anything yet" and "the documentation does not cover this" are different problems and only one of them is yours to fix. --- # Authoring a documentation package A documentation package is a normal npm package whose payload is Markdown rather than code. `docspack init` scaffolds one, `docspack build` regenerates it, `docspack doctor` checks it and `docspack preview` shows what an agent would get back. ## Starting from nothing Run `init` inside the library you want to document: ```bash npx docspack init ``` It reads the surrounding project first — the library name and version, a `docs/` directory, an OpenAPI document, the git remote, the license — and proposes a package built from what it found. On a terminal it asks six questions, each pre-filled, so the common path is six times Enter. In CI or a script, `--yes` takes the same defaults without asking: ```bash npx docspack init --name @acme/docspack --from ./docs --yes ``` `--dry-run` prints the file tree and writes nothing. Files that already exist are kept unless you pass `--force`, so re-running `init` in a real project is safe. ## What init writes ```text docspack/ ├── package.json name, version, build settings, publish scripts ├── README.md written for whoever installs the package ├── .gitignore the generated payload is a build output └── .github/workflows/publish-docspack.yml ``` Two lines in the generated `package.json` matter more than the rest. `files` includes `.llms`, because omitting it publishes a package that installs fine and indexes nothing. And `prepublishOnly` runs `docspack build && docspack doctor --strict`, so a stale or broken payload cannot reach the registry by accident. When no documentation is detected, `init` seeds `docs/` with three templates shaped the way retrieval works: one heading per question. They contain `TODO:` markers, and `doctor --strict` fails while those remain — a half-filled template cannot be published as though it were documentation. ## Build settings live in package.json `init` writes a `docspack` key: ```json "docspack": { "documents": "@acme/sdk@1.4.0", "from": "../docs", "maxChunkTokens": 800 } ``` `docspack build` with no arguments reads it, which keeps the build script, the release workflow and `prepublishOnly` from drifting apart. Paths are relative to the package directory. Command-line flags always win over the key. Supported fields: `documents`, `from`, `openapi`, `source`, `feedback`, `maxChunkTokens`, `minChunkTokens` and `pages`. `feedback` is how you opt in to receiving documentation problems — see the chunk on trust and safety. `init` does not write it, because opting you in by default would defeat the point. ## What the package documents `documents` names the libraries a package describes, each optionally with a version: ```json "docspack": { "documents": ["@acme/core@0.17.0", "@acme/react@0.17.0", "@acme/i18n@0.14.2"] } ``` Keeping the docs package version equal to the library version is the simplest way to scope a release, and it only works when there is one library. A monorepo publishing eighteen packages at four versions from one documentation surface has no single version to mirror, which is what the list is for. `build` writes it into `.llms/manifest.json`, so it is part of the payload rather than build configuration a consumer never sees. Every answer then states it: ```text ## @acme/docspack@2026.1.0/button-props Source: @acme/docspack@2026.1.0 — chunks/button-props.md · documents @acme/core@0.17.0 ``` Without it, an agent has to assume the docs package version is the library version. State it instead. ## Checking that the docs still describe the code `docspack verify` compares the identifiers your chunks name against what the libraries actually declare. It reads `documents` from the manifest to know which libraries those are, and checks against their combined type surface — a name belongs to whichever of them exports it: ```bash npx docspack verify ``` ```text drift @acme/docspack@1.4.0 documents @acme/sdk@1.4.0 · 31 names checked, 30 declared client.setKey() is not declared; did the API become setApiKey? @acme/docspack@1.4.0/api-auth · chunks/api-auth.md ``` It reads the library's `.d.ts` files and never imports the package, so nothing you depend on executes. Exit code 1 means something was reported, which makes it usable in CI beside `doctor`. What it checks, what it deliberately leaves alone, and how to scope one chunk to one library are in [verifying documentation against code](https://docspack.dev/docs/verify). `verify` reports a name only when it is absent *and* something very like it is declared — the signature of a rename. An absent name with no close relative is far more often an example about a different library, so it is counted and ignored. Measured against 69 published packages, that rule reported nothing at all, while catching 99.9% of simulated qualifier renames such as `setKey` becoming `setApiKey`. Findings stay on your machine. docspack has no code that can send them anywhere. ## Building from Markdown ```bash npx docspack build --from ./docs --name @acme/docspack --pkg-version 1.4.0 ``` The generator splits each document at `##` headings. If a section is still larger than the chunk budget it splits again at `###`, and then at paragraph boundaries, until every chunk fits. Retrieval works best when one chunk answers one question, so headings are worth writing with that in mind. Tables are written with their alignment padding collapsed. A generated props table padded into columns measures four times its real weight, because tokens are counted from characters — and that weight is spent on a reader's response budget, counted against `chunk-too-large`, and can force a table to be split into fragments that then compete for the same query. Collapsing it changes nothing for a reader or a Markdown renderer. Tables inside a code fence are left exactly as written. ### Generated reference: packing instead of splitting One chunk per heading is right for prose and wrong for generated reference, where a heading is a field name. An API generator emits pages shaped `Install`, `Category`, `Variants`, `Sizes`, `Props`, `Examples` — `Category` is one line and `Sizes` is three. Split per heading across two hundred component pages, that is hundreds of chunks flagged `chunk-too-small`, plus a `duplicate-chunk` for every near-identical `Sizes` section, and `doctor --strict` then blocks the publish. `minChunkTokens` packs the other way first: ```bash npx docspack build --from ./reference --min-chunk-tokens 400 --max-chunk-tokens 900 ``` Adjacent sections are merged until a chunk reaches the floor, and never past the ceiling. A whole component lands in one chunk, headed by the page title, with each section's own heading kept inside it. Nothing is packed when `minChunkTokens` is unset, so prose keeps splitting at every heading. Each chunk records the heading words as tags, and the identifiers appearing in inline code as entities. Both member accesses (`Client.setApiKey`) and bare identifiers written as two or more words (`TwoColumn`, `useSlide`, `text-accent`) count, so a component library's API is indexed and checkable by `verify` like any other. Both are indexed, which is why a chunk can be found by an API name that appears only in a code sample. ### Aiming one section Front matter `title` and `tags` belong to a whole document. A single section takes its own with a comment under the heading, which is what to reach for when one page holds eighty of them: ```md ## Two-column layout ``` The comments are removed from the generated chunk. Directive tags are added before the heading words, and tags outweigh prose in the ranking, so this is the lever that makes one chunk the answer to one question. ## Building from an OpenAPI document ```bash npx docspack build --openapi ./openapi.json ``` Every operation becomes one chunk carrying everything needed to make the call: the base URL, the credential, the inputs with their types, the body shape, the response and the failures. Add `--from` as well to publish prose and an API in one package. JSON or YAML, decided by what the file holds rather than by its extension. The whole of it — what a chunk contains, how to ask for one endpoint, and what is not read — is in [documenting an HTTP API](https://docspack.dev/docs/documenting-an-http-api). ## Building from a published llms.txt Vendors are only beginning to publish documentation packages. To try docspack against a library that has not, build a package from the project's public [llms.txt](https://llmstxt.org): ```bash npx docspack sources npx docspack init --mirror hono --name @docspack-community/hono --yes ``` This fetches over the network, unlike every other command. It is an authoring convenience, and the result is a mirror: publish it under `@docspack-community`, not as the vendor's own `@vendor/docspack`. A vendor redistributing another project's documentation alongside its own has a third option: `@vendor/-docspack`, a second pack in a scope it already owns, which keeps the two corpora separately licensed, versioned and attributed. ## Checking the package ```bash npx docspack doctor ``` `doctor` reads the package the way the indexer and a reviewer would, and reports what would otherwise fail silently: - errors: an invalid manifest, a missing or empty chunk, a chunk path escaping `.llms/`, a version that disagrees with `package.json`, or a `files` array that would publish an empty package - warnings: chunks too large to fit a response budget or too small to answer anything, chunks with no tags or entities, template placeholders, and a payload older than its source - notes: prose style — narration a model does not need, and sentences over forty words `--strict` turns warnings into failures, which is what `prepublishOnly` and CI use. It does not fail on prose style: `--strict` is the gate `init` scaffolds, and documentation written by hand is not a reason to block a first publish. `--pedantic` is `--strict` plus the prose notes, for authors who want the tool to hold that line. `--json` prints the findings for a machine. ## Seeing what an agent would get ```bash npx docspack preview "how do I authenticate" ``` `preview` indexes the package in memory and answers through the same ranking and token budget an agent gets — without publishing, installing, or writing to the global store. Most authoring mistakes are obvious the moment you see what comes back: a heading that should have been two, or a chunk with no distinguishing words in it. ## Measuring retrieval, and gating on it `preview` answers one query. `eval` answers a set of them and reports a number: ```bash npx docspack eval ./eval/queries.json --min-hit-rate 90 ``` The set is a JSON array of questions and the chunk ids that would answer them: ```json [ { "query": "how do I verify a webhook signature", "expect": "webhooks-signing" }, { "query": "rate limits", "expect": ["rate-limits", "errors"] } ] ``` ```text 1 how do I verify a webhook signature 3 rate limits expected rate-limits | errors returned charges, webhooks-signing, rate-limits top-3 2/2 (100%) top-1 1/2 (50%) ~1453 tokens per answer ``` This is the only check that can fail on retrieval. Every chunk in a package can be well-formed, well-tagged and the right size while the package answers the wrong question — `doctor` reads structure and cannot see it. `--min-hit-rate` exits 1 below the threshold, which makes it a CI gate beside `doctor --strict`, and the mean answer size is what tells you whether a change to `maxChunkTokens` bought anything. ## Every command's flags are on its own help page ```bash npx docspack build --help npx docspack doctor --help npx docspack eval --help ``` The global `docspack --help` lists the commands and the options common to all of them. What each command takes is on its own page. ## The loop ```bash npx docspack init # scaffold, build and check $EDITOR docs/02-getting-started.md # write npx docspack build # regenerate npx docspack doctor # what is still wrong npx docspack verify # do the docs still match the code npx docspack preview "how do I authenticate" npx docspack eval ./eval/queries.json # does it still answer npm publish # prepublishOnly rebuilds and checks again ``` With one library, keep the package version equal to the library version it documents, so `@acme/docspack@1.4.0` describes `acme@1.4.0`. The generated workflow does this for you on every GitHub release. With more than one, or with a version that cannot line up, declare `documents` and let the manifest carry the mapping. ## Fixing documentation you have already published Publish a new version of the documentation package. A documentation fix is never a reason to release the library again. It also never reaches anyone without a version bump: npm versions are immutable, and `docspack sync` keys the index on name and version. A consumer's report reaches you as a GitHub issue, from `docspack feedback submit` on their machine. The fix is the ordinary loop, ending in a bump: ```bash $EDITOR docs/03-authentication.md npm version patch # 1.4.0 becomes 1.4.1 npm publish # prepublishOnly rebuilds and checks again ``` A consumer on `^1.4.0` resolves `1.4.1` on the next install. `docspack sync` indexes it as a new entry and leaves the old one for projects still on it. Neither side needs `--force`, which is for content that changed without a version bump. ### The package version stops matching the library Keeping the two equal is a convention. `doctor` checks the manifest and `package.json` against each other; nothing checks either against the library. So `@acme/docspack@1.4.1` may document `acme@1.4.0`, and the manifest is where it says so: ```json "docspack": { "documents": ["acme@1.4.0"] } ``` Every answer then carries the mapping instead of leaving it to be inferred: ```text Source: @acme/docspack@1.4.1 — chunks/api-auth.md · documents acme@1.4.0 ``` Declare `documents` before the first correction rather than after. A package that relied on the versions matching has nothing to state once they stop matching. ### Corrections collide with the next release The generated workflow publishes under the release tag, so a docs-only `1.4.1` takes the version `acme@1.4.1` will want. That release then fails to publish, because npm rejects a version that already exists. Pick one of two schemes before it happens. | Scheme | The version means | Suits | | --- | --- | --- | | Independent versions | The documentation's own release | Docs fixed between library releases | | Calendar — `2026.4.1` | When the documentation was cut | Frequent fixes, on any cadence | A calendar version can never collide with a semver release tag, which is the whole of its advantage here. Both schemes need `documents`. Publish a docs-only version with `workflow_dispatch`, which takes a version as input. The `release` trigger has no tag to give it. ## Writing the manifest by hand `build` is a convenience, not a requirement. Any package with a valid `.llms/manifest.json` can be indexed: ```json { "$schema": "https://docspack.dev/schema/v1.json", "name": "@acme/docspack", "version": "1.4.0", "chunks": [ { "id": "api-auth", "file": "chunks/api-auth.md", "tokens": 150, "tags": ["authentication", "bearer", "api key"], "entities": ["Client.setApiKey"] } ] } ``` `id` must be unique within the package. `file` is relative to `.llms/`. `tokens` may be omitted and is estimated at index time — omit it rather than writing `0`, which is refused. `tags` and `entities` are optional, and are the cheapest way to make retrieval noticeably better. `documents` belongs at the top level, beside `name` and `version`. The whole format — naming, layout, every field, and the rules a consumer enforces when reading a manifest it did not write — is specified at [docspack.dev/spec](https://docspack.dev/spec), and the JSON Schema is served at the URL `$schema` points to. --- # Documenting an HTTP API An HTTP API is the one documentation topic most projects already describe in a machine-readable file. `docspack build --openapi` reads that file and writes one chunk per operation: ```bash npx docspack build --openapi ./openapi.json --name @acme/docspack --pkg-version 1.4.0 ``` Combine it with `--from` to publish prose and an API in one package, which is what a library with both usually wants: ```bash npx docspack build --from ./docs --openapi ./openapi.json ``` Or put both in the `docspack` key of the package's `package.json`, so `docspack build` with no arguments does the same thing: ```json { "docspack": { "from": "./docs", "openapi": "./openapi.json", "documents": "acme@1.4.0" } } ``` ## What one chunk contains Everything needed to make the call, and nothing else: the base URL, the credential, the inputs with their types, the body shape, the response, the failures, the types they reference, and a runnable `curl` line. The heading is the endpoint, the operation's own prose comes first, and the rest is one block: ```lapis [meta] api: Invoice Service base: https://api.example.com/v2 auth: bearer header:Authorization [ops] createInvoice POST /invoices Creates a draft invoice. > body: InvoiceCreate < Invoice # 201 The created invoice. @auth bearerAuth [types] InvoiceCreate: customer_id: str lines: [LineItem] due_date?: date [errors] 401 Error # The token is missing, expired or invalid. 422 Error # The line items did not validate. ``` That notation is [LAPIS](https://github.com/cr0hn/LAPIS), an open format for describing an API to a model rather than to a code generator. It is used here rather than JSON because the same facts cost far less. Measured against Stripe's published document — 594 operations, 1,454 schemas — one operation's chunk is around 830 tokens. The document it came from is two million. A chunk carries what the operation reaches, bounded. Where a shape was too large to expand, the chunk says so — `+32 more`, `{...}`, `# not expanded here:` — rather than presenting a partial shape as a whole one. ## Asking for one request An endpoint is a key, not a phrase. Ranked as prose, `POST /v1/charges` matches every chunk that mentions charges, and the overview page usually wins. So every operation chunk answers to both spellings of its endpoint, looked up exactly: ```bash docspack ask "POST /v1/charges" docspack ask "createCharge" ``` A concrete URL reaches the template the document declared, which is the useful direction: what you have is an id out of a log line, and what the document contains is `{charge}`. ```bash docspack ask "GET /v1/charges/ch_3OxTmP2eZvKYlo2C" # finds GET /v1/charges/{charge} ``` Naming no method matches every method on that path: ```bash docspack ask "what can I do with /v1/charges" ``` The MCP tool answers the same way, because it is the same code path. ## Chunk ids are the endpoint An operation's chunk id comes from its method and path — `post-v1-charges` — not from its summary and not from its `operationId`. A summary gets reworded and an `operationId` gets renamed; the endpoint changes only when the endpoint changes. That matters because a chunk id is what `docspack feedback add --chunk` pins and what a published link resolves to. ## JSON or YAML Either, decided by what the file contains rather than by its extension — so a `.yaml` file holding JSON works too. The reader is [`@docspack/lapis`](https://www.npmjs.com/package/@docspack/lapis), which is also a command of its own if all you want is the conversion: ```bash npx @docspack/lapis ./openapi.yaml --stats ``` ## What is not read **Swagger 2.0.** Convert it first: `npx swagger2openapi openapi.json -o openapi3.json`. **A document split across files.** An external `$ref` is refused rather than quietly dropped, because a schema replaced by "unknown" is a chunk claiming an endpoint takes no body. Bundle first: `npx @redocly/cli bundle openapi.yaml -o openapi.json`. Each of these stops the build and names the command to run. ## The page a person reads The same document renders a reference — parameters, schemas, responses, code samples in cURL, JavaScript, Python and Go, and a console that sends the request — through `@docspack/sheaf-react`. docspack.dev does this with its own API: [the reference](https://docspack.dev/api) for a reader, [`/api.lapis`](https://docspack.dev/api.lapis) for an agent that needs the map, and `/api/.md` for an agent that already knows which endpoint it wants. That Invoice Service is an actual document in this repository, and it renders as a page too: [docspack.dev/api/example](https://docspack.dev/api/example) — a bearer token, request bodies, a `DELETE` and an error table, which this site's own read-only API has none of. The chunk above is cut down to fit; the page is the whole of it. Its document is [`/api/example.json`](https://docspack.dev/api/example.json) and its LAPIS is [`/api/example.lapis`](https://docspack.dev/api/example.lapis): the whole API in about 900 tokens against 5,700 for the JSON. Open the two beside each other — that pair is the entire idea. The console never stores a credential. It is read from the field when you press Send and kept in nothing that outlives the request. ## Endpoints are not exported names `docspack verify` compares a chunk's entities against the names a library exports. `POST /v1/charges` is not one, so an operation chunk is neither accused of drift nor cleared of it. Proving an endpoint still exists means asking the server, which is a different tool from this one. --- # Verifying documentation against code Documentation drifts when a library is renamed and its prose is not. `docspack verify` catches that class of error before a reader does: it compares the identifiers a package's chunks name against what the libraries it documents actually declare. ```bash npx docspack verify ``` ```text drift @acme/docspack@1.4.0 documents @acme/sdk@1.4.0 · 31 names checked, 30 declared client.setKey() is not declared; did the API become setApiKey? @acme/docspack@1.4.0/api-auth · chunks/api-auth.md ``` Run without arguments it checks every docs package this project has installed, which is the consumer's question: do the docs I am about to hand an agent still describe the code I have? `--package-dir ` checks one package on disk instead, which is the publisher's. ## What it reads, and what it never runs `verify` reads the `.d.ts` files of each documented library. It never imports the package and never executes it, so a `postinstall` or a module side effect cannot run because a check was run. A library that ships no type declarations is reported as unchecked rather than as passing. Two things are compared. The `entities` on each chunk, which are the identifiers the documentation names, and the export surface of the libraries in the manifest's `documents` field. A name is fine if any library in scope declares it. Exit code 1 means something was reported. Nothing was reported is exit code 0, so `verify` belongs in CI beside `doctor`. `--json` prints the whole report for a script to read. ## Where the entities come from `docspack build` fills `entities` from the source documentation. A name in `inline code` that looks like an identifier is one, whether it carries a dot or not, so `client.setKey()`, `TwoColumn` and `useSlide` are all checked. Add one the generator missed with a directive under the heading: ```md ## The layout primitive ``` Prose that names an API in `inline code` is therefore doing two jobs: it retrieves better, and it is what `verify` has to work with. A page that describes an API without naming it cannot be checked. ## Which libraries a chunk is checked against The manifest's `documents` field names them. On a package it is the union of everything the pack covers, which is the right answer for a single library: ```json { "documents": ["@acme/sdk@1.4.0"] } ``` A monorepo publishing many libraries from one documentation surface needs the narrower answer, because a name exported by any one of eighteen libraries would otherwise excuse a rename in any chunk. A chunk that carries its own `documents` is checked against those libraries alone: ```json { "id": "api-auth", "file": "chunks/api-auth.md", "documents": ["@bpmnkit/api@1.2.0"] } ``` `docspack build` writes that from front matter on a document, or from a `` directive under a heading. A package that declares no `documents` anywhere is skipped, with the reason stated. ## Why some drift is not reported A finding is produced only when a documented name is absent **and** a very similar declared name exists. That is the signature of a rename. An absent name with nothing like it is far more likely to come from an example about some other library, so it is counted and ignored. Two shapes count as similar, and only two: - a near-identical spelling of the same leading word, which catches typos and small edits - a qualifier inserted between the same first and last word — `setKey` against `setApiKey` A different first word is never a match. `includeLanguages` and `excludeLanguages` are two edits apart and mean opposite things. Names shorter than four characters are skipped, as are file extensions such as `wrangler.jsonc` and platform members such as `document.cookie` or `stream.Duplex`. They belong to the language or the runtime, not to the library, and the library never declared them. The trade is deliberate: precision over recall. A check that cries wolf is worse than no check, because the report stops being read. ## Where it sits among the other checks Three commands answer three different questions about a package, and none of them substitutes for another: | Command | Question | | --- | --- | | `docspack doctor` | Is the package well-formed, and will it retrieve? | | `docspack eval` | Does it still answer the questions it is supposed to answer? | | `docspack verify` | Do the names in it still exist in the code? | `doctor` can pass on a package whose every chunk describes an API that was renamed two releases ago. `verify` is the check for that, and it is the one that makes a community package — generated from someone else's `llms.txt`, and drifting from the day it was built — worth depending on. --- # Trust and safety Documentation packages are third-party input that ends up in a model's context. docspack treats them accordingly. ## Vendor and community packages Two naming conventions carry different guarantees. `@vendor/docspack` and its siblings `@vendor/-docspack` are published by the vendor, under a scope only the vendor can publish to. `@docspack-community/` is published by whoever maintains it. The suffix changes nothing about trust, because the npm scope is what trust rests on: whoever can publish `@vendor/docspack` can publish `@vendor/-docspack` and nobody else can. What the shape is *not* is a way for a package to nominate itself — a `docspack` field in `package.json`, or a `.llms/` directory, would let any transitive dependency at any depth declare itself documentation and be indexed as trusted. The name check is what prevents that, and it is the reason discovery is a name check at all. Community results are labelled `(community)` in `docspack search` and `docspack list`. When a community chunk is returned over MCP, the response ends with a notice telling the model that the source is unverified and that its content is data, not instructions. That does not make prompt injection impossible, but it means an agent is never handed unvetted text without being told what it is. Prefer vendor packages where they exist. Read a community package before adding it, exactly as you would read any other dependency. ## Path traversal A manifest is a routing table written by someone else, and every `file` entry is resolved before being read. A path that resolves outside the package's own `.llms/` directory is refused and reported, not read: ```json { "id": "escape", "file": "../../../../etc/shadow" } ``` The rest of the package still indexes. One bad entry does not take down the whole package, and it does not read a file outside the boundary either. ## Context exhaustion An agent that retrieves too much context is as broken as one that retrieves nothing. Every response is bounded, using the token count stored with each chunk rather than an estimate made after the fact. See the chunk about connecting an agent for the exact limits and how to change them. ## What docspack does not do over the network `sync`, `search`, `list` and `mcp` never make a network request. They read `node_modules` and a local SQLite file. An agent using docspack works on a plane, in a locked-down CI runner, or inside a network-isolated container. The single exception is `docspack build` when given a remote source, which is an authoring command run deliberately, not part of serving documentation to an agent. ## Recording documentation problems An agent reading documentation is unusually well placed to notice it is wrong: it has the docs, the installed library and a failing program in front of it at once. `docspack feedback` captures that, and the whole design is built so it cannot turn into a flood of machine-written issues aimed at maintainers. ```bash docspack feedback add --chunk @acme/docspack@1.4.0/api-auth \ --kind drift --evidence "client.setKey is not exported; setApiKey is" docspack feedback list docspack feedback remove ``` Agents on MCP use the `record_docs_problem` tool instead, which takes the same fields and runs the same checks. Both interfaces call one function, so neither can accept a finding the other would refuse. Findings go to `.docspack/feedback.jsonl` in the project: one JSON object per line, readable, diffable, and deleted with `rm`. **Nothing leaves the machine.** docspack contains no code that can transmit a report — no tokens, no endpoints, no background requests. That guarantee comes from something not being built, which is the strongest kind. A finding has to be falsifiable, and that is enforced when it is written rather than when a maintainer reads it: - The chunk must exist in a package installed here. A reader that cannot point at the chunk did not read it. - `drift` must name the identifier that drifted, so it cannot be used to file an opinion. - `incorrect` and `missing` must carry `--expected`, `--actual` and `--repro`. A claim that cannot show its work is not recorded at all. - There is no kind for "this page is confusing". Unfalsifiable claims are infinitely generatable and of no use to anyone. Recording the same problem again increments a counter instead of adding a second entry. The fingerprint covers the package, version, chunk, kind and the normalized claim, so the same problem worded differently still lands on the same line. Twelve agents hitting one bug produce one finding that says `seen 12×` — stronger evidence than twelve reports, and far cheaper to read. A repeat only ever adds to the record; it never rewrites wording a human may already have reviewed. ## Sending a problem upstream `docspack feedback submit` prints a prefilled GitHub issue URL for every finding whose vendor asked to receive it. It routes nothing anywhere else, and it sends nothing at all: a human opens the link, reads what is prefilled, edits it, and files it under their own name. A vendor opts in from their own `package.json`, the way [security.txt](https://www.rfc-editor.org/rfc/rfc9116.html) solved the same problem for vulnerability reports — a machine-discoverable contact point that the recipient declares: ```json "docspack": { "feedback": { "github": "acme/sdk", "labels": ["documentation", "agent-reported"], "accepts": ["drift"], "policy": "https://acme.dev/docs-feedback" } } ``` No `feedback` block means no channel, and `submit` lists those findings as **not routed** rather than guessing a destination. That is the default, and it is the correct one: a maintainer who has not asked for this gets nothing. `accepts` is the field that matters most. A vendor drowning in noise narrows it to `["drift"]` and receives only the mechanically verifiable class — reports where no judgment was exercised and so no judgment can be wrong. Omitting `accepts` means all kinds are welcome. The person who bears the cost sets the threshold. `policy` is shown before anything is filed, so the vendor's own rules are read first. Reports state their provenance rather than hiding it, because maintainers have asked to be able to tell machine-written reports apart, and concealing it is what makes them toxic: ```md **Chunk**: `@acme/docspack@1.4.0/api-auth` **Kind**: drift — a name the docs use that the package does not declare **Seen**: 12 times in one project client.setKey is not exported; setApiKey is --- Recorded with docspack 0.1.0, read and filed by a human. ``` When a finding carries a reproduction, `submit` warns you first: the reproduction is often your own code, which is exactly what makes it useful and what makes it sensitive. You see the whole report before anything is filed, because a human is in the loop by construction. There is no automatic filing, at any level of confidence. There is no hosted service, no telemetry, and no credentials. The strongest guarantee that docspack cannot become an automated flood is that docspack contains no code that can send a report. --- # How docspack compares docspack overlaps with several things an agent can already do. The differences are worth being precise about, because most of them are not about capability but about which version of the truth reaches the model, and how much of it. ## docspack is an MCP server, and does not need to be "Why not just use MCP?" is a category error, and a common one. MCP is the protocol an agent uses to call a tool; it says nothing about what is behind the tool. docspack ships an MCP server — `docspack mcp` — so the comparison is not docspack versus MCP. It also does not require one. `docspack ask ""` returns the same text from the same code path, so the recommended setup is a line in AGENTS.md rather than a server. That costs no per-client configuration, no resident process, and no tool definition sitting in the context window for a session that may never ask a question. The real comparison is with documentation MCP servers that fetch from the web on each call. Those return whatever the vendor publishes today, which is rarely the version in your lockfile; they need the network on every query; and they usually return whole pages, because they have no chunk-level index to rank against. docspack answers from packages installed in `node_modules`, so the version matches by construction, nothing is fetched, and the response is ranked and bounded. ## Agent skills solve a different problem A skill teaches an agent how to do something: your review checklist, your deploy runbook, the house style for a commit message. It is authored once, lives with the agent, and is loaded as a whole document when it applies. Dependency documentation has a different shape. It changes when you bump a version, it is far larger than any context window, and it should be identical for everyone on the team regardless of which agent they use. Encoding it as a skill means one person maintaining a copy that silently drifts from the installed version. docspack ships it as a dependency instead, so it updates with `npm update` and reads the same for every agent on the team — whether they call `docspack ask` or the MCP tool. Skills and docspack compose: a skill that says "check the docs before answering API questions" is more useful when the docs are actually there. ## Vendoring documentation into the repository Copying a library's Markdown into `docs/vendor/` or pasting it into `CLAUDE.md` gets the content local, which is the right instinct. Two things go wrong. The copy has no version attached, so nobody notices when it describes a release you no longer run. And an agent either loads the whole file, which spends context on prose that answers nothing, or greps it, which finds the word but not the section. docspack keeps the content local and adds the two missing pieces: a version, and an index that returns the passage rather than the file. ## Vector databases and RAG A hosted vector store gives you semantic search, which finds a passage that means the same thing in different words — genuinely better recall than keyword search for vague questions. The costs are an embedding pipeline, a service to run, an ingestion step that has to be repeated whenever a dependency changes, and a network round trip per query. For documenting the libraries a project already installs, that machinery is aimed at a problem you do not have: the corpus is small, already written in the vocabulary the user types, and already versioned by the package manager. SQLite's FTS5 with porter stemming handles it with no service, no ingestion pipeline, and no round trip. ## The llms.txt convention `llms.txt` is a publishing convention: a Markdown index a project puts at a well-known URL, listing its documentation. It solves discovery, and docspack builds on it — `docspack build` can generate a package from one. It is not a retrieval system. Reading an llms.txt still means fetching the pages it links to, over the network, at whatever version the site serves. docspack turns that content into versioned, chunked, locally indexed packages. ## What it costs in context The concrete case, measured against Hono's published documentation: - everything the project publishes, as one file: about 92,000 tokens - the same content as a documentation package: 309 chunks, about 62,000 tokens on disk - what one query returns: 833 to 1,040 tokens across three chunks The middle number never reaches the model. The index holds it; a query spends about one percent of the first number, and the response is capped at 3,000 tokens no matter what is asked. Token counts are estimated at four characters per token. --- # Writing documentation for agents A documentation package is read by a model, not a person browsing a website. That changes what belongs in it. `docspack doctor` enforces most of this, so the rules below are what the warnings mean. ## Cut filler, not prose The instinct to compress everything to terse notes is wrong, and the effect is measurable. Stripping every prose sentence from a 309-chunk package — keeping only headings, code and lists — produces this: - corpus on disk: 62,366 tokens becomes 43,807 - what a query returns: 14% fewer tokens - top result correct: 17 of 20 queries instead of 20 of 20 The index matches on the words someone types. Those words live in sentences. A query for "basic authentication" found the basic-auth chunk in full text, and a generic middleware page once the prose was gone. Losing 15% of retrieval accuracy to save 14% of a budget that is already capped is a bad trade. What is free to remove is filler: narration and marketing that carries neither a fact nor a word anyone would search for. `doctor` flags these: ``` In this guide we will explore… nobody searches for it Let's dive in! nobody searches for it As you can see, … nobody searches for it A blazing-fast, world-class router nobody searches for it ``` Delete them. Keep "Keys are scoped per environment, so a test key fails against production" — every word of that is a fact, and half of them are search terms. ## One claim per sentence `doctor` warns at 40 words. A long sentence usually contains two facts and a conjunction, and both facts retrieve better apart. Splitting also gives the ranker more distinct chunks to work with. ## Show, do not describe `doctor` counts chunks with no code, no inline identifier, no list and no table. A chunk that only describes is hard to act on and hard to find, because API names are the terms users search for. ```md Bad: The client must be configured with credentials before use. Good: Call `client.setApiKey(key)` before any other request. ``` Names in `inline code` are extracted as entities and indexed, which is how a query for `setApiKey` finds a chunk that never spells the word out in prose. ## One heading, one question docspack splits chunks at `##`. Write headings that match how someone would ask: "Verifying a webhook signature" retrieves better than "Advanced usage". A section that answers five questions becomes one chunk that half-answers all of them. `doctor` warns above roughly 1,500 tokens, because a chunk that large crowds out everything else in a response, and below 30, because a chunk that small answers nothing. ## How the ranking treats what you write Two facts about bm25, the ranking, decide more than any advice on this page. Tags outweigh prose — the weights are `(content 1.0, tags 3.0)` — so the words in a heading and in a `` directive are the strongest lever you have. Function words are filtered out of heading-derived tags for exactly that reason: a section titled "How to use it" otherwise outscored a 605-chunk corpus on every question beginning "how do I". **And bm25 normalizes for length: a short chunk outranks a long one on the same match.** A thorough page therefore loses to a stub that says less about the same thing. This pulls against the response budget, which prefers few large chunks over many small ones, and there is no setting that is right for every corpus — so measure it: ```bash npx docspack eval ./eval/queries.json ``` `eval` answers a set of questions from the package and reports hit rate, top-1 rate and mean answer size. Sweeping `maxChunkTokens` against 40 questions on one real corpus showed 400 tokens retrieving best (35/40 in the top 3) and 900 answering best (27/40 top-1, at twice the tokens per answer). The number to pick depends on your corpus; the tradeoff is always there. ## Do not ship the same thing twice `doctor` reports two chunks that normalize to the same text. Duplicates compete for the same query and waste the response budget on repetition. This happens most often when a package is built from a site that publishes both a full document and per-page versions of it. ## What build removes for you `docspack build` strips front matter, badges, and site chrome such as "Edit this page" or "Was this page helpful?" out of your Markdown. Front matter is not simply deleted: `title` becomes the document title and `tags` or `keywords` are indexed alongside the prose. Table alignment padding is collapsed as well, which is lossless for a reader and for a renderer but can be three quarters of a generated reference page's measured tokens. ## Checking your work ```bash npx docspack doctor # what is wrong npx docspack doctor --strict # structural warnings as failures — what prepublishOnly runs npx docspack doctor --pedantic # the same, plus prose style npx docspack preview "how do I authenticate" npx docspack eval ./eval/queries.json --min-hit-rate 90 ``` `preview` is the honest check. It answers through the same ranking and token budget an agent gets, so a chunk that reads well but retrieves badly shows up immediately. `eval` is `preview` over a whole question set, with a number at the end. It is the only check that can fail on retrieval: every chunk in a package can be well-formed, well-tagged and the right size while the package still answers the wrong question, and `doctor` cannot see that. With `--min-hit-rate` it belongs in CI beside `doctor --strict`. Every command's own flags are on its help page — `docspack eval --help`, `docspack doctor --help` — which is where to look rather than at the global one. --- # Answers from the installed build Half of a well-documented library's exported names are mentioned nowhere in its own documentation. Measured against two projects that document themselves well, hono's site covers 51% of what hono exports and zod's covers 27%. Every one of those uncovered names is declared in the package sitting in `node_modules`. So `docspack sync` indexes that too. Alongside the documentation packages, it reads each installed library's own type declarations and records one entry per exported name, carrying the import a caller writes and the declaration as the installed build states it. ``` + @acme/docspack@1.4.0 128 chunks indexed + hono@4.13.3 272 chunks indexed (declarations) ``` This is what a documentation site cannot serve, because the site is not the artifact, and what `llms.txt` cannot serve for the same reason. docspack is already running in the directory where the answer is. ## Declarations are looked up by name, never ranked A library exports far more names than its documentation has pages. Ranking declarations against prose would answer every question with type machinery, so they are not in the ranking at all: a declaration reaches an answer only when the question names it exactly. Matching is case-sensitive. Someone asking about an API writes it as it is spelled, because they read it in a stack trace or an editor. Matching loosely would let "how do I send data" pin a type called `Data` ahead of the page that answers the question. ## Saying what the documentation does not cover A ranker always returns its best matches, and "best" is not "relevant". Ask about a name no documentation mentions and the old answer was three unrelated passages, formatted exactly like a correct answer, with nothing to say the subject was never covered. Now the answer leads with the declaration and ends with the difference: ```text ## hono@4.13.3/ValidationTargets Source: hono@4.13.3 — declared in dist/types/types.d.ts, read from the installed package # ValidationTargets Declared by `hono/types`. This is the installed build's own declaration, not prose. import { ValidationTargets } from "hono/types"; export type ValidationTargets = { … }; --- NOTE: `ValidationTargets` is exported by the installed library but mentioned in no documentation package here. The declaration above is the installed build's own, not prose anyone wrote. ``` A question that names no symbol — "how do I stream a response" — has no name to be absent, so nothing changes for it. This closes the half that can be checked mechanically. ## Turning it off ```bash npx docspack sync --no-artifacts ``` Declarations are read from packages this project depends on directly, and only from names an entry point in the `exports` map actually publishes. A package that ships no type declarations is skipped without comment: most of a dependency tree is not what anybody asks questions about. ## What changed between two versions The store is global, so a version indexed for one project on this machine is available to every other. `docspack changed` compares two of them: ```bash npx docspack changed hono ``` ``` hono 3.12.12 → 4.13.3 246 added, 3 removed gone COMPOSED_HANDLER, UndefinedIfHavingQuestion, childrenToString new ALBProcessor, Accept, AddedSSGDataRequest … and 226 more (--json for all) 237 new names that no documentation package here mentions: … ``` Upgrades are overwhelmingly additive. The gap between what a model remembers and what is installed is mostly **things that did not exist yet**, not things that broke — and the useful report is what exists now that an older release did not have. Nothing is fetched: both versions come from the local index, and a version that is not indexed is an error saying so rather than a network request. ## Coverage The same comparison, pointed at a documentation package instead of a version: ```bash npx docspack list --coverage ``` ``` @acme/docspack@1.4.0 128 chunks indexed documents 189/367 (51%) of acme's exports missing: ValidationTargets, NotFoundHandler, ErrorHandler, … ``` `docspack doctor` reports the same number as a note. It is never a failure and `--strict` never promotes it: a page listing every export and explaining none would score full marks, so this is a number for an author to read rather than a gate. A package documenting a command line rather than a library API will score near zero and be perfectly correct — the note's own suggested fix says so. --- # One source, two audiences Documentation has two readers with incompatible needs. A person wants a page: navigation, prose, a table of contents. An agent wants a passage: small, self-contained, and matched to a question. Most projects serve them from two pipelines, and the two drift. This page is the worked example. It is rendered as HTML at , served as Markdown at the same path with `.md` appended, and shipped as a chunk inside `@docspack/docspack` — all from this one file. ## The two halves `docspack` is the agent's half. It indexes documentation packages into a local SQLite database and answers from the versions a project actually installed. Nothing it does requires a website to exist. **Sheaf** is the reader's half. It turns a directory of Markdown into a *content graph* — an ordered, serializable object with a slug, title, summary and headings for every document — and that graph renders the site. It is four packages, and you take only the ones you need. | Package | What it does | | --- | --- | | `@docspack/sheaf` | The content graph. No dependencies, no rendering. | | `@docspack/sheaf-astro` | Renders the graph with Astro's own Markdown pipeline. | | `@docspack/sheaf-react` | The docs shell: sidebar, page, contents, pager. | | `@docspack/sheaf-emit` | Turns the graph into a docspack package. | The graph is the contract. Everything else reads it, and nothing reads anything else. ## Why one parse rather than two A folder of Markdown does not say which documents are published, in what order, or under what URLs. Two tools reading that folder each decide for themselves, and their answers diverge the first time a page is renamed or a draft is added. The site drops a page; the index keeps it. A graph decides once. Drafts, ordering, slugs and headings are settled before either consumer sees them, so the site and the index cannot disagree about which documents exist or what they are called. ## What a chunk cites `docspack build` reads a folder, so the most it can record about a chunk's origin is the file it came from: ```text ``` That is no use to whoever reads the answer. A graph knows where its pages are published, so a chunk emitted from one cites the page and the section: ```text ``` The answer an agent gives now carries a link a person can open. This package is built that way: all of its chunks cite a URL, and every one of those URLs is a route this site serves. ## Building this package from the graph The whole build is one file. It reads the directory, emits the payload, and says nothing else: ```js import { buildGraph } from "@docspack/sheaf"; import { emitDocspackPackage } from "@docspack/sheaf-emit"; const graph = await buildGraph("./docs"); await emitDocspackPackage(graph, { out: ".", name: "@docspack/docspack", version: "1.0.0", documents: ["docspack"], baseUrl: "https://docspack.dev/docs", }); ``` `baseUrl` is what turns a filename into a citation. Without it the emitter falls back to the source path, which is what a folder-based build would have written anyway. Sections split at every `##`. A section over roughly 800 tokens splits again at its own `###` headings, because an oversized chunk crowds the rest of an answer out of the response budget — the same escalation `docspack build` performs, and the reason `docspack doctor` warns above 1500. ## Rendering the same graph as a site Rendering belongs to the host, not to Sheaf. `@docspack/sheaf-astro` binds the graph to Astro's own Markdown processor — the same one that renders a `.md` file in `src/pages` — so a project's Markdown configuration keeps applying: ```js import { createPageRenderer } from "@docspack/sheaf-astro"; const renderPage = createPageRenderer({ syntaxHighlight: false }); const { html, headings } = await renderPage(page); ``` The graph deliberately carries no HTML. That is what lets a React app, a Preact app and a static Astro page each use their own renderer, and what lets the emitter above ship Markdown rather than markup. ## Blocks, where the two renderings differ The list of packages above is a Markdown table. A reader sees it drawn; an agent is given the table. Neither rendering invents anything the other does not have, because the source wraps the table in a block rather than replacing it: ```text :::packages | Package | What it does | | --- | --- | | `@docspack/sheaf` | The content graph. No dependencies, no rendering. | ::: ``` `toAgentMarkdown` removes the block and keeps what it wrapped, and every agent-facing output goes through it: the chunks in this package, the `.md` route for this page, and `llms-full.txt`. The site's renderer takes the same parse and replaces the block with a component. The rule is structural, so the emitter needs to know no component names: **A block wraps content; it never replaces it.** A block with nothing inside it is refused by file and line, because there would be nothing to give an agent where the page shows something — and a component the site has no renderer for fails the build for the same reason from the other side. A block that reached either audience as an absence is the one outcome neither half allows. ## Anchors are the seam A table of contents is built from the graph's heading slugs. The `id` attributes it links to are stamped by the renderer. Different code produces them, and a mismatch fails silently: a fragment that matches nothing scrolls to the top of the page instead of raising an error. It is worth guarding deliberately, because nothing reports it. The graph's slugs are checked against `github-slugger` and against the real Astro processor; the rendered page is checked for the ids the graph predicted; and the URLs this package publishes are checked against the routes the site builds. ## Which half do you need You can take either half alone. - **docspack only** — you consume other people's documentation packages, or you publish one from a folder with `docspack build`. No site, no graph, no Sheaf. - **Sheaf only** — you want a documentation site inside an app you already have, and you do not publish anything for agents. Take the graph and a renderer. - **Both** — one directory of Markdown becomes the site people read and the package their agents answer from, and the two cannot describe different documentation. --- # Indexing your own sources `docspack sync` indexes documentation somebody published. `docspack index` indexes material this project already has — notes, decision records, an export, rows out of a query — and `docspack recall` answers from it. ```bash docspack index --from ./notes docspack recall "what did we decide about retries" ``` The reason is the one this tool exists for. A corpus larger than the context window cannot be read into it, and an agent handed a folder reads the wrong files or exhausts its budget. The index returns the passages that answer the question and nothing else, bounded by the same 3,000-token ceiling `ask` uses. Indexing needs no model, so building the index costs no tokens at all. ## Records, not only files Anything that can emit JSON can be indexed, so there is no database driver here: ```bash sqlite3 -json shop.db 'select id, title, body as text from posts' | docspack index --from-json - psql -At -c "select ..." --json | docspack index --from-json - docspack index --from-json ./export.json ``` Each record is `{ "title", "text" }`, with optional `id`, `tags` and `entities`. A record that carries an `id` becomes exactly one chunk under that id — a row's identity is its key, and splitting it would either duplicate that key or discard it. A record without an `id` is split by heading and size like any other document. `tags` are worth supplying. They weigh three times as much as prose in ranking, and for rows the useful ones are usually the columns you did not put in the text: an author, a status, a date. ## What keeps it honest **The corpus changes underneath the index.** Published documentation is immutable for the life of its version; your own files are not, and are often edited by the same agent that is asking about them. So `index` records each source's size, modification time and hash, and `recall` leads with a warning when any of them no longer match: ``` NOTE: the corpus is out of date. 1 indexed source has changed since it was built: notes/retries.md. The passages below may be superseded — run `docspack index` again before relying on them. ``` A stale answer is quoted correctly and is still wrong, which is worse than no answer, so the warning comes before the passages rather than after them. Re-running `index` does nothing when no source has changed, so it is cheap to run often. A corpus piped in through `--from-json -` has nothing on disk to compare against later, and an answer from one says so instead of implying it is current. **`recall` is not `ask`.** They are separate commands on purpose. `ask` promises an answer from the versions this project installed, and that promise is kept by one filter in the query path. Your notes are never an installed version, so they cannot reach an answer about a dependency — and a dependency's documentation cannot reach an answer about your notes. Neither `--all` nor pointing both at the same database changes that. **The index is a copy of your sources.** It lives in `.docspack/local.db` as plaintext, so it is as sensitive as whatever went into it. `docspack index` writes a `.gitignore` beside it so it is not committed by accident. ## Whether it is worth it Not obvious, and measured rather than asserted. An agent already has glob, grep and read, and below some corpus size those are better than an index: exact, no setup, and incapable of going stale. On one 269-chunk corpus of prose nobody wrote for retrieval, the index returned the answering passage for 15 of 17 held-out questions at about 1,170 tokens per answer. Reading a window around grep's best hit managed 8 of 17 at a similar cost; reading grep's top three files whole found all 17, but cost about 16,000 tokens — roughly fourteen times as much. So the index is not more accurate than reading everything grep suggests. It is much cheaper at close to the same accuracy, and that gap is what compounds across a session. Where the crossover sits for a smaller corpus is not known.