Verifying documentation against code

Documentation drifts when a library is renamed and its prose is not. docspack verify catches that class of error before a reader does: it compares the identifiers a package’s chunks name against what the libraries it documents actually declare.

npx docspack verify

drift @acme/docspack@1.4.0 documents @acme/sdk@1.4.0 · 31 names checked, 30 declared
  client.setKey() is not declared; did the API become setApiKey?
  @acme/docspack@1.4.0/api-auth · chunks/api-auth.md

Run without arguments it checks every docs package this project has installed, which is the consumer’s question: do the docs I am about to hand an agent still describe the code I have? --package-dir <dir> checks one package on disk instead, which is the publisher’s.

What it reads, and what it never runs

verify reads the .d.ts files of each documented library. It never imports the package and never executes it, so a postinstall or a module side effect cannot run because a check was run. A library that ships no type declarations is reported as unchecked rather than as passing.

Two things are compared. The entities on each chunk, which are the identifiers the documentation names, and the export surface of the libraries in the manifest’s documents field. A name is fine if any library in scope declares it.

Exit code 1 means something was reported. Nothing was reported is exit code 0, so verify belongs in CI beside doctor. --json prints the whole report for a script to read.

Where the entities come from

docspack build fills entities from the source documentation. A name in inline code that looks like an identifier is one, whether it carries a dot or not, so client.setKey(), TwoColumn and useSlide are all checked.

Add one the generator missed with a directive under the heading:

## The layout primitive

<!-- docspack: entities=TwoColumn,useBreakpoint -->

Prose that names an API in inline code is therefore doing two jobs: it retrieves better, and it is what verify has to work with. A page that describes an API without naming it cannot be checked.

Which libraries a chunk is checked against

The manifest’s documents field names them. On a package it is the union of everything the pack covers, which is the right answer for a single library:

{ "documents": ["@acme/sdk@1.4.0"] }

A monorepo publishing many libraries from one documentation surface needs the narrower answer, because a name exported by any one of eighteen libraries would otherwise excuse a rename in any chunk. A chunk that carries its own documents is checked against those libraries alone:

{
  "id": "api-auth",
  "file": "chunks/api-auth.md",
  "documents": ["@bpmnkit/api@1.2.0"]
}

docspack build writes that from front matter on a document, or from a <!-- docspack: documents=@bpmnkit/api@1.2.0 --> directive under a heading. A package that declares no documents anywhere is skipped, with the reason stated.

Why some drift is not reported

A finding is produced only when a documented name is absent and a very similar declared name exists. That is the signature of a rename. An absent name with nothing like it is far more likely to come from an example about some other library, so it is counted and ignored.

Two shapes count as similar, and only two:

• a near-identical spelling of the same leading word, which catches typos and small edits
• a qualifier inserted between the same first and last word — setKey against setApiKey

A different first word is never a match. includeLanguages and excludeLanguages are two edits apart and mean opposite things.

Names shorter than four characters are skipped, as are file extensions such as wrangler.jsonc and platform members such as document.cookie or stream.Duplex. They belong to the language or the runtime, not to the library, and the library never declared them.

The trade is deliberate: precision over recall. A check that cries wolf is worse than no check, because the report stops being read.

Where it sits among the other checks

Three commands answer three different questions about a package, and none of them substitutes for another:

doctor can pass on a package whose every chunk describes an API that was renamed two releases ago. verify is the check for that, and it is the one that makes a community package — generated from someone else’s llms.txt, and drifting from the day it was built — worth depending on.