4. Putting Zensical in CI, and What Strict Mode Cannot See

A generator that builds correctly on your laptop and cannot fail a build in CI will eventually publish something broken. This part puts zensical in a pipeline, looks at the workflow it writes for you, and reports exactly which failures it can and cannot catch.

We got this wrong on the first pass, which is worth saying up front because the mistake is easy to make from the help text alone.

Two flags with the same name and different answers

$ zensical serve --help
  -s, --strict    Strict mode (currently unsupported)

$ zensical build --help
  -s, --strict    Enable strict mode - abort the build on warnings

Same flag, two subcommands, opposite descriptions. Read only the first and you conclude strict mode does not exist. It does, on the subcommand CI actually runs, and it works.

The dev server one being unsupported is defensible: aborting a live-reload server on a warning would be unhelpful. But the inconsistency is a trap, and we fell in it before checking both.

What strict actually catches

We broke the same projects four ways and recorded the exit code with and without the flag. Exit codes are all CI reads.

Broken input	Plain	--strict
Link to a page that does not exist	0	1
nav entry pointing at a missing file	0	0
Unsupported MkDocs plugins	0	0
Clean site, as a control	0	0

Row one is the good news and it is real: a broken internal link prints Warning: page does not exist, and under --strict that becomes exit 1. That is the check most documentation pipelines need most.

Rows two and three are the finding. Strict mode escalates warnings, and the silent failures never produce a warning to escalate. A nav entry pointing at a file that does not exist builds clean and reports "No issues found" whether or not you pass the flag. The plugins from part 3 disappear just as quietly.

The workflow it scaffolds, and what is missing from it

zensical new . writes .github/workflows/docs.yml. Here is the substance of it:

- uses: actions/setup-python@v6
  with:
    python-version: 3.x
- run: pip install zensical
- run: zensical build --clean
- uses: actions/upload-pages-artifact@v5
  with:
    path: site
- uses: actions/deploy-pages@v5

It wires up GitHub Pages with the right permissions block and the modern deploy actions, which saves real time. Three things about it are worth changing before you rely on it.

It does not pass --strict. The flag exists, works on build, and the default pipeline does not use it. So the workflow you get for free ships with the one available safety net switched off.

python-version: 3.x is unpinned. Given part 1, this is the trap waiting to happen: 3.x resolves to whatever the runner considers current. It is fine today. On a runner that resolved to something below 3.10 you would silently install the 1.0 kB stub and every later step would fail in a way that points nowhere near the cause.

pip install zensical is unpinned. At version 0.0.63, floating your dependency on the latest release is asking for a build that changes under you.

The version we would actually run:

- uses: actions/setup-python@v6
  with:
    python-version: '3.12'
- run: pip install zensical==0.0.63
- run: zensical build --clean --strict

Three edits, all one line, and the pipeline goes from "probably fine" to "fails when it should".

Covering what strict cannot see

Strict handles warnings. For the silent cases you need an assertion of your own, and the cheapest useful one is counting output against input:

expected=$(find docs -name "*.md" | wc -l)
actual=$(find site -name "index.html" | wc -l)

if [ "$actual" -lt "$expected" ]; then
  echo "::error::built $actual pages from $expected sources"
  exit 1
fi

Crude, and it catches the nav case from the table above, which strict does not. If you are migrating from MkDocs, the file tree diff from part 3 is the stronger version of the same idea and worth running once rather than on every build.

Neither is elegant. Both are the difference between noticing and not, and both can be deleted if the silent cases start producing warnings, at which point --strict would catch them for free.

What we would actually run

Putting the four parts together, the pipeline we would be comfortable pushing:

1. Pin Python explicitly. Part 1's failure is far worse in CI than locally: a runner below 3.10 installs the 1.0 kB stub and every later step fails confusingly.
2. Pin the zensical version. At 0.0.x, floating is asking for a surprise on a day you are not looking.
3. Pass --strict. It works on build, it catches broken links, and the scaffolded workflow leaves it off.
4. Assert the page count against the markdown file count, to cover what strict structurally cannot see.
5. Diff the file tree once against your old build if you are migrating, per part 3.

Steps 4 and 5 exist only because of the silent failures. If those ever produce warnings, strict catches them and both steps can be deleted.

Error messages, good and bad

Two of our five deliberate breakages produced genuinely good output. A missing docs directory gives Error: Docs directory does not exist: plus the absolute path, and exits 1. That is what you want from a log at 2am.

One did not. Malformed TOML produces a raw Python traceback:

tomli._parser.TOMLDecodeError: Expected ']' at the end of a ...

An internal exception reaching the user rather than a handled error. It exits 1 so CI catches it, and the message does name the real problem, so this is a rough edge rather than a defect. At 0.0.63 we would not weigh it heavily.

How we got this wrong first

Worth reporting, because the mistake is built into the tool's own interface and you will hit it too.

Our first pass at this part concluded that strict mode did not exist. We had run zensical serve --help, read "Strict mode (currently unsupported)", and written a whole section around the idea that broken links could not fail a build. It was a reasonable reading of the evidence we had gathered and it was wrong.

What caught it was noticing that the scaffolded workflow runs zensical build --clean, and --clean was not in the options list we had read. That meant build had its own flag set, which meant we had only ever checked half the interface.

The lesson generalises past this tool: subcommands can carry different flags with the same name, and checking one is not checking the other. Run --help on every subcommand you intend to use, not just the first one you tried.

It also changed the recommendation materially. "Strict mode does not work" would have been a reason to wait. "Strict mode works and catches broken links, but cannot see errors that produce no warning" is a reason to use it with one extra assertion, which is a very different conclusion for a reader deciding today.

Where this leaves the series

Four parts in, the honest position.

Genuinely good: one 13.7 MB wheel, no compiler, no Node, no separate theme. 203 pages in 1.37 seconds of real work. It reads mkdocs.yml directly and gets nav, metadata and markdown extensions right. sitemap.xml and objects.inv for free. A dev server that worked first try. And strict mode does exist where it counts.

Not ready: several classes of error produce no warning at all, so no flag can catch them. MkDocs plugins are dropped without a word. The search index is 1.4 MB at 203 pages and scales linearly. The scaffolded pipeline pins nothing and skips the safety net.

Use it now if you have a straightforward markdown site, you want a fast build and a small toolchain, and you will pin your versions and add the page count check.

Wait if your site depends on plugins for content, or you cannot absorb a breaking change in a 0.0.x dependency.

We would watch this one. A Rust core from the team behind the most widely used documentation theme in open source is a serious starting position, and most of what is missing looks like the kind of thing that arrives with time rather than the kind that needs a redesign.

One thing we could not check

We ran every test on a local machine, never on a GitHub runner. The scaffolded workflow was read rather than executed, so our criticisms of it are about what it specifies, not about observed behaviour in Actions.

Specifically, we have not verified that python-version: 3.x resolves to something below 3.10 on any current runner image. We think it resolves to a recent Python today, which is why the unpinned version is a latent risk rather than a live bug. Treat that as a reason to pin, not as a report that the default workflow is broken.

Related

If deliberately breaking tools to see what they admit to is interesting, our piece on Claude Code hooks failing open is the same exercise elsewhere and found the same shape of problem: a guard reporting success while doing nothing.