The noyalib ecosystem

This document does two things. It describes what the ecosystem is today, and it states — with numbers anyone can reproduce — how good it actually is. The second part is the harder one, and the reason scripts/ecosystem-scorecard.sh exists.

Why a harness instead of a claim

A rating is worth something only if it can be shown to be wrong. Most “production-ready / battle-tested / enterprise-grade” language in this corner of the industry cannot be, because nothing is attached to it. So every number below is produced by a command, and the command is printed next to the number.

Three rules govern the harness:

1. Falsifiable. Each metric names the exact command and the exact threshold. “Docs are good” is not a metric. “rustdoc emits 0 warnings under -D warnings” is — run it and see.
2. No credit for unmeasured work. A probe that cannot run scores N/A and is removed from the denominator. It never scores 0, and it never silently scores 1. The report states what fraction of the rubric actually executed, because a perfect score over a third of the rubric is not a perfect score.
3. Reproducible. The header records rustc/cargo versions, host triple, each repo’s commit SHA, and whether the worktree was dirty.

Run it:

scripts/ecosystem-scorecard.sh                  # local probes
scripts/ecosystem-scorecard.sh --network        # + crates.io, GitHub, OpenSSF
scripts/ecosystem-scorecard.sh --deep           # + all-features build probes
scripts/ecosystem-scorecard.sh --with-coverage  # + cargo-llvm-cov
scripts/ecosystem-scorecard.sh --json out.json

CI runs the same harness weekly and on every change to it or to this page (.github/workflows/ecosystem-scorecard.yml), against fresh checkouts of all six repos; the workflow fails below the 0.90 floor, and the JSON report is attached to the run.

It exits non-zero below SCORE_FLOOR (default 0.90), so the rating can be a CI gate rather than a boast.

What it deliberately does not measure

• Adoption. Downloads are recorded as context below, not scored. Popularity is not quality, and scoring it would let the rating drift with marketing rather than engineering.
• Benchmark numbers. The harness checks that docs/BENCHMARKS.md discloses host, toolchain and repro command — the things that make a speed claim falsifiable — but does not re-run criterion. Timing a laptop under variable load and calling the delta a score would be measurement theatre.
• Code review. OpenSSF scores this 0 across every repo in the family, correctly: these are single-maintainer repos with squash merges. That is a real limitation and is left visible rather than excluded.

The competitive landscape, 2026

YAML tooling is not one market. It is five, and each has a different incumbent in a different language.

The Rust library tier alone has fragmented since serde_yaml was archived. Recent-download counts from the crates.io API:

The honest reading: noyalib is roughly 0.3% of the deprecated incumbent and 6.8% of serde-saphyr. On measured capability it leads; on measured reach it does not. The constraint is distribution, not engineering — which is exactly where the gaps below sit.

For a feature-by-feature comparison of the Rust libraries — and instructions for verifying any row of it yourself — see COMPARISON.md. This document is about the suite, not the library.

The structural advantage is that noyalib is the only one of these that spans all five surfaces from a single core. yq, yamllint and yaml-language-server are three unrelated codebases in three languages with three different YAML parsers and three different sets of edge-case behaviour. A team that adopts them gets three answers to “is this document valid?”.

Measured gaps

Every row below was produced by a filesystem or API probe on 2026-08-20, not by judgement.

Capability the suite cannot express today

Both are surfacing work, not engine work. The lossless CST already tracks every span these rules need, and already backs remove/set.

Distribution

Supply-chain posture

OpenSSF Scorecard, scans taken the same day at the current HEAD of each repo:

The satellite deficit is concentrated in four checks:

• Fuzzing 10 -> 0. No satellite has a fuzz/ directory. Worth about a point each and cheap to fix.
• CII-Best-Practices 5 -> 0. The badge is registered for the core only.
• Contributors 3 -> 0, Signed-Releases 3 -> 1.

Code-Review is 0 on all five, including the core — single-maintainer squash merges. Honest, and not something a script can fix.

The credibility gap

Three crates in the same portfolio still depend on YAML libraries their own authors have marked dead:

Already migrated: frontmatter-gen, html-generator, static-site-generator, and the five satellites.

Shipping a YAML library while three of your own crates depend on an archived one is the most falsifiable credibility claim in the set, and the cheapest to close.

Proposed additions

Ranked by measured gap times incumbent strength.

Tier 1 — capability

1. noyaq (new bin in noya-cli) — query and in-place transform with a yq-compatible expression subset.
2. noyalint (new bin in noya-cli) — yamllint’s rule set over the CST.
3. noyalib-rules (new crate) — the rule engine behind noyalint, reused by the LSP for diagnostics and by the MCP server as an agent-facing lint tool, so three surfaces do not re-implement the same checks.

Tier 2 — distribution

4. First-party GitHub Action. CI is where YAML tooling is actually consumed; this is the highest-leverage single artefact on the list.
5. .pre-commit-hooks.yaml — one file, unlocks the entire pre-commit user base.
6. VS Code extension wrapping noyalib-lsp.
7. Homebrew tap (shipped, with Scoop and AUR) and a CLI container image.

Tier 3 — reach beyond Rust

8. noyalib-py (PyO3) — the linting and round-trip audience is Python-native.
9. noyalib-node (napi-rs) — native throughput where WASM loses.

Tier 4 — cheap credibility

10. Fuzz targets in each satellite.
11. OpenSSF Best Practices badge for the five satellites.
12. Migrate libmake, nucleusflow, wiserone; finish metadata-gen.

Rating

Rating: A- (92.5% weighted) — 134 executed probes, 12 scored N/A.

Measured 2026-10-15T20:47:55Z on aarch64-apple-darwin, rustc 1.99.0 (b940084d7 2026-09-28). Rubric executed: 92%.

_{Generated by scripts/ecosystem-scorecard.sh v1.0.0. Regenerate rather than edit.}

How to disagree with this rating

Pick the row you doubt and run its probe. If the number differs, the harness is wrong and should be fixed — open an issue with the command output. That is the entire contract.

Two caveats the harness prints about itself and this document repeats, because a score that hides its own limits is a marketing number:

• Rubric coverage. The percentage of probes that actually executed is reported alongside the score. Network and coverage probes are opt-in; with them off, a large slice of the rubric is N/A, and the headline is correspondingly less meaningful.
• Worktree state. The header records whether each repo was dirty. A score measured against uncommitted changes describes those changes, not the published artefact.

Known weaknesses of the harness itself

• A timed-out probe used to score 0, asserting a failure it had not observed. The run recorded above shows noyalib · tests · timeout after 3600s · 0.00, which dragged correctness to 83.3%. That is an artefact of the ceiling firing on a cold target dir, not a test failure: the same suite run directly on the same commit passed 5739 / 0, and spec_conformance in this very run reports 406/406. It also contradicted rule 2 above — an unmeasured probe is supposed to score N/A and never 0. Fixed: a timeout now records N/A, which keeps it visible (N/A rows print, and they lower the reported rubric coverage) without claiming a failure that did not happen. The next regeneration will show N/A here rather than a false ✗.

• It measures the presence of fuzz targets, not fuzzing effort. Two targets that have never found anything score the same as two that have.

• dependency_closure used one flat budget of 60 for every repo. That is why noya-cli scored 0 at 130 crates. The obvious fix — a per-repo budget — turned out to be the wrong one: any number picked now would be a number picked so the current value clears it. The rule that survived scrutiny is about propagation. A library’s dependencies are inherited by every downstream consumer, so their size is a cost imposed on other people and belongs in the score; noyalib sits at 12. A leaf binary’s are not inherited, and 130 of noya-cli’s crates are miette’s fancy renderer giving noyavalidate source excerpts and carets — which for a validator is the job, not bloat. Leaves are therefore recorded and not scored: the count stays in the table and the JSON, so a leaf that doubles its tree is obvious, but it is not judged against a line nobody can defend.

• Code-Review cannot be fixed by any script, and no probe pretends otherwise.

• Test pass ratio is scored, but not test quality. A suite of 4,000 trivial assertions scores the same as 4,000 sharp ones. Coverage (--with-coverage) partly compensates and is off by default because it is slow.

Family layout

Every repository in the family carries the same skeleton, enforced per push by the Family layout contract step of shared-docs-lint.yml — a repo that drops one of these files fails its own CI:

Satellites additionally carry supply-chain/; the core additionally carries the shared-*.yml workflows the family consumes by SHA and the manual for the library itself. Until 2026-09-05 the citation record, AGENTS.md, and the devcontainer lived in the core only; every satellite now carries its own, so the contract lists them.

Where the family is published

All six crates release in lockstep at the identical =0.0.X; the npm packages and container images carry the same version. Homebrew, Scoop, and AUR carry noya-cli (see its pkg/). A repository’s GitHub Packages sidebar counts only GitHub-hosted registries, so it is not the map; this table is.