Detailed README reference

This document preserves the extended technical material from the primary README before the workspace-wide canonical README template was adopted. The concise README.md is the project entry point. This reference retains the deeper rationale, feature catalogue, security discussion, and examples so the template migration does not discard documentation.

noyalib

A YAML 1.2 parser and serialiser for Rust, with full serde integration and zero unsafe code.

Contents

Getting started

• Install — Cargo, source
• Requirements — toolchain floor, platforms, no_std
• Quick Start — parse and serialise in ten lines

The noyalib ecosystem (library + five satellite crates)

• The noyalib ecosystem — noyalib, noya-cli, noyalib-lsp, noyalib-mcp, noyalib-wasm, noyalib-serde-yaml at a glance

Library reference

• One-minute migration from serde_yaml (and the wider ecosystem) — name-for-name mapping for serde_yaml 0.9, serde_yml, yaml_serde, serde-yaml-ng, serde-norway, serde-yaml-bw, serde-saphyr, yaml-spanned
• Why this approach? — design rationale
• Capabilities at a glance — the current surface by theme
• Two APIs, one parser — data binding vs. tooling
• Ecosystem comparison — short matrix; full table at docs/COMPARISON.md
• Benchmarks — headline numbers; full table at docs/BENCHMARKS.md
• Features — module-level capability list
• Custom tags (“just data”) — Value::Tagged, untag, registry
• Library Usage — deserialise, serialise, values, spans
• Configuration — parser and serialiser options
• Examples — runnable example index

Operational

• When not to use noyalib — limitations
• Development — make targets, fuzzing, CI
• Security — guarantees and compliance
• Documentation — all reference docs
• Stability guarantees — SemVer axis, output stability, MSRV discipline
• License

Install

As a Rust library (crates.io)

[dependencies]
noyalib = "0.0.58"

As a CLI tool

The noyafmt and noyavalidate binaries ship from the noya-cli companion crate (the noyalib library crate itself contains no binaries — the split keeps clap + miette + validate-schema out of the library’s dependency graph for downstream embedders).

cargo install noya-cli --locked builds both binaries by default (via the noyavalidate Cargo feature). To install only the formatter and skip the schema-validation toolchain, use cargo install noya-cli --locked --no-default-features --features noyafmt.

GitHub Releases additionally publish pre-built tarballs for Linux (gnu + musl), macOS (Intel + Apple Silicon + universal), and Windows (x86_64, i686, aarch64). Each archive ships with the binaries, man pages, shell completions, license bundle, and a cosign keyless signature + SLSA Build L2 attestation.

See pkg/VERIFY.md for verification commands and pkg/PUBLISH.md for the per-channel maintainer runbook.

no_std support

[dependencies]
noyalib = { version = "0.0.58", default-features = false }

Requires alloc. Core data binding (from_str, to_string, Value, schemas) and the streaming deserializer all compile and run without the standard library. from_reader, to_writer, the Spanned<T> deserialization helper (which uses thread-local storage), and the CST module require the std feature, which is enabled by default.

Build from source

git clone https://github.com/sebastienrousseau/noyalib.git
cd noyalib
make          # check + clippy + test

Cargo features

All optional integrations are off by default. Enable only what the application needs.

# Example: rich diagnostics + schema validation
[dependencies]
noyalib = { version = "0.0.58", features = ["miette", "validate-schema"] }

Optional features: lossless-u64 preserves YAML integer scalars above i64::MAX as Number::Unsigned(u64) instead of lossy f64 widening — useful for distributed-system IDs, content hashes, and timestamp fields. Enable the Cargo feature, then opt in at runtime with ParserConfig::lossless_u64_integers(true) and SerializerConfig::lossless_u64_integers(true). See docs/adr/0004-lossless-u64-integers.md for rationale and migration notes.

Requirements

• Rust 1.89.0 or newer. Every crate manifest carries rust-version, and CI enforces the floor on every push: msrv-core builds the core surface on 1.89.0, and msrv-per-crate checks each satellite against its own declared floor.
• Any tier-1 platform. The test matrix runs on Linux, macOS, and Windows with the stable, beta, and nightly toolchains; stable is the gate, beta and nightly are early warning.
• std is optional. With default-features = false the core builds on core + alloc; CI checks wasm32-unknown-unknown, thumbv7em-none-eabihf, riscv32imac-unknown-none-elf, and aarch64-unknown-none on every push.

MSRV by crate. Each workspace crate carries its own rust-version; CI’s msrv-per-crate job (Phase 7) gates each crate independently so a satellite never silently breaks downstream users pinned to the core’s floor.

As of v0.0.58 the whole lockstep set shares one floor, 1.89.0, the same floor as the maintainer’s other Rust crates. The historical split (core at a lower floor than the satellites) has been gone since v0.0.16.

The number is the floor we verify, not the floor the library happens to reach. At the old 1.86.0 floor, cargo +1.85.0 check --lib succeeded, but cargo +1.85.0 check --all-targets fails because criterion 0.8 requires 1.86, so no test, bench or coverage run can execute there. We do not advertise an MSRV that CI cannot exercise. We also guarantee never to require a rustc newer than 12 months old at release time — see docs/POLICIES.md §1.

The opt-in, bench-only compare-saphyr feature needs rustc 1.88+ (serde-saphyr uses let-chains), which the 1.89.0 floor covers. It is still not built by the MSRV gate, which checks the core feature sets.

rust-toolchain.toml itself selects stable for local development; the 1.89.0 floor on the core surface is enforced by the dedicated msrv-core CI job (Ubuntu, no-default-features + default-features build paths).

Quick Start

use noyalib::{from_str, to_string};

#[derive(Debug, serde::Serialize, serde::Deserialize, PartialEq)]
struct Config {
    name: String,
    port: u16,
    features: Vec<String>,
}

fn main() -> Result<(), noyalib::Error> {
    let yaml = "
name: myapp
port: 8080
features:
  - auth
  - api
";

    let config: Config = from_str(yaml)?;
    let output = to_string(&config)?;
    let roundtrip: Config = from_str(&output)?;
    assert_eq!(config, roundtrip);

    Ok(())
}

The noyalib ecosystem

Six crates make up the family. The library is the core; five satellites wrap it for specific delivery surfaces, each in its own repository, all releasing in lockstep at the identical =0.0.X (ADR-0005) — the version number is the compatibility contract.

Install the pieces

# CLI tools (noyafmt + noyavalidate)
cargo install noya-cli

# LSP server
cargo install noyalib-lsp

# MCP server
cargo install noyalib-mcp

# WASM bundle
npm install @sebastienrousseau/noyalib-wasm

# serde_yaml drop-in — the whole migration is this one line:
serde_yaml = { package = "noyalib-serde-yaml", version = "=0.0.58" }

Per-crate READMEs cover the surface specific to each artifact:

• CLI: noya-cli — flags, exit codes, recipes.
• LSP: noyalib-lsp — capabilities, editor configs.
• MCP: sebastienrousseau/noyalib-mcp — tools, host configs (split repo since v0.0.13; strict lockstep per ADR-0005).
• WASM: sebastienrousseau/noyalib-wasm — JS API, bundling (split repo since v0.0.12).

Per-host quick links

How good is this, really?

The suite carries a rating, and the rating is a program.

scripts/ecosystem-scorecard.sh --network

It runs ~30 probes across the five repos — tests, spec conformance, clippy, rustdoc under -D warnings, cargo audit/deny/vet, REUSE, dependency-closure size, SHA-pinning of GitHub Actions, fuzz targets, release signatures, OpenSSF Scorecard — and prints, for every one, the command that produced the number next to the number itself.

Three properties make the score worth quoting:

• Falsifiable. Every metric names its command and its threshold. Disagree with a row? Run the command in that row.
• No credit for unmeasured work. A probe that cannot run (tool absent, no network, opt-in gate not passed) scores N/A and leaves the denominator. It is never silently counted as a pass. The report states what fraction of the rubric actually executed.
• Reproducible. The header records rustc/cargo versions, host triple, and each repo’s commit SHA and dirty flag.

--json emits the same data for CI. The script exits non-zero below SCORE_FLOOR (default 0.90), so the rating can gate a merge instead of decorating a README.

Full methodology, the competitive landscape, and the measured gaps — including the ones that do not flatter this project — are in docs/ECOSYSTEM.md.

The rest of this README covers the library surface (noyalib itself). For the satellite crates, jump straight to their READMEs above.

One-minute migration from serde_yaml (and the wider ecosystem)

The one-minute version is a package rename. Change zero source lines:

[dependencies]
serde_yaml = { package = "noyalib-serde-yaml", version = "=0.0.58" }

noyalib-serde-yaml re-exports noyalib’s serde_yaml 0.9 shim and releases in lockstep with the core, so the exact pin is the compatibility contract. The shim is behavioural since v0.0.29: << stays a literal key, 0123 stays a string, u64::MAX keeps precision, alias bombs fail with upstream’s repetition limit exceeded, and the pinned error classes carry libyaml’s phrasing and locations, verified by an 18-case contract suite captured live from serde_yaml 0.9.34. Every type is noyalib-native; there is no transitive dependency on the archived upstream. The same shim is available without the rename: depend on noyalib with features = ["compat-serde-yaml"] and replace use serde_yaml with use noyalib::compat::serde_yaml.

Moving to the native API

Take this route when you want what the shim does not expose: from_str_strict, Spanned<T>, the Tagged variant, and the lossless CST. Most call sites are mechanical to update. The full guide, covering serde_yaml 0.9 plus every actively published fork and adjacent crate, is docs/MIGRATION-FROM-SERDE-YAML.md. The headline mapping for serde_yaml 0.9 is below; the same guide has per-crate sections for serde_yml, yaml_serde, serde-yaml-ng, serde-norway, serde-yaml-bw, serde-saphyr, and yaml-spanned with verified function tables for each.

-[dependencies]
-serde_yaml = "0.9"
+[dependencies]
+noyalib = "0.0.58"

-use serde_yaml::Value;
-let v: Value = serde_yaml::from_str(input)?;
-let s        = serde_yaml::to_string(&v)?;
+use noyalib::Value;
+let v: Value = noyalib::from_str(input)?;
+let s        = noyalib::to_string(&v)?;

Coming from a different YAML crate?

Each crate has a standalone migration guide with TL;DR diff, function-mapping table, behavioural notes, and a checklist. Crates.io state verified 2026-09-05:

The umbrella index is docs/MIGRATION.md — start there if you’re not sure which guide applies, or pick the row above.

The three behavioural differences worth knowing about (YAML 1.2 strict booleans, Tagged variant, multi-doc API): MIGRATION-FROM-SERDE-YAML.md covers each in detail.

Why this approach?

noyalib targets the niche serde_yaml / serde_yml / libyml occupy — read YAML into typed Rust structs, write Rust structs back as YAML — and is written from scratch against the YAML 1.2 spec. The implementation runs the official YAML test suite to 100% strict compliance: 406/406 attempted cases pass, 0 failures, 0 skips, on every push and with no skip list. Every companion crate runs the same vendored suite through its own entry point in its own CI (see Conformance), and since v0.0.36 every valid case also runs as a multi-document stream to pin error positions. It is not a fork of serde_yaml; the parser, scanner, serialiser, and CST are independent code.

Two architectural choices motivate the rewrite:

1. Streaming-first deserialise. The default from_str<T> path walks parser events directly into the typed target. The serde_yaml-shaped pattern is parse → Value → T::deserialize(&Value), which allocates every key, scalar, and nested mapping into an intermediate AST that the typed target then throws away. noyalib bypasses that AST when the caller asked for a typed T. The dynamic Value tree is still available when callers want it, but it is no longer in the hot path.

2. #![forbid(unsafe_code)] at the workspace root. There is no FFI to a C library, no raw-pointer dereferences, and no unsafe blocks in the parser, scanner, formatter, or CST. CI enforces the attribute on every push. Most popular Rust YAML crates wrap libyaml via C-FFI; the unsafe blocks involved are usually well-vetted, but their existence makes a security-conscious downstream audit meaningfully harder.

A few features built on top of those choices:

• SIMD-accelerated structural discovery. Stable Rust dispatches to memchr (SSE2 / NEON) for arity-1/2/3 needles and SWAR for arity-4+. With nightly-simd on, the structural-bitmask scanner widens to a 32-byte Simd<u8, 32> chunk and walks delimiters via mask.trailing_zeros() — the same shape that powers simdjson.
• SWAR decimal parsing. The plain-scalar integer resolver folds 8 ASCII digits per u64 cycle via three pair-wise multiply-add phases. ~2× faster than <i64 as FromStr>::from_str on big numbers.
• Lossless CST. A side-table green tree (noyalib::cst::Document) reproduces the source byte-for-byte and supports surgical edits. doc.set("version", "0.0.2") rewrites only the touched span; comments, indentation, and sibling entries are left alone.

The default profile compiles eight crates: five unconditional (serde, indexmap, rustc-hash, memchr, smallvec) plus three default-on but optional (itoa via fast-int, ryu via fast-float, serde_ignored via strict-deserialise). Disabling the default features drops the graph to the five unconditional crates. No archived or unmaintained crate appears in the graph — serde_yaml 0.9 (archived), libyaml (C-FFI), and thiserror are all absent. cargo audit, cargo deny, and cargo vet are CI gates on every push.

Capabilities at a glance

The library has covered a complete YAML 1.2 stack since its first release. See CHANGELOG.md for the per-release inventory; the table below groups the current surface by theme.

Two APIs, one parser

noyalib exposes two complementary surfaces over the same scanner and strictness rules:

• Data binding — from_str, to_string, Value, StreamingDeserializer, BorrowedValue. Read YAML into typed Rust data, write Rust data back out. The round-trip travels through a Value/struct, so comments and exact whitespace are not preserved. Use this for config loaders, RPC payloads, and the 95% of YAML workloads that just want data.
• Tooling / automation — noyalib::cst::parse_document, parse_stream, and the Document handle. Read YAML into a side-table CST that reproduces the source byte-for-byte, then run targeted edits like doc.set("version", "0.0.2") — only the touched span is rewritten, every comment and the original indentation is left alone. Use this for Renovate-style version bumps, manifest patchers, formatters, and schema-driven linters. See examples/lossless_edit.rs.

The CST also lets a tool break aliases by inlining the anchored content at every reference site — useful when a manifest needs to become self-contained before being shipped to a system that does not resolve YAML aliases:

#![allow(unused)]
fn main() {
use noyalib::cst::parse_document;

let yaml = "a: &shared 7\nb: *shared\nc: *shared\n";
let mut doc = parse_document(yaml).unwrap();

// Replace every `*shared` reference with the bytes of the
// anchored value. The `&shared` declaration stays in place.
let n = doc.materialise_aliases_of("shared").unwrap();
assert_eq!(n, 2);
assert!(!doc.to_string().contains('*'));
}

Document::materialise_alias_at(byte_pos) is the single-site variant for callers that already know the alias’s source position.

Ecosystem comparison

noyalib is the only Rust YAML implementation that passes all 406 active YAML 1.2 Test Suite cases under strict comparison and ships a lossless CST, native LSP, MCP server, WASM bundle, JSON Schema validator, and schema-driven autofix in one workspace.

The full feature matrix — every row, every column, with the reading-the-table notes — lives at docs/COMPARISON.md so the README stays fast to scan.

Quick orientation:

Per-crate migration guides at docs/MIGRATION.md.

Benchmarks

Headline numbers (Apple M4 / aarch64, Rust 1.95 stable, --release with LTO=fat, codegen-units=1, panic=abort, criterion --warm-up-time 2 --measurement-time 4):

(serde_yml was previously a comparison column — retired in v0.0.6 since RUSTSEC-2025-0068 flagged it unsound + unmaintained; the bench arm was dropped so the OpenSSF Scorecard Vulnerabilities check stays clean. The headline ratios above are unchanged for the remaining four competitors.)

noyalib is faster than every other pure-Rust YAML library on every deserialize fixture measured. Speedup ranges across the four competitors above: 1.69×–2.00× vs serde-saphyr, 1.42×–1.84× vs serde_yaml_ng, 1.38×–1.74× vs yaml-spanned, 1.11×–1.36× vs yaml-rust2. Serialize is 3.00×–4.34× ahead of serde_yaml_ng.

The narrowest gap is yaml-rust2, which doesn’t carry the Spanned<T> plumbing, the per-tag Cow<'a, str> propagation, or the Value::Tagged preservation that noyalib does — closing the remaining gap to ≥ 2× over yaml-rust2 is a separate effort because the levers needed (CompactString keys in Mapping, bump-arena event lifetimes, eliminating the Value AST on the typed path) require SemVer-breaking refactors.

./scripts/pgo.sh runs the LLVM profile-guided optimisation pipeline and adds 5–15% on top of the numbers above; recommended for production deployments.

The full breakdown — every workload, every comparison library, the SWAR pipeline explanation, parallel multi-doc scaling, and the project-metrics table — lives at docs/BENCHMARKS.md.

Algorithmic-complexity guarantees (O(n) parser, O(d) stack, etc.) live in POLICIES.md §4.

Features

Governance: schema-driven autofix

YAML written by hand often has type slips that strict validation catches but humans don’t notice — port: "8080" (string) where the schema declares port: integer. noyalib ships a one-call fix-up engine that rewrites these to match the schema:

#![allow(unused)]
fn main() {
use noyalib::{coerce_to_schema, from_str, schema_for, validate_against_schema, JsonSchema};

#[derive(serde::Serialize, serde::Deserialize, JsonSchema)]
struct ServerConfig {
    /// Port the server binds on.
    port: u16,
    /// Hostname or IP literal.
    host: String,
}

let schema = schema_for::<ServerConfig>().unwrap();

// Hand-written YAML with the port quoted by mistake.
let mut data = from_str("port: \"8080\"\nhost: api.example.com\n").unwrap();

// Strict pass — fails because `port` is a string.
assert!(validate_against_schema(&data, &schema).is_err());

// Apply schema-driven coercions.
let n = coerce_to_schema(&mut data, &schema).unwrap();
assert_eq!(n, 1, "one fix expected");

// Re-validation passes — port is now an integer.
validate_against_schema(&data, &schema).unwrap();
}

Library engine behind noyavalidate --fix. Handles String → Integer, String → Number, String → Boolean coercions; the fix-loop iterates until convergence. Unparseable inputs (e.g. port: "abc" against type: integer) are left in place so the caller can surface the residue via a follow-up validate_against_schema call.

Policy enforcement (Safe YAML)

The pluggable policy::Policy trait + ParserConfig::with_policy(p) lets you reject documents that violate organisational constraints at parse time, before any data flows to downstream code. Built-in policies cover the most common asks; custom policies implement the trait directly.

#![allow(unused)]
fn main() {
use noyalib::policy::DenyAnchors;
use noyalib::{from_str_with_config, ParserConfig, Value};

let cfg = ParserConfig::new().with_policy(DenyAnchors);

// Anchors / aliases are the classical billion-laughs vector and a
// readability hazard in audited configs. With `DenyAnchors`
// registered, any document that defines `&name` or dereferences
// `*name` is rejected before deserialise begins.
let res: Result<Value, _> =
    from_str_with_config("key: &x 1\nval: *x\n", &cfg);
assert!(res.is_err());

// Anchor-free input passes through unchanged.
let v: Value = from_str_with_config("key: 1\nval: 1\n", &cfg).unwrap();
assert!(matches!(v, Value::Mapping(_)));
}

Built-ins: DenyAnchors, DenyTags (rejects custom tags while preserving YAML 1.2 core tags), MaxScalarLength(n). Compose freely — cfg.with_policy(DenyAnchors).with_policy(DenyTags) runs both in registration order.

Strict deserialise (typo detection)

noyalib::from_str_strict<T> errors if the YAML carries any keys the target type T doesn’t declare — closing the silent-data-loss gap when a config-key typo (e.g. replicass: 3) deserialises into a struct whose replicas field stays at its Default:

#![allow(unused)]
fn main() {
#[derive(Debug, serde::Deserialize)]
struct Config {
    port: u16,
    host: String,
}

let yaml = "port: 8080\nhost: api.example.com\nporrt: 9090\n";

// Lenient path silently drops `porrt`.
let cfg: Config = noyalib::from_str(yaml).unwrap();
assert_eq!(cfg.port, 8080);

// Strict path surfaces the typo as a typed error.
let err = noyalib::from_str_strict::<Config>(yaml).unwrap_err();
assert!(err.to_string().contains("porrt"));
}

The strict path walks nested structs — a typo at server.unknown is reported with its parent path so the user sees exactly where the bad key lives.

The same check is available on every input shape:

The slice and reader variants share from_str_strict’s semantics — they exist so callers that already hold bytes (a buffer, a network frame, a bytes::Bytes) don’t have to round-trip through String to opt in.

Diagnostics output

Every parse error carries an exact (line, column, byte offset) triple. Error::format_with_source(input) renders a rustc-style snippet without pulling in any extra dependency:

error: expected ',' or ']'
  --> input.yaml:2:7
   |
 1 | host: localhost
 2 | port: [broken
   |       ^^^^^^ here
 3 | db: postgres
   |

Enabling --features miette makes noyalib::Error implement miette::Diagnostic; wrapping the error in miette::Report gives terminals with ANSI support the same snippet with a colour-coded label, and lets a CLI surface error codes (noyalib::parse, noyalib::duplicate_key, …) and actionable help text from the same call site:

#![allow(unused)]
fn main() {
let cfg: Config = noyalib::from_str(yaml)
    .map_err(|e| miette::Report::new(e).with_source_code(yaml.to_owned()))?;
}

See examples/diagnostic.rs for the full integration pattern.

For sinks with hard length budgets — Slack messages, Sentry tags, structured-log fields — the same diagnostic is available capped at a caller-supplied character count:

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};
let source = "a: [unclosed";
let err = from_str::<Value>(source).unwrap_err();

// Cap at 60 chars; UTF-8-aligned cut; ASCII `...` ellipsis.
let short = err.format_with_source_truncated(source, 60);
assert!(short.len() <= 60);

// Multi-line context (rustc-style) capped at 200 chars:
let with_ctx = err.format_with_source_radius_truncated(source, 1, 200);
assert!(with_ctx.len() <= 200);
}

The truncation contract: UTF-8-aligned cut, ... appended on truncation (dropped only when max_chars < 3), no ANSI escapes, deterministic across builds.

Custom tags (“just data”)

YAML tags (!Custom, !!python/object, !Color) attach a type label to any node. noyalib surfaces them as Value::Tagged on the default from_str::<Value> path, never as code paths — they are pure data the caller dispatches on:

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};

let v: Value = from_str("!Color '#ff8800'\n").unwrap();
match &v {
    Value::Tagged(t) => {
        assert_eq!(t.tag().as_str(), "!Color");
        assert_eq!(t.value().as_str(), Some("#ff8800"));
    }
    _ => unreachable!(),
}
}

Three escape hatches when the wrapper isn’t what you want:

For lossless emit, use to_string_value / to_writer_value:

#![allow(unused)]
fn main() {
use noyalib::{from_str, to_string_value, Value};
let v: Value = from_str("!Color '#ff8800'\n").unwrap();
let yaml = to_string_value(&v).unwrap();           // "!Color '#ff8800'\n"
let back: Value = from_str(&yaml).unwrap();
assert!(matches!(back, Value::Tagged(_)));         // round-trips
}

The generic to_string (and the rest of the Serialize-trait family) routes Value::Tagged through serialize_map for serde-bridge interop with serde_json etc., which is lossy on the YAML-tag wire form. The dedicated *_value variants short- circuit the Serialize pipeline.

Typed targets (#[derive(serde::Deserialize)] struct Foo { ... }) see through tags transparently — from_str::<Foo>("!Foo {x: 1}") yields Foo { x: 1 } regardless of the tag. The tag wrapper is only surfaced when the deserialise target is Value itself, detected at the entry point via TypeId.

Compact list indentation

Two camps disagree on whether YAML lists under a mapping key should indent (the default) or align with the key column. noyalib ships both styles; opt into the compact form via SerializerConfig::compact_list_indent(true):

#![allow(unused)]
fn main() {
use noyalib::{to_string_with_config, SerializerConfig, Value};

let yaml = "items:\n  - a\n  - b\n";
let v: Value = noyalib::from_str(yaml).unwrap();

// Default — list items indented one level past the key.
let std = noyalib::to_string(&v).unwrap();
// Output:
//   items:
//     - a
//     - b

// Compact — list items align with the key column. Style preferred
// by Kubernetes manifests and GitHub Actions workflows.
let cfg = SerializerConfig::new().compact_list_indent(true);
let compact = to_string_with_config(&v, &cfg).unwrap();
// Output:
//   items:
//   - a
//   - b
}

Key interning (memory footprint)

For Kubernetes-shaped streams that repeat the same keys (metadata, labels, name, apiVersion, selector) thousands of times, noyalib::interner::KeyInterner deduplicates the heap allocations:

#![allow(unused)]
fn main() {
use noyalib::interner::KeyInterner;
use std::sync::Arc;

let mut interner = KeyInterner::new();
let a = interner.intern("metadata");
let b = interner.intern("metadata");
// Same `Arc` — second call returned a clone of the cached entry.
assert!(Arc::ptr_eq(&a, &b));
}

The Mapping public API stays String-keyed for v0.0.1 — the interner is the explicit opt-in primitive. A future major version may switch the internal storage to Arc<str> and use the interner transparently during parse.

The “Norway problem”

YAML 1.1 resolved the bare scalar no as the boolean false. The country code for Norway is NO, so a YAML 1.1 parser silently rewrites country: NO to country: false. Real-world data corruption.

YAML 1.2’s official fix is to drop those bare-word booleans — only true and false count. noyalib defaults to YAML 1.2 strict semantics, so country: NO round-trips as the string "NO".

For migrating from Docker Compose / GitHub Actions / pre-1.2 YAML toolchains that expect the legacy behaviour, opt back in via legacy_booleans:

#![allow(unused)]
fn main() {
use noyalib::{from_str_with_config, ParserConfig, Value};

// Default (YAML 1.2 strict): "NO" → string.
let v: Value = noyalib::from_str("country: NO\n").unwrap();
assert_eq!(v["country"].as_str(), Some("NO"));

// Opt-in legacy mode: "NO" → false. Use only when migrating
// from a toolchain that depended on the YAML 1.1 behaviour.
let cfg = ParserConfig::new().legacy_booleans(true);
let v: Value = from_str_with_config("country: NO\n", &cfg).unwrap();
assert_eq!(v["country"].as_bool(), Some(false));
}

Conversely, strict_booleans = true tightens the YAML 1.2 contract — only the lowercase forms (true / false) are recognised; True, TRUE, Yes, etc. all stay strings. Use this for schema-strict pipelines where boolean-shape ambiguity is a contract violation.

Library Usage

#![allow(unused)]
fn main() {
// API surface synopsis — substitute your own `Config`, `yaml`, `bytes`, etc.
use noyalib::{from_str, from_slice, from_reader, from_value, ParserConfig};

// From string, byte slice, reader, or Value
let config: Config = from_str(yaml)?;
let config: Config = from_slice(bytes)?;
let config: Config = from_reader(file)?;
let config: Config = from_value(&value)?;

// With security limits
let parser = ParserConfig::strict();
let config: Config = noyalib::from_str_with_config(yaml, &parser)?;
let config: Config = noyalib::from_slice_with_config(bytes, &parser)?;
let config: Config = noyalib::from_reader_with_config(reader, &parser)?;
}

#![allow(unused)]
fn main() {
// API surface synopsis — substitute your own `config` value + `file` writer.
use noyalib::{to_string, to_writer, to_fmt_writer, to_value, SerializerConfig};

// To string, io::Write, or fmt::Write
let yaml: String = to_string(&config)?;
to_writer(&mut file, &config)?;
let mut buf = String::new();
to_fmt_writer(&mut buf, &config)?;

// To Value
let value: noyalib::Value = to_value(&config)?;

// With custom config
let ser_config = SerializerConfig::new()
    .indent(4)
    .quote_all(true)
    .document_start(true);
let yaml = noyalib::to_string_with_config(&config, &ser_config)?;
}

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};

let value: Value = from_str("
name: test
items:
  - one
  - two
settings:
  debug: true
")?;

// Field access
let name = value.get("name").and_then(|v| v.as_str());

// Path-based traversal
let debug = value.get_path("settings.debug");

// Sequence indexing
let first = value.get("items").and_then(|v| v.get(0));

// Missing keys return None (never panic)
assert!(value.get("nonexistent").is_none());
assert!(value.get_path("a.b.c").is_none());

// Path queries: wildcards and recursive descent
let all_names = value.query("items[*]");       // all items
let deep = value.query("..debug");              // find at any depth
}

#![allow(unused)]
fn main() {
use noyalib::borrowed::from_str_borrowed;

let yaml = "host: localhost\nport: 8080\n";
let value = from_str_borrowed(yaml).unwrap();

// Strings borrow directly from input — zero heap allocation
assert_eq!(value.as_mapping().unwrap().get("host").unwrap().as_str(), Some("localhost"));

// Convert to owned Value when needed
let owned = value.into_owned();
}

#![allow(unused)]
fn main() {
use noyalib::{from_str, Spanned};

#[derive(serde::Deserialize)]
struct Config {
    port: Spanned<u16>,
}

let config: Config = from_str("port: 8080")?;
assert_eq!(config.port.value, 8080);
assert_eq!(config.port.start.line(), 1);
assert_eq!(config.port.start.column(), 6);
}

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};

let yaml = "
defaults: &defaults
  timeout: 30
  retries: 3

production:
  <<: *defaults
  timeout: 60
";

let mut value: Value = from_str(yaml)?;
value.apply_merge()?;
// production -> {timeout: 60, retries: 3}
}

#![allow(unused)]
fn main() {
// API surface synopsis — substitute your own `Config` type + `config1`/`config2` values.
use noyalib::{load_all, to_string_multi};

let docs = load_all("---\na: 1\n---\nb: 2\n")?;
for doc in docs {
    println!("{:?}", doc?);
}

let items: Vec<Config> = noyalib::load_all_as::<Config>(yaml)?;
let yaml = to_string_multi(&[config1, config2])?;
}

#![allow(unused)]
fn main() {
#[derive(serde::Serialize, serde::Deserialize)]
enum Action { StartServer, StopServer }

#[derive(serde::Serialize, serde::Deserialize)]
struct Task {
    #[serde(with = "noyalib::with::singleton_map")]
    action: Action,
}
}

Configuration

#![allow(unused)]
fn main() {
// API surface synopsis — substitute your own `input` string.
use noyalib::{from_str_with_config, ParserConfig, DuplicateKeyPolicy};

let config = ParserConfig::new()
    .max_depth(64)
    .max_document_length(1_000_000)
    .max_alias_expansions(1000)
    .max_mapping_keys(10_000)
    .max_sequence_length(10_000)
    .duplicate_key_policy(DuplicateKeyPolicy::Error)
    .strict_booleans(true);

let value: noyalib::Value = from_str_with_config(input, &config)?;
}

#![allow(unused)]
fn main() {
// API surface synopsis — substitute your own `value` to serialise.
use noyalib::{to_string_with_config, SerializerConfig, FlowStyle, ScalarStyle};

let config = SerializerConfig::new()
    .indent(4)
    .flow_style(FlowStyle::Auto)
    .scalar_style(ScalarStyle::DoubleQuoted)
    .quote_all(true)
    .document_start(true)
    .document_end(true)
    .block_scalars(true)
    .block_scalar_threshold(3);

let yaml = to_string_with_config(&value, &config)?;
}

Examples

Run all examples:

cargo run --example all

When not to use noyalib

A few cases where another tool fits better, listed because the short answer is “we don’t do that yet” rather than because of a disagreement on priorities.

• You need to round-trip comments through the data-binding API. The YAML data model excludes comments by spec. The lossless CST (noyalib::cst::Document) preserves them byte-for-byte for the tooling path, but from_str::<T> → to_string(&T) does not. No Rust YAML library currently round-trips comments through a typed deserialise / serialise pair.
• You need a different YAML 1.1 resolver behaviour than the three forms the spec actually disagrees with 1.2 on. ParserConfig::version(YamlVersion::V1_1) flips the bundle of resolver-table differences (yes / no / on / off booleans, bare-0 octal 0644, sexagesimal 60:00) on as a single preset; fine-grained legacy_* flags remain for mix and match. Other 1.1-isms (mandatory !! tag prefix, broader timestamp parsing) are not version-gated in either direction.
• You have a hard dependency budget that cannot tolerate a Grisu / Ryu float formatter and a hash-randomised lookup table. Default profile carries 8 runtime deps. noyalib = { version = "0.0.58", default-features = false, features = ["std"] } (or the equivalent features = ["minimal"]) drops to 5 — itoa, ryu, and serde_ignored become opt-in via the fast-int / fast-float / strict-deserialise features. Numeric formatting falls back to core::fmt (slower; output remains valid YAML); the from_str_strict typo-detection helpers go away.

If you hit a case that should be on this list, please open an issue — that’s how it gets fixed or moved into the supported set.

Development

make              # check + clippy + test
make test         # run all tests
make clippy       # lint with Clippy
make fmt          # check formatting
make examples     # run all examples
make doc          # build API documentation
make deny         # supply-chain audit
make miri         # Miri memory checking (requires nightly)
make sbom         # generate software bill of materials
make clean        # remove build artifacts

Fuzzing

Thirteen cargo-fuzz targets ship under fuzz/fuzz_targets/, including differential targets that compare noyalib against other ecosystem parsers and a structure-aware target that builds Value trees with Arbitrary and proves round-trip, CST agreement, and alias-budget invariants rather than only surviving input. Two more under fuzz-nostd/ drive the core + alloc build, so the no_std code shape is fuzzed and not only compiled. Every push replays the seed corpus plus the minimized crash inputs of previously-fixed findings (fuzz/regressions/), so a fixed crash cannot silently return; onboarding to Google’s OSS-Fuzz is in review (google/oss-fuzz#16093).

# Generic surface
cargo +nightly fuzz run fuzz_parse              # arbitrary YAML parsing
cargo +nightly fuzz run fuzz_roundtrip          # parse → serialize → re-parse
cargo +nightly fuzz run fuzz_value_roundtrip    # Arbitrary Value → emit → parse == Value; alias budget oracle
cargo +nightly fuzz run fuzz_from_value         # Value → typed deserialise
cargo +nightly fuzz run fuzz_multi_doc          # multi-document streams
cargo +nightly fuzz run fuzz_strict             # tight security limits

# Targeted regression coverage
cargo +nightly fuzz run fuzz_borrowed_alias     # BorrowedValue + alias resolution
cargo +nightly fuzz run fuzz_diff               # owned-vs-borrowed parity
cargo +nightly fuzz run fuzz_double_quoted      # double-quoted scalar escapes
cargo +nightly fuzz run fuzz_yaml_v1_1          # YAML 1.1 resolver toggle

Seed corpus included in fuzz/corpus/seed/. The alloc-only crate:

cargo +nightly fuzz run --fuzz-dir fuzz-nostd fuzz_roundtrip_alloc_only

Miri (UB / aliasing / leak verification)

noyalib is #![forbid(unsafe_code)] so Miri does not police noyalib’s own code — every byte is checked at compile time. The reason a Miri job exists is to verify the interaction with the runtime dependencies (indexmap, rustc-hash, ryu, itoa, memchr, smallvec — all of which use unsafe internally) is sound, plus to validate that the SWAR decimal parser and the structural-bitmask iterator behave correctly under simulated big-endian targets.

make miri              # focused suite — parser, scanner, value, interner, simd
make miri-full         # full lib test suite under Miri (slow)
make miri-bigendian    # focused suite simulated on mips64 big-endian

# Or invoke the script directly:
./scripts/miri.sh                    # full focused suite
./scripts/miri.sh simd               # subset
MIRI_TARGET=mips64-unknown-linux-gnuabi64 ./scripts/miri.sh
MIRI_MODEL=tree ./scripts/miri.sh    # Tree Borrows (weekly in CI)
MIRI_ALIGN=1 ./scripts/miri.sh simd  # symbolic alignment check on the SWAR module (weekly in CI)

The CI matrix runs the focused suite on every PR (miri-focused) and the full + big-endian sweep on a weekly schedule (miri-full).

CI

See CONTRIBUTING.md for signed commits and PR guidelines.

Security

Reporting: never open a public issue for a vulnerability — see SECURITY.md for the private reporting channel and disclosure policy.

YAML parsers are notorious attack surface — libyaml-based wrappers have shipped multiple critical CVEs over the years (deserialisation RCE in PyYAML, billion-laughs amplification in dozens of language ecosystems, code-execution-via-tag in legacy Ruby YAML). noyalib’s posture is built around closing each of those vectors at the architectural level, not via opt-in flags.

RCE prevention (no arbitrary object instantiation)

noyalib does not instantiate arbitrary objects via tags. YAML’s custom-tag mechanism (!Foo, !!python/object) is the historical RCE vector — a malicious tag can load arbitrary code in legacy parsers. noyalib only deserialises into Rust types you’ve defined at compile time, with #[derive(serde::Deserialize)].

Custom tags surface as pure data through the Value tree, never as code paths. Three options, in order of decreasing strictness:

• Tagged data is fully preserved. from_str::<Value>("!Custom 'hello'\n") returns Value::Tagged(t) where t.tag() == "!Custom" and t.value() == Value::String("hello") — the same shape that already covered tagged sequences and tagged mappings. Downstream code can read the tag via Value::Tagged(t) pattern matching, dispatch on it, or step through it via Value::untag_ref for transparent reads. No global type registry, no runtime code lookup, no attacker-controlled instantiation — just data.
• Typed targets see through tags. A #[derive(serde::Deserialize)] struct Foo { x: u8 } against !Foo {x: 1} yields Foo { x: 1 }. The typed visitor never observes the tag string, so an attacker forging a tag to trigger a different Deserialize impl is impossible.
• TagRegistry for opt-in routing. Register the specific tags your application understands; unregistered custom tags still surface as Value::Tagged(...) data. See TagRegistry.
• policy::DenyTags for hard rejection. Reject any document carrying a non-core tag at parse time, before any data flows downstream. See Policy enforcement.

There is no path from a parsed YAML document to running attacker- chosen code. Period.

Configurable resource budgets

The ParserConfig::strict() preset enforces explicit resource budgets that cap every dimension of input size — designed to make billion-laughs and similar memory-amplification attacks mathematically impossible:

Alias-byte accumulation uses saturating_add so a crafted input that overflows a usize counter still triggers the limit cleanly — no integer-wrap escape hatch.

Secret redaction (${VAR} interpolation)

Value::interpolate_properties (see examples/env.rs) substitutes ${name} references inside string scalars from a property map. By default it is strict: an unknown placeholder returns Error::Custom so a misconfigured deployment surfaces immediately rather than silently degrading. Two opt-in variants change that contract:

For secrets, pair the strict path with secrecy::Secret<T> on the target field to keep the substituted value out of Debug / log output.

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};
use std::collections::HashMap;

let yaml = "db_url: ${DATABASE_URL}\napi_key: ${API_KEY}\n";
let mut map: HashMap<String, String> = HashMap::new();
map.insert("DATABASE_URL".into(), "postgres://...".into());
map.insert("API_KEY".into(), "redacted-token".into());

let mut value: Value = from_str(yaml).unwrap();
// Strict by default — every placeholder must resolve.
value.interpolate_properties(&map).unwrap();
// `api_key` now holds the substituted secret — wrap downstream
// in `secrecy::Secret<String>` to keep it out of Debug formatting.

// Hide the placeholder name in error output (audit-trail safe).
let mut value: Value = from_str(yaml).unwrap();
let _ = value.interpolate_properties_redacted(&map);

// Treat unknown placeholders as empty strings (env-var style).
let mut value: Value = from_str(yaml).unwrap();
value.interpolate_properties_lossy(&map);
}

Compile- and runtime-safety guarantees

• #![forbid(unsafe_code)] across the entire workspace — no FFI, no raw-pointer dereferences, no unsafe blocks anywhere.
• #[non_exhaustive] on ParserConfig, SerializerConfig, FlowStyle, ScalarStyle, Error — adding a variant is not a semver break.
• #[must_use] on 83 query methods — silent result-discarding bugs caught at compile time.
• #[track_caller] on 13 Index/IndexMut panic paths — panic diagnostics point at your call site, not noyalib internals.
• Pluggable parser policies (DenyAnchors, DenyTags, MaxScalarLength) — opt into “Safe YAML” enforcement at parse time. See noyalib::policy.

Supply chain

• cargo audit clean — zero advisories.
• cargo deny clean — license / advisory / ban / source checks.
• cargo vet clean — every dependency in the graph has a local audit or imported coverage from Mozilla / Google / Bytecode Alliance / Embark / ISRG audit sets.
• OpenSSF Scorecard tracked, badge in the header — v0.0.6 lifts the score from 6.5/10 to ~9/10 by demoting workflow tokens to least privilege, pinning every GitHub Action by SHA, wiring Dependabot, and retiring the unmaintained serde_yml / libyml bench dev-deps. See docs/POLICIES.md § OpenSSF Scorecard posture for the per-check breakdown.
• SLSA Build L2 provenance + sigstore signing on every release.
• REUSE.software 3.3 compliant — every source file carries SPDX headers.
• Signed commits (SSH ed25519) enforced via CI.

Notes

• Comment round-tripping for the data-binding API is not supported (YAML spec excludes comments from the data model). Commented<T> provides write-only comment injection. The CST API (noyalib::cst::Document) preserves comments byte-for-faithfully for the lossless-tooling path.

Documentation

The four entry points, identical across every repo in the family:

• User Manual — the rendered book: user guide, migrations, architecture, policies, ADRs
• API reference — rustdoc on docs.rs
• Developer docs — toolchain, task map, reproducing every CI gate locally
• Ecosystem map — the six crates, the lockstep model, the scorecard

The per-crate READMEs at crates/noyalib, noya-cli, noyalib-lsp, and the split-out satellite repos sebastienrousseau/noyalib-mcp and sebastienrousseau/noyalib-wasm document the surface specific to each artifact (binaries, MCP tools, LSP capabilities, WASM bindings).

Acknowledgements

Independent implementations find things a library’s own test suite cannot, because they exercise it against files nobody thought to write down.

@zoosky, author of yqr, has been noyalib’s most consistent source of that signal — eight merged commits across the CST edit API and the scanner, and the reports behind #221, #269, #280, #283, #285 and #288. The pattern is the same each time: adopt a published release in a real consumer, find where it produces or refuses something the library itself disagrees with, and arrive with a reproduction, a diagnosis naming the responsible function, a fix, and the tests that must keep failing.

Two contributions are worth naming specifically. #269 was filed as a defect and then re-framed by its author once he read the tests — the behaviour was deliberate, so the argument became a design question about whether two mutators may disagree about who owns a comment, which is a better question than the one originally asked. And #261’s CRLF fix was proven by disabling yqr’s local workaround and showing the suite pass against the branch while failing against the released version — evidence rather than assertion.

Thanks also to @kshpytsya and @EdJoPaTo for reports and clippy follow-ups.

Stability guarantees

• Versioning. Strict SemVer. Pre-1.0 the breaking-change axis is the patch number during the 0.0.x series; every breaking change is marked with ! in its commit and CHANGELOG entry, and cargo semver-checks gates every PR against the last release.
• Output stability. For a parser, output is API: a change to how a document parses (scalar resolution, error-vs-accept, the resulting Value shape) is treated as a breaking change even when no Rust signature moves — spec-alignment fixes ship as fix(spec)!: on the breaking axis, never in a compatible release.
• MSRV. Raised only on the breaking axis with the reason recorded in the CHANGELOG, never silently — policy and history in doc/MSRV-AND-DEPRECATION.md.
• Deprecations live for at least two releases with a #[deprecated] note naming the replacement before removal.

License

Dual-licensed under Apache 2.0 or MIT, at your option.

See CHANGELOG.md for release history.

Back to Top