noyalib user guide

A long-form companion to the README. The README is the elevator pitch and reference; this guide walks through every public-facing surface in the order you typically encounter it.

If you’ve used serde_yaml 0.9 before, also see MIGRATION-FROM-SERDE-YAML.md — most call sites are mechanical to update.

Contents

1. Reading YAML
2. Writing YAML
3. The dynamic Value tree
4. Source spans (Spanned<T>)
5. Strict deserialise (typo detection)
6. Parser policies (defence in depth)
7. Structured diagnostics
8. Lossless edits (cst::Document)
9. Schema validation and autofix
10. Multi-document streams + parallel parse
11. The CLI tools (noyafmt, noyavalidate)
12. WASM, MCP, and LSP

1. Reading YAML

The simplest case — read into a typed struct via serde:

#![allow(unused)]
fn main() {
use noyalib::from_str;

#[derive(Debug, serde::Deserialize)]
struct Server {
    host: String,
    port: u16,
}

let yaml = r#"
host: api.example.com
port: 8080
"#;
let s: Server = from_str(yaml)?;
assert_eq!(s.port, 8080);
}

Every input shape serde_yaml supports has a noyalib equivalent:

Each has a _with_config(input, &ParserConfig) variant for when you need to tighten parser limits or opt into legacy behaviours (see section 6).

2. Writing YAML

Mirror surface for the write side:

#![allow(unused)]
fn main() {
use noyalib::to_string;

#[derive(serde::Serialize)]
struct Server { host: String, port: u16 }

let yaml = to_string(&Server {
    host: "api.example.com".into(),
    port: 8080,
})?;
println!("{yaml}");
// host: api.example.com
// port: 8080
}

to_string_with_config(&v, &cfg) accepts a SerializerConfig that tunes indent, flow style, scalar style, document markers, and block-scalar thresholds. See crates/noyalib/examples/emit.rs for every option in action.

3. The dynamic Value tree

When the schema isn’t fixed at compile time, work with Value:

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};

let yaml = "
server:
  port: 8080
items:
  - name: first
    debug: true
  - name: second
";

let v: Value = from_str(yaml)?;

// Dot-path traversal.
let port = v.get_path("server.port").and_then(|n| n.as_u64());

// Sequence indexing.
let first = v.get("items").and_then(|s| s.get(0));

// Path queries — wildcards and recursive descent.
let names = v.query("items[*].name");          // every name in the list
let any   = v.query("..debug");                // every `debug` at any depth

// A key the grammar would read as structure is bracket-quoted;
// `join_keys` spells the path from literal keys.
let app = v.get_path(r#"labels["app.kubernetes.io/name"]"#);
let same = v.get_path(&noyalib::path::join_keys(["labels", "app.kubernetes.io/name"]));

// Mutation.
let mut v = v;
v["server"]["port"] = Value::from(9090u16);
}

Missing keys return None; never panic on lookup.

For lower-overhead reads, the borrowed AST shares scalar bytes with the input:

#![allow(unused)]
fn main() {
use noyalib::borrowed::from_str_borrowed;

let yaml  = "host: localhost\nport: 8080\n";
let v     = from_str_borrowed(yaml)?;
let host  = v.as_mapping().unwrap().get("host").unwrap().as_str();
// host points into `yaml` — no String allocation for the scalar.
}

BorrowedValue<'a> doesn’t follow YAML aliases (*name); use the owned path for documents that anchor.

4. Source spans (Spanned<T>)

Wrap any deserialise target in Spanned<T> to record where in the input it came from:

#![allow(unused)]
fn main() {
use noyalib::{from_str, Spanned};

#[derive(serde::Deserialize)]
struct Cfg {
    port: Spanned<u16>,
    host: Spanned<String>,
}

let cfg: Cfg = from_str("port: 8080\nhost: api\n")?;
assert_eq!(cfg.port.value, 8080);
assert_eq!(cfg.port.start.line(), 1);
assert_eq!(cfg.port.start.column(), 6);
}

Spanned<T> round-trips through serde::Serialize as T — the span info is read-side only. Combine with miette to render the exact offending region:

#![allow(unused)]
fn main() {
use noyalib::Spanned;
#[derive(serde::Deserialize)]
struct Cfg { port: Spanned<u16>, host: Spanned<String> }
let yaml = "port: 80\nhost: api\n";
let cfg: Cfg = noyalib::from_str(yaml)?;
let report = noyalib::diagnostic::spanned_error(
    yaml,
    &cfg.port,
    "port must be >= 1024 (privileged ports not allowed)",
);
eprintln!("{report:?}");
}

See crates/noyalib/examples/validation.rs.

5. Strict deserialise (typo detection)

The single biggest config-file footgun: a typo in a key name silently deserialises into the field’s Default. from_str is lenient by design (extras are ignored), from_str_strict errors out:

#![allow(unused)]
fn main() {
#[derive(Debug, serde::Deserialize)]
struct Cfg { port: u16, host: String }

let yaml = "port: 8080\nhost: api\nporrt: 9090\n";

// Lenient: silently drops the typo.
let c: Cfg = noyalib::from_str(yaml)?;
assert_eq!(c.port, 8080);

// Strict: typed error pointing at `porrt`.
let err = noyalib::from_str_strict::<Cfg>(yaml).unwrap_err();
assert!(err.to_string().contains("porrt"));
}

The strict path walks nested structs — a typo at server.unknown is reported with its parent path. Available on every input shape:

Strictness is opt-in per call site, not global — be strict about config you own and lenient about a third party’s payload in the same program. Runnable walk-through: crates/noyalib/examples/strict_deserialise.rs (cargo run --example strict_deserialise --features strict-deserialise).

6. Parser policies (defence in depth)

YAML’s anchor / merge-key / custom-tag features have historically been amplification vectors (billion-laughs) and remote-code- execution vectors (!!python/object). noyalib doesn’t instantiate arbitrary objects via tags, but you may still want to reject documents that abuse the spec at parse time:

#![allow(unused)]
fn main() {
use noyalib::{from_str_with_config, ParserConfig};
use noyalib::policy::{DenyAnchors, DenyTags, MaxScalarLength};

let cfg = ParserConfig::new()
    .with_policy(DenyAnchors)             // no `&name` / `*name`
    .with_policy(DenyTags)                // reject custom !tag scalars
    .with_policy(MaxScalarLength(64_000)) // cap individual scalar size
    .max_alias_expansions(100)            // billion-laughs guard
    .max_depth(64);

let input = "host: api.example.com\nport: 8080\n";

let res: Result<noyalib::Value, _> =
    from_str_with_config(input, &cfg);
}

Use ParserConfig::profile(ParserProfile::Standard), Strict, or SerdeYaml when the selected trust and compatibility contract should be explicit in code. ParserConfig::strict() remains an alias for the strict profile.

ParserLimits groups the resource budgets independently from YAML semantics. config.limits() extracts the active budgets, and config.with_limits(limits) replaces all of them while preserving version, key, policy, property, registry, and resolver settings.

7. Structured diagnostics

Every parse error carries (line, column, byte_offset) plus a machine-readable error code. The default API exposes a renderer-neutral payload for editors, services, and custom user interfaces:

#![allow(unused)]
fn main() {
use noyalib::{DiagnosticCode, Value};

let err = noyalib::from_str::<Value>("port: [unclosed").unwrap_err();
let diagnostic = err.diagnostic();
assert_eq!(diagnostic.code(), DiagnosticCode::Parse);
assert_eq!(diagnostic.code().as_str(), "noyalib::parse");

if let Some(label) = diagnostic.primary_label() {
    let byte_range = label.span().offset()..label.span().end();
    println!("{} at {byte_range:?}", label.message());
}
}

SourceSpan offsets and lengths are bytes in the original UTF-8 input. Protocol adapters must convert them when a client uses another coordinate system, such as the UTF-16 positions required by LSP.

The minimum, no-feature path also renders a rustc-style snippet:

#![allow(unused)]
fn main() {
use noyalib::Value;

let input = "port: [unclosed";
let err = noyalib::from_str::<Value>(input).unwrap_err();
println!("{}", err.format_with_source(input));
// error: expected ',' or ']'
//   --> input.yaml:1:7
//    |
//  1 | port: [unclosed
//    |       ^^^^^^^^^ here
}

Enable --features miette to render the same payload through the miette::Diagnostic interface with cargo / rustc-style ANSI output. Enable --features ariadne for an ariadne::Report; both adapters consume the canonical code, help, severity, and label list rather than maintaining their own mappings.

// Needs `miette` in *your* Cargo.toml alongside noyalib's `miette`
// feature, so this block is shown rather than compiled here.
fn main() -> miette::Result<()> {
    let cfg: Config = noyalib::from_str(yaml)
        .map_err(|e| miette::Report::new(e)
            .with_source_code(yaml.to_owned()))?;
    Ok(())
}

See crates/noyalib/examples/diagnostic.rs.

8. Lossless edits (cst::Document)

from_str → to_string round-tripping discards comments and exact whitespace by design — the YAML data model excludes both. When you need byte-faithful editing (Renovate-style version bumps, manifest patchers, formatters), use the lossless CST instead:

#![allow(unused)]
fn main() {
use noyalib::cst::parse_document;

let mut doc = parse_document(r#"
Production config
server:
  host: api.example.com
  port: 8080      # bind to public IP
"#)?;

// Surgical edit — only the touched span is rewritten.
doc.set("server.port", "9090")?;
println!("{doc}");
//
// # Production config
// server:
//   host: api.example.com
//   port: 9090      # bind to public IP
//
// (Comment and trailing whitespace preserved byte-for-byte.)
}

The CST exposes:

See crates/noyalib/examples/lossless_edit.rs and crates/noyalib/examples/entry_api.rs.

9. Schema validation and autofix

Under --features validate-schema:

#![allow(unused)]
fn main() {
use noyalib::{from_str, schema_for, validate_against_schema, JsonSchema};

#[derive(serde::Serialize, serde::Deserialize, JsonSchema)]
struct ServerConfig {
    /// Port the server binds on.
    port: u16,
    /// Hostname or IP literal.
    host: String,
}

let schema = schema_for::<ServerConfig>()?;
let data: noyalib::Value = from_str("port: 8080\nhost: api")?;
validate_against_schema(&data, &schema)?;
}

The matching coerce_to_schema(&mut data, &schema) engine rewrites obvious type slips — e.g. port: "8080" (quoted string where the schema declares integer) — back to the schema’s expected type, then re-validates. This is the library engine behind noyavalidate --fix. See crates/noyalib/examples/schema_validation.rs.

9b. Custom YAML tags (Value::Tagged)

YAML lets any node carry a “tag” — a type label like !Color, !Celsius, !!python/object. noyalib surfaces these on the default from_str::<Value> path so downstream code can dispatch on them safely (no global type registry, no runtime code lookup, no attacker-controlled instantiation):

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};

let v: Value = from_str("!Color '#ff8800'\n")?;
match &v {
    Value::Tagged(t) => {
        println!("tag: {}", t.tag().as_str());          // "!Color"
        println!("inner: {:?}", t.value().as_str());    // Some("#ff8800")
    }
    _ => unreachable!(),
}
Ok::<(), noyalib::Error>(())
}

This contract holds for all three tag-bearing shapes:

With the lossless-u64 feature enabled and ParserConfig::lossless_u64_integers(true), !!int scalars in (i64::MAX, u64::MAX] resolve as Value::Number(Unsigned(_)).

Reading through the wrapper. Two helpers on Value step through the tag for transparent reads:

#![allow(unused)]
fn main() {
use noyalib::{from_str, Value};
let v: Value = from_str("!Color '#ff8800'\n")?;
assert_eq!(v.untag_ref().as_str(), Some("#ff8800"));    // borrow-friendly
let owned = v.untag();                                  // consumes
Ok::<(), noyalib::Error>(())
}

Typed targets see through tags transparently. A #[derive(serde::Deserialize)] struct Foo { ... } against !Foo {x: 1} yields Foo { x: 1 } — the typed visitor never observes the tag string. This is what lets schema-tagged YAML inputs deserialise into bare structs.

Opt out via TagRegistry. When you want noyalib to strip known tags inline on the streaming path (no AST detour, no wrapper to step through), register them:

#![allow(unused)]
fn main() {
use std::sync::Arc;
use noyalib::{from_str_with_config, ParserConfig, TagRegistry};

let cfg = ParserConfig::new()
    .tag_registry(Arc::new(TagRegistry::new().with("!Color")));
// Now `!Color "#ff8800"` deserialises directly into a String,
// no Tagged wrapper, no AST fallback.
}

Reject custom tags entirely. For schema-strict contexts — audit pipelines, security-sensitive ingest — policy::DenyTags fails any document carrying a non-core tag at parse time:

#![allow(unused)]
fn main() {
use noyalib::{from_str_with_config, ParserConfig, Value};
use noyalib::policy::DenyTags;

let cfg = ParserConfig::new().with_policy(DenyTags);
let bad: Result<Value, _> = from_str_with_config("k: !Foo 1\n", &cfg);
assert!(bad.is_err());
}

Round-trip a Tagged value back to YAML. The generic to_string serialises through serde, which routes Value::Tagged(...) as a single-entry map (the right shape for serde_json interop) — losing the YAML-tag wire form. For lossless emit, the dedicated to_string_value / to_writer_value family bypasses the serde pipeline:

#![allow(unused)]
fn main() {
use noyalib::{from_str, to_string_value, Value};
let v: Value = from_str("!Color '#ff8800'\n")?;
let yaml = to_string_value(&v)?;
// `yaml` re-parses into an equivalent `Value::Tagged`.
assert!(matches!(noyalib::from_str::<Value>(&yaml)?, Value::Tagged(_)));
Ok::<(), noyalib::Error>(())
}

Known limitation: the tag-preserving fast path on the deserialise side is engaged only when the target is exactly T = Value (detected via [std::any::TypeId]). Wrapper targets like Spanned<Value>, Vec<Value>, Option<Value>, HashMap<_, Value> route the inner deserialise through the standard transparent-unwrap path. If you need both spans and tag preservation, parse twice (once into Value for tag-aware queries, once into Spanned<T> for the span-aware view).

10. Multi-document streams + parallel parse

YAML’s --- document separator lets one file carry many independent documents (Kubernetes manifests, audit-event streams). The eager API:

#![allow(unused)]
fn main() {
use noyalib::Value;
#[derive(serde::Deserialize)]
struct MyConfig { a: i64 }
let stream = "a: 1\n---\na: 2\n";
use noyalib::{load_all, load_all_as};

// `load_all` yields a lazy iterator of `Result<Value>`; collect it.
let docs: Vec<Value>      = load_all(stream)?.collect::<Result<_, _>>()?;
// `load_all_as` deserializes every document and returns a `Vec<T>` directly.
let typed: Vec<MyConfig>  = load_all_as::<MyConfig>(stream)?;
}

For large streams, the parallel feature unlocks linear-with- cores throughput:

#![allow(unused)]
fn main() {
#[derive(serde::Deserialize)]
struct MyConfig { a: i64 }
let stream = "a: 1\n---\na: 2\n";
// Drop-in for `load_all_as`. Pre-scans `---` boundaries on a
// single thread, then deserialises each document concurrently.
let docs: Vec<MyConfig> = noyalib::parallel::parse(stream)?;
}

The pre-scan is O(input_len); the per-document work parallelises across the Rayon thread pool. Services can supply a bounded, caller-owned pool instead of using Rayon’s global pool:

#![allow(unused)]
fn main() {
let stream = "---\na: 1\n---\na: 2\n---\na: 3\n---\na: 4\n";
let pool = noyalib::parallel::ThreadPoolBuilder::new()
    .num_threads(2)
    .build()?;
let docs = noyalib::parallel::parse_with_config_in_pool::<noyalib::Value>(
    stream,
    &noyalib::ParserConfig::default(),
    &pool,
)?;
assert_eq!(docs.len(), 4);
}

The pool owns the concurrency limit and lifecycle. Inputs with fewer than four documents remain sequential to avoid scheduling overhead.

10b. Error-recovering parser for LSP / IDE (recovery feature)

The default from_str family returns Err at the first syntax violation. LSP / IDE consumers need the opposite contract: keep going past errors, build a best-effort partial tree, collect every error encountered so the editor can show a complete diagnostics list and offer autocomplete on the recoverable subtrees.

#![allow(unused)]
fn main() {
// Cargo.toml: noyalib = { version = "0.0.58", features = ["recovery"] }
use noyalib::recovery::parse_lenient;

let half_typed = "name: noyalib\nfeatures: [recovery, sval\n# ^ unclosed\n";
let r = parse_lenient(half_typed);
assert!(!r.is_complete);
println!("errors: {}", r.errors.len());
// r.value is the best-effort recovered tree.
}

Three-pass recovery: strict pass → DuplicateKeyPolicy::Last retry → line-truncation retry. Multi-document input split on ---; each document recovered independently. Error collection bounded by LenientConfig::max_errors.

See crates/noyalib/examples/recovery_lenient.rs.

10c. Native async parsing on tokio (tokio feature)

For high-concurrency services parsing YAML from network sources, the tokio feature provides bounded drain helpers and a backpressured document stream:

// Needs an async runtime and, for pattern 2, `tokio-util` in *your*
// Cargo.toml, so this block is shown rather than compiled here.
// Cargo.toml: noyalib = { version = "0.0.58", features = ["tokio"] }
use noyalib::tokio_async::{async_yaml_stream, from_async_reader_multi};

// Pattern 1: drain-and-parse
let docs: Vec<MyDoc> = from_async_reader_multi(&mut reader).await?;

// Pattern 2: backpressured stream, one parsed document per item
let documents = async_yaml_stream::<_, MyDoc>(reader);

Per-document boundaries follow the YAML 1.2.2 §9.1.2 --- grammar: column-0 marker followed by whitespace or EOL. Parsing each complete document is synchronous when the stream is polled; the stream prevents additional reads until the consumer requests the next item.

See crates/noyalib/examples/tokio_async_reader.rs.

10d. sval streaming adapter (sval feature)

Alternative to the default serde route for callers wanting to skip serde_derive’s compile-time overhead or the binary-size cost of serde monomorphisation. The adapter implements sval::Value for the noyalib value graph so any sval::Stream consumer can read it:

// `sval` and the `sval::Stream` you hand it are *your* dependencies,
// so this block is shown rather than compiled here.
// Cargo.toml: noyalib = { version = "0.0.58", features = ["sval"] }
let value: noyalib::Value = noyalib::from_str("name: noyalib")?;
sval::Value::stream(&value, &mut my_stream)?;

serde remains the default route; sval is opt-in.

See crates/noyalib/examples/sval_streaming.rs.

11. The CLI tools (noyafmt, noyavalidate)

Two command-line companions ship from the split-out sebastienrousseau/noya-cli repo (split at v0.0.13, ADR-0005; released in strict lockstep):

# Format YAML in-place via the lossless CST.
noyafmt --write config.yaml

# Verify formatting without writing — for CI gates.
noyafmt --check ci/*.yaml

# Validate syntax + (optionally) JSON Schema 2020-12.
noyavalidate --schema schema.yaml in.yaml

# Validate + auto-fix obvious type slips.
noyavalidate --schema schema.yaml --fix in.yaml

Both use the same noyalib::cst::format + coerce_to_schema engines exposed to library callers. Distro-installable via crates.io (cargo install noya-cli --locked), Homebrew, AUR, and the other channels in pkg/PUBLISH.md.

12. WASM, MCP, and LSP

Three satellite crates — each split to its own repo (ADR-0005) and released in strict lockstep with this library — target specific deployment shapes:

• noyalib-wasm (sebastienrousseau/noyalib-wasm). wasm-pack output published to npm as @sebastienrousseau/noyalib-wasm. Browser IDEs use it for live YAML formatting / validation; the bundle is ~338 KB after LTO. Split to its own repo in v0.0.12 (ADR-0005); releases in strict lockstep with this workspace.

• noyalib-mcp (sebastienrousseau/noyalib-mcp). Model Context Protocol server speaking JSON-RPC over stdio. AI agents (Claude Code, GitHub Copilot, …) call parse, format, get, set, validate tools without needing a Rust toolchain. Available as npx @sebastienrousseau/noyalib-mcp (the wrapper bootstraps the binary) and as a GHCR container image ghcr.io/sebastienrousseau/noyalib-mcp. Split to its own repo in v0.0.13 (ADR-0005); releases in strict lockstep with this workspace.

• noyalib-lsp (sebastienrousseau/noyalib-lsp). Language Server Protocol implementation. Editors get format-on-save, inline diagnostics, schema-driven hover docs. Bundled into the noyalib VS Code / Open VSX extension (pkg/vscode/).

Where next

• Source spans + miette diagnostics: crates/noyalib/examples/diagnostic.rs
• Anchor manipulation: crates/noyalib/examples/anchor_shared.rs
• Schema codegen: crates/noyalib/examples/schema_validation.rs
• Figment / garde / validator integration: crates/noyalib/examples/figment.rs, crates/noyalib/examples/validation_garde.rs, crates/noyalib/examples/validation_validator.rs
• CST architecture: ARCHITECTURE.md
• Migrating from serde_yaml: MIGRATION-FROM-SERDE-YAML.md

Reporting issues / sending PRs

Filed at https://github.com/sebastienrousseau/noyalib. Reproducer YAML + the parser config you used together is the fastest path to a fix.