Benchmarks¶
Two kinds, both with JMH 1.37, for every public method.
- Client-side cost (
ClientSideBenchmark, no network): building each typed method's request (the JSON body, its serialisation, theHttpRequestwith its headers) and decoding each typed method's response (parse plus record mapping). Inputs are the conformance kit's synthetic fixtures; the responses are what pain001 0.0.87 answered for them, recorded byscripts/capture-bench-responses.shintosrc/bench/resources/. - Loopback round trip (
LoopbackBenchmark): one full call per method, plus the rawrequest, againstpain001 serveon 127.0.0.1. This is mostly the server's work; the client-side numbers above are the SDK's share of it.
make bench runs both (the loopback ones only with PAIN001_BASE_URL
set; they are skipped with a message otherwise) and writes
target/jmh.txt. CI smoke-runs every benchmark once per pull request
(make bench-smoke) to keep them compiling and running; nothing is
asserted on the numbers.
Environment¶
| Machine | Apple A18 Pro, 6 cores, 8 GB, macOS 27.2 |
| JVM | Temurin 21.0.13+9 (OpenJDK 64-Bit Server VM), default flags |
| JMH | 1.37, average time, one thread |
| Server | pain001 0.0.87 (pain001 serve, one process) on 127.0.0.1 |
| Date | 2026-11-14 |
Read these numbers as rough. The machine was shared with other
heavy builds while they were taken: the load average was between 80
and 245 on 6 cores during the client-side run (it was not recorded for
the loopback run). The confidence intervals are wide, and some pairs
that do the same work (buildValidate and buildGenerate) differ by
more than the work explains. Re-run make bench on an idle machine
before quoting a number.
Client-side cost¶
3 forks, 5 x 1 s warm-up and 10 x 1 s measurement iterations each (30 samples per benchmark); score is the mean, error the 99.9 % confidence interval.
| Benchmark | Mean (µs/op) | Error (µs/op) |
|---|---|---|
buildHealth |
6.194 | ± 2.432 |
buildValidate |
16.969 | ± 9.631 |
buildGenerate |
46.192 | ± 19.500 |
buildCheck |
171.845 | ± 75.407 |
buildFaces |
7.498 | ± 3.288 |
buildProjectFace |
145.208 | ± 48.511 |
decodeHealth |
1.713 | ± 1.100 |
decodeValidate |
2.386 | ± 1.267 |
decodeGenerate |
27.020 | ± 12.548 |
decodeCheck |
2.679 | ± 1.188 |
decodeFaces |
2.654 | ± 0.614 |
decodeProjectFace |
58.208 | ± 21.214 |
decodeRawRequest |
75.940 | ± 29.825 |
build* for validate and generate serialise the 5-row kit CSV;
check and projectFace serialise the 4.4 KB kit XML. decodeGenerate
and decodeRawRequest parse the 4.5 KB generate answer (the raw one
without record mapping); decodeProjectFace the 5.6 KB Berlin Group
projection.
Loopback round trip¶
1 fork, 3 x 1 s warm-up and 5 x 1 s measurement iterations
(make bench defaults); 5 samples per benchmark.
| Benchmark | Mean (ms/op) | Error (ms/op) |
|---|---|---|
health |
4.912 | ± 6.638 |
validate |
26.833 | ± 51.713 |
generate |
374.956 | ± 542.977 |
check |
37.159 | ± 74.779 |
faces |
12.839 | ± 29.933 |
projectFace |
51.476 | ± 94.871 |
rawRequest (GET /api/v1/health) |
10.741 | ± 28.349 |
The client-side cost of every method is under 0.25 ms in these runs, against 5 to 375 ms for the round trip: the time a caller sees is the server's.
Reproducing¶
make bench # client-side only
PAIN001_BASE_URL=http://127.0.0.1:8000 make bench # both kinds
make bench BENCH_OPTS="-f 3 -wi 5 -w 1s -i 10 -r 1s" # the client-side settings above