Benchmarks¶
Two questions, two benchmark classes in
benchmarks/Pain001.Sdk.Benchmarks (BenchmarkDotNet):
- Client side (
ClientSideBenchmarks): what the SDK itself costs per call, with no network. Each method builds and serializes its request from the conformance kit's fixtures (the five-payment credit-transfer CSV, its pain.001.001.09 document) and decodes a recorded answer frompain001 serve0.0.87 to those fixtures, through an in-memoryHttpMessageHandler. - Loopback (
LoopbackBenchmarks): one full round trip per method to a realpain001 serveon 127.0.0.1, with the same fixtures. This is the client, the loopback TCP stack and the server's own work together. It is skipped, with a message, whenPAIN001_BASE_URLis unset.
Method¶
make build
PAIN001_BASE_URL=http://127.0.0.1:8000 make bench # full run, results in artifacts/benchmarks/
make bench-smoke # one iteration each, as CI runs it
BenchmarkDotNet's default job, in process (--inProcess), with
[MemoryDiagnoser]. The numbers below are one run on one machine; they
show orders of magnitude, not a guarantee. CI runs the smoke job on
every pull request to keep the benchmarks compiling and running; it
asserts nothing.
Environment¶
| Item | Value |
|---|---|
| Machine | Mac with an Apple A18 Pro (6 cores), 8 GB |
| OS | macOS 27.2 (26B123) |
| .NET | SDK 10.0.404, runtime 10.0.15, Arm64 RyuJIT |
| BenchmarkDotNet | 0.15.8, InProcessEmitToolchain |
| Server | pain001 serve 0.0.87 (Python 3.12, uvicorn), same machine, 127.0.0.1 |
| Date | 2026-11-14 |
The machine was not otherwise idle during the loopback run, and the server shares its CPU with the benchmark, so the loopback numbers have wide error bars.
Results: client side (no network)¶
| Method | Mean | Error | StdDev | Median | Allocated |
|---|---|---|---|---|---|
| Health | 1.899 μs | 0.060 μs | 0.164 μs | 1.875 μs | 2.83 KB |
| Validate | 6.380 μs | 0.408 μs | 1.151 μs | 6.085 μs | 8.95 KB |
| Generate | 30.942 μs | 1.789 μs | 4.986 μs | 29.578 μs | 28.66 KB |
| Check | 26.164 μs | 1.361 μs | 3.772 μs | 25.153 μs | 23.17 KB |
| Faces | 4.735 μs | 0.223 μs | 0.614 μs | 4.575 μs | 5.84 KB |
| ProjectFace | 50.056 μs | 3.067 μs | 8.498 μs | 47.086 μs | 49.25 KB |
| RawRequest | 1.577 μs | 0.092 μs | 0.254 μs | 1.481 μs | 2.53 KB |
Results: loopback round trip¶
| Method | Mean | Error | StdDev | Median | Allocated |
|---|---|---|---|---|---|
| Health | 3.939 ms | 1.334 ms | 3.742 ms | 2.518 ms | 4.89 KB |
| Validate | 11.757 ms | 1.758 ms | 4.813 ms | 9.991 ms | 7.66 KB |
| Generate | 55.645 ms | 12.233 ms | 35.295 ms | 49.719 ms | 27.44 KB |
| Check | 18.644 ms | 3.390 ms | 9.561 ms | 15.559 ms | 12.6 KB |
| Faces | 7.200 ms | 1.703 ms | 4.832 ms | 5.238 ms | 7.92 KB |
| ProjectFace | 35.881 ms | 9.547 ms | 27.392 ms | 29.745 ms | 38.69 KB |
| RawRequest | 1.191 ms | 0.138 ms | 0.386 ms | 1.092 ms | 4.55 KB |
Reading them¶
- The client's own cost is microseconds: at most about 50 μs (a face projection, whose answer is the largest body) against milliseconds for the round trip. Its share of a loopback call is under 0.2 % for every method here; on a real network it is smaller still.
- The round trip is the server's work: generation and projection, which build and validate a document, are the slowest.
HealthandRawRequestsend the sameGET /api/v1/health; their loopback means differ because of the noise noted above, and their medians are within 1.5 ms of each other.