0004. Async model: futures from one owned event-loop thread, and the retry table¶
- Status: Accepted
- Date: 2026-11-16
- Deciders: Sebastien Rousseau
Context¶
Every Crypto Service SDK offers an asynchronous API first and a
blocking one second, with identical resilience semantics. C++17 has no
standard event loop or coroutines; its standard asynchronous result is
std::future. Callers range from single-threaded tools to servers that
fan out many requests at once and want them multiplexed over few
connections. Retrying must never repeat an operation that may already
have changed state on the server.
Options considered¶
std::asyncper call over a blocking client. One thread per call in flight, one connection per thread, no multiplexing.- Callbacks or C++20 coroutines over a caller-supplied event loop (Asio). Efficient, but it imposes a loop and, for coroutines, C++20.
- An
AsyncClientthat owns one event-loop thread driving libcurl's multi interface, returning astd::futurefor every operation; a blockingClientbeside it with the same methods.
Decision¶
Option 3.
- Shape.
Operations<Derived, Ret>holds one method per operation;AsyncClientderives fromOperations<AsyncClient, std::future>,ClientfromOperations<Client, Same>, so both have the same names, arguments and result types. - The loop. A call validates its arguments, queues its request under
a mutex, wakes the loop and returns its future at once. The loop
thread alone touches the
AsyncTransport(by default one libcurl multi handle: shared connection cache, keep-alive, HTTP/2 multiplexing over TLS where the server offers it). Each turn it adopts new calls, ends cancelled ones, starts attempts that are due, polls the transport until the next timer, decodes finished responses and fulfils their promises. Backoff waits are timers on the loop, so a waiting retry blocks no thread and no other call. - Thread safety.
AsyncClient: every member function except the destructor is thread-safe; many threads may share one instance. The destructor must not run concurrently with a call. Futures may be waited on from any thread, and a ready future's value is complete (decoding happens on the loop thread).AsyncClientis neither copyable nor movable.Client: thread-compatible, not thread-safe; it owns one libcurl easy handle whose connection is reused between calls, so use one per thread or guard it with a mutex. It is movable.CancelSource::cancel()may be called from any thread. - Cancellation.
CallOptions::canceltakes aCancelTokenfrom aCancelSource. The loop notices a cancelled token within about 10 ms and ends the call withCancelledError; the blocking client notices it within about a second (libcurl's progress callback) and while waiting between attempts. A cancelled call is not retried. - Destruction.
~AsyncClientstops the loop, cancels every transfer still in flight, ends each pending future withCancelledError(never a broken promise), and joins the thread; no completion runs afterwards. ThreadSanitizer runs the tests that destroy a client with calls in flight. - Time.
timeout(default 30 s) bounds each attempt;deadline, when set, bounds the whole call including retries and waits, and shortens the last attempt's timeout to what remains.
The retry table¶
Ported from the TypeScript SDK's request.ts, identical in every SDK.
Retries are off by default (max_retries 0). With max_retries set,
attempt n+1 waits a uniformly random time in
[0, min(initial_delay * 2^(n-1), max_delay)] (full jitter; defaults
200 ms and 2 s), or what a 429, 503 or 504 response's Retry-After
asks for (delta-seconds or an HTTP-date); a Retry-After longer than
max_delay makes that response final.
| Outcome | Idempotent or pure operation (every GET, every other POST) | Stateful operation |
|---|---|---|
| 429 Too Many Requests | Retry | Retry |
| 503 Service Unavailable | Retry | Retry |
| 504 Gateway Timeout | Retry | Never |
| Connection error or attempt timeout | Retry | Never |
| Any other status | Final | Final |
| Cancelled, or deadline reached | Final | Final |
Stateful operations are those whose path contains /keys/generate,
/keys/delete, /keygen (every key generation route: ML-KEM, hybrid,
ML-DSA, SLH-DSA, HPKE), /kms/create-key or /opaque/register/. After
a 429 or 503 the server certainly did nothing; after a 504 or a lost
connection it may already have created or deleted a key.
Consequences¶
- An
AsyncClientcosts one thread and one multi handle; create one per server and share it, not one per call. - Many concurrent calls share connections and, over HTTP/2, multiplex on one.
std::future::get()blocks the caller's thread; callers that need callbacks or coroutines wrap the future or implement anAsyncTransport(a candidate in docs/ROADMAP.md).- The worst case for one call without a deadline is
(max_retries + 1) * timeoutplus the waits; setdeadlineto bound it.