Batching and Lifecycle

prepared = TypeSafeSDK.prepare!(urgent: TypeSafeSDK.noul("Urgent?"))
results = TypeSafeSDK.evaluate_many(client, states, prepared,
  max_concurrency: 8, ordered: true, on_error: :collect,
  task_timeout_ms: 40_000, attempt_timeout_ms: 5_000)

evaluate_many collects a lazy evaluate_stream. Input can be any Enumerable. Shared questions/options are validated before work starts and invalid shared configuration raises even under :collect. Individual state validation/API failures return one error in the result stream. Prepared is reused, not rebuilt for each state. Calling evaluate_stream alone performs no HTTP and starts no supervisor; every enumeration has independent lifecycle state.

Do not combine attempt_timeout_ms with timeout or timeout_ms. Ordinary request timeout remains seconds. Retry budget/backoff remain seconds in RetryPolicy. Set the task budget to accommodate your intended retry budget; a task timeout can terminate a request before retry exhaustion.

TypeSafeSDK.evaluate_stream(client, large_enumerable, prepared, ordered: false)
|> Enum.each(fn
  {:ok, response} -> IO.inspect({response.batch_index, response.answers})
  {:error, error} -> IO.inspect({error.details.batch_index, error.type})
end)

Batch indexes are zero-based and exist on errors too. Do not zip unordered results positionally with inputs. on_error: :raise stops enumeration at the first observed failure. Unlinked tasks isolate exits from the consumer; task exits become sanitized :task_exit errors, timeouts become :timeout with details.scope == :batch. Raw exit reasons can contain state and are omitted.

An owned supervisor and stream continuation shut down outstanding tasks on early halt, exceptions and caller termination. Ordered execution prefetches at most max_pending inputs into a finite window. All results from that window are delivered before the next window begins. This bounds completed-result buffering behind a slow predecessor as well as input prefetch. The tradeoff is a window boundary: some workers can be idle while a window's final slow request finishes. Tune max_pending to balance bounded buffering against that utilization cost. Unordered execution has no window boundary and uses the normal demand-driven OTP task stream. These bounds count values, not bytes.

Early halt may discard prefetched inputs; do not use a destructive/acknowledging source unless the application explicitly owns its reprocessing semantics. Collecting all results necessarily uses memory proportional to output size; use streaming for large inputs. Escaping worker exceptions are normalized before OTP's Task crash reporter can print their state-bearing arguments or reasons.

This does NOT establish a global transport queue bound, peer stream limit, or streaming response-byte cap. Nor does killing a worker prove the remote server stopped processing it. See runtime guarantees and retry ambiguity.

Run mix run examples/live_batching.exs for a complete live walkthrough. See the live example catalog for setup and API-call costs.

Shared cancellation in 0.3

Pass one Pristine.Cancellation token to the batch:

cancel = Pristine.Cancellation.new()
stream = TypeSafeSDK.evaluate_stream(client, states, prepared,
  cancellation: cancel,
  max_concurrency: 8)

:ok = Pristine.Cancellation.cancel(cancel)

The same token is forwarded to every started unary request. Before scheduling another item, the batch source checks the token; once cancellation is observed, no new request work is started. Completed results keep the existing ordered or indexed result contract, while in-flight cancellations remain typed :cancelled errors. The owned lifecycle still shuts down tasks/monitors on halt or caller death.

Physical local HTTP cancellation and cleanup are Pristine transport guarantees, not batch-worker guarantees. Remote processing may already have begun; cancelled operations are not automatically safe to replay.