NOTE 02 · 26 Sep 2026
Failing over without failing the patient
An AI provider is a dependency like any other: it can be slow, rate-limited, deprecated or simply wrong. The useful question is not whether the model is impressive on a good day. It is what the system does on a bad one.
MediVault extracts clinical information from uploaded records. A single provider in that path is a single point of failure, so the pipeline has an explicit fallback and a confidence gate.
The happy path is the easy part
The primary extractor is Gemini 3.1 Flash Lite. The fallback is NVIDIA NIM running Llama 3.2 11B Vision. They are not interchangeable in quality, latency or failure modes, so the system treats them as two attempts with different budgets rather than one magical model.
Each attempt has a timeout. The first success wins. If both fail, the pipeline raises an extraction-unavailable error instead of inventing an answer.
async function extractWithFallback(image: File) {
const attempts = [
{ name: "gemini-3.1-flash-lite", timeout: 2_500 },
{ name: "nvidia-nim-llama-3.2-11b-vision", timeout: 4_000 },
];
for (const attempt of attempts) {
try {
const result = await withTimeout(
extract(attempt.name, image),
attempt.timeout,
);
return { ...result, provider: attempt.name };
} catch (error) {
log.warn("extraction.provider_failed", {
provider: attempt.name,
error,
});
}
}
throw new ExtractionUnavailableError();
}The confidence gate
A successful response is not the same as a trustworthy extraction. Below 90% confidence, the result is flagged NEEDS_REVIEW and never auto-saved into the clinical record. The clinician sees the source image beside the extracted fields and confirms or corrects them.
The threshold is a product decision, not a model parameter. It encodes how much error the workflow can absorb and who is accountable when it does not.
const reviewRequired = result.confidence < 0.9;
await db.extractions.insert({
...result,
status: reviewRequired ? "NEEDS_REVIEW" : "ACCEPTED",
});Audit what the fallback did
A fallback that silently degrades is worse than an outage, because nobody knows to distrust the output. Every attempt logs the provider, latency, confidence and model version. The record keeps the provider that actually produced the result.
That log is not just observability. It is how a clinician, an auditor or a future maintainer reconstructs why a field contains what it contains.
- Log the provider that succeeded and every provider that failed.
- Record latency and confidence with the extraction, not beside it.
- Keep the source document reference for review.
- Alert on fallback rate, not only on total failures.
What I would change
The current request-bound pipeline is fine for demos and small uploads. Under batch use, it is the first thing that breaks. I would move ingestion onto a durable queue with idempotent workers, version the extractor prompts, and add contract tests around the review gate before touching the policy layer again.
A fallback that silently degrades is worse than an outage. At least an outage is honest.