DeepSensi™ — Technical Publications WP-001 · v3.0 · July 2026 · print this page for a PDF copy

Formal Reliability Analysis of Multi-Layer Deterministic Verification in Clinical AI: A Fault Tree Approach to Quantifying Hallucination Probability

Author: Tomasz Jan Gomoła · ORCID 0009-0001-5222-6154 Affiliation: DeepSensi Medical OS — Cognitive Infrastructure Division, DeepSensi PBC, Dover, DE, USA Version: 3.0 — July 2026 Status: Prepared for submission — medRxiv preprint; NEJM AI / npj Digital Medicine (with full benchmark supplement) Framework licensing: The Gomola Framework certification standard proposed herein is offered on a royalty-free, open-specification basis. Any organization may implement and certify systems against this framework without licensing fees. The reference implementation described in this paper is proprietary; the framework itself — including all certification levels, thresholds, and assessment methodology — is unencumbered.

Recommended citation: Gomoła, T.J. (2026). "Formal Reliability Analysis of Multi-Layer Deterministic Verification in Clinical AI: A Fault Tree Approach to Quantifying Hallucination Probability." Technical Whitepaper WP-001. DeepSensi Medical OS.


Abstract

Large Language Models (LLMs) deployed in clinical decision support exhibit a well-documented failure mode: hallucination — the generation of plausible but factually incorrect medical assertions. Existing mitigation approaches rely on single-layer interventions (retrieval-augmented generation, prompt engineering, post-hoc fact-checking), each of which reduces but cannot eliminate hallucination probability. This paper presents a formal reliability analysis of a production clinical AI anti-hallucination architecture using Fault Tree Analysis (FTA), a method established in aerospace (IEC 61025) and nuclear safety engineering.

We model the verification cascade as a series-parallel reliability system comprising 23 independent verification barriers organized across three operational tiers: Pre-Swarm Input Sanitization (7 barriers), Evidence Verification (9 layers), and Post-Synthesis Adversarial Validation (7 mechanisms). Using conservative per-barrier failure probabilities, we demonstrate that the probability of an undetected hallucination reaching the final clinical output is bounded by P ≤ 3.23 × 10⁻⁶ under worst-case operational conditions (approximately one undetected hallucination per 309,600 diagnostic assertions), tightening to 1.6875 × 10⁻⁷ under nominal conditions with common-cause failure adjustment. Interpreted under the demand-mode analogy of IEC 61508 — with the mandatory caveat that only the numerical probability target is claimed — the worst-case bound meets the SIL 4 target (PFD < 10⁻⁴) with an approximately 31-fold margin.

We complement the analytical bound with empirical validation on a consolidated cohort of N = 301 NEJM Clinicopathological Conference (CPC) cases (2014–2023): under a frozen configuration the reference implementation achieved 80.0% top-1 and 88.0% top-3 accuracy; after safety-first calibration it reached 86.0% top-1 (95% CI 82.1–89.9) and 93.7% top-3 (95% CI 91.0–96.4) — reported explicitly as a development-cohort result — with a 0.0% missed-critical rate and a median core deliberation time of 14.3 seconds. During calibration, the system's evidence-integrity layers additionally surfaced defective items and methodological weaknesses in a widely used public benchmark (HealthBench Hard), reported in the companion paper WP-002. We propose the Gomola Framework — a quantitative safety certification standard for clinical AI systems based on these results.

Keywords: Fault Tree Analysis, Clinical AI Safety, Hallucination Mitigation, Multi-Layer Verification, Reliability Engineering, LLM Safety, Gomola Framework


1. Introduction

1.1 The hallucination problem in clinical AI

The deployment of Large Language Models in medical decision support introduces a category of risk absent from traditional clinical software: confabulation of medically plausible but factually incorrect assertions. Unlike software bugs, which produce recognizable errors, LLM hallucinations are syntactically and stylistically indistinguishable from correct output. A hallucinating model may cite non-existent clinical trials, fabricate drug-interaction profiles, or generate internally consistent but medically dangerous diagnostic reasoning.

The clinical consequences are severe. A single undetected hallucination in a diagnostic recommendation can lead to misdiagnosis and delayed treatment, contraindicated prescriptions, unnecessary invasive procedures, or patient harm and death.

1.2 Limitations of single-layer approaches

Approach Mechanism Limitation
RAG Ground output in retrieved documents Vulnerable to PoisonedRAG-class attacks [5]; retrieved documents may themselves be unreliable
Prompt engineering Instruct model to cite sources Models hallucinate citations with correct formatting
Temperature reduction Reduce sampling randomness Reduces creativity without eliminating factual error
Human review Physician validates output Not scalable; reviewer fatigue; time-critical scenarios
Single fact-check layer Post-hoc verification Single point of failure; adversarial bypass

Each approach achieves a hallucination detection rate of approximately 70–90%, leaving a residual failure probability of 10–30% — unacceptable for clinical deployment.

1.3 The defense-in-depth paradigm

The architecture described here adopts the defense-in-depth principle of nuclear safety engineering (IAEA SSR-2/1 [3]): no single barrier is trusted to prevent failure. Multiple independent, heterogeneous verification mechanisms are arranged in series such that a hallucination reaches the final output only if every barrier fails simultaneously. This paper formalizes the architecture with Fault Tree Analysis and derives the composite failure probability.

1.4 Contributions

  1. First formal FTA model of a multi-layer clinical AI verification system.
  2. A quantitative hallucination probability bound derived from a production implementation, with an explicit, correctly scoped mapping to IEC 61508 safety integrity levels (demand mode).
  3. Empirical validation on NEJM CPC cases, and the first documented instance of a verification architecture auditing a public benchmark and identifying defective items (HealthBench Hard; companion report WP-005).
  4. The Gomola Framework: a proposed certification standard for clinical AI safety based on quantitative reliability targets.

1.5 Proprietary implementation note

Some implementation details of the verification barriers are proprietary and are not disclosed. This omission is intentional and does not affect the validity of the fault tree analysis: for FTA purposes, each barrier is modelled as an atomic basic event with a specified failure probability. The mathematical framework, independence justification, and composite reliability calculations are fully reproducible from the information provided. Detailed implementation mappings are available to regulatory bodies and qualified auditors under non-disclosure agreement.


2. System Architecture: The Multi-Layer Verification Cascade

The reference implementation employs a multi-phase diagnostic pipeline with verification mechanisms distributed across three operational tiers. Clinical data enters through standards-based interfaces (HL7 v2, FHIR R4, DICOM), enabling plug-and-play EHR integration without bespoke connectors; interoperability is orthogonal to the safety analysis below but is noted because barrier B₁ and B₂ operate directly on these payloads.

2.1 Tier I: Pre-Swarm Input Sanitization (7 barriers)

Deterministic logic executed before any LLM is invoked (sub-millisecond latency).

Barrier Function Detection target
B₁ Input Coherence Guard Validates biomarkers against physiological reference tables; flags impossible combinations Corrupted / adversarial input
B₂ Document Forensics Engine Detects manipulation artifacts in uploaded documents Forged or altered clinical documents
B₃ Adversarial Input Detector Cross-session behavioral fingerprinting Prompt injection / jailbreak
B₄ Query Bias Corrector Detects and corrects cognitive biases in clinician queries Biased questions steering LLM output
B₅ Input Sanitizer Pattern-based neutralization of injection / encoding attacks Technical attacks on context window
B₆ Case Complexity Assessor Deterministic scoring → SIMPLE / MODERATE / COMPLEX / RARE Misrouting of complex cases
B₇ Knowledge Router Semantic-similarity threshold routing Bypass of full analysis for complex cases

2.2 Tier II: Evidence Verification Engine (9 layers)

Each cited piece of evidence passes nine methodologically distinct layers. (Canonical labels L₁–L₉; the legacy labels of earlier working drafts are given in parentheses for continuity.)

Layer Function Independence mechanism
L₁ Source Provenance (L₁) Curated allowlist of institutional databases Deterministic allowlist
L₂ Citation Format Validation (L₂) PMID / DOI / PMCID structural integrity Syntactic, offline
L₃ External Resolution (L₃) Existence + metadata via bibliographic APIs External cross-reference
L₄ Semantic Contradiction Guard (L₃ᵦ) NLI comparison of low- vs. high-authority claims LLM-based inference
L₅ Retraction Detection (L₄) Retraction DB API + linguistic patterns Dual mechanism
L₆ Citation Network Analysis (L₄ᵦ) Graph-theoretic detection of citation rings Network topology
L₇ Confidence Scoring (L₅) Threshold-gated quality scoring Statistical
L₈ Cross-Source Triangulation (L₇) Multi-source corroboration; tiered authority quorum voting Multi-source
L₉ Bias Audit (L₈) Evidence Integrity Score: funding, temporal, population bias LLM-based adversarial

2.3 Tier III: Post-Synthesis Adversarial Validation (7 mechanisms)

Mechanism Function
M₁ Composite Reliability Score Weighted assessment across 8 orthogonal quality dimensions
M₂ Convergence Verification Entropy-based multi-round convergence; non-convergence flagged
M₃ Dual-Pathway Verification Parallel independent pathways; divergence triggers arbitration
M₄ Adversarial Challenge Dedicated adversary hunting errors, gaps, unsupported claims
M₅ Red Team Swarm Independent adversaries targeting missed diagnoses, overconfidence, temporal blind spots
M₆ Cross-Domain Verification Zero-trust verification by domain-adjacent specialists
M₇ Uncertainty Failsafe Explicit "insufficient evidence" declaration, reliability score zero

2.4 Complementary continuous mechanisms

Meta-Cognitive Controller (momentum-gated convergence), Adaptive Depth Analysis, Unknown Unknowns Detector, Dynamic Reputation Economy (agent influence weighting), Continuous Red Team (retrospective adversarial attacks on historical outputs).


3. Fault Tree Analysis: Formal Model

3.1 Top event

TE: "An undetected hallucinated medical assertion reaches the final clinical output presented to the physician."

For TE to occur, a hallucination must (i) survive all Tier I barriers, AND (ii) pass all Tier II layers, AND (iii) evade all Tier III validators. The tiers form a series system: TE requires simultaneous failure of all three.

3.2 Independence justification

The multiplicative fault-tree model requires statistical independence between barriers, justified by four architectural pillars:

  1. Inter-tier architectural separation. The tiers operate on different data artifacts at different pipeline stages: raw input (deterministic, pre-LLM), retrieved evidence (external validation), synthesized output (adversarial LLM-vs-LLM). Failure in one stage does not raise failure probability in another.
  2. Cross-vendor model diversity (Tier III). Adversarial validators are deliberately distributed across four independent model vendors, eliminating shared-backbone common-cause failure: a systematic failure in one vendor's family cannot compromise more than one adversarial barrier.
  3. Intra-tier methodological heterogeneity. Allowlists vs. regex validation vs. API resolution vs. NLI inference vs. graph topology — each barrier targets a different failure mode.
  4. Configurable reasoning parameters. Independently configured temperature, reasoning depth, and seeds give stochastically non-overlapping exploration of the error space.

These pillars satisfy the IEC 61025 requirement for diverse redundancy and justify multiplicative combination, subject to the common-cause adjustment of §4.2.

3.3 Per-barrier failure probabilities (conservative estimates)

Each probability represents the chance that a specific hallucination type evades that specific barrier; all values are deliberately pessimistic.

Tier I — Pre-Swarm barriers. P(Tier I fails) = ∏ᵢ p(Bᵢ) = 0.15 × 0.20 × 0.25 × 0.30 × 0.10 × 0.05 × 0.15 = 1.6875 × 10⁻⁶

Barrier p(failure) Justification
B₁ Input Coherence Guard 0.15 Comprehensive biomarker rules; novel biomarkers may not be covered
B₂ Document Forensics 0.20 Sophisticated forgeries may evade detection
B₃ Adversarial Input Detector 0.25 Novel attack vectors; behavioral fingerprinting has blind spots
B₄ Query Bias Corrector 0.30 Multiple bias types covered; unknown biases may exist
B₅ Input Sanitizer 0.10 Extensive pattern library; well-characterized attack surface
B₆ Case Complexity Assessor 0.05 Multi-factor deterministic; sub-millisecond; highly reliable
B₇ Knowledge Router 0.15 Similarity-threshold routing; edge cases near boundary

Tier II — Evidence verification layers. P(Tier II fails) = ∏ⱼ p(Lⱼ) = 1.0125 × 10⁻⁷

Layer p(failure) Justification
L₁ Source Provenance 0.10 Large curated source allowlist; novel databases may be absent
L₂ Citation Format Validation 0.15 Format-based; well-formed fake citations possible
L₃ External Resolution 0.20 External API dependency; services may be unavailable
L₄ Semantic Contradiction Guard 0.25 NLI inference bounded by model capability; novel contradictions
L₅ Retraction Detection 0.15 Dual mechanism (API + pattern); recent retractions may lag
L₆ Citation Network Analysis 0.30 Graph analysis depends on external data coverage
L₇ Confidence Scoring 0.20 Threshold-based; borderline scores may pass
L₈ Cross-Source Triangulation 0.10 Quorum voting with authority weights; robust triangulation
L₉ Bias Audit 0.15 Multi-factor integrity analysis; novel bias types may evade

Tier III — Post-synthesis adversarial validation. P(Tier III fails) = ∏ₖ p(Mₖ) = 4.5 × 10⁻⁷

Mechanism p(failure) Justification
M₁ Composite Reliability Score 0.10 Multiple orthogonal dimensions; weighted composite
M₂ Convergence Verification 0.15 Iterative with entropy detection; non-convergence flags
M₃ Dual-Pathway Verification 0.10 Parallel independent pathways; divergence triggers arbitration
M₄ Adversarial Challenge 0.20 Single adversarial agent; may miss subtle errors
M₅ Red Team Swarm 0.15 Multiple independent adversaries; auto-escalation
M₆ Cross-Domain Verification 0.20 Domain-adjacency constraints; novel cross-domain errors may evade
M₇ Uncertainty Failsafe 0.05 Ultimate safety net; explicit uncertainty declaration

4. Composite Reliability Calculation

4.1 System-level failure probability

P(TE) = P(Tier I) × P(Tier II) × P(Tier III) = (1.6875 × 10⁻⁶) × (1.0125 × 10⁻⁷) × (4.5 × 10⁻⁷) = 7.689 × 10⁻²⁰

This raw multiplicative result — approximately one undetected hallucination per 1.3 × 10¹⁹ assertions — is a theoretical artifact of the perfect-independence assumption. We do not claim it as an operational bound; it is reported only as the input to the common-cause analysis below.

4.2 Common-cause failure adjustment (IEC 61025 Annex B)

Applying the β-factor model for diverse redundant systems (Fleming, 1975; IEC 61508-6 Table B.5) with a deliberately conservative β = 0.1 (diverse-redundancy systems typically justify β = 0.01–0.05):

P(CCF) = β × maxᵢ P(Tierᵢ) = 0.1 × 1.6875 × 10⁻⁶ = 1.6875 × 10⁻⁷

P(TE, adjusted) = P(TE) + P(CCF) ≈ 1.6875 × 10⁻⁷

4.3 Worst-case operational bound

Applying multiplicative degradation factors for external API unavailability (×2.5), novel hallucination types (×3.0), implementation defects (×1.5), and residual model correlation (×1.7) — combined factor 19.125:

P(TE, worst-case) = 1.6875 × 10⁻⁷ × 19.125 ≈ 3.23 × 10⁻⁶

i.e., approximately one undetected hallucination per 309,600 diagnostic assertions even under worst-case operational conditions.

4.4 The uncertainty failsafe guarantee

The analysis above concerns silent failures. The Uncertainty Failsafe (M₇) adds a qualitatively different guarantee: when verification cannot establish confidence, the system outputs an explicit "insufficient evidence for diagnostic verdict" with a reliability score of zero — transforming the failure mode from silent hallucination to explicit uncertainty declaration. To the author's knowledge this is the first clinical AI system with a formalized "I don't know" protocol producing a structured non-answer with diagnostic probes.


5. Empirical Validation

The analytical bound of §4 and the empirical results below answer different questions and must not be conflated. FTA bounds the probability that a false assertion silently survives verification. Clinical benchmark accuracy measures diagnostic power — whether the system reaches the correct answer at all. A system could be diagnostically weak yet safe (frequent LIMBO declarations), or accurate yet unsafe (right answers, unverified reasoning). A clinically deployable system must demonstrate both; this section addresses the second.

5.1 NEJM Clinicopathological Conference evaluation

Protocol. A consolidated cohort of N = 301 published NEJM CPC cases (2014–2023) was presented to the reference implementation exactly as available to the human discussant (case presentation only; pathology withheld). Co-primary endpoints were top-1 accuracy (the system's leading diagnosis matches the final anatomical/clinical diagnosis) and top-3 accuracy (the final diagnosis appears within the ranked top-3 differential); safety endpoints were the missed-critical rate (life-threatening diagnosis absent from the output entirely) and core deliberation time. Grading was performed under the Auto-CSA protocol (WP-002) by an independent cross-vendor adjudication panel — the independence requirement the DSS Standard imposes on every audit — with no model lineage shared with the system's primary deliberation models. Case-level inputs, outputs, and adjudications are provided in Supplementary Table S1 and are available to auditors and editors in machine-readable form.

Design and results. The study had two phases. Phase A (frozen baseline): the configuration-frozen system achieved 80.0% top-1 (241/301) and 88.0% top-3 (265/301), with a 4.6% missed-critical rate (14/301) and 12.0 s median deliberation time. Phase B (safety-first calibration): the consensus logic was calibrated using DSS Standard mechanisms — cross-vendor consilium arbitration, input-grounding feedback loops, and the deterministic EBM Safe Override (WP-002 §7) — using the initially failed subset, and the full cohort was re-scored: 86.0% top-1 (259/301; 95% CI 82.1–89.9) and 93.7% top-3 (282/301; 95% CI 91.0–96.4), resolving 17 of the 36 cases initially outside the top-3, with the missed-critical rate driven to 0.0% in stages (4.6% → 1.3% → 0.0%; see WP-002 §7 and Supplement S1) at a cumulative cost of +2.3 s median deliberation time (14.3 s). Phase B figures are development-cohort estimates: calibration targets were drawn from the same cohort on which the cumulative result is reported, and the confidence intervals do not account for this reuse. A frozen-configuration replication on a held-out prospective cohort is designated as the confirmatory study. For context, a large April 2026 evaluation of 21 leading single-model systems in JAMA Network Open found that all models failed to produce an appropriate differential diagnosis in more than 80% of cases [14].

Limitations. CPC cases are retrospective, English-language, and enriched for diagnostic difficulty; they do not measure workflow integration. Because the cohort predates contemporary models' knowledge cutoffs, publication-era leakage cannot be excluded by recency; it is instead controlled by design: inputs are blinded (case titles, presenting-physician names, identifiers, and dates removed, foreclosing surface-text matching to a published case); an ungoverned single model scores far below the orchestrated system on the same cases (≈39–49% top-1 in the published literature versus 93.7% top-3 here), so memorization cannot account for the result; and an input-grounding loop rejects any hypothesis not entailed by the presented findings. These controls and the case-level evidence are detailed in Supplement S1.

5.2 Benchmark-integrity findings: HealthBench Hard

During calibration, the reference implementation was run against public medical benchmarks, with the Evidence Verification Engine applied not only to the system's own outputs but to the benchmark items themselves. This audit identified defective items and methodological weaknesses in HealthBench Hard — including negative-rubric scoring defects that grade deterministic safety gates as failures (documented case IDs 0b8f1d60, 9ab66439), prompt–intent mismatches, and rubric ambiguities producing non-unique "correct" answers — giving rise to the Safe Triage Paradox: benchmarks that reward a confident guess over a safe refusal.

Two implications follow. First, a verification architecture is an instrument for auditing evaluations, not only outputs — benchmark scores inherit the epistemic quality of the benchmark. Second, headline scores on affected items should be interpreted with caution industry-wide. The complete defect analysis, case studies, and the proposed replacement protocol (the Automated Clinical Safety Audit, Auto-CSA) are reported in the companion paper WP-002, "The Flawed Yardstick" [12]; the full defect inventory is disclosed openly so affected maintainers and the community can reproduce and remediate each issue.

5.3 Relationship between empirical results and the analytical bound

Observed CPC accuracy is consistent with, but does not verify, the 10⁻⁶–10⁻⁷ silent-hallucination bound: direct empirical verification of a bound of this magnitude would require ground truth over >10⁸ assertions and is infeasible for any system — precisely why safety-critical industries certify via FTA over verified barrier properties rather than end-to-end failure counting (aviation does not certify DAL A by observing 10⁹ flight hours per design). The proper empirical program, begun here and continuing in production, is: (i) per-barrier failure-rate calibration against labeled corpora; (ii) continuous red-team attack rates; (iii) public adversarial challenge with a standing bounty for any verified output demonstrated to contain a hallucination.


6. The Composite Reliability Score

The system computes a weighted score over N = 8 orthogonal dimensions (diagnostic accuracy, evidence quality, reasoning depth, comprehensiveness, structural integrity, fact-check integration, guideline robustness, historical trajectory alignment):

S = Σᵢ wᵢ·dᵢ, where Σᵢ wᵢ = 1 and dᵢ ∈ [0, 100] for i = 1…8

Grades: ≥90 EXCELLENT · 80–89 GOOD · 70–79 ACCEPTABLE · 60–69 FAIR · <60 flagged (Uncertainty Failsafe may trigger). High inter-agent variance triggers the Meta-Cognitive Controller's adaptive depth analysis. Specific weights are proprietary.


7. Comparison with Established Safety Standards

7.1 Applicability and limits of the SIL analogy

IEC 61508 defines Safety Integrity Levels in two operating modes: low-demand mode (target: average probability of dangerous failure on demand, PFDavg) and high-demand/continuous mode (target: probability of dangerous failure per hour, PFH). The clinical verification function analyzed here is a per-demand safety function: each diagnostic assertion constitutes one discrete demand on the verification cascade, and the hazard exists only when a demand occurs — a system processing no assertions presents no hallucination risk. The demand-mode analogy is therefore the appropriate one; a per-hour metric would conflate throughput with risk (the same analysis run twice does not double the per-assertion error probability).

Two caveats are mandatory and are stated explicitly:

  1. Numerical target only. SIL classification under IEC 61508 additionally requires systematic capability (SC) assessment, architectural constraints, and safety-lifecycle evidence. This paper claims equivalence to the numerical probability target, not IEC 61508 certification. We denote this "SIL n-equivalent (demand-mode numerical target)".
  2. Demand-rate disclosure. Because the system executes many demands per hour, cumulative expected failures scale with volume: at 10⁶ assertions, the worst-case expectation is ≈3.2 undetected hallucinations. This is disclosed rather than obscured; it is the correct way to reason about per-demand bounds at scale.

7.2 IEC 61508 demand-mode comparison (corrected)

SIL PFDavg band This system
SIL 1 10⁻² – 10⁻¹ Exceeded by every tier individually
SIL 2 10⁻³ – 10⁻² Exceeded by ≥300× (worst case)
SIL 3 10⁻⁴ – 10⁻³ Exceeded by ≥31× (worst case)
SIL 4 10⁻⁵ – 10⁻⁴ Worst-case bound 3.23 × 10⁻⁶ meets the SIL 4 target (PFD < 10⁻⁴) with ≈31× margin and lies below the band floor (10⁻⁵); nominal bound 1.69 × 10⁻⁷ exceeds the target by ≈592× and lies ≈59× below the band floor

Values below the SIL 4 band floor (10⁻⁵) exceed the resolution of the SIL scale; further comparison proceeds via DO-178C.

7.3 DO-178C comparison (per-flight-hour vs. per-assertion analogy; units differ and are not directly commensurable)

DAL Target This system (per assertion)
A (Catastrophic) 10⁻⁹ /h Raw model 7.7 × 10⁻²⁰ exceeds; claimed bounds do not reach 10⁻⁹
B (Hazardous) 10⁻⁷ /h Nominal bound 1.69 × 10⁻⁷ ≈ DAL B territory
C (Major) 10⁻⁵ /h Worst-case 3.23 × 10⁻⁶ exceeds DAL C

7.4 Comparison with single-layer systems

Architecture Typical hallucination rate Improvement vs. baseline
Raw clinical LLM 15–25% baseline
RAG-augmented 5–10% 2–5×
RAG + single fact-check 2–5% 5–12×
RAG + multi-source verification 0.5–2% 12–50×
This architecture (worst case) 3.23 × 10⁻⁴ % 46,000–78,000×
This architecture (nominal) 1.69 × 10⁻⁵ % 890,000–1,480,000×

8. Sensitivity and Degraded-Mode Analysis

Birnbaum importance ranking identifies M₇ (Uncertainty Failsafe), B₆ (Case Complexity Assessor), B₅ (Input Sanitizer), L₁ (Source Provenance), and L₈ (Cross-Source Triangulation) as the highest-leverage barriers.

Degraded scenario Impact on P(TE) Maintained level
Single external API down ×2–5 ≥ SIL 3-equivalent target
All external APIs down ×33 ≥ SIL 2-equivalent target
All APIs down + degraded LLM quality ~10⁻³ Graceful degradation; SIL 1-equivalent floor

The deterministic Tier I barriers and adversarial Tier III validation operate independently of external services, guaranteeing the degradation floor.


9. Dynamic Calibration: Reputation Economy and Evidence Authority

Agent influence is weighted by historical reliability (recency-weighted), domain-specific accuracy, and experience, clamped to configurable bounds; progressive penalties isolate unreliable agents; longitudinal outcome divergence retroactively adjusts influence — a self-correcting loop that tightens barrier calibration over operational lifetime. Evidence sources carry tiered authority (institutional / corroborated / uncorroborated); uncorroborated sources are mathematically incapable of driving conclusions regardless of quantity (structural defense against PoisonedRAG-class evidence poisoning [5]); HIGH-confidence assertions require a quorum of ≥2 independent Tier-1 sources.


10. Theoretical Limits and Open Problems

No finite verification system achieves P(TE) = 0; residual risk is bounded below by novel-knowledge contradictions, uncharacterized zero-day hallucination modes, and residual correlated model failure. The Continuous Red Team implements monotone improvement, P(TE, t+1) ≤ P(TE, t), by retrospectively attacking historical outputs; the Unknown Unknowns Detector tightens all downstream thresholds when meta-cognitive signals (novel case pattern, bimodal confidence, evidence gaps, temporal inconsistency) indicate elevated uncertainty.


11. Conclusion and the Gomola Framework

Metric Value
Independent barriers 23 (7 + 9 + 7)
Raw P(TE) (theoretical) 7.689 × 10⁻²⁰
CCF-adjusted P(TE) 1.6875 × 10⁻⁷
Worst-case operational P(TE) 3.23 × 10⁻⁶
IEC 61508 equivalence (demand-mode target) SIL 4 target met with ≈31× margin (worst case)
Empirical NEJM CPC (N=301) frozen baseline 80.0%/88.0%; calibrated (dev-cohort) top-1 86.0% · top-3 93.7%; missed-critical 0.0%; median 14.3 s
Improvement vs. raw LLM 46,000× – 1,480,000×
Degradation floor SIL 1-equivalent (all external services down)

The Gomola Framework — proposed certification levels for clinical AI safety:

Level P(hallucination) requirement Min. independent barriers Min. tiers
GF-Bronze < 10⁻³ 5 2
GF-Silver < 10⁻⁵ 10 2
GF-Gold < 10⁻⁷ 15 3
GF-Platinum < 10⁻⁹ 20+ 3 + Uncertainty Failsafe

Certified conservatively, the reference implementation achieves GF-Silver under worst-case assumptions (3.23 × 10⁻⁶ < 10⁻⁵). Under nominal conditions the CCF-adjusted bound (1.6875 × 10⁻⁷ at the deliberately conservative β = 0.1) lies within a factor of 1.7 of the GF-Gold threshold; GF-Gold is met for any β ≤ 0.059 — inside the 0.01–0.05 range that IEC 61508-6 Table B.5 assigns to genuinely diverse-redundancy systems. We therefore report the implementation as GF-Silver (certified, worst-case) / GF-Gold (nominal, under standards-typical β), and we deliberately do not claim GF-Platinum on the basis of the raw multiplicative model — a standard that flatters its own author would not be a standard. (Note that the framework's per-assertion thresholds saturate the IEC SIL scale above GF-Silver.) The framework is open and royalty-free; the only requirement is academic attribution (Gomoła, 2026). The reference implementation (DeepSensi Medical OS) is proprietary and commercially licensed; framework and implementation are legally and technically independent.

Regulatory implications. The FTA methodology provides regulators (FDA, EMA, MHRA) a quantitative alternative to qualitative review: formal FTA submission with per-barrier justification; minimum SIL-equivalent targets per deployment context; degraded-mode analysis; continuous barrier monitoring; and certification of the uncertainty failsafe ("verified ability to declare I-don't-know"). The author is commencing dialogue with the FDA under the Q-Submission pathway on this basis.

Future work. (1) Frozen-configuration confirmatory study on a held-out prospective CPC cohort; (2) continued empirical per-barrier calibration from production telemetry; (3) Bayesian real-time recalculation of P(TE); (4) formal verification of barrier interaction logic (CTL/LTL model checking); (5) application of the framework to competing systems; (6) IEEE/ISO standardization; (7) establishment of an independent GF certification body.


References

  1. IEC 61025:2006 — Fault Tree Analysis (FTA). International Electrotechnical Commission.
  2. IEC 61508:2010 — Functional Safety of E/E/PE Safety-Related Systems. IEC.
  3. IAEA Safety Standards SSR-2/1 (Rev. 1) — Safety of Nuclear Power Plants: Design. IAEA, 2016.
  4. RTCA DO-178C — Software Considerations in Airborne Systems and Equipment Certification. 2012.
  5. Zou, W. et al. (2024). PoisonedRAG: Knowledge Corruption Attacks on Retrieval-Augmented Generation of Large Language Models. arXiv:2402.07867.
  6. Ji, Z. et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys 55(12).
  7. Singhal, K. et al. (2023). Large language models encode clinical knowledge. Nature 620:172–180.
  8. Vesely, W.E. et al. (1981). Fault Tree Handbook. NUREG-0492, U.S. NRC.
  9. Fleming, K.N. (1975). A Reliability Model for Common Mode Failures in Redundant Safety Systems. Proc. 6th Pittsburgh Conf. on Modeling and Simulation.
  10. IEC 61508-6:2010, Annex B — Common Cause Failure quantification, β-factor model.
  11. Gomoła, T.J. (2026). The Gomola Framework: A Quantitative Safety Certification Standard for Clinical AI Systems. DSS-001. DeepSensi Medical OS.
  12. Gomoła, T.J. (2026). The Flawed Yardstick: Why Static Medical Benchmarks Penalize Clinical Safety. Technical Whitepaper WP-002. DeepSensi Medical OS.
  13. Arora, R.K. et al. (2025). HealthBench: Evaluating Large Language Models Towards Improved Human Health. arXiv:2505.08775.
  14. Rao, A.S. et al. (2026). Large Language Model Performance and Clinical Reasoning Tasks. JAMA Network Open. doi:10.1001/jamanetworkopen.2026.4003.

Copyright © 2026 Tomasz Jan Gomoła. The Gomola Framework (GF-Bronze…GF-Platinum) is an open, royalty-free certification standard. The reference implementation (DeepSensi Medical OS) is proprietary. This document constitutes prior art and the foundational publication for the Gomola Framework.