Epistemic Fingerprints · Research Program · Pilot 01
Do AI ensembles share a blind spot?
Multiple AI instances are increasingly consulted as if they were independent expert opinions. We tested whether that trust is earned — on real, previously investigated cases with a true, non-obvious cause.
The Program
False epistemic redundancy
It is becoming common to query several differently-configured instances of the same AI model and treat their combined output as carrying the weight of independent judgment — cross-checking a diagnosis, auditing an engineering conclusion, triaging a security incident with a panel of AI reviewers instead of one. Epistemic Fingerprints is a research program built around one question: is that trust earned? Do nominally independent AI agents actually explore different parts of a problem's hypothesis space, or does the panel just restate a shared, narrow search in different wording?
We call the risk false epistemic redundancy, by analogy to common-mode failure in redundant engineered systems — a backup only protects you if it can fail for a different reason than the system it backs up. This page reports the program's first pilot: does persona conditioning alone move a single model's search toward the less obvious “tail” of a hypothesis space, where rare but consequential explanations live?
Pilot 01 · Headline Result
Given only the facts at the scene
A language model, prompted as a neutral baseline and separately under four reasoning personas, tried to explain three real cases whose true cause was non-obvious — and two easier cases as controls.
Both hard cases passed a capability control — the model rated the true cause plausible once it was simply named — so this is a search failure, not a recognition failure.
The Case File
Five cases, two domains of safety investigation, one of clinical medicine
| Case | Established explanation | Why non-obvious |
|---|---|---|
Aviation hard case |
Overweight departure combined with ice-protection workload degrading airspeed monitoring | Multifactor cause, not implied by the scene alone |
Industrial fire hard case |
Corroded fire-suppression sprinkler system contacted stored oxidizers | Requires inverting a safety system's causal role |
Newborn illness hard case |
Congenital rubella syndrome | One specific pathogen within a broad infection family |
Jet engine failure control |
Bird ingestion | — direct evidentiary signature |
Foodborne illness control |
Botulinum toxin, home-canned food | — direct evidentiary signature |
Pilot 01 · Central Finding
Given the complete file investigators used
We supplied the exact records human investigators used to solve each hard case — embedded among equally detailed irrelevant records, never phrased as a conclusion. Recovery stopped being uniformly zero. It became case-dependent, in opposite directions.
| Case | Baseline | Personas |
|---|---|---|
| Aviation | 0 / 10 | 0 / 10 |
| Industrial fire | 0 / 10 | 4 / 10 |
| Newborn illness | 4 / 10 | 0 / 10 |
Personas recovered the industrial cause where baseline could not. Baseline recovered the medical cause where personas could not. Aviation stayed at zero either way. If personas only restyled a fixed underlying search, all three cases would move together — they don't.
Exhibits
What the numbers look like
Zero recovery across every hard case under scene-level evidence; a case-dependent picture once the complete file is supplied; full recovery on both controls regardless of condition.
Persona conditioning reliably raised semantic diversity — in seven of eight case-by-evidence comparisons — regardless of whether it helped, hurt, or made no difference to actual recovery. High-diversity points sit at both 0% and 100% recovery.
Pooled across all 160 generations: confidence carries a real but limited signal (AUC 0.72). A threshold-based escalation policy catches most wrong answers, but still lets roughly one in four through at high reported confidence.
Findings
Three things this pilot suggests
Personas change substantive search, not just wording
Industrial recovery improved under personas, medical recovery worsened, aviation stayed unchanged — against identical evidence. A framing that only restyled a fixed answer would move every case the same way.
Useful diversity looks case-dependent
The industrial case needed synthesizing scattered facts into a mechanism — a persona built to trace causal chains supplied exactly that. The medical case was a well-learned diagnostic pattern a neutral prompt already retrieved by name — and personas built to avoid convergent retrieval worked against it.
Agreement isn't evidence of coverage
Multiple AI agents can disagree in wording, mechanism, and confidence while still sharing the one omission that mattered most — the common-mode failure a redundant system is supposed to prevent. If AI consultation substitutes for expert access entirely, that shared blind spot becomes the deployed system's blind spot.
Where This Goes Next
The research program beyond this pilot
Three cases and one model establish a signature worth pursuing at scale, not a general law. This is where Epistemic Fingerprints goes from here.
Replication & scale
More real cases, more domains, larger samples per condition, additional model families — testing whether case-dependence generalizes beyond these three cases.
What produces real redundancy
Repeated sampling, brainstorming instructions, accumulated interaction histories, different foundation models, and fine-tuned — rather than prompted — persona weights as candidate mechanisms.
Characterizing blind spots
Do shared omissions cluster around multifactor explanations, causal-role inversions, or evidence requiring synthesis across distant records?
Interactive investigation
Letting a model request a specific record, revise its hypothesis, and follow its own correctly-directed questions, rather than being scored on one unrevisable guess.
Better epistemic metrics
Which plausible hypothesis families no agent ever proposed, marginal hypothesis coverage, and benchmarking diversity against an external reference distribution.
Human–AI systems
Humans alone, AI alone, humans with homogeneous AI assistance, humans with heterogeneous AI assistance — does AI expand collective epistemic coverage, or concentrate it?
Governance
If shared blind spots are a general property of single-model ensembles, high-stakes deployments may need standards for epistemic independence — analogous to audit and redundancy requirements elsewhere.