Shrink Solves Murder - REVIEW

by Philippa Perry

When a body is found near Beachy Head, the police chalk it up to suicide — a tragic but not uncommon end in these parts. But local psychotherapist Patricia Phillips isn’t convinced. The victim? Her three o’clock patient, Henry Clayton. The cause of death is supposedly self-inflicted. Yet Pat can’t shake the belief that someone wanted Henry Clayton dead. She spends her working life listening to histories and secrets, and she has a nose for when a story doesn’t quite ring true. Drawn from the therapy room to the crime scene, Pat begins to notice what others appear to overlook. At her side is her best friend Prichard — a home-brewer of fearsome, stomach-turning concoctions, an excellent cook, and a man who seems to get along with everyone. Which makes him useful for infiltrating village life. As Pat and Prichard look beneath the village’s thin veneer of normality — one that barely conceals its appetites — they discover a killer hiding in plain sight.

Shrink Solves Murder is a warm, witty, and perceptive crime caper from the nation’s favorite therapist.

Penguin Random House
ISBN-13: 978-1529155327

#PenguinRandomHouse

#ShrinkSolvesMurder

What is RAG ?

What Is Retrieval-Augmented Generation (RAG), and Is It a Groundbreaking Way of Interacting with Historical Documents?

by Alvin Blackshear | Historian & Researcher

For as long as archives have existed, the historian's craft has depended on a single, stubborn skill: finding the right document among thousands of wrong ones. Card catalogs gave way to keyword search engines, which gave way to digitized finding aids, yet the underlying task never changed. A researcher formulates a question, searches a collection using the vocabulary available at the time, and hopes the terms they chose overlap with the terms an archivist or a scanner happened to preserve. Retrieval-Augmented Generation, or RAG, is the first technology in this long lineage that promises to close the gap between what a historian actually wants to know and what a search box is capable of returning. Whether it fully delivers on that promise, and under what conditions, is the question this article sets out to examine.

Why Historians Need RAG Rather Than a Generic Chatbot

It is tempting to assume that any large language model, given enough context, can answer historical questions competently. This assumption is mistaken, and understanding why is essential to understanding what RAG actually contributes.

A generic LLM answers from its training data, a fixed and often outdated snapshot of text scraped from the internet. It has no reliable way to point to a specific archival folder, newspaper issue, or census page as the source of its claim, and it has every incentive, structurally speaking, to produce a fluent and confident answer even when no such source exists in its memory. For a historian, whose entire discipline rests on the traceability of a claim back to a primary document, this is disqualifying. An answer without a citation is not history. It is a plausible-sounding guess.

RAG changes the architecture in a specific way. Instead of relying solely on what a model memorized during training, a RAG system retrieves relevant passages from an external, defined collection, be it a digitized newspaper archive, a set of scanned letters, or a database of military records, and then asks the language model to generate an answer grounded in those retrieved passages. The research group behind the paper "Improving Access to Historical Archives with Real-time RAG-based Systems," led by Stergios Konstantinidis and colleagues, demonstrates this architecture at scale, applying it to a Swiss newspaper archive of roughly half a million segments spanning nearly two and a half centuries. Their system pairs a semantic retrieval and reranking pipeline with grounded answer generation, which means the model's output is tethered to specific archival text rather than floating free of any documentary anchor. This is the essential difference that makes RAG, rather than an ordinary chatbot, the appropriate tool for archival research.

How RAG Differs from Conventional Archival Search

Traditional archival search tools, including many of the platforms libraries and museums have relied on for decades, are built on lexical matching. A researcher types a term, and the system returns documents containing that exact term or a close variant. This works reasonably well when the researcher already knows the vocabulary used by the source material, but historical vocabulary shifts constantly. Place names change. Institutional terminology evolves. Spelling was inconsistent long before anyone standardized it. A search for a term as it is understood today may simply miss a document that describes the same event using nineteenth-century phrasing.

RAG systems address this through semantic retrieval, which represents both the query and the archival text as vectors in a shared conceptual space, allowing the system to surface documents that are meaningfully related even when the wording differs substantially. A master's thesis from the University of Twente, authored by Gauri Bhatnagar and focused on the CLARIAH Media Suite, an audiovisual heritage collection, tested exactly this proposition with real humanities scholars and archival professionals. The study found that RAG genuinely helps with cross-lingual access and exploratory, interpretive queries that lexical search struggles with, but it also surfaced a critical limitation: the system performed well on factual retrieval and named-entity identification, while struggling with interpretive reasoning and the kind of complex synthesis that sits at the heart of humanities scholarship. In other words, RAG can help a historian find the needle, but it cannot yet be trusted to explain what the needle means within a broader historiographical argument.

A separate, more industry-facing piece published by Veridian Software, a company that builds digital newspaper and archive platforms, frames RAG in more practical terms for institutions considering adoption, describing it as a natural extension of the search tools archives already maintain rather than a wholesale replacement of them.

Common Failure Modes in Archival RAG Systems

No technology arrives without its own characteristic failures, and archival RAG is no exception. Several recurring problems deserve explicit attention from anyone applying these tools to historical research.

OCR errors.  Most archival text exists as scanned images that have been converted to machine-readable text through optical character recognition, a process that is notoriously imperfect on older, damaged, or low-quality print. The Konstantinidis paper directly measures this problem and reports that an LLM-based refinement step reduced character error rates by up to roughly forty-five percent and word error rates by close to sixty-one percent, compared to the raw OCR output. That is a substantial improvement, but it is not perfection, and any residual error can silently distort a retrieved passage or an answer built on top of it.

Incomplete retrieval.  Even the best semantic search will not surface every relevant document, particularly when a collection is heterogeneous or when relevant material uses unusual phrasing, an obscure genre, or a minority language. A RAG system that returns five confident-sounding sources may be quietly ignoring a sixth that contradicts them.

Provenance loss.  When a language model synthesizes multiple retrieved passages into a single fluent answer, it is easy for the boundary between one source and another to blur. A historian needs to know precisely which claim came from which document, on which page, from which collection. A system that merges sources into an undifferentiated paragraph has failed a basic archival standard even if every individual fact happens to be accurate.

Temporal confusion.  Archives frequently contain material spanning centuries, and a retrieval system optimized purely for topical similarity has no inherent understanding of chronological sequence. It is entirely possible for a RAG system to present an 1850s account and a 1950s retrospective as though they were contemporaneous, a mistake that would mislead any historian relying on the system's framing rather than checking dates independently.

Fabricated citations.  Perhaps the most consequential failure mode is the language model's tendency to generate a citation that looks correct in form, a plausible archive box number or a newspaper date, but does not correspond to any actual retrieved document. Even well-grounded RAG systems are not fully immune to this, especially when a query has no good answer in the underlying collection and the model is inclined to produce something anyway rather than admit the gap.

The Missing Piece: A Historian's Evaluation Standard

Here is where the existing research reveals a striking gap. Every technical paper reviewed for this article evaluates RAG systems using measures native to information retrieval and natural language processing: retrieval precision, recall, embedding quality, OCR accuracy, system latency, and hallucination rate. A recent survey of RAG evaluation methods by Aoran Gan catalogs this landscape comprehensively, and a companion technical guide from Toloka lays out practical metrics for groundedness and answer faithfulness. These are legitimate and necessary measures. But none of them ask the questions a professional historian actually asks when assessing whether a piece of evidence is trustworthy.

Did the system retrieve the most authoritative version of a source, or merely the most textually similar one? Did it note when retrieved evidence contradicted itself, or did it quietly favor the passage that supported a cleaner narrative? Did it preserve the chain of custody and provenance for each claim? Did it distinguish a primary account from a later secondary interpretation of that account? Did it respect the order of events rather than collapsing decades into an undifferentiated blend? Did it accurately synthesize material drawn from separate collections without conflating them? Did it acknowledge uncertainty where the evidence was genuinely ambiguous, rather than resolving that ambiguity for the sake of a tidy answer? And did it surface evidence that was difficult to find but still relevant, rather than defaulting to whatever ranked highest by similarity score?

This gap between technical evaluation and historiographical evaluation is the foundation for what can be called a Historian's AI Evaluation Framework, or HAIEF. Rather than asking whether a RAG system is efficient, HAIEF asks whether it is trustworthy by the standards the historical profession has used for generations: provenance, contextualization, chronology, corroboration, and transparency about uncertainty. A system might score extremely well on NDCG or answer-correctness metrics, the kind reported by Konstantinidis and colleagues, whose reranking pipeline improved NDCG@10 from roughly sixty-six percent to eighty-seven percent, and still fail a historian's basic sniff test if it cannot show its provenance trail or if it silently smooths over contradictory testimony.

Toward a Historian's Testing Framework

A workable HAIEF would need to translate these five values into repeatable, scorable tests. Provenance could be scored by checking whether every claim in a generated answer links back to a specific, verifiable source with collection, folder, and page-level detail. Contextualization could be scored by asking whether the system situates a document within its original archival and historical setting rather than presenting it as free-floating text. Chronology could be tested with queries that require ordering events correctly across a span of decades or centuries. Corroboration could be tested by deliberately including contradictory sources in a test collection and observing whether the system flags the disagreement or resolves it artificially. Transparency about uncertainty could be measured by presenting queries with genuinely unresolved historical questions and checking whether the system hedges appropriately rather than manufacturing false confidence.

Benchmark Collections and a Comparative Rubric

Testing such a framework requires benchmark collections that reflect the actual diversity of archival material historians work with: personal letters and correspondence, census and vital records, digitized newspapers, military service and pension records, oral history transcripts, and photograph collections with accompanying metadata. Each of these genres carries its own provenance conventions and its own failure risks, and a system that performs well on newspapers may perform poorly on oral histories, where nuance, tone, and interviewer influence complicate straightforward retrieval.

A repeatable scoring rubric applying HAIEF criteria could then be used to compare general-purpose assistants such as ChatGPT, Claude, and Gemini against dedicated archival RAG systems purpose-built for a single institution's holdings. Such a comparison would likely show that general-purpose assistants excel at fluent synthesis but lag on provenance transparency, while dedicated archival systems, closer in spirit to the Konstantinidis pipeline, perform better on traceability but may still struggle with the interpretive reasoning that Bhatnagar's thesis identified as a persistent weakness across the board.

Recommendations for Archives Building AI-Assisted Discovery Tools

Institutions developing these tools should treat historiographical evaluation as a first-class requirement rather than an afterthought bolted on after a system already works technically. This means involving historians and archivists directly in system design, not merely as end-user testers after launch. It means building citation and provenance display into the interface itself, so that a user can see, at a glance, exactly which archival item supports each claim. It means deliberately including contradictory or ambiguous material in test collections so that a system's handling of disagreement can be assessed before deployment, rather than discovered by an unlucky researcher months later. And it means resisting the temptation to optimize purely for fluent, confident-sounding answers, since fluency and historical reliability are not the same thing and can, in the worst cases, actively work against one another.

Conclusion

RAG is not a gimmick, and the evidence gathered here, from a large-scale newspaper archive study to a humanities-focused thesis to a growing body of evaluation literature, makes clear that it represents a genuine advance over keyword search for anyone working with digitized historical material. But calling it groundbreaking requires a qualification. It is groundbreaking as a retrieval technology. It is not yet groundbreaking as a historiographical instrument, because none of the existing research has built evaluation standards around the values historians actually rely on to judge evidence. Closing that gap, through a framework like HAIEF, is not a minor refinement. It is the difference between a tool that finds documents quickly and a tool that a historian can actually trust.

____________________________ 

Sources:

Bhatnagar, Gauri. "Enhancing Multimodal Archival Search and Discovery in the Sound and Vision Archive Using Large Language Models and Retrieval-Augmented Generation." Master's thesis, University of Twente, August 2025. https://purl.utwente.nl/essays/108957.

Gan, Aoran. "Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey." arXiv, April 21, 2025. https://doi.org/10.48550/arXiv.2504.14891.

Jancovic, Marek. "RAG for Historians: A Radically New Way of Interacting with Historical Documents." LinkedIn, March 29, 2026. https://www.linkedin.com/pulse/rag-historians-radically-new-way-interacting-marek-jancovic-ew3xe/.

Konstantinidis, Stergios, Hayman Lotfy, Alexis Erne, Faruk Zahiragic, Min-Yen Kan, and Michalis Vlachos. "Improving Access to Historical Archives with Real-time RAG-based Systems." arXiv, July 3, 2026. https://arxiv.org/html/2607.03440v1.

Toloka. "RAG Evaluation: A Technical Guide to Measuring Retrieval-Augmented Generation." Toloka Blog, August 15, 2025. https://toloka.ai/blog/rag-evaluation-a-technical-guide-to-measuring-retrieval-augmented-generation.

Veridian Software. "What Is RAG and How Could It Support Digital Collection Search?" Veridian Software Knowledge Base, June 29, 2025. https://veridiansoftware.com/knowledge-base/a-new-way-to-search-digital-collections-introducing-rag.

Why Metadata Matters in AI Systems

by Alvin Blackshear  | Historian & Researcher  <ablackshear@gmail.com>

There is a quiet assumption embedded in most conversations about artificial intelligence: that the intelligence lives in the model. Bigger parameter counts, cleverer architectures, more refined training regimes. These are the variables that dominate public discussion and marketing copy alike. Yet anyone who has built or evaluated a retrieval-augmented generation system knows that this framing leaves out something essential. A language model, however capable, is only as trustworthy as the information it is given to reason over. And the quality of that information is determined less by the words on the page than by the structure surrounding them: the metadata.

Metadata is often treated as an administrative afterthought, the digital equivalent of a filing label. This framing badly undersells what metadata actually does inside a modern AI system. It is the connective tissue that turns a pile of documents into something resembling knowledge. It carries context, provenance, chronology, and relationships. It tells a system not just what a piece of information says, but where it came from, when it was true, how much to trust it, and how it relates to everything else the system knows. Strip metadata away, and even the most sophisticated retrieval pipeline is reasoning in the dark.

Metadata Is the Context Behind the Data

Every document a system retrieves carries an implicit history. It was written by someone, at some point, for some purpose, and it exists in some relationship to other documents. Without metadata capturing that history, an AI system has no way to distinguish a peer-reviewed clinical guideline from an anonymous forum post, or a policy that was superseded last year from one still in force. Both simply become "text" to a retrieval engine, indistinguishable in weight or authority.

This is not a hypothetical failure mode. Research on metadata generation within relational data warehouse processes has shown that when systems are built to actively generate and manage metadata as part of the retrieval pipeline, rather than treating it as a static byproduct of storage, the downstream reasoning improves measurably. Framing metadata as a first-class citizen in system design, rather than an afterthought bolted onto search indexes, appears to be one of the more consistent findings across recent work in this area.

Metadata Drives Retrieval in RAG Systems

Retrieval-augmented generation (RAG) depends entirely on finding the right passage at the right moment. This sounds simple until one considers how much ambiguity lives inside ordinary language, especially around time. A query about "the current guidelines" or "recent developments" contains a temporal reference that a keyword search cannot resolve on its own. Recent work on handling fuzzy time expressions in RAG systems tackled exactly this problem, introducing temporal metadata filtering that allows a system to reason about vague date references rather than ignoring them. The reported result was notable: a 15.7 percent improvement in retrieval hit rate when temporal metadata was incorporated into the filtering process. That is not a marginal gain. It represents the difference between a system that reliably surfaces the correct document and one that gets there by chance roughly one in six fewer times.

This finding deserves attention because it reframes what "better retrieval" actually means. Much of the public conversation about improving RAG systems centers on embedding quality or chunking strategy. Those matter, but the temporal robustness research suggests that metadata-aware filtering can produce gains of a similar order of magnitude, often at a fraction of the computational cost of retraining or fine-tuning an embedding model.

Metadata Improves AI Accuracy and Reduces Hallucination

Hallucination remains the most persistent trust problem in generative AI, and much of the discourse around it treats the phenomenon as a model-level flaw to be solved through better training. But a meaningful portion of hallucination in RAG systems is a retrieval problem wearing a generation costume. If a system retrieves irrelevant, outdated, or low-authority material and passes it to the language model as context, the model will do exactly what it is designed to do: synthesize a fluent answer from what it was given. The result looks like a hallucination, but its root cause is upstream.

Work on reliable retrieval-augmented feature generation has examined precisely this relationship, showing that the reliability of downstream reasoning is closely tied to the quality of what gets retrieved in the first place. This is a useful corrective to any evaluation strategy that only scores final outputs. If a benchmark measures answer accuracy without also scoring retrieval precision and metadata fidelity, it risks rewarding systems that are lucky rather than sound, and it makes diagnosing failure far harder than it needs to be.

Provenance and Source Trustworthiness

Few domains illustrate the stakes of provenance more clearly than clinical decision support. A framework for auditable, source-verified clinical AI, integrating retrieval-augmented generation with explicit data provenance, was designed around a simple but demanding principle: every recommendation the system makes should be traceable back to a verifiable source, with that source's reliability made visible rather than assumed. This is provenance metadata doing real work, not as a compliance checkbox but as a functional requirement for a system operating in a high-stakes domain.

The broader lesson generalizes well beyond medicine. Any AI system that produces recommendations, summaries, or answers benefits from being able to answer the question "where did this come from" with something more specific than "the training data" or "a document somewhere in the index." Provenance metadata is what makes that specificity possible.

Temporal Reasoning: Knowing When Something Was True

Facts have expiration dates, even when the sentences describing them do not visibly change. A statement that was accurate in 2022 may be false today, and a document that has been formally superseded may still sit in a retrieval index looking exactly as authoritative as its replacement. The fuzzy time expression research referenced earlier addresses this directly, and it is worth returning to because temporal reasoning touches nearly every other capability discussed here. Explainability depends on knowing when a claim was made. Trust depends on knowing whether a claim still holds. Citation depends on being able to tell a user not just what a source says but whether it remains current. Metadata that encodes validity windows, supersession relationships, and publication dates gives a system the raw material to reason about all of this rather than presenting every retrieved passage as equally timeless.

Metadata Enables Explainable AI

Explainability is frequently discussed as though it were purely a matter of model interpretability: attention visualizations, feature attributions, and the like. But for retrieval-augmented systems, a more practical and arguably more useful form of explainability comes from metadata itself. If a system can show which document it drew from, who authored it, when it was published, and how authoritative that source is judged to be, it has already given a user most of what they need to evaluate the answer's credibility. Research introducing governance-driven agentic retrieval chains has pushed this idea further, proposing explicit evidence ledgers that log authority scores, temporal validity, and conflict resolution decisions across a multi-step retrieval process. This kind of ledger transforms explainability from a static disclosure into an auditable trail, which is a meaningfully different and more rigorous standard.

Metadata Supports Citation Generation

Citation is where metadata's value becomes most visible to an end user. A system that can point to the exact source of a claim, along with its date, authorship, and context, is offering something categorically different from a system that simply asserts a fact. Work applying retrieval-augmented generation to art provenance research within a major cultural heritage index demonstrates this well. In a domain like art history, where the chain of ownership and attribution is often the entire substance of the inquiry, metadata is not a supporting feature. It is the subject matter itself. The success of that research in surfacing accurate, explainable provenance chains for historical artworks offers a useful proof of concept for citation generation in far less specialized domains.

Metadata Helps AI Understand Document Relationships

Individual documents rarely stand alone. They respond to one another, build on one another, and sometimes contradict one another. Research on adaptive information management for retrieval-augmented generation has explored how systems can maintain coherent reasoning across multiple retrieval steps by tracking these relationships in working memory rather than treating each retrieval as an isolated event. This has particular relevance for historical research, where understanding how one document relates to, revises, or is revised by another is often the actual analytical task. Metadata that captures these relational links allows an AI system to support that kind of layered inquiry rather than flattening it into a series of disconnected facts.

Metadata Enables Better AI Evaluation

Evaluation of AI systems has tended to focus heavily on final-answer accuracy, but a growing body of work argues for scoring the retrieval and reasoning process itself. Research on self-evaluation driven strategy optimization in agentic retrieval introduced a self-assessment step in which the system evaluates the quality of retrieved evidence before generating a final answer. This kind of built-in scoring rubric, applied to metadata quality and source reliability rather than only to output fluency, represents a more rigorous approach to benchmarking. It suggests that future evaluation standards for RAG systems should include explicit metrics for provenance completeness, temporal accuracy, and citation traceability, not only for the correctness of the final generated text.

Metadata Improves Security and Governance

Metadata also does quiet but essential work in access control. Permission-aware retrieval, in which a system respects who is allowed to see which documents based on classification, ownership, or sensitivity metadata, is what makes AI systems viable in enterprise and regulated environments. The governance-driven framework mentioned earlier extends this idea by building authority scoring and audit trails directly into the retrieval chain, treating governance not as a separate layer bolted on top of the system but as something woven through its metadata architecture from the start.

The Future: Metadata-Aware AI Agents

Looking ahead, the most interesting frontier is not a smarter language model but a more metadata-literate one. Future agentic systems will likely reason over metadata with the same deliberateness they currently apply to text: weighing provenance, tracking temporal validity, assigning confidence scores, generating citations automatically, integrating with knowledge graphs, handling multimodal metadata across images, audio, and video, and preserving chain of custody across long, multi-step workflows. This shift would move AI systems away from merely producing plausible-sounding text and toward producing answers that are transparent, traceable, and genuinely defensible under scrutiny.

That is a meaningful distinction, and it is worth sitting with. Plausibility and defensibility are not the same standard. A fluent answer can be plausible and still wrong. A defensible answer, grounded in well-structured metadata, gives a user the means to check it. As AI systems take on more consequential tasks, from clinical support to historical research to enterprise decision-making, that difference is likely to matter more, not less. The path toward more trustworthy AI runs directly through the unglamorous, essential work of getting metadata right.

---

Sources

Chen, L.-C., Chen, H.-W., & Chen, M.-S. (2026). DeFuzzRAG: Handling Fuzzy Time Expressions for Temporal Robustness in Retrieval-Augmented Generation. Proceedings of AAAI-26. https://ojs.aaai.org/index.php/AAAI/article/view/40276

An Auditable and Source-Verified Framework for Clinical AI Decision Support: Integrating Retrieval-Augmented Generation with Data Provenance. Frontiers in Artificial Intelligence (2026). https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2026.1737532/full

Reliable Retrieval-Augmented Feature Generation with Large Language Model Reasoning. Knowledge and Information Systems (Springer), 2026. https://link.springer.com/article/10.1007/s10115-026-02792-4

A ReAct- and RAG-Based Framework for Metadata Generation and Access in Relational Data Warehouse Processes. Big Data and Cognitive Computing (2026). https://doi.org/10.3390/bdcc10060172

LedgerRAG: Governance-Driven Agentic Chain of Retrieval for Dynamic Knowledge Scenarios. Electronics (MDPI), 2026. https://www.mdpi.com/2079-9292/15/7/1376

Reasoning with Memory: Adaptive Information Management for Retrieval-Augmented Generation. Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.1834

Reflective RAG: Self-Evaluation Driven Strategy Optimization in Agentic Retrieval-Augmented Generation. Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.648

Retrieval-Augmented Generation for Natural Language Art Provenance Searches in the Getty Provenance Index (2026). https://eprints.whiterose.ac.uk/id/eprint/238115

I Asked ChatGPT to Find My Professor Profile. It Failed. Here's Why It Matters.

by Alvin Blackshear  |  Historian & Researcher   <ablackshear@gmail.com>

Why Historians Should Never Trust "I Couldn't Find It."

I asked ChatGPT to find my profile on Rate My Professor. It told me, with no hedging at all, that it could not find a professor profile under that name. That answer was wrong. When I added the name of the college where I taught, the model found my profile immediately, in the same database it had just searched a moment before.

When I asked why its first answer had been incorrect, ChatGPT explained that the initial search had been too narrow and that it had overstated the result. It went further, telling me that it had not been appropriate to state definitively that no profile existed. That admission is more interesting than the original mistake. It names, almost exactly, the failure that researchers of every kind run into constantly: absence of evidence quietly becomes evidence of absence, and a system built to retrieve information ends up asserting a negative it has no real basis for.

This is a small experiment. But small experiments are often where large problems become visible. What follows is an attempt to take this one seriously, not as a complaint about a chatbot, but as a case study in how retrieval systems fail, why those failures are harder to catch than ordinary hallucinations, and what a better evaluation standard would look like.

The Failure Was Not a Hallucination

It is worth being precise about what did not happen here. The model did not invent a fake Rate My Professor page. It did not fabricate reviews, ratings, or a biography. Hallucination, in the sense the AI research community usually means it, involves generating content that has no basis in the retrieved evidence or in reality. That is not what happened in this case.

Instead, the model committed a quieter and, in some ways, more dangerous error. It ran an incomplete retrieval, treated the incomplete result as sufficient, and then converted that insufficient evidence into a categorical claim. Four distinct failures stacked on top of each other:

1. The retrieval step was incomplete, since the search used only a name and not the additional identifying detail (the institution) that would have disambiguated the result.
2. The evidence gathered was insufficient to support any firm conclusion, positive or negative.
3. The model's confidence should have been low, given how thin the search actually was.
4. The final answer failed to reflect that low confidence, presenting a tentative non-result as though it were a settled fact.

Hallucinations tend to draw attention because they are often vivid and easy to catch. A fabricated citation or an invented quote can be checked against a source and shown to be false. This kind of error is harder to catch precisely because it looks like careful, responsible behavior. The model appeared to be reporting a fact rather than making one up. It sounded modest. It just was not modest enough about the right thing.

Why This Matters

Strip the incident down to its underlying logic and it reads like this: I searched. I did not find it. Therefore it does not exist. Stated plainly, that reasoning is invalid, and most people would recognize the problem immediately if it were presented as a syllogism in a logic class. Yet this is exactly the shortcut that retrieval systems take when they are not explicitly designed to resist it.

Anyone who has done serious research recognizes this pattern because it is the default condition of research itself. Archives are incomplete. Metadata is inconsistent. Names appear under variant spellings, married names, initials, or transliterations. Sources are catalogued under headings that make sense to an archivist but not to a later researcher. Databases frequently require an additional piece of context, an institution, a date range, a middle name, before they will surface a record that has been sitting there the entire time.

None of this is exotic. It is the ordinary condition of information work. The question is not whether an AI system will encounter it. The question is whether the system is built to recognize when it has hit this wall, or whether it will simply report the wall as an empty room.

Historians Have Solved This Problem for Centuries

There is a useful comparison to be made with historical methodology, which has spent a very long time developing habits of mind for exactly this situation. A historian who cannot find a record does not usually conclude that the record never existed. Instead, the working assumption is procedural: did I search enough archives, are there alternate spellings of this name, did the person's name change through marriage or naturalization, is this source catalogued under a different heading, is there another repository that might hold a more complete record.

The distinction that matters here is between two very different sentences. "It doesn't exist" is a conclusion about the world. "I haven't found evidence yet" is a conclusion about the search. Good historians are trained, often through painful experience, to default to the second sentence and to treat the first as a claim that requires an extraordinary amount of additional work before it can be responsibly made.

This is not a minor stylistic habit. It is the entire discipline's defense against a kind of error that predates computers by centuries: mistaking the limits of one's own search for the limits of what actually happened.

What Actually Failed

Returning to the four evaluation points named earlier, each deserves a closer look from an evaluator's perspective rather than a user's perspective.

Retrieval was incomplete because the search relied on a single identifier, a name, in a database where names alone are frequently ambiguous or thin. A single search pass is rarely sufficient in any system where entities can share names, use variant spellings, or be indexed under partial records.

Evidence was insufficient because a single unsuccessful query does not constitute a thorough investigation. One null result from one search string tells a system almost nothing about whether a record exists elsewhere in the same database under a slightly different query.

Confidence should have been low because the retrieval process itself provided no strong signal either way. Low retrieval coverage should map to low confidence in any well calibrated system, and a wide gap between actual coverage and stated confidence is itself a measurable defect.

The final answer should have expressed that uncertainty rather than erasing it. The correct output was never "no profile exists." The correct output was closer to "I could not locate one with the search I ran, and here is what might help me look further."

How a Better AI Should Respond

It is useful to imagine the same interaction handled well. Instead of stating that there is no professor profile, a more careful system might have said something like this: I could not locate one using your name alone. If you can tell me the institution where you taught, I can search more precisely.

Notice what that sentence does. It preserves uncertainty instead of erasing it. It invites the user to supply the missing retrieval key rather than closing the inquiry. It treats the null result as information about the search process, not information about the world. This is a small rewording, but it represents a genuinely different epistemic posture, one that treats retrieval as provisional until proven otherwise.

Why This Matters for RAG Systems

This incident connects directly to a broader question about how Retrieval-Augmented Generation (RAG) systems are built and evaluated. RAG systems are only as good as the retrieval pipeline underneath them, and that pipeline depends on several interacting components: retrieval quality, metadata structure, search strategy, query expansion, ranking, and the underlying coverage of the source material itself.

Recent research on RAG evaluation has tried to formalize exactly this dependency. A comprehensive review of RAG evaluation dimensions by Knollmeyer and colleagues works through how retrieval quality, question types, and evaluation metrics interact, and it makes a case that evaluation frameworks need to look separately at retrieval performance and generation performance rather than treating a RAG system's output as a single undifferentiated block. That separation matters here, because the language model itself did not fail at composition or fluency. The retrieval layer beneath it simply stopped searching too early, and the generation layer then dressed that shortfall up as a confident answer.

Work on uncertainty-aware retrieval reinforces the same point from a different angle. Research on uncertainty-aware dynamic retrieval argues that retrieval decisions should be driven by the model's own uncertainty rather than by fixed, one-shot retrieval rules, so that a system recognizes when its confidence is too low to stop searching. Adaptive multi-source retrieval frameworks make a related argument, showing that incorporating query complexity and confidence-aware fusion allows a system to keep pulling in additional sources precisely in the situations where a single retrieval key, like a name without an institution, is not enough to settle the question. Hypothesis-and-verification approaches to retrieval push this further still, treating an initial null result not as a final answer but as a hypothesis to be tested against additional evidence before any conclusion is reported. Self-evaluation approaches to agentic retrieval add another layer, having the system review its own retrieval strategy before committing to a final answer, which is exactly the internal check that would have caught this failure before it reached the user.

None of this is exotic academic theorizing removed from practical stakes. It describes, almost mechanically, what would have needed to happen for ChatGPT to say something more honest than "no profile exists."

An AI Evaluation Rubric

It is possible to turn this single incident into a reusable evaluation checklist, one that applies whenever an AI system reports that something cannot be found. Work on faithfulness-aware uncertainty quantification for fact-checking RAG outputs provides useful scaffolding here, since it separates the question of whether an answer is faithful to its retrieved evidence from the separate question of whether that evidence is factually sufficient in the first place. Applied to a "not found" claim, the following questions become a practical audit:

Were multiple search strategies attempted, or did the system stop after a single query. Were alternate identifiers used, such as an institution, a date, or a variant spelling. Was metadata expanded to account for inconsistent cataloguing. Was uncertainty reported explicitly, rather than folded silently into a flat statement. Did the model distinguish between "not found" and "does not exist." Could one additional retrieval key plausibly change the result.

That last question turned out to be the entire story in this case. One additional identifier, the name of the institution, was all it took to convert a false negative into a correct answer. A system that had asked itself that question before answering would have caught its own mistake.

Implications Beyond Historians

This failure mode is not confined to historians looking up their own teaching profiles. It shows up wherever retrieval sits underneath a serious decision. Legal researchers checking whether a precedent exists face the same risk of concluding "no such case" from an incomplete search of case law. Genealogists working with inconsistent historical spellings run into it constantly. Medical literature searches can miss relevant studies indexed under different terminology, with consequences far more serious than a mistaken professor lookup. Journalists checking whether a claim has prior reporting, researchers conducting literature reviews, and enterprise knowledge bases used inside companies all depend on retrieval systems that can quietly convert "I didn't find it in my index" into "it isn't true." A recent Nature article on synthesizing scientific literature with retrieval-augmented language models makes a related point in the scholarly context specifically, arguing that trustworthy synthesis of academic evidence depends on retrieval systems that are transparent about their own coverage limits rather than presenting synthesized answers as though they rested on complete evidence.

In every one of these domains, the underlying mechanism is identical to what happened with a single Rate My Professor search. One missing retrieval key can completely change the answer, and a system that does not know how to say "I haven't looked hard enough yet" will eventually say something false with total confidence.

Conclusion

Return to where this started. ChatGPT eventually found my profile once I supplied the one piece of context it needed, the institution where I taught. The information had been there the entire time. The system's retrieval capability was never really in question. What failed was something quieter and, in the end, more consequential: the model treated an incomplete search as a complete answer and reported a negative conclusion before it had exhausted reasonable ways of looking.

The most important lesson here was not that ChatGPT failed to find my professor profile. It was that, for a moment, it mistook an incomplete search for a finished one. For historians, for researchers in any field, and for anyone building AI systems meant to be trusted with real questions, that distinction is not a technicality. It is the whole matter.

---

References

Asai, Akari, et al. "Synthesizing Scientific Literature with Retrieval-Augmented Language Models." Nature (2026). https://www.nature.com/articles/s41586-025-10072-4

Fadeeva, Ekaterina, et al. "Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval-Augmented Generation." Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.338

Knollmeyer, Simon, et al. "Evaluating Retrieval Augmented Generation: A Comprehensive Review of Evaluation Dimensions, Question Types, and Application." SN Computer Science 7 (2026). https://link.springer.com/article/10.1007/s42979-026-05134-x

Wu, Haiyan, et al. "Reflective RAG: Self-Evaluation Driven Strategy Optimization in Agentic Retrieval-Augmented Generation." Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.648

"AdaKG-RAG: Adaptive KG-guided Retrieval-Augmented Generation with Hypothesis-and-Verification Retrieval for Multi-hop Question Answering." Journal of King Saud University Computer and Information Sciences (2026). https://link.springer.com/article/10.1007/s44443-026-01028-3

"UA-RAG: Uncertainty-aware Dynamic Retrieval-Augmented Generation." Neurocomputing (2026). https://www.sciencedirect.com/science/article/pii/S0925231226017558

AI Fact Verification for Historians: Building a Discipline-Specific Protocol for a New Research Environment

by Alvin Blackshear | Historian & Researcher  <ablackshear@gmail.com>

Historians have always worked at the boundary between evidence and interpretation, testing claims against sources, weighing provenance, and living comfortably with uncertainty. Generative artificial intelligence has entered this terrain quickly and unevenly, offering historians an extraordinary tool for discovery while introducing new categories of risk that the discipline has not yet fully reckoned with. The question facing the field is not whether AI belongs in historical research. It already does. The real question is how historians verify what AI produces, and whether existing habits of source criticism are sufficient for a technology that can fabricate footnotes, invent quotations, and drift toward revisionist narratives when prompted to do so.

This article argues that AI can meaningfully accelerate historical research, but it cannot replace professional source criticism, and it proposes the outline of a repeatable verification protocol suited to the particular demands of archival research, primary source authentication, citation validation, chronology checks, and historiographical consistency.

The stakes of getting this right extend beyond any single article or dissertation. History as a discipline depends on a shared confidence that its claims can be traced back to something real, a document, an object, a testimony that survived. When that chain of evidence is compromised, even quietly and even with good intentions, the damage does not stay contained to one footnote. It erodes the credibility of the discipline as a whole, at a moment when public trust in expertise is already under strain. Historians who adopt AI tools without adopting equally rigorous verification habits are not simply risking their own reputations. They are testing the durability of a field that has spent centuries building methods for distinguishing memory from evidence and evidence from invention.

The Promise and the Ceiling

Large language models can summarize secondary literature, suggest starting bibliographies, identify patterns across large text corpora, and help historians locate materials they might otherwise have missed. Used well, these tools compress hours of preliminary searching into minutes. The American Historical Association's Ad Hoc Committee on Artificial Intelligence in History Education, in its 2025 guiding principles, acknowledges this potential while insisting on a firm boundary: generative AI can mimic some of the work historians do, but this should never be mistaken for the work itself. The committee's language is worth sitting with. AI produces texts, images, audio, and video, not truths. It selects words and patterns from training data rather than comprehending the past in its full complexity and contradiction. When a model encounters a gap in its training, it does not pause to acknowledge uncertainty. It fills the gap with something plausible sounding, a process the field now calls hallucination.

For historians, hallucination is not an abstract technical curiosity. It is a direct threat to the evidentiary basis of scholarship. A fabricated citation that looks correctly formatted, attached to a real-sounding archive or journal, can pass a cursory glance and enter a footnote before anyone notices the source does not exist. The AHA's guiding principles are unambiguous on this point: any reference generated by AI must be checked against the original before it is used, and a reference included in a footnote without that check is never acceptable practice, regardless of how the historian frames their working process.

Why Automated Fact-Checking Falls Short

It is tempting to assume that the solution to AI hallucination is simply more AI, in the form of automated fact-checking systems layered on top of generative tools. Recent research complicates that assumption considerably. A 2026 study led by Kelly Amaddio examined how people respond to automated fact-checkers correcting misinformation, using a sample of more than thirteen hundred participants exposed to a piece of gun control misinformation and a subsequent AI-generated correction. The findings revealed a persistent and troubling pattern: prior beliefs shaped both continued endorsement of the misinformation after correction and perceptions of the fact-checking system's reliability. When the fact-checker was described as only moderately accurate, close to the accuracy levels people already associate with existing AI systems, prior beliefs predicted whether the correction actually changed minds. Even when participants were told the system was highly accurate, the effect of prior belief did not disappear, only lessened.

This matters for historians because it demonstrates that automated verification does not operate in a neutral space. It is filtered through the same human biases that shape how any evidence is received, and it can be defeated by claims that already align with what someone wants to believe. Historical scholarship, particularly on contested and politically charged topics, is especially vulnerable to this dynamic. A fact-checking layer bolted onto a language model cannot substitute for the disciplinary training that teaches historians to interrogate their own assumptions as rigorously as they interrogate a source.

Revisionism as a Distinct Problem

Contested historical claims present a challenge that generic fact-checking frameworks were never designed to solve, because the disagreement is not always about a discrete, verifiable fact but about interpretation, framing, and the political stakes attached to memory. A 2026 study by Francesco Ortu and collaborators addresses this directly. The researchers built a benchmark called Historical Misinfo, drawing on five hundred contested historical events from forty five countries, each paired with a documented factual narrative and a documented revisionist counter-narrative. They then tested how large language models responded under neutral prompting compared with prompting that explicitly requested the revisionist version of events.

The results are instructive. Under neutral conditions, most models leaned closer to the factual reference. But when users explicitly requested a revisionist framing, every model tested showed a sharp increase in revisionist output, with little resistance or self-correction. This finding should give historians real pause. It suggests that a model's apparent reliability on a given topic says little about how that same model will behave when a user, deliberately or not, nudges it toward a distorted account. For historians working on genocide, colonialism, contested territorial claims, or any subject where competing national narratives exist, this is not a marginal concern. It is central to responsible use.

What Verification-First Research Looks Like Elsewhere

Historians are not alone in confronting this problem. Related work emerging from adjacent fields offers useful models for how disciplines might restructure their relationship with AI around verification rather than convenience. Lei You, Lele Cao, and Iryna Gurevych have argued that AI-assisted peer review in the sciences should be verification-first rather than review-mimicking, proposing that AI tools function as adversarial auditors generating checkable evidence rather than as systems that simply predict what a human reviewer might conclude. Belinda Mo extends this argument to the broader landscape of autonomous AI research agents, warning of a widening verification gap between what AI systems can produce and what any human can realistically check, and calling for verification infrastructure built around observable workflows and clear attribution.

A study by Raymond Solga and Mohammed Sarwar offers perhaps the closest parallel to the historian's own situation. Comparing traditional historical methods against several generative AI tools in identifying and validating primary sources in ancient history, the researchers found that AI performed well at broad content discovery and thematic synthesis but struggled significantly with genre boundaries, provenance transparency, and the kind of contextual interpretation that trained historians bring almost instinctively to a source. Their proposed solution, a human-in-the-loop framework built around model pluralism and provenance-first protocols, maps closely onto what historians already know from decades of archival practice, simply extended to include AI as one input among many that must be checked rather than trusted.

There is also a temporal dimension worth naming. Carlo Iacono's recent audit of AI-assisted research output found that the models cited in published work are frequently outdated by the time readers encounter them, sometimes by close to a year, with model versions superseded and behaviors shifting in ways that are rarely documented. For historians, this is a reminder that any claim about what a given AI tool can or cannot do should be dated and treated as provisional rather than as a permanent property of the technology.

Toward a Repeatable Protocol

Drawing these threads together, a workable verification protocol for historians should rest on several commitments. Every AI-generated citation must be checked against the primary source or an authoritative secondary source before it appears in any written work, with no exceptions for citations that merely look plausible. Chronology and provenance claims generated by AI require independent confirmation, since these are precisely the areas where current tools show the weakest performance. Contested or politically charged claims deserve heightened scrutiny, given the demonstrated tendency of models to drift toward whatever framing a prompt implies. Verification work should be documented as it happens, noting which tool was used, when, and what was checked, both for the historian's own accountability and because the tools themselves change quickly enough that undocumented claims about their reliability age poorly. And throughout, the historian's own expertise must precede rather than follow AI use, since evaluating a model's output well requires the same depth of subject knowledge that historians have always needed to evaluate any source.

None of this diminishes the real value AI offers historians willing to use it carefully. It does insist that the discipline's oldest habit, treating every claim as something to be verified rather than assumed, remains the right response to a genuinely new technology. Generative AI can accelerate the search for evidence. It cannot substitute for the judgment that turns evidence into history.

There is a version of this moment that history has lived through before, in smaller ways, each time a new technology promised to change how the past is studied. Microfilm, digitization, keyword-searchable databases, all reshaped the pace and reach of archival work without ever touching the core discipline of asking what a source can and cannot tell us. Generative AI is different in degree, not necessarily in kind. It compresses years of searching into minutes, but it also compresses the distance between plausible and true, and that compression is precisely where a historian's training earns its keep. The AHA's committee put it plainly: there are no shortcuts to expertise, and the skills that let a scholar recognize a hallucinated citation or a subtly revisionist framing can only be built through the same sustained engagement with sources that has always defined the craft. A historian who verifies rigorously will find in AI a genuine research partner. A historian who does not will find, sooner or later, that the past they have written no longer holds.

______________________________

Sources

American Historical Association, Ad Hoc Committee on Artificial Intelligence in History Education. "Guiding Principles for Artificial Intelligence in History Education." Approved by AHA Council, July 29, 2025. https://www.historians.org/resource/guiding-principles-for-artificial-intelligence-in-history-education/

Amaddio, Kelly M., et al. "Prior Beliefs and Automated Fact Checking: Limits on the Accuracy of AI Verification." February 5, 2026. https://doi.org/10.1371/journal.pone.0342332

Iacono, Carlo. "The Half-Lives of Generative-AI Evidence." July 27, 2026. https://doi.org/10.48550/arXiv.2607.24032

Mo, Belinda. "The Age of AI Agents Demands a New Scientific Paradigm to Sustain Trustworthy Science." June 18, 2026. https://doi.org/10.48550/arXiv.2607.26064

Ortu, Francesco, et al. "Preserving Historical Truth: Detecting Historical Revisionism in Large Language Models." February 22, 2026. https://doi.org/10.48550/arXiv.2602.17433

Solga, Raymond S., and Mohammed J. Sarwar. "Evaluating Generative AI in Historical Research." AI & Antiquity (2026). https://doi.org/10.64946/aiantiquity.v2i1.003

You, Lei, Lele Cao, and Iryna Gurevych. "Preventing the Collapse of Peer Review Requires Verification-First AI." February 12, 2026. https://doi.org/10.48550/arXiv.2601.16909
 

Beyond Correlation: How Historians Distinguish Causation, Coincidence, and Contingency in the Age of AI

by Alvin Blackshear | Historian & Researcher  <ablackshear@gmail.com>

There is a particular kind of intellectual vertigo that comes from watching a machine narrate the past with total confidence. Ask an AI system to explain why a revolution happened, why a movement gained momentum, or why a community's history unfolded as it did, and it will almost always answer. The prose will be fluent, the timeline will be clean, and the causal claims will arrive stated as fact rather than as argument. What the answer will rarely do is pause to ask whether the events it has linked together were actually connected at all, or whether they simply happened to occur near one another in time. This is the quiet crisis facing historical writing in an era of generative AI: the technology is exceptionally good at producing narratives that sound causal, and considerably less good at knowing whether those narratives are true.

Historians have spent more than a century developing tools to guard against exactly this failure. Long before anyone worried about large language models hallucinating a tidy story out of scattered facts, historians were worried about themselves doing the same thing. The discipline's entire methodological apparatus, footnotes, source criticism, peer review, historiographical debate, exists in large part to slow down the impulse to declare that because one thing followed another, the first caused the second. What AI-generated history reintroduces, at scale and with new fluency, is an old temptation: the temptation to mistake sequence for cause, and pattern for proof.

The Seduction of Sequence

Correlation is easy to notice and hard to interpret responsibly. Two trends move together, two events happen in the same decade, two figures cross paths, and the mind supplies a connection because connection is a more satisfying story than coincidence. Sarah Woodhouse's recent work on correlation reminds us that not all statistical relationships imply causation, and that there are several distinct varieties of correlation, only one of which reflects a genuine causal link between the variables involved.[1] The others might reflect a shared underlying cause, a coincidence of timing, a feedback loop running in the opposite direction from what intuition suggests, or simply noise that resembles a pattern if you squint at it long enough.

Historians face this same interpretive challenge, except their raw material is not a spreadsheet but the messy residue of human lives: letters, court records, newspapers, oral testimony, material culture. When two developments appear together in this record, a rise in labor unrest alongside a shift in immigration policy, a change in religious practice alongside an economic downturn, a legal victory alongside a change in public rhetoric, the historian's task is to ask what actually connects them, if anything does. Johannes Nagel's comparative analysis of causal reasoning in historical scholarship makes a useful observation here: historians rarely operate with the clean methodological categories that philosophers of science might prefer. Real historical practice is often described as vaguely causal-analytical, blending causal-intentional explanation, attention to unintended consequences, and selective use of theory and comparison, without ever fully resolving into a single rigorous method.[2] This is not a failure of the discipline. It is an honest reflection of how difficult the underlying problem actually is.

AI systems, by contrast, do not experience this difficulty as friction. A model trained to produce plausible continuations of text will happily supply a causal mechanism because causal mechanisms make for better sentences than admissions of uncertainty. The danger is not that AI invents facts out of nothing, though it sometimes does that too. The deeper danger is that it takes real facts, arranges them in a narrative shape, and lets the shape itself imply a causal claim that no historian has actually verified.

Mechanism, Not Just Chronology

If there is a single principle that separates serious historical causal argument from mere narrative sequencing, it is the requirement of an explicit mechanism. It is not enough to say that event A preceded event B and therefore contributed to it. A historian is expected to explain how A produced B: through what institutions, what decisions, what material pressures, what beliefs held by which people.

Daniel Little's work on the microfoundations of social explanation is instructive here. Little argues that sweeping historical forces such as capitalism, the state, or industrialization do not themselves act as causal agents in any direct sense. They are abstractions that describe the aggregate result of countless individual decisions, made by people operating within specific institutions and constrained by specific incentives and beliefs. A causal historical explanation, on this view, has to eventually cash out in terms of those individuals: what they knew, what they wanted, what options were actually available to them, and how their choices interacted with those of others to produce a larger outcome. This is demanding work. It requires the historian to move between scales, from the granular texture of a single decision to the broader pattern that many such decisions eventually create, without losing the thread that connects the two.[3]

Mark Hewitson's argument about the marginalization of causal thinking within the historical profession adds an important wrinkle to this picture. Hewitson suggests that the turn toward language, discourse, and representation that reshaped historical writing over the past several decades, whatever its genuine intellectual gains, also made many historians reluctant to speak plainly about why things happened. Causal explanation came to seem naive, or worse, complicit in older positivist ambitions to discover fixed laws of history. And yet, as Hewitson points out, historians never actually stopped making causal arguments. Every account of a revolution, an election, a migration, or a social movement is saturated with implicit claims about why it occurred. What was lost was not the practice of causal reasoning but the willingness to examine that practice openly and hold it to a clear standard.[4]

This matters enormously for evaluating AI-generated history, because a system that has absorbed decades of scholarship in which causal claims are made implicitly, without explicit justification, will likely reproduce that same implicitness. It will offer causal-sounding narratives without ever surfacing the mechanism that supposedly connects cause to effect, because the training material it draws from often does the same thing. The corrective, for both human historians and the tools they now use, is the same: state the mechanism, or admit that you cannot.

What Coincidence and Contingency Demand of Us

Not everything that happens in history happens for a deep structural reason. Some things happen because a particular person was in a particular place at a particular moment, because a letter arrived a day late, because an illness struck one leader and spared another. Historians call this contingency, and distinguishing it from genuine causation is one of the discipline's more delicate tasks.

The stakes of this distinction are not merely academic. When causal narratives are built where only coincidence existed, the resulting history can flatten the genuine unpredictability and agency that shaped real events into something that looks inevitable, as though it could only have happened the way it did. This has particular consequences for histories that have too often been treated as footnotes to a larger, supposedly more central narrative. African American history, feminist history, and the history of gay and queer communities have all, at various points, been subjected to accounts that either erase contingency entirely, presenting outcomes as foregone conclusions of impersonal social forces, or that erase causation entirely, treating hard-won structural change as a matter of isolated coincidence rather than sustained organizing and argument. Getting the distinction right, separating what was truly contingent from what was the product of deliberate, structural, causal effort, is not a technical nicety. It is a matter of according historical actors the accuracy they deserve, whether that means recognizing the genuine unpredictability they navigated or recognizing the genuine causal power of what they built.

Tay Jeong's work on counterfactual reasoning offers historians one of their sharpest tools for making this distinction. The basic move is to ask what would have happened in the absence of the proposed cause. If removing a particular event or decision from the historical record would plausibly have left the outcome largely unchanged, the causal claim weakens considerably. If removing it would plausibly have produced a genuinely different outcome, the causal claim gains support. Jeong's more technical contribution is to note that counterfactual reasoning is most useful not as a blanket method applied indiscriminately, but in specific situations, particularly where two or more candidate causes seem to be doing similar explanatory work and our untrained intuitions about which one truly mattered are likely to mislead us. This is precise, careful reasoning, and it is exactly the kind of reasoning that a fluent AI narrative can obscure simply by never raising the question in the first place.[5]

The Discipline of Alternative Explanations

Good historical argument does not simply assert a cause and move on. It considers what else might explain the same outcome, and it explains why the preferred causal account is more persuasive than the alternatives. This habit of mind, developed across the humanities, has an unexpected ally in fields that seem far removed from history. Hannah Correia and her coauthors, writing about causal inference in environmental research, describe systematic approaches for distinguishing genuine causal relationships from mere observational association, approaches built around identifying confounding variables and establishing plausible mechanisms before accepting a causal claim.[6] Linbo Wang's comparative overview of causal inference frameworks, spanning the traditions associated with Rubin, Pearl, and structural equation modeling, makes a similar point from a more statistical angle: rigorous causal reasoning requires explicit assumptions, and different frameworks make different assumptions visible in different ways.[7] Though these authors are not writing about history, their underlying discipline, of refusing to accept a causal story until alternatives have been seriously weighed, translates directly into the historian's craft.

This is precisely the discipline that AI-generated narratives tend to skip. A well-trained model can produce a fluent account of why a particular reform succeeded, but it rarely volunteers the alternative explanations a careful historian would consider and reject: that the reform succeeded for reasons largely unrelated to the actors typically credited, that its apparent success was partly an artifact of how success was later measured, or that multiple contributing factors were entangled in ways that resist a single tidy story.

Evidence, Uncertainty, and the Historian's Honesty

Underneath all of this sits the question of evidence. Every causal claim a historian makes rests on some body of evidence, and that evidence varies enormously in quality, completeness, and reliability. Xinyue Chen and colleagues, in their study of how historians actually use visualization in their published work, found that historians face persistent practical and epistemological barriers when trying to represent uncertainty and justify their conclusions to readers and peers.[8] The justification burden, as they describe it, is not a peripheral concern. It is central to what makes a piece of historical writing trustworthy.

Georg Iggers, reflecting late in his career on the historian's role as an engaged intellectual, insisted that this burden cannot be set aside even by historians who write from a place of moral or political conviction. Iggers rejected the idea that historians must choose between detached objectivity and partisan advocacy.[9] Instead, he argued for a middle position: historians can and should be engaged with the pressing concerns of their time, but that engagement must never come at the cost of rigorous evidentiary standards. A historian writing feminist history, or African American history, or the history of gay and queer life, is no less bound by these standards than any other scholar, and indeed the stakes of getting the causal story right, of neither overstating structural inevitability nor understating hard-won agency, are often higher precisely because these histories have so frequently been distorted or dismissed.

Frank Ankersmit pushes this even further by reminding us that the past itself no longer exists in any directly accessible form. Historians do not reproduce it; they represent it, using an inherited semantic apparatus of meaning, truth, and reference that Ankersmit argues must be handled with care rather than assumed to work automatically.[10] If historical writing is, at some level, an act of representation rather than simple description, then AI-generated historical narrative inherits this same burden. It too is representing the past, not retrieving it, and representation always involves choices about emphasis, framing, and inclusion that deserve scrutiny rather than passive acceptance.

A Standard Worth Holding

What emerges from this body of scholarship is not a single formula but a disposition, a way of approaching any historical claim, human-authored or machine-generated, with a consistent set of questions. Is there a stated mechanism connecting cause to effect, or only a sequence dressed up as causation? Have alternative explanations been seriously considered and set aside for identifiable reasons? Has the line between structural causation and genuine contingency been drawn carefully, rather than assumed? Would the outcome have looked meaningfully different if the proposed cause were absent? And does the account acknowledge the limits of its own evidence, rather than presenting a confident narrative where the underlying record is thin or contested?

These questions predate artificial intelligence by decades, and they will outlast any particular model or platform. What AI has changed is the volume and fluency with which causal-sounding narratives can now be produced, and therefore the urgency of applying these questions consistently. The historians whose work informs this discussion were not writing with AI in mind. They were writing because the problem of causation in history has always been genuinely hard, and because getting it wrong has always carried real costs, especially for the histories of communities whose stories have too often been told either as inevitable or as accidental, when the truth usually lies in the harder, more interesting space between. That space, between correlation and causation, between coincidence and contingency, between chronology and mechanism, is where careful historical work has always lived. It is also, increasingly, where the responsible use of AI in historical research will need to live too.

---

Notes:

[1] Sarah Woodhouse, "When Two Things Seem Linked (but Aren't): Understanding Different Types of Correlations," Clearer Thinking, July 2, 2025, https://www.clearerthinking.org/post/when-two-things-seem-linked-but-aren-t-understanding-different-types-of-correlations.

[2] Johannes Nagel, "Causal Explanation in History: An Analysis of Research Strategies," Analyse & Kritik, March 12, 2026, https://doi.org/10.1007/s11577-026-01059-8.

[3] Daniel Little, Microfoundations, Method, and Causation: On the Philosophy of the Social Sciences (New Brunswick, NJ: Transaction Publishers, 1998).

[4] Mark Hewitson, History and Causality (Basingstoke: Palgrave Macmillan, 2014).

[5] Tay Jeong, "When to Use Counterfactuals in Causal Historiography," Sage Journals, February 2, 2025, https://doi.org/10.1177/00491241251314039.

[6] Hannah E. Correia et al., "Best Practices for Moving from Correlation to Causation in Environmental Research," Nature Communications, February 24, 2026, https://rdcu.be/fxl0y.

[7] Linbo Wang, "Causal Inference: A Tale of Three Frameworks," Journal of Data Science, February 11, 2026, https://doi.org/10.6339/25-JDS1211.

[8] Xinyue Chen et al., "How Historians Use Visualization: A Corpus-Backed Taxonomy and Analysis for Cross-Disciplinary Practice," July 10, 2026, https://doi.org/10.48550/arXiv.2605.01456.

[9] Georg Iggers, "The Historian as an Engaged Intellectual: Historical Writing and Social Criticism – A Personal Retrospective," in The Engaged Historian: Perspectives on the Intersections of Politics, Activism and the Historical Profession, ed. Stefan Berger (New York: Berghahn Books, 2019), 277–299.

[10] Frank Ankersmit, Historical Representation (Stanford, CA: Stanford University Press, 2002).

---

Works Cited

Ankersmit, Frank. Historical Representation. Stanford, CA: Stanford University Press, 2002.

Chen, Xinyue, et al. "How Historians Use Visualization: A Corpus-Backed Taxonomy and Analysis for Cross-Disciplinary Practice." July 10, 2026. https://doi.org/10.48550/arXiv.2605.01456.

Correia, Hannah E., et al. "Best Practices for Moving from Correlation to Causation in Environmental Research." Nature Communications, February 24, 2026. https://rdcu.be/fxl0y.

Hewitson, Mark. History and Causality. Basingstoke: Palgrave Macmillan, 2014.

Iggers, Georg. "The Historian as an Engaged Intellectual: Historical Writing and Social Criticism – A Personal Retrospective." In The Engaged Historian: Perspectives on the Intersections of Politics, Activism and the Historical Profession, edited by Stefan Berger, 277–299. New York: Berghahn Books, 2019.

International Commission for the History and Theory of Historiography (ICHTH). 2026 Congress Proceedings. https://www.ichth.net/content/home.html.

Jeong, Tay. "When to Use Counterfactuals in Causal Historiography." Sage Journals, February 2, 2025. https://doi.org/10.1177/00491241251314039.

Little, Daniel. Microfoundations, Method, and Causation: On the Philosophy of the Social Sciences. New Brunswick, NJ: Transaction Publishers, 1998.

Nagel, Johannes. "Causal Explanation in History: An Analysis of Research Strategies." Analyse & Kritik, March 12, 2026. https://doi.org/10.1007/s11577-026-01059-8.

Wang, Linbo. "Causal Inference: A Tale of Three Frameworks." Journal of Data Science, February 11, 2026. https://doi.org/10.6339/25-JDS1211.

Woodhouse, Sarah. "When Two Things Seem Linked (but Aren't): Understanding Different Types of Correlations." Clearer Thinking, July 2, 2025. https://www.clearerthinking.org/post/when-two-things-seem-linked-but-aren-t-understanding-different-types-of-correlations.

AI for Older Adults

AI for Older Adults

by Alvin Blackshear | Historian & Researcher

Artificial intelligence tools like ChatGPT, Claude, Gemini, Perplexity, and Copilot have moved from novelty to necessity faster than almost any technology in memory. For adults over 60, that speed can feel less like an invitation and more like being left behind at the station. But research on how older adults actually use, and struggle with, these tools tells a more encouraging story: the gap isn't about ability, it's about approach. A 2026 study of older adults' technology support requests found that when queries were rewritten to add missing context and clearer language, the accuracy of AI-generated solutions jumped from 46% to 69%, and related Google search results improved from 35% to 69% accuracy. In other words, the tools already work well enough. The skill worth learning is how to talk to them.

This article walks through four practical steps: choosing the right AI for your needs, writing prompts that get better answers, understanding what AI "agents" are and when to trust them, and avoiding the mistakes that trip up new users of every age, but especially those who didn't grow up parsing tech jargon.

Choosing the Right AI

There is no single "best" AI assistant. Each is tuned for different strengths, and picking the right one for the task at hand saves time and frustration. A simple way to think about it:

·   Best for everyday questions: ChatGPT is the most general-purpose assistant, well suited to answering quick questions, drafting emails, or explaining a topic in conversation.

·   Best for writing and long documents: Claude tends to produce clear, well-organized long-form writing and is a strong choice for letters, essays, or reviewing lengthy documents.

·   Best for web research: Perplexity is built around search, pairing its answers with cited sources so you can click through and verify what it tells you, which is useful when accuracy matters most.

·   Best for Google users: Gemini integrates directly with Gmail, Google Docs, and Google Search, which makes it a natural fit if that's already your everyday ecosystem.

·   Best for Microsoft 365 users: Copilot is built into Word, Outlook, and Excel, so if your family or workplace runs on Microsoft software, it can work inside the programs you already know.

None of these choices is permanent or exclusive. Many people end up using two: one for quick daily questions and another for anything they want double-checked with sources. The point isn't to find the "correct" one, but to match the tool to the task, the same way you'd choose a phone call over a text message depending on what you're trying to accomplish.

Learning to Write Better Prompts

The single biggest lever an older adult has for getting better answers from AI isn't a special app or setting. It's the wording of the question itself. Research on communication between older adults and AI systems identified four recurring patterns that reduce answer quality: verbosity (explaining too much surrounding detail), incompleteness (leaving out a key fact the AI needs), over-specification (adding assumptions that aren't actually true), and under-specification (being too vague about what's wanted). The good news from that same research is that these patterns are fixable, not through retraining how you think, but through a few reliable habits when you type or speak your question.

Ask for plain English. AI models default to whatever register they think fits the question, and that can drift into jargon. Simply adding "explain this in plain English, no technical terms" or "explain it like you would to a smart friend who isn't a computer person" changes the entire tone of the reply.

Request step-by-step explanations. Instead of asking "how do I attach a photo to an email," try "walk me through this one step at a time, and wait for me to say 'next' before moving to the next step." This breaks a wall of instructions into a manageable back-and-forth, and it also gives you a natural point to ask a follow-up question without losing the thread.

Ask for examples. Abstract instructions are harder to follow than concrete ones. Adding "give me a real example" or "show me what that would actually look like on my screen" turns a vague description into something you can match against what you're actually seeing.

Ask the AI to critique its own answer. This is a lesser-known but powerful technique: after receiving an answer, ask "is there anything wrong with this answer, or a simpler way to do it?" AI models are often able to catch their own errors or oversimplifications when prompted to double-check themselves, and this habit builds in a layer of quality control you don't have to supply yourself.

The underlying finding from the research is worth repeating: rephrased, well-specified questions were understood far better by others (93.7% versus 65.8% comprehension in one comparison) and led to solutions people felt confident following. A better-written question isn't a nicety. It's the difference between a correct answer and a confusing one.

Using AI Agents

You may have started to hear the term "AI agent," distinct from a regular chatbot. It's worth understanding the difference, because agents come with real convenience and real risk.

What an agent is. A standard AI chat answers a question and stops. An agent is designed to take a goal and carry out a sequence of actions on its own (searching the web, filling out a form, booking something, or working through a multi-step task) before reporting back. Industry analysis on the state of AI agents in 2026 notes that the defining shift in newer systems is their ability to handle progressively longer and more complex tasks, maintaining context across many steps rather than answering a single quick question.

When to use one. Agents are useful for tasks that are repetitive, well-defined, and low-stakes if something goes slightly wrong, for example, summarizing a folder of documents, comparing prices across several websites, or drafting a set of similar emails. They save the most time exactly where a task has many small, mechanical steps that would otherwise take you an hour of clicking around.

Risks of autonomous agents. The tradeoff for that convenience is reduced oversight. Because an agent acts across many steps without checking in at each one, a small misunderstanding early on can compound into a larger error by the end, the AI equivalent of a wrong turn early in a road trip that takes you far off course before anyone notices. This matters most for anything involving money, personal information, medical decisions, or sending communications on your behalf. Never give an agent standing access to your bank accounts, passwords, or ability to make purchases without a manual confirmation step at the end.

Practical everyday examples. Reasonable, lower-risk uses include: asking an agent to research and summarize options for a specific purchase (with you making the final decision), having it organize digital photos or files, or using it to draft, but not send, a batch of similar messages, such as holiday cards or appointment reminders, for you to review individually before anything goes out.

Avoiding Common Mistakes

Even with the right tool and better prompts, a handful of habits cause most of the frustration people report with AI, regardless of age.

Trusting AI too quickly. AI models can sound confident even when they're wrong. They occasionally generate plausible-sounding but incorrect information, a phenomenon often called "hallucination." Treat a confident tone as no guarantee of accuracy, especially for anything involving health, legal, or financial specifics.

Asking vague questions. As the communication research above showed, vague or incomplete questions are the most common source of poor answers. If a response feels off-target, the fastest fix usually isn't switching tools. It's adding the missing context and asking again.

Accepting the first answer. The first response an AI gives is a starting point, not a verdict. Asking a follow-up like "are you sure?" or "what would make this answer wrong?" often surfaces caveats or corrections the first answer left out.

Failing to verify important facts. For anything that matters, such as a medication interaction, a legal deadline, or a financial figure, cross-check the AI's answer against a trusted human source, such as your doctor, pharmacist, financial advisor, or a reputable website. Tools like Perplexity, which show their sources directly, make this step easier by letting you click through and confirm the claim yourself rather than taking it on faith.

The Bigger Picture

None of this requires becoming a technical expert. The research is fairly consistent on this point: older adults who use AI report high confidence and ease once they receive well-matched, clearly explained answers. The 2026 diary study found 89.8% of older adult participants felt able to answer AI's follow-up questions, and 94.7% felt able to follow the solutions they were given. The barrier isn't capability; it's a mismatch in how questions get asked and answered. Broader research on aging and generative AI echoes the same conclusion, pointing to real promise in health and cognitive support as long as tools are built and used thoughtfully, with attention to privacy and genuine ease of use. Choose a tool that matches the task, ask questions the way outlined above, use agents cautiously and only for low-stakes chores, and verify anything that truly matters. That combination, not fluency in tech jargon, is what determines whether AI becomes a genuinely useful assistant in daily life.

___________________________ 

Sources:

Dhanuka, N. (2026, February 26). State of AI agents 2026: Autonomy is here. Prosus. https://www.prosus.com/news-insights/2026/state-of-ai-agents-2026-autonomy-is-here

Ernst & Young Global Limited. (2026, March). Understanding older generations' adoption of AI [Ripples Research Report]. https://www.ey.com/content/dam/ey-unified-site/ey-com/en-gl/about-us/corporate-responsibility/documents/ey-gl-how-older-generations-are-engaging-with-ai-03-2026.pdf

Liu, T. (2026, May 29). The role of multimodal generative AI in older adults' health management: Systematic scoping review. JMIR AI. https://ai.jmir.org/2026/1/e84695

Shomee, H. H. (2026, January 15). Empowering older adults in digital technology use with foundation models. arXiv. https://doi.org/10.48550/arXiv.2601.10018