Ten Common Hallucinations in Genealogical AI

Ten Common Hallucinations in Genealogical AI

by Alvin Blackshear | Historian & Researcher <ablackshear@gmail.com>

If you've started using AI tools to help build out your family tree, you've probably noticed something exciting: AI can search patterns, suggest connections, and summarize records faster than any human researcher. But you may have also noticed something unsettling. Sometimes the AI is simply wrong, and it's wrong with total confidence. It doesn't hedge. It doesn't say "I'm not sure." It states fabricated facts as if they were pulled straight from a courthouse archive.

This isn't a quirk specific to genealogy tools. It's a well-documented feature of how large language models work. Researchers have found that AI systems are, in a sense, rewarded for guessing rather than admitting uncertainty, much like a student on a hard exam who writes down a plausible-sounding answer rather than leaving the question blank. The same dynamic that produces fabricated legal citations in court filings and nonexistent references in academic papers also produces invented ancestors, false marriages, and imaginary historical explanations in your family tree.

For beginning genealogists who are comfortable with AI tools but new to the discipline of genealogical research, this creates a real risk. Family history has its own standards of evidence (rules developed over generations of professional practice), and AI, for all its fluency, doesn't inherently know or follow them. Below are ten (plus a bonus eleventh) hallucinations you're likely to encounter, along with how to spot each one.

#1. Invented Ancestors

The most obvious and probably most common hallucination is the flat-out invented person. If your records show John Smith born in 1846, married to Sarah Jones, and died in 1912, an AI tool may cheerfully add "William Smith, father of John" to your tree, simply because a father-shaped gap exists and the model has learned that fathers usually appear in these slots.

No birth record names him. No probate record mentions him. No document anywhere establishes his existence. He is, in the most literal sense, a statistical guess dressed up as a fact.

How to catch it: Ask the AI directly, "What specific document supports this person's existence?" If it can't name one, or names one that turns out not to exist (see #5), strike the individual from your tree.

#2. False Parentage

This is a subtler and more dangerous error because it doesn't invent anyone; it just connects the right person to the wrong family. Imagine two men named James Brown, both born in Virginia in 1818, but belonging to two entirely unrelated families. An AI tool, pattern-matching on name, place, and approximate date, may merge them, attaching one man's documented parents to the other man's documented children.

Because online family trees copy from each other constantly, an error like this doesn't stay contained. It can spread across dozens or hundreds of public trees within months, becoming "true" simply through repetition.

How to catch it: Whenever two individuals share a name, county, and rough birth year, ask the AI to list the specific evidence distinguishing them: land records, church parishes, associates named as witnesses, distinguishing middle names. If it can't distinguish them, you probably have two people, not one.

#3. Collapsing Two People into One

Related to false parentage, this hallucination happens when the AI treats "same name, similar era, same county" as sufficient proof of "same person." Common names are especially vulnerable: John Williams, William Johnson, Mary Jones, Elizabeth Brown. In many rural communities of the eighteenth and nineteenth centuries, several unrelated people carried the same name at the same time.

How to catch it: Before accepting that two records refer to one individual, ask what independent evidence (land transactions, church membership, military service, or family testimony) ties them together, not just shared name and geography.

#4. Splitting One Person into Several

The mirror-image error: one real person recorded inconsistently, as "Robert," "Robt.," "Bob," and "R.J.," gets treated by the AI as four different people. Suddenly your family tree has artificially multiplied, with duplicate siblings or duplicate spouses cluttering the record.

Nineteenth-century clerks were not consistent spellers, and names were frequently abbreviated, translated, or anglicized. AI models, trained to treat text strings literally, can miss the obvious inference that a human researcher would make instantly.

How to catch it: When a "new" individual appears with a name variant of someone already in your tree, in the same location and timeframe, check whether the records could plausibly describe one person before treating them as separate.

#5. Invented Sources

Perhaps the easiest hallucination to catch once you know to look for it, and one of the most dangerous when you don't. AI tools will sometimes cite a source that sounds entirely plausible: "County Probate Book 14, page 327" or "Virginia Marriage Register, Volume 8." The citation has the right shape, the right format, the right level of specificity. It just doesn't exist.

This mirrors a broader, well-documented problem. A 2026 analysis of AI-generated academic writing found a sharp rise in nonexistent references following widespread adoption of large language models, with a conservative estimate of nearly 147,000 hallucinated citations appearing in 2025 alone. Similar patterns have shown up in legal filings, where courts have caught lawyers submitting briefs built on cases that were never decided because an AI tool invented them. Genealogy is not immune to the same failure mode; if anything, it may be more vulnerable, since archival citation formats are formulaic and easy for a model to imitate convincingly.

How to catch it: Never accept a citation at face value. Look the source up yourself, in the actual archive, library catalog, or online record collection, before recording it as fact. If you can't locate it, assume it doesn't exist until proven otherwise.

#6. Imaginary Historical Context

AI loves to explain. "The family likely moved west because of the Panic of 1837." It's a tidy, plausible sentence. It may even be historically reasonable. But plausible is not the same as documented, and an AI tool rarely marks the difference clearly. Historical interpretation, offered without evidence specific to your family, quietly becomes historical fiction wearing the authority of fact.

How to catch it: Treat any sentence containing words like "likely," "probably," or "given the context of the time" as a hypothesis to test, not a fact to record. Ask what document (a land sale, a migration record, a letter) actually supports the claimed motive.

#7. Overconfident Translation

If your ancestors left records in German, Polish, Latin, French, or Dutch, AI translation can feel like a superpower, until it isn't. AI models translate old church and civil records with impressive fluency, but that fluency can mask serious errors: invented words to fill illegible gaps, misread names, or entire phrases reconstructed from context rather than actually read. Genealogists working with foreign-language parish records have reported AI tools fabricating names, places, and even whole narrative details that don't exist in the original document.

How to catch it: For any translation affecting a name, date, or relationship, ask for a word-by-word rendering alongside the smooth translation, and compare it against the original image yourself, or have a human speaker of the language check ambiguous passages.

#8. Fabricated Relationships

A record lists a witness, Samuel Jones, at a wedding or a will signing. The AI concludes he must be an uncle, brother, father, or cousin. But witnesses in historical documents were frequently neighbors, business associates, fellow church members, or simply friends, not relatives at all. The AI's eagerness to complete the family picture leads it to assign a relationship where the record specifies none.

How to catch it: Treat every unlabeled name in a record (witness, executor, sponsor, bondsman) as a relationship of unknown type until independent evidence establishes otherwise.

#9. Timeline Repair

Large language models dislike contradictions, and historical records are full of them. If a record shows a birth in 1820 and a marriage in 1832, and something else nearby seems inconsistent, an AI tool may quietly adjust one of the dates so everything appears internally consistent. The apparent contradiction vanishes, but so does the actual historical evidence, replaced by a smoothed-over fiction that fits the model's sense of what "should" be true.

How to catch it: When a date changes between one AI response and another, ask why, and check both figures against the original record. Real historical records are often genuinely inconsistent, and that inconsistency is itself information, not a bug to be silently fixed.

#10. Source Laundering

This is probably the least recognized hallucination, and arguably the most insidious. AI tools trained on the vast ocean of user-submitted online family trees absorb whatever errors already exist there, including all the mistakes described above, made by human researchers over the past two decades. The AI then presents those errors back to you as though they were settled historical fact.

Nothing here is invented from scratch. That's what makes it dangerous. The information already circulates online, attached to seemingly credible trees and family history websites, so it carries an unearned appearance of authority. The AI hasn't lied exactly; it has simply repeated community folklore as though it were consensus.

How to catch it: Be especially skeptical of any claim the AI attributes to "commonly accepted" genealogy or "widely documented" family history. Ask what primary source underlies the claim, not what other online trees say about it.

An Eleventh Hallucination: Certainty Inflation

There's one more pattern worth naming, even though it's rarely discussed: certainty inflation. Professional genealogists are trained to say, when the evidence doesn't fully support a conclusion, "the evidence is insufficient." AI tools are much less comfortable with that kind of humility. Instead, they tend to write things like "based on available evidence, John was almost certainly the son of..."

That phrase, "almost certainly," can represent an enormous leap well beyond what the underlying evidence supports. This isn't accidental. Research into why language models hallucinate has found that the training and evaluation processes used to build these systems reward confident, plausible-sounding answers over honest expressions of uncertainty, the AI equivalent of a student guessing on a test rather than leaving a blank. The result is a model that is statistically inclined to sound more certain than the facts warrant.

Toward a Genealogical AI Evaluation Framework

None of this means AI is useless for family history research; far from it. It means AI output needs to be treated the way a careful historian treats any single source: as a claim to be verified, not a fact to be recorded.

One useful discipline is to treat every AI-generated genealogical statement as a research hypothesis, never as evidence in itself. Accept only what can be traced back to an identifiable primary or reliable secondary source, and preserve uncertainty wherever the historical record itself remains uncertain. In practice, that means running each AI claim through a short mental checklist:

Criterion	Key Question
Source authenticity	Is every factual claim traceable to a real, locatable source?
Evidence chain	Can the conclusion be reconstructed from the cited evidence?
Identity confidence	Has the AI distinguished between similarly named individuals?
Chronological consistency	Are dates plausible without silent correction?
Geographic plausibility	Do locations fit historical boundaries and migration patterns?
Uncertainty disclosure	Does the AI distinguish evidence from inference?
Genealogical Proof Standard alignment	Would the conclusion satisfy professional genealogical standards?

Applying a framework like this won't eliminate hallucinations (no current AI system is free of them), but it will keep you from mistaking a fluent, confident answer for a well-documented one. AI can be a genuinely powerful research assistant for genealogy: it can suggest search strategies, summarize long documents, translate unfamiliar scripts, and spot patterns across large record sets faster than any human could alone. Just remember that its job is to help you find the evidence, not to become the evidence itself.

________________________________

Sources:

Kalai, Adam, et al. "Why Language Models Hallucinate." arXiv, 4 Sept. 2025, https://arxiv.org/abs/2509.04664.

Newsham, Jack. "AI Hallucinations in Court Documents Are a Growing Problem, and Data Shows Lawyers Are Responsible for Many of the Errors." Business Insider, 27 May 2026, https://www.businessinsider.com/increasing-ai-hallucinations-fake-citations-court-records-data-2025-5.

Yin, Yian, et al. "LLM Hallucinations in the Wild: Large-Scale Evidence from Non-Existent Citations." arXiv, 8 May 2026, https://arxiv.org/abs/2605.07723.

 

How Historians Should Evaluate AI

The Historian's AI Evaluation Framework (HAIEF): Twelve Questions Before You Trust a Machine-Written Past

by Alvin Blackshear, retired Historian   <ablackshear@gmail.com>
______________________________________________________________

In the summer of 2026, two of the world's largest consulting firms had to pull back work they had already delivered to clients. PwC withdrew AI-generated reports after discovering they contained fabricated citations and references to sources that did not exist.[1] KPMG ran into the same problem: investigators reviewing one of its AI-assisted reports found that most of its citations were fabricated or misleading.[2] Neither firm works in history. Neither incident involved archives, manuscripts, or a single primary source. That is exactly why they matter here.

It would be easy for historians to read those stories and conclude they describe someone else's problem, a corporate reporting failure, a compliance issue, a cautionary tale about consulting culture. That reading is a mistake. What failed at PwC and KPMG was not domain knowledge. It was sourcing discipline: the basic, mechanical trustworthiness of a citation. If large language models fabricate references in financial and regulatory reporting, where the source material is comparatively narrow and well-indexed, there is no reason to expect better behavior in historical research, where the source base is vast, unevenly digitized, and full of the kind of ambiguity these systems are worst at handling. This is not a story about historians being unusually vulnerable to bad AI output. It is a story about a structural failure mode that will show up wherever citation and provenance matter, and few disciplines depend on citation and provenance more than history does.

There is a second, related problem, and it may be more dangerous than outright fabrication because it is harder to catch. Recent reporting on AI accuracy has noted a troubling pattern: as these systems have become more fluent, they have not necessarily become more honest about what they don't know. Instead, they increasingly deliver incorrect information with the same confident tone as correct information, rather than flagging uncertainty or hedging where hedging is warranted. Any historian will recognize this failure mode immediately, because it is precisely what they are trained to distrust in human sources. A memoir written decades after the fact, a diplomatic cable reporting secondhand rumor as established fact, a witness testifying with total conviction about details that turn out to be wrong — historians have centuries of methodological practice built around the fact that confidence is not evidence. A machine that states falsehoods fluently and without hedging is not a new problem for historical method. It is an old problem wearing a new interface.

Put these two failure modes together, fabricated sourcing and confident error, and you arrive at the central argument of this piece: a language model can perform well on the accuracy benchmarks the AI industry currently uses, and still fail every test a professional historian would apply to a piece of research. General-purpose evaluation metrics measure things like factual accuracy rates, hallucination frequency, and human preference scores. None of those metrics ask whether a source actually exists in the form cited, whether a quotation can be independently located, whether a narrative acknowledges live scholarly disagreement, or whether the chronology holds together under scrutiny. A model can score 95 percent on general factual accuracy and still produce a historical narrative that treats a contested interpretation as settled, leans on secondary literature while ignoring available primary sources, and quietly imports present-day assumptions into a past context that didn't share them. None of that shows up in a hallucination-rate benchmark. All of it would get a paper rejected by a competent peer reviewer.

 That gap, between what general AI evaluation measures and what historical method actually requires, is the subject of this article. This is not a piece about whether historians should use AI, or how to prompt it more effectively; that ground has already been covered well by other writers, including practical workflow guides for AI-assisted historical research. This is a piece about something narrower and, so far, largely unaddressed: how a working historian should grade what the AI hands back. It is meant to function less like an essay and more like a QA protocol, a practical checklist you could actually run against a piece of AI-generated research before you decide whether to trust, cite, or build on it. 

What the research actually shows

The clearest empirical precedent for this kind of evaluation comes from a 2026 comparative study that pitted trained historians against generative AI tools: ChatGPT-4, Claude, Gemini, and Perplexity, on parallel case studies in ancient history. The researchers found a consistent pattern: AI tools were genuinely strong at broad content discovery and thematic synthesis, the kind of work that benefits from fast pattern-matching across large bodies of text. But they were consistently weaker than human historians on source verification, chronological reasoning, provenance, and contextual interpretation, which are the parts of the discipline that depend on judgment rather than retrieval. The study's authors argued for a human-in-the-loop approach that treats AI as a research accelerant rather than a substitute for scholarly adjudication.

A separate line of research has approached the problem from the model side rather than the historian's side, building benchmarks specifically designed to test historical reasoning. That work has found that even the strongest available models struggle with the kind of sophisticated evidentiary reasoning that is routine for a trained historian, the weighing conflicting accounts, recognizing genre and authorial bias, and reconstructing plausible sequences of events from incomplete evidence. This is a useful corrective to the assumption that model capability will simply catch up with historical judgment as systems improve. The gap is not just about scale; it is about a kind of reasoning these systems are not currently built to do well.

Not every recent contribution to this literature is critical of AI's role in historical work. Case studies of experienced historians integrating AI into their actual research workflows, mapping specific tasks to language models, to conceptual work the historian retains, and to conventional computational tools, that describe an approach where AI expands capacity without replacing judgment, provided the historian maintains rigorous cross-checking at every stage. That distinction matters for this piece: the argument here is not that AI is useless for historical work. It's that historians need a way to grade the output that general AI evaluation frameworks were never designed to provide. 

Introducing the framework: HAIEF[3]

What follows is a two-part tool. The first is a practitioner-facing checklist, including twelve diagnostic questions meant to be run quickly against any piece of AI-generated historical research, the way you'd run a gut-check before deciding whether something is worth building on. The second is a more formal, citable rubric that breaks the same underlying concerns into eight scored criteria, useful when you need to document *why* a piece of AI output was accepted or rejected, not just whether it was.

The twelve questions:

1. Are all citations real?

2. Can every quotation be independently located?

3. Is the chronology internally consistent?

4. Does the AI distinguish primary from secondary sources?

5. Are archival collections correctly identified?

6. Are names, dates, and places cross-checked?

7. Does the narrative reflect historiographical debate?

8. Does the AI acknowledge uncertainty?

9. Does it introduce present-day assumptions into historical contexts?

10. Can another historian reproduce the research?

11. Which claims require archival verification?

12. What percentage of the output would I actually trust without checking?

That last question is deliberately blunt, and it's the one I'd argue matters most in practice. Every other question on the list is diagnostic.  It tells you where a piece of AI research is likely to break down. Question twelve forces a number out of you, an honest estimate of how much of the output you'd actually stand behind without independent verification. In my own use, that number is rarely above 40 or 50 percent for anything involving primary-source claims, and treating that as a normal, expected ceiling, rather than a failure of the tool, is the right posture to bring to this work.

The formal rubric maps the same territory onto eight criteria: citation authenticity (do the cited works exist and actually support the claim made), provenance (is every factual assertion traceable to a reliable source), chronological accuracy (do the dates and sequences hold together internally), primary-source fidelity (are quotations and archival references accurate to the original), historiography (does the output acknowledge that scholarly consensus is contested where it is contested), context (are events read within their own historical setting rather than through present-day assumptions), uncertainty (does the output distinguish established fact from inference), and reproducibility (could another historian, working from the same cited evidence, independently reach the same conclusions).

The value of running both versions side by side is that they serve different moments in the workflow. The twelve questions are what you ask while you're reading the output, in real time, deciding whether to keep going or start over. The eight-criterion rubric is what you fill out afterward, when you need a defensible record of why a source was trusted or discarded, which is useful for peer review, for training research assistants, or for documenting due diligence in a professional or editorial context. 

Who this is for

Historians and academic researchers are the most obvious audience, and the stakes for them are the highest: a fabricated citation or an unacknowledged historiographical dispute that makes it into a submitted paper is a professional liability, not just an inconvenience.

Genealogists are arguably the highest-volume, lowest-scrutiny users of AI-assisted historical research, and they are almost entirely unaddressed by the academic literature on this problem. Family history research routinely involves exactly the kind of claims, be it names, dates, places, relationships, that AI systems handle with the least reliability, and genealogists are less likely than academic historians to have institutional habits of source verification already built in.

Journalists covering historical topics under deadline face a particular version of this problem: the twelve-question checklist doubles as a fast fact-checking pass when there isn't time for a full archival dig, and question twelve, what percentage would you trust without checking, is a useful discipline for anyone filing on a clock.

Educators like myself have perhaps the most durable use case. Detecting AI-generated text is a losing, ever-shifting battle. Teaching students to interrogate AI output using a framework like this one, and not asking "did a machine write this" but rather, "would this survive a historian's scrutiny", is a skill that remains useful regardless of how the underlying technology changes. 

Closing

None of this should be mistaken for a claim that a checklist can do a historian's job. A framework like HAIEF can catch a fabricated citation, flag an inconsistent chronology, or surface a claim that needs archival verification before it gets treated as settled. What it cannot do is listen. Jan Burzlaff has argued that AI systems summarize but do not listen, reproduce but do not interpret.  These tools  achieve coherence while faltering at contradiction, and that historical writing is not simply an output to be optimized but a form of presence, a risk taken in the act of trying to make meaning where no prewritten frame will do. That distinction is worth sitting with rather than resolving. A verification framework can tell you whether the sourcing holds up. It cannot tell you whether the history has been understood. Those are different questions, and confusing them, mistaking a clean citation audit for genuine historical interpretation, may turn out to be the more consequential failure mode of all.

Sources

Burzlaff, Jan. "Fragments Not Prompts: Five Principles for Writing History in the Age of AI." Rethinking History (2026). https://doi.org/10.1080/13642529.2025.2546174.

Campbell, Chris. "The Historian in the Age of AI." Transactions of the Royal Historical Society (December 10, 2025). https://www.cambridge.org/core/journals/transactions-of-the-royal-historical-society/article/historian-in-the-age-of-ai/37E3B742A2983DF4DAA2E38D48252F89.

"Generative AI as a Historical Source: Source Criticism, Citation Integrity, and the Jagged Frontier of Digital History." (March 20, 2026). https://acnsci.org/journal/index.php/cte/article/view/1438.

Fox, Yaniv. Using Generative AI in Historical Practice. Cambridge Elements. Cambridge: Cambridge University Press, 2026. https://www.cambridge.org/core/elements/abs/using-generative-ai-in-historical-practice/7C1392A6E9DBD379FAA42E2D16A6D45B.

Henriot, Christian. "The AI-Augmented Research Process: A Historian's Perspective." Preprint, arXiv, August 3, 2025. https://arxiv.org/abs/2508.01779.

"HistoRAG: Embedding Historical Methodology in Retrieval-Augmented Generation." Preprint, arXiv, June 16, 2026. https://arxiv.org/abs/2606.18103.

"Can LLMs Act as Historians?" Preprint, arXiv, April 27, 2026. https://arxiv.org/abs/2604.24690.

Ruchniewicz, Krzysztof. "AI in History Education: The American Historical Association's Guidelines, the Views of Their Authors, and Their Reception within the Historical Community." Studia Historica 62, no. 1 (2026). https://doi.org/10.14746/sh.2026.62.1.003.

Solga, Raymond S., and Mohammed J. Sarwar. "Evaluating Generative AI in Historical Research: A Comparative Study on Identifying Primary Source Evidence in Ancient History." AI & Antiquity 2, no. 1 (February 26, 2026). https://doi.org/10.64946/aiantiquity.v2i1.003.

Financial Times. "PwC Withdraws AI-Generated Reports over Fabricated Citations." July 29, 2026. https://www.ft.com/content/7e149ac8-2ce2-4266-8940-192f9821b33c.

TechRadar Pro. "A Major KPMG Report on AI Was Found to Be Chock-Full of AI Hallucinations." June 12, 2026. https://www.techradar.com/pro/a-major-kpmg-report-on-ai-was-found-to-be-chock-full-of-ai-hallucinations.

Axios. "AI Chatbots Are Getting More Confident — and Sometimes More Wrong." May 30, 2026. https://www.axios.com/2026/05/30/ai-accuracy-chatbots-hallucinations.

---

[1] https://www.ft.com/content/7e149ac8-2ce2-4266-8940-192f9821b33c (2026-07-29)

[2] https://www.techradar.com/pro/a-major-kpmg-report-on-ai-was-found-to-be-chock-full-of-ai-hallucinations (2026-06-12)

[3] HAIEF = Historian's AI Evaluation Framework