The Ultimate Arbiter: Human Epistemic Authority in AI-Augmented Research
By Michael Kelman Portney
In May 2023, attorney Steven Schwartz made modern legal history for all the wrong reasons. Representing a client in Mata v. Avianca, he used ChatGPT to find supporting case law. The model gave him six cases—detailed, persuasive, completely fabricated. When opposing counsel flagged the citations as nonexistent, Schwartz did what only a true believer would do: he asked the same chatbot if it had lied. ChatGPT assured him it hadn’t. He believed it.
Judge P. Kevin Castel fined Schwartz and his co-counsel $5,000, describing their brief as “replete with citations to non-existent cases.”[^1] The ruling was measured. The situation wasn’t. Schwartz had made the fundamental epistemic error of the AI age: he treated the output of a language model as a declaration of truth, not a probabilistic guess.
By early 2025, this had become a pattern. Morgan & Morgan—the largest personal injury firm in America—submitted briefs citing nine cases, eight of which were hallucinations.[^2] Michael Cohen’s attorney filed motions based on rulings that Google Bard had simply invented.[^3] Different firms, same mistake. The problem isn’t the AI. The problem is human abdication of epistemic responsibility.
Because here’s the hard truth: you are ultimately responsible for being truthful, not the AI. Your job is not to believe or disbelieve what AI tells you. Your job is to verify it. That’s not paranoia—it’s professionalism. It’s also the only way to prevent an epistemic collapse.
When the Oracle Hallucinates
Law: When Trust Becomes Liability
The Mata fiasco opened the floodgates. In February 2025, U.S. District Judge Kelly Rankin sanctioned three Morgan & Morgan attorneys for submitting briefs with eight fabricated cases in Wadsworth v. Walmart.[^4] They had relied on the firm’s in-house AI, MX2.law, without checking a single citation. “While technology continues to evolve,” Judge Rankin wrote, “one thing remains the same—checking and verifying the source.”
Those lawyers paid $5,000 in fines. But the reputational damage—having “AI hallucination” attached to your name in federal case law—is forever.
Academia: Peer Review on Autopilot
In late 2024, Neurosurgical Review retracted 129 papers in less than two months.[^5] One Indian university produced 87 of them. Two researchers authored 35. All were AI-generated junk.
That same winter, Frontiers in Cell and Developmental Biology published AI-generated images labeled “protemns” instead of “proteins.”[^6] These made it through peer review, because the journal used its own AI “reviewer”—AIRA—to screen submissions. AIRA didn’t catch the gibberish, because AIRA doesn’t know what a protein is. We’ve now reached the point where machines are generating bad science and other machines are rubber-stamping it.
Business: Corporate AI Meets Legal Reality
Air Canada learned this lesson the hard way. In 2024, its chatbot told customer Jake Moffatt that bereavement fares could be applied retroactively within 90 days. That wasn’t true. When he sued, the airline’s lawyers argued that the chatbot was “a separate legal entity” responsible for its own words. The tribunal didn’t buy it: “This is a remarkable submission.”[^7] The court found Air Canada responsible and ordered restitution. The company quietly shut the chatbot down.
The Pattern Is Predictable
Across law, academia, and business, the failure mode is identical:
Blind trust: Users treat AI output as authoritative.
AI doubles down: When questioned, it insists it’s correct.
Authority substitution: Humans stop thinking critically.
Public embarrassment: The fallout is immediate and expensive.
A 2025 Nature Communications Medicine study tested six major language models on 300 clinical scenarios, each with a hidden false detail. Hallucination rates ranged from 50% to 83%. Even the best-performing model, GPT-4o, was wrong 23% of the time—with carefully engineered prompts.[^8]
That means even under ideal conditions, one in four confident AI answers is false. If that doesn’t scare you, it should.
The Epistemology of Not Being an Idiot
Hallucination Is Structural
You can’t “fix” hallucinations. They’re baked into the architecture. Research by OpenAI scientists Kalai and Nachum shows that models hallucinate because their training rewards confident prediction, not honesty.[^9] They’re designed to complete patterns, not to distinguish truth from plausible fiction. A model that admits uncertainty gets penalized; a model that bluffs gets rewarded.
As long as we optimize for fluency and confidence, we’re training systems to lie persuasively. That’s not a glitch—it’s an incentive structure.
Epistemic Authority and Abdication
Philosopher Linda Zagzebski defines epistemic authority as the power to provide reasons that preempt our own reasoning.[^10] When you defer to an authority, you replace your judgment with theirs. That’s fine when the authority is competent. It’s catastrophic when the “authority” is a probabilistic generator trained to imitate competence.
Alvin Goldman’s social epistemology explains the trap: laypeople can’t always evaluate expertise. AI exploits that blind spot. It sounds authoritative about everything, never signals uncertainty, and doesn’t carry the reputational cost of being wrong.[^11] It’s epistemic authority without accountability—a philosopher’s nightmare and a marketer’s dream.
The Extended Mind and the Broken Prosthetic
Andy Clark and David Chalmers argued that our cognitive processes extend into the tools we use.[^12] Your notes app is an external memory. Your calculator extends your reasoning. But when one of those tools unpredictably fabricates data, the whole cognitive system becomes unreliable. A defective prosthetic doesn’t absolve you of walking off a cliff.
Epistemic Responsibility
Lorraine Code called the pursuit of accurate knowledge an ethical obligation.[^13] Verification isn’t optional—it’s part of moral life. Trust, she argues, must be earned through reliability. AI has not earned it.
Research confirms this. Horowitz and Kahn found that people with moderate AI knowledge are the most likely to over-trust it.[^14] They know just enough to be reckless. Meanwhile, Vicente and Matute showed that once people use AI, its errors infect their reasoning even after the AI is gone.[^15] The contamination sticks.
So the epistemic bottom line is this: you can’t offload the moral burden of truth. You’re responsible for being truthful, even when using tools that aren’t.
Four Ethical Frameworks Walk Into a Bar
If an action violates virtue ethics, consequentialism, deontology, and care ethics simultaneously, it’s not “complex.” It’s wrong. Unverified AI use clears that bar easily.
Virtue Ethics: The Death of Intellectual Character
Virtue epistemology values humility, diligence, and intellectual courage.[^16] Blind trust in AI annihilates all three. It’s the opposite of intellectual virtue. Shannon Vallor calls this “moral deskilling”—technology eroding the very habits that make us moral agents.[^17] Lawyers filing hallucinated briefs aren’t just negligent; they’re epistemically lazy.
Consequentialism: The Wreckage Adds Up
The Ponemon Institute found that major AI system failures cost an average of $3.7 million per incident—more than double when no human oversight is involved.[^18] But the real damage is cultural.
Misinformation spreads faster.
Biases compound.
Safety systems fail.
Trust in institutions collapses.
Over 10,000 papers were retracted in 2023 alone.[^19] They were cited more than 35,000 times, embedding falsehoods into the scholarly record—and into AI training data. Every unverified claim makes the next generation of AI worse.
Deontology: Kant Would Call You a Liar
Kant’s categorical imperative says lying is always wrong because it destroys the very conditions for trust.[^20] Sharing unverified AI output is lying by negligence. It violates your duty to yourself (you’re complicit in self-deception), to others (you’re misinforming them), and to humanity (you’re eroding the reliability of communication).
AI can’t bear moral duty because it lacks autonomy. When Air Canada argued its chatbot was a “separate legal entity,” it was attempting moral outsourcing. Kant would’ve laughed them out of the room.[^21]
Care Ethics: Negligence as Indifference
Care ethics asks who gets hurt when you stop verifying. The answer: everyone least equipped to defend themselves. Children misled by misinformation. Elderly people trusting fake health data. Marginalized groups targeted by biased models.
AI widens moral distance. The further removed you are from the people affected by your errors, the easier it becomes to stop caring.[^22][^23] Jake Moffatt didn’t just lose a refund; he lost dignity when a corporation replaced compassion with code.
The Convergence
Four frameworks. Different languages. Same verdict: failure to verify isn’t just sloppy—it’s unethical.
How to Actually Not Screw This Up
1. Primary Sources First
The rule is simple: feed AI verified information; don’t ask it to verify for you. Traditional research goes: find sources → read → synthesize → verify. AI-augmented research goes: gather verified sources → feed them to the model → identify patterns → verify again.
You stay the arbiter. The AI remains a tool.
The American Bar Association’s Formal Opinion 512 makes this explicit: lawyers must verify AI outputs and understand that models “lack reasoning ability.”[^24] The American Medical Association echoes it: “qualified human intervention required in clinical decisions.”[^25]
2. Cross-Model Triangulation (With Limits)
Cross-checking multiple models helps when exploring ideas, not when verifying facts. ChatGPT, Claude, and Gemini were all trained on similar data. If they all hallucinate the same reference, it’s still fake. Consensus among parrots doesn’t make it true.
3. The Socratic Method
Princeton researchers found that Socratic prompting—asking a model to explain its reasoning step by step—reduces hallucination rates.[^26] It doesn’t eliminate them. It just forces the AI to expose its thought process so you can see where it leaps off the rails.
4. Professional Standards Exist for a Reason
The professional codes are clear:
ABA (Legal): Verify everything. Don’t bill for AI time saved.[^27]
APA (Psychology): Preserve prompts, don’t list AI as an author.[^28]
AMA (Medical): Human oversight mandatory.[^29]
ACM (Computing): Reassess machine-learning risks regularly.[^30]
If you fail to follow these, you’re not being “innovative.” You’re being reckless.
5. Use Verification Tiers
Not every claim deserves the same level of scrutiny.[^31]
High stakes: Primary sources, multiple confirmations, expert review, full documentation.
Medium stakes: Check stats, confirm key facts, cross-reference anything unusual.
Low stakes: Read skeptically, spot-check surprises, know you’re brainstorming.
The Mata lawyers used a high-stakes system with zero verification. That’s why they’re in the footnotes of history as examples of what not to do.
Conclusion: Responsibility Is Not Delegable
AI can extend your cognition, but it cannot assume your conscience. Every hallucination it produces is still your lie if you repeat it.
The lawyers, researchers, and corporations who’ve already learned this lesson the hard way didn’t fail because they were stupid. They failed because they outsourced the moral act of truthfulness to a system incapable of ethics.
AI is a cognitive prosthetic, not a conscience. It can accelerate research, but it can’t guarantee honesty. It can generate arguments, but it can’t shoulder responsibility.
You are ultimately responsible for being truthful. Not the AI.
And when the next hallucination sneaks through—and it will—don’t expect “the chatbot said so” to save you.
Check your sources. Keep your authority. Remember what’s at stake.
Because when the noise gets this loud, truth depends on whoever’s still willing to think.
[^1]: Mata v. Avianca, Inc., 2023 WL 4114965 (S.D.N.Y. June 22, 2023)
[^2]: Wadsworth v. Walmart Inc., Order on Sanctions, 2:23-CV-118-KHR (D. Wyo. Feb. 24, 2025)
[^3]: United States v. Michael Cohen, Case 1:18-cr-00602 (S.D.N.Y. 2023)
[^4]: Reason.com, “Sanctions on Lawyers for Filing Motion Containing AI-Hallucinated Cases” (Feb. 25, 2025)
[^5]: Retraction Watch, “As Springer Nature journal clears AI papers, one university's retractions rise drastically” (Feb. 10, 2025)
[^6]: MIT Technology Review, “AI models are using material from retracted scientific papers” (Sep. 23, 2025)
[^7]: Moffatt v. Air Canada, 2024 BCCRT 149 (Feb. 14, 2024)
[^8]: Omar, M., Sorin, V., Collins, J.D. et al., “Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks,” Communications Medicine 5, 330 (2025)
[^9]: Kalai, A.T. & Nachum, O., “Why Language Models Hallucinate,” arXiv:2509.04664 (Sep. 2025)
[^10]: Zagzebski, L.T., Epistemic Authority: A Theory of Trust, Authority, and Autonomy in Belief (Oxford University Press, 2012)
[^11]: Goldman, A.I., Knowledge in a Social World (Oxford University Press, 1999)
[^12]: Clark, A. & Chalmers, D., “The Extended Mind,” Analysis 58(1), 1998
[^13]: Code, L., Epistemic Responsibility (State University of New York Press, 1987/2020)
[^14]: Horowitz, M.C. & Kahn, L., “Bending the Automation Bias Curve: A Study of Human and AI-based Decision Making in National Security Contexts,” International Studies Quarterly 68(2) (2024)
[^15]: Vicente, L. & Matute, H., “Humans inherit artificial intelligence biases,” Scientific Reports 13, 15737 (2023)
[^16]: Stanford Encyclopedia of Philosophy, “Virtue Epistemology” (2025)
[^17]: Vallor, S., Technology and the Virtues: A Philosophical Guide to a Future Worth Wanting (Oxford University Press, 2016)
[^18]: Ponemon Institute AI Cost Study (2024)
[^19]: Retraction Watch Database (2024)
[^20]: Kant, I., Groundwork of the Metaphysics of Morals (1785)
[^21]: Korsgaard, C.M., “Kantian Ethics and AI” discussions (2020–2024)
[^22]: Gilligan, C., In a Different Voice (Harvard University Press, 1982); Noddings, N., Caring: A Feminine Approach to Ethics and Moral Education (University of California Press, 1984)
[^23]: Research on moral distance in AI decision-making (2023–2024)
[^24]: American Bar Association Formal Opinion 512, “Generative Artificial Intelligence Tools” (July 29, 2024)
[^25]: American Medical Association, “AMA Principles for Augmented Intelligence” (Nov. 12, 2024)
[^26]: Princeton NLP Group, “The Socratic Method for Self-Discovery in Large Language Models” (2024)
[^27]: ABA Formal Opinion 512
[^28]: American Psychological Association, “Artificial Intelligence and the Field of Psychology” (Aug. 2024)
[^29]: AMA Principles for Augmented Intelligence
[^30]: ACM Code of Ethics and Professional Conduct (Updated June 22, 2018)
[^31]: International Fact-Checking Network Code of Principles; various fact-checking methodologies
