In brief
The survey of 360 judges discovered widespread use of AI hallucinations, with 80% reporting encounters with nonexistent or false AI-generated precedents. Most judges called for institutional measures, while a court official warned that such citations can delay proceedings.
In an eye‑opening report released Monday, South Korea’s Supreme Court disclosed that eight in ten judges have faced court cases where artificial‑intelligence‑generated legal precedents, none of which exist, were cited as legitimate authority by parties or even by the judges themselves.
The study, conducted from October 6 to 16, surveyed 360 judges handling civil, family, and administrative matters and found that 196 judges—or 80.7%—had seen the use of nonexistent precedents, while 159 judges—or 65.4%—spotted cases involving false precedents. The phenomenon, dubbed “AI hallucination” by the court, refers to AI models presenting fabricated data as fact.
A striking 41 judges (16.9%) reported encounters with false or nonexistent laws and regulations, and a further 10 judges (4.1%) noted citations of fake or non‑existent theses and publications. In total, 164 judges admitted to having cited false precedents themselves, and 136 judges—or 57.6%—confirmed that attorneys on either side had made such citations.
The Scope of AI Hallucinations in Korean Courts
“AI hallucinations have infiltrated our courtroom, not only undermining the integrity of legal arguments but potentially delaying proceedings,” a National Court Administration official told reporters, reflecting widespread concerns among judiciary officials.
The citation of AI‑generated false or nonexistent legal precedents may lead to delays in court proceedings, underscoring a need for countermeasures that are both effective and grounded in empirical data.
In the survey, nearly 92% of judges called for the implementation of institutional safeguards. Proposed measures include the legal establishment of fines for repeated falls into using false precedents, the requirement of verifiable sources for all cited materials, and disciplinary action against attorneys who persistently cite fabricated information.
Calls for Measures and Challenges Ahead
Supreme Court officials emphasized that the findings expose a systemic vulnerability: with AI tools increasingly accessible, the risk that non‑existent precedents can influence judicial outcomes grows sharply. The court’s own analysis suggested that these hallucinations could erode public confidence in the justice system if left unchecked, especially as citizens rely on courts to uphold the rule of law.
Concurrently, the survey underscored a potential widening of the knowledge gap among legal practitioners. Attorneys who lack proficiency in discerning AI‑generated content may inadvertently or deliberately rely on false citations, compounding the risk of miscarriages of justice. The discussion has, therefore, extended beyond the judiciary to the broader legal profession and the public at large, for who ultimately receives justice depends on the accuracy of legal precedents.
While the Supreme Court is poised to develop a framework to penalize improper citations, experts warn that the rapid evolution of AI models will outpace any static regulatory approach. Continuous education for judges and attorneys, coupled with real‑time validation tools, will be critical components of any comprehensive strategy.
In the coming months, the court is expected to announce specific guidelines and implement training modules for both judges and legal teams. The announcement will also likely set the precedent for other common‑wealth jurisdictions grappling with similar AI‑related disruptions.
As South Korea moves forward, the challenge lies in balancing technological innovation with the uncompromising integrity of legal precedent—an endeavor that will shape how citizens, corporations, and the judiciary navigate the intersection of law and artificial intelligence.
