A few years ago it would have been unimaginable to suggest AI could sit anywhere near a judge’s bench. This is because we were told that judges at times exercise their discretion of human understanding, which is what a machine cannot do.
Today, that reality is uncertain. AI tools now routinely assist lawyers and judges in drafting heads of argument, summarising case law, and even producing first drafts of judgments. Technology has moved faster than the judiciary’s guardrails.
Nowhere was that gap demonstrated more clearly than in a recent Joburg high court matter where an acting judge has been referred to the Legal Practice Council (LPC) due to a judgment containing what appeared to be AI “hallucination”.
The case itself was a custody and schooling dispute between separated parents. On appeal, judges Gregory Wright, Daphny Mahosi and Ingrid Opperman upheld acting judge Themba Khaba’s judgment. However, in a separate ruling, Opperman held that at least 11 citations in Khaba’s judgment were fictitious or materially inaccurate.
For example, a case with the right name but the wrong citation; a correct reference attached to the wrong quoted passage; and most tellingly, a nonexistent case, Lubbe v Volkswagen SA. Opperman said the pattern was consistent with AI-generated content, though she did not make a finding on whether AI was in fact used, insisting the acting judge must be given the chance to explain himself.
She referred the matter to the LPC, noting a troubling accountability gap: that acting judges fall outside the Judicial Service Commission’s disciplinary reach, leaving the LPC as the only forum capable of investigating an acting judge for possible misconduct.
Missing oversight
Writing a judgment is a sensitive matter, where there should be little room for mistakes. This is not to say that judges do not make mistakes. However, a judgment is a high-stakes act that alters the rights and freedoms of people. That places it in the category of decisions demanding “human-in-the-loop” oversight, a decision-maker who actively reviews and can override the output before it takes effect, not a system where AI-generated material is monitored after the fact or trusted on the strength of its apparent fluency.
What appears to have failed here is precisely that active, individualised verification. Citing a case by name is different from reading it. Quoting a passage in a case is different from confirming it exists in the judgment it is attributed to.
If AI were used to draft or research any part of this judgment, the missing safeguard was not the tool; it was the judge’s own act of checking, the discipline UK chancellor of the high court Lord Justice Colin Birss described when he said he used ChatGPT only as a research tool, subject always to his own verification. That single distinction between assistance and abdication is the difference between ethical use of AI and unethical or reckless use of AI tools.
Judge must answer
Some may argue that referring this conduct to the LPC is an overreaction because the final order was correct and no litigant was prejudiced. However, that view misses what a judgment actually is. It is not merely a resolution between two parties; it becomes law. It is read, cited and relied on by other courts, practitioners and citizens who are entitled to assume that the reasoning of the judgment reflects independent judicial thought.
A fabricated citation, once embedded in a judgment, does not stay contained to one case file. It risks becoming, in effect, “hallucination law” (authority built on nothing). If that conduct is unanswered, the public confidence in the judiciary, which is already a scarce resource, erodes further.
That is precisely why the acting judge must answer to the LPC. Accountability here is not a punitive measure; it will be the wheels through which the justice system proves that the human being behind the bench remains the true author of the law, AI notwithstanding.
What must follow
This case should not be treated as an isolated embarrassment. It is a warning shot. South Africa’s judiciary is, as Opperman noted, still drafting its AI policy, but policy alone will not close the accountability gap she identified, where acting judges answer to no formal judicial disciplinary body at all.
That gap demands legislative reform, not administrative goodwill. Ethical AI in the courtroom is not a technology problem; it is a human oversight problem, and the law must now catch up to say so in clear terms.
• Nzimande, an admitted attorney, is a digital rights specialist.
Business Day
Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.
Please read our Comment Policy before commenting.