The Supreme Court on Thursday set aside a National Company Law Tribunal (NCLT) judgment that relied on non-existent, fake, and hallucinated precedents generated by artificial intelligence (AI), warning that such practices are a catastrophic threat to judicial integrity. A bench of Justices PS Narasimha and Alok Aradhe declared that courts must adopt zero tolerance against citing or using AI-generated precedents without verification.
Court Condemns Use of Fake AI Precedents
Comparing the use of fabricated AI material to the release of methyl isocyanate, the bench stated: "The production of fake, non-existent, and hallucinated material and its utilisation as precedents in law, is like the release of methyl isocyanate in the province of law and justice: invisible, insidious, and catastrophic by the time anyone notices. It not only contaminates but takes away the very lifeblood of judicial determination." The court emphasized that such decisions are void ab initio, regardless of whether the fake material had a direct or indirect bearing on the outcome.
Details of the Case: Essel Infraprojects Insolvency
The case originated from an insolvency dispute involving Pooja Ramesh Singh, Jammu and Kashmir Bank Ltd, and Essel Infraprojects Ltd. The appellant challenged an NCLT Mumbai order that admitted a Section 7 insolvency application under the Insolvency and Bankruptcy Code. Upon review, the Supreme Court discovered that several "precedents" cited by the NCLT to justify its decision simply did not exist. These included fabricated case names such as ICICI Bank Ltd vs Urban Infrastructure Real Estate Ltd (2019) 16 SCC 528 and Sarbjit Singh vs Union Bank of India (2022) 7 SCC 464, both of which were entirely non-existent citations. The court noted that the NCLT had obtained these through its "own research," as confirmed by an affidavit from Jammu and Kashmir Bank stating that their counsel had not cited these cases.
Zero Tolerance for AI-Generated Fabrications
The Supreme Court declared that it is a "misconduct on the part of an advocate to cite such judgments without verification" and a "serious lapse" if a judge relies on such material. "We have no hesitation in declaring that such a decision is no decision in the eyes of the law, irrespective of whether such material had a direct or indirect bearing on the decision-making. Such decisions are to be set aside even if an iota of fake or hallucinated material enters the decision-making process, as it would violate the sanctity of adjudication," the bench said. The court clarified that its judgment does not preclude the rightful use of AI but strictly prohibits reliance on fake or hallucinated material as court precedents.
Directive to Bar Council of India
Recognizing that mere prohibitory action is insufficient, the court directed the Bar Council of India (BCI) to constitute a committee to deliberate on the issue of advocates submitting fake and hallucinated material before courts. The BCI must prescribe guiding principles to prevent such occurrences and outline disciplinary actions for violations. The court urged the BCI to take up the matter with utmost seriousness.
Broader Implications for AI in Judiciary
The Supreme Court observed that while AI technology can aid adjudication, total human control must be maintained at every stage. "More than the inevitable consequence of setting aside such judgments, what is significant for our decision-making is our resolve to adopt AI technology in aid of adjudication, while at the same time asserting and declaring total and absolute control over adjudication, with a human in the loop at every stage," the bench noted. This case underscores the dangers of unchecked AI use in legal proceedings and reinforces the need for strict verification of legal citations.
