The Supreme Court of India has warned that using Artificial Intelligence (AI) to fabricate legal citations is as dangerous as the toxic gas that caused the Bhopal disaster. In a recent judgment, the Court set aside orders of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) after finding that the lower tribunal relied on fictitious AI‑generated case law.
Key Developments
- Supreme Court likened AI hallucinations to methyl isocyanate – invisible, insidious, and catastrophic.
- Orders of NCLT and NCLAT were vacated in an insolvency case due to reliance on fake AI citations.
- Bench of Justices P.S. Narasimha and Alok Aradhe termed such reliance as judicial misconduct, not a mere error.
- Draft Regulations for Use of Artificial Intelligence (AI) in Courts, 2026 proposes a ban on AI in adjudication, sentencing, bail decisions, and credibility assessment.
- The Court directed the Bar Council of India (BCI) to form a committee for strict norms and disciplinary action against lawyers citing unverified AI material.
Important Facts
AI hallucination refers to a false output generated by an AI model that appears plausible but has no factual basis. The Court emphasized that any judgment influenced by even a single hallucinated AI fragment is "no decision in the eyes of law". The draft regulations, currently open for public consultation, explicitly prohibit AI from performing core judicial functions such as adjudication, sentencing, bail eligibility, and witness credibility evaluation.
Exam Relevance
Understanding the Court’s stance is vital for GS 2 (Polity) as it deals with the separation of powers, judicial independence, and regulation of emerging technologies. The issue also touches upon GS 4 (Ethics) – the need for professional accountability and the ethical use of AI in public institutions. Moreover, the insolvency context links to GS 3 (Economy) where corporate restructuring mechanisms are crucial for financial stability.
Way Forward
1. Human Oversight: Courts must retain human reasoning and discretion; AI can only assist in research, not decision‑making.
2. Regulatory Framework: The draft regulations need swift finalisation, clear penalties, and regular updates as AI evolves.
3. Legal Profession Training: BCI’s committee should educate advocates on verifying AI outputs and the consequences of professional misconduct.
4. Public Awareness: Stakeholders, including law students and policymakers, must be aware of AI hallucinations to prevent misuse.
By treating AI as a tool rather than a substitute for judicial judgment, India aims to safeguard the integrity of its justice delivery system while embracing technological efficiency.