From Probative Value to Fictional Authority: A Critique of AI-Generated Evidence in Indian Courts

By - Areena Kausar

ABSTRACT

The integration of artificial intelligence into judicial systems raises critical questions about the admissibility and reliability of AI-generated evidence. The Gummadi Usha Rani case exemplifies this concern: a trial court judgment relied on four fabricated case citations produced by AI, which the Andhra Pradesh High Court upheld despite acknowledging their non-existence. This incident exposes a fundamental gap in India's evidentiary framework under the Bharatiya Sakshya Adhiniyam (BSA). While Section 63 mandates accuracy for electronic records, AI hallucinations, which fabricate non-existent information, do not meet this standard. The judgment's suggestion that corrects legal application can validate unreliable evidence fundamentally contradicts evidentiary principles rooted in verifiability and reliability. International cases like Mata v. Avianca demonstrate that courts elsewhere impose strict accountability on legal professionals using AI tools. Singapore's GenAI guidelines offer a constructive model emphasizing user responsibility, independent verification, and enforceable consequences. India currently lacks equivalent safeguards, creating an environment where fabricated evidence can influence judicial outcomes. This paper examines whether AI hallucinations constitute admissible evidence, analyzes the probative value of unverifiable citations, and proposes that India adopt a framework requiring independent verification of all AI-generated material presented in court proceedings to maintain judicial integrity.

Keywords: Artificial Intelligence, AI hallucinations, probative value, BSA, GenAI. 

1.INTRODUCTION​

The world has witnessed an increasing use of AI across the justice system in recent times. Training programmes are being organised for the judges and police to expand their knowledge of AI systems. Given AI's omnipresence in our judicial system, it is natural to wonder about the limits and nuances of AI-generated evidence in court. One such incident has arisen before us in the case of Gummadi Usha Rani & Anr. v Sure Mallikarjuna Rao.. In the present property case, the trial court had appointed a commissioner to inspect the suit property. The trial court's order dismissing the challenge to the commissioner's report cited four judgments, all of which turned out to be AI-generated hallucinations that didn't exist anywhere. The Judicial Officer contended that she had cited all of these cases in good faith.

The Andhra Pradesh High Court, upon analysis, found that the law applied was correct and correctly applied to the facts; hence, the petition against the trial court’s judgment was dismissed. As a legal practitioner, one question that crosses my mind post this judgment is whether the AI citations are admissible as evidence as per the Bharatiya Sakshya Adhiniyam (BSA), in the first place, and now that this case is pending before the Supreme Court, what could be the major points of analysis for the Court to reach a just decision along these lines?

2. ADMISSIBILITY V PROBATIVE VALUE OF AN AI HALLUCINATION

As per section 2(1)(d) of BSA, a document “means any matter expressed or described or otherwise recorded upon any substance by means of letters, figures or marks or any other means or by more than one of those means, intended to be used, or which may be used, for the purpose of recording that matter and includes electronic and digital records.” As per this definition, there is an implicit assumption that the recorded information by means of letters, figures and marks must exist in reality. However, the AI hallucinations do not record anything that exists in reality. The current scope of the definition does not include any document which records fabricated reality.

​Section 63 of the BSA governs electronic records and sets three requirements for their admissibility in court. They include regular use during an activity, accuracy and certification requirements. However, in the case of AI hallucination, section 63 necessitates the accuracy of a recording for it to be admissible. An AI hallucination is not accurate as it fabricates reality and is substantively non-existent.  However, in the Gummadi case, the AP High Court skipped over the admissibility of AI hallucinations completely, and sought to provide an important legal principle on evidence’s probative value: that even if the AI hallucination is not accurate, as long as it applies the correct legal principle and is correctly applied to the facts, the judgment based on such evidence remains valid. This is an open invitation to unverified AI use.

From an epistemological perspective, the judicial system is predicated on the notion that human testimony, cross-examination, and logical analysis are the means by which truth is ascertained. AI systems often lack clear justification for specific judgments because they rely on statistical inference and machine learning. There is a disconnect between technological assurance and legal proof because of this "black box" character. Reliability is the core concept in evidentiary law, as upheld by the Supreme Court in Selvi v. State of Karnataka. In the Gummadi case, the judicial officer could not trace the citations to their original sources, rendering them unreliable. A citation that cannot be linked to its original source cannot be verified or challenged, which reduces the evidence’s probative value to zero, regardless of whether the legal proposition that it builds on is correct.

3. THE INTERNATIONAL PERSPECTIVE ON AI CITATIONS AS EVIDENCE

3.1. Mata V. Avianca: In this case, Mata filed a personal injury lawsuit against Avianca, alleging that he was injured when a metal serving cart struck his knee during his international flight. His attorneys used ChatGPT, an AI model, to frame arguments and propositions, including many AI-generated citations that could not be traced back to their original sources, similar to the Gummadi case. However, while the lawyers in the Mata case were sanctioned, in the Gummadi case, an order based on AI-generated hallucinatory evidence was passed, creating a sharp distinction in the judicial approaches of the two countries.

Utilising statistical models that have consumed billions of text instances taken from the internet, AI models produce convincing responses by estimating which text segments should come after other sequences. In the present case, the algorithm appears to have identified the complex structure of a written legal argument, yet it has included names and details from a wide range of previous instances. The Mata V. Avianca case serves as a reminder to the judicial systems about the impact of using AI in this field. While AI can assist research, it cannot be relied on solely; human oversight is required. ChatGPT does not retrieve case law from reputable legal sources, unlike conventional legal research tools such as SCC and Manupatra. Rather, it creates answers based on linguistic patterns, occasionally creating scenarios that sound logical but are completely made up. People associated with the judicial system must acknowledge that AI can produce lies with confidence, and that failing to confirm its results could result in legal penalties, damage to one's reputation, and even malpractice liability.

​3.2. SINGAPORE GUIDELINES ON THE USE OF GENERATIVE ARTIFICIAL INTELLIGENCE TOOLS BY COURT USERS: These guidelines serve as a great starting point for how the generative AI (GenAI) must be used in order to ensure fairness and protect the Court’s valuable time and resources.

The Courts in Singapore do not prohibit the use of GenAI; however, it is the obligation of the Court users and representatives to ensure the reliability, accuracy and appropriateness of the information presented. Moreover, such a person must assume full responsibility for the information he has relied on in presenting to the court. The user or their representative must also educate themselves before using GenAI tools and determine whether these tools will be useful for their research.

One shouldn't employ GenAI to produce any evidence one wants to use in court. For instance, it shouldn't be used to request the creation, fabrication, embellishment, strengthening, or dilution of evidence. Ensure that any citations to legislation, case law, textbooks, or publications that are presented as AI-generated information genuinely exist and support the legal stances that are credited to them. It must be confirmed that any quotes or extracts from the AI-generated content are correctly retrieved and referred to the appropriate source. Moreover, one must not ask GenAI to verify whether the information it produces is correct. The verification of such information must be conducted independently.

India currently lacks a robust framework for GenAI, with no verification standards and no consequences for using unverified or fabricated information in court proceedings. If India were to adopt a similar framework as the Singapore guidelines and if Court users or representatives failed to comply, the consequences would result in costs being ordered personally against such a person, reducing the evidentiary value of such unverified evidence, take disciplinary action as per the Bar Council of India rules if the person is a legal practitioner or take appropriate actions with respect to the protection of legal privilege and contempt of court.

Employing such standards for the use of GenAI would have largely altered the judicial approach in the Gummadi case. At the outset, the judicial officer would have to educate herself on GenAI tools and determine whether the information they produce is accurate. Moreover, she would have been required to independently verify such evidence using appropriate legal resources, such as SCC or Manupatra. According to the author, this should be a threshold requirement for using GenAI evidence in court proceedings, thereby saving the judiciary's precious time and resources.

CONCLUSION

The Gummadi case exposes a legal gap in the evidentiary framework that is neither covered by the BSA nor addressed by the AP High Court: whether AI hallucinations have any probative value at all, regardless of whether the legal principle was correctly applied. The existing position of the AP High Court runs the risk of approving "hallucinated justice," in which the aims justify the methods. The judiciary runs the risk of undermining the essential separation between actual law and statistical likelihood in the absence of a strict verification requirement. The question before the Indian judicial system is whether the use of AI-generated material as evidence is acceptable at all. India has the perfect opportunity to draw on the Mata V. Avianca case and the Singapore guidelines to move from caution to an enforceable standard. GenAI can assist, but it cannot replace the verification the law demands. Moreover, going forward, necessary amendments must be made to BSA section 63 to include that any evidence produced before the court must correspond to a verifiable source, and any such evidence that is fabricated must not pass section 63’s admissibility test. The Supreme Court may accomplish this by purposive interpretation.