Can AI make mistakes in legal documents? Yes, and Indian courts are now saying who is responsible.
Short answer. Yes. On 2 July 2026 the Supreme Court of India set aside NCLT and NCLAT orders because the NCLT had relied on non-existent, AI-generated precedents, declared “zero tolerance” for unverified AI citations, and asked the Bar Council of India to frame disciplinary principles. Studies explain why: Stanford researchers found commercial legal research tools built on retrieval hallucinated between 17% and 33% of the time, and general-purpose models between 58% and 88% on specific questions about federal cases. AI can still help with drafting and review, but every citation, quotation and date must be checked against the source by a lawyer. This is general information, not legal advice.
01
What the Supreme Court found
In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., decided on 2 July 2026 by Justices P S Narasimha and Alok Aradhe, the Court examined an insolvency order in which the NCLT had relied on six precedents. As reported by Hindustan Times, some citations did not exist, some were real but the paragraph attributed to them did not exist, and one citation number belonged to a different judgment. The Court held that a decision resting on fake or hallucinated material is “no decision” in law, that even an iota of such reliance vitiates it, and that artificial intelligence may still be used as an aid.
The Court directed the Bar Council of India to constitute a committee, frame guiding principles and prescribe disciplinary consequences for advocates who rely on or submit such material. Hindustan Times reports that the bench called citing hallucinated judgments professional misconduct for an advocate and a serious lapse for a judge. It was not the first warning: on 27 February 2026 the same bench took cognisance of a trial court relying on AI-generated, non-existent verdicts.
02
Why it keeps happening
Both studies used US law, but the failure is the one the Indian Court describes: plausible, well-formatted authority that does not exist. The Stanford team noted that vendors had described retrieval as eliminating or avoiding hallucinations.
Study
What it tested
Result
Magesh et al., Stanford, 2024
Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI, which retrieve sources
Each hallucinated between 17% and 33% of the time
Dahl et al., Journal of Legal Analysis, 2024
General-purpose models asked specific, verifiable questions about random federal court cases
Hallucination between 58% (GPT-4) and 88% (Llama 2)
03
Where mistakes show up in drafting
Invented citations, or real citations attached to the wrong case.
A real judgment with a paragraph or quotation it does not contain.
Wrong court, wrong jurisdiction or superseded law.
Clause summaries that leave out exceptions or conditions.
Party names, dates and amounts carried between documents.
A confident tone that hides the uncertainty. NIST warns that confabulated logic or citations can mislead people into trusting an output.
04
How to use AI in legal work safely
Treat output as a draft from a junior colleague. A lawyer signs.
Verify every citation against a primary source, such as the law report, the gazette or the court’s own site, before it goes anywhere.
Ask for the quoted passage and read it in the original, paragraph number included.
Use AI on your own documents for summarising, comparing and finding, where the source passage sits beside the answer.
Keep a record of what was generated, by whom and what was checked.
Do not paste client confidences into tools you do not control.
05
Where ANVE fits
ANVE shows the source passage next to answers over the office’s own documents and logs every model call. A legal research agent is something ANVEAI would build only with a paid pilot; it is not a shipped product. ANVE is pre-release and holds no certifications.
06
Questions people ask
Is it misconduct to cite an AI-invented case in India?
Hindustan Times reports that the Supreme Court said so in July 2026, and the Court asked the Bar Council of India to frame guidelines and disciplinary consequences. Check the Bar Council’s current position and ask your own counsel.
Can AI check contracts for errors?
It can flag candidates, such as inconsistent defined terms or missing clauses, but a lawyer must confirm each one. The error rates above come from legal research and citation questions; contract review has its own failure modes.
Do legal AI tools that show citations avoid hallucinations?
No. The Stanford study found that tools which retrieve sources and cite them still hallucinated between 17% and 33% of the time, because a citation can exist and still not support the claim.
Does running the AI locally reduce errors?
No. Where the model runs changes who can see your data, not how often it errs. Larger models and answers grounded in your own documents help, but neither removes the need to check.
07
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
