Delhi High Court has cautioned the Tax Department (both GST and IT) about the use of Artificial Intelligence (AI) in show cause notices.
“The GST Department as well as the IT Department must exercise utmost caution while citing judgements and must take full responsibility in case the same is cited or generated by using Artificial Intelligence software,” a division bench comprised of Justices Prathibha M Singh and Shail Singh said in a recent ruling while disposing a petition filed to quash a show cause notice (SCN) issued by GST Departments.
Though the bench held that the SCN is completely pre-mature and said that the petitioner ought to reply to the notice and participate in the proceedings, but also expressed strong views on use of AI in issuing the notice. It found discrepancies in the judgments which are cited by the GST Department.
It quoted from a ruling on the matter of Christian Louboutin SAS and Anr. vs M/S Shoe Boutique where it was said that responses from ChatGPT as also the one relied upon by the Plaintiffs shows that the said tool cannot be the basis of adjudication of legal or factual issues in a court of law. The response of a Large Language Model (LLM) based chatbots such as ChatGPT, which is sought to be relied upon by ld. Counsel for the Plaintiff, depends upon a host of factors including the nature and structure of query put by the user, the training data etc. Further, there are possibilities of incorrect responses, fictional case laws, imaginative data etc. generated by AI chatbots
“Accuracy and reliability of AI generated data is still in the grey area. There is no doubt in the mind of the Court that, at the present stage of technological development, AI cannot substitute either the human intelligence or the humane element in the adjudicatory process. At best the tool could be utilised for a preliminary understanding or for preliminary research and nothing more,” the ruling had said.
Accordingly, the bench observed that the said judicial precedents “clearly demonstrate the risk of AI hallucinating, by citing fake and non-existent judgements. Under such circumstances, the GST Department as well as the IT Department must exercise utmost caution while citing judgements and must take full responsibility in case the same is cited or generated by using AI software. Moreover, before issuing SCNs or finalising assessments, all judgements ought to be verified,” it said.
Commenting on the ruling, an tax expert said, that the Delhi High Court while acknowledging that AI can assist in analysing evidence and preparing summaries, has cautioned that government departments must verify all AI-generated content before citing it in official documents. “The ruling highlights the risk of ‘AI hallucinations’ i.e., fake or non-existent judgments being cited and underscores that presumptions under the Income Tax Act cannot automatically apply to GST proceedings,” he said.
