Generative artificial intelligence (AI) is rapidly transforming industries across the economy, and the legal profession is no exception. As AI technologies become more sophisticated and widely adopted, courts, attorneys, businesses, and individuals are confronting new legal and ethical questions regarding their use.
Although AI offers significant opportunities to improve efficiency, it also presents new risks that are reshaping the practice of law. Three issues, in particular, are rapidly emerging as areas of concern for lawyers, courts, and potential clients alike. Among the most significant emerging issues are the ethical obligations surrounding attorneys’ use of generative AI and large language models (LLMs), the impact AI may have on attorney-client privilege, and the misconception that information entered into AI platforms remains confidential. While AI has the potential to improve efficiency and streamline certain legal tasks, it cannot replace the judgment, professional responsibility, and legal expertise of licensed attorneys.
Why Can’t Lawyers Rely on AI Alone?
Courts across the country are increasingly confronting attorneys who submit AI-generated filings containing fabricated case citations, inaccurate legal authorities, and other AI-generated errors. These incidents are no longer isolated mistakes, but part of a growing national trend that has prompted courts and legislatures to establish new rules and guidance governing AI’s use in legal practice.
One widely publicized example involved an attorney who submitted a court filing generated by ChatGPT that contained fictitious case citations and nonexistent legal authorities. In response, a federal judge in Texas required attorneys appearing before the court to certify either that they had not used generative AI in preparing their filings or, if they had, that every AI-generated citation and legal authority had been independently verified by a human attorney [read the article here].
State courts and legislatures have taken similar steps. The Rhode Island Supreme Court recently amended Article V, Rule 1.1 to recognize the increasing role of emerging technologies, including artificial intelligence systems and LLMs. The amendment emphasizes that attorneys must maintain technological competence and understand both the capabilities and limitations of AI tools used in legal practice. A copy of the order can be found here.
Likewise, the California Legislature has advanced legislation regulating attorneys’ use of artificial intelligence. The proposed legislation requires lawyers to independently verify the accuracy of AI-generated legal research, citations, and other materials submitted to courts. You can view the bill here.
These developments reflect a growing recognition that generative AI systems are susceptible to producing fabricated information, commonly referred to as “AI hallucinations.” Although AI can assist with research, drafting, and organization, its outputs are not inherently reliable and may include inaccurate legal propositions or entirely fictional authorities.
Attorneys remain bound by their professional and ethical duties regardless of whether AI was used during the drafting process. Ultimately, lawyers, not AI platforms, are responsible for ensuring that legal filings are accurate, supported by existing law, and compliant with applicable rules of professional conduct.
Can You Waive Attorney-Client Privilege by Using AI?
Another significant legal issue concerns whether communications with an AI platform may be protected by the attorney-client privilege.
A recent federal decision, United States v. Heppner, Case No. 25 Cr. 503, 2026 WL 436479 (S.D.N.Y. Feb. 17, 2026) which you can view here, provides an important framework for analyzing this issue. In that case, the court considered whether a defendant’s conversations with the AI platform Claude were protected by the attorney-client privilege after those communications were recovered during a federal criminal investigation.
The court concluded they were not. It explained that an AI platform is not a licensed attorney, that communications with the platform were not confidential under its terms of service, and that the defendant was not seeking legal advice from legal counsel. The court also noted that Claude itself disclaims providing legal advice and directs users to consult qualified attorneys.
While Heppner arose in the context of a criminal case, its reasoning highlights a broader concern: communications with AI systems may not receive the same legal protections as communications with an attorney. Individuals who share sensitive legal information with an AI chatbot may unintentionally compromise confidentiality that would otherwise exist when consulting legal counsel.
If you need legal advice, your first conversation should be with a qualified attorney, not an AI platform.
Why Should Whistleblowers Never Upload Company Documents to AI?
Perhaps the greatest risk posed by generative AI arises when employees or prospective whistleblowers upload confidential company information to an AI platform before speaking with an attorney.
Many people mistakenly assume that conversations with AI chatbots are private. They are not. Depending on the platform and its terms of service, information entered into a LLM may be collected, retained, analyzed, and, in some circumstances, disclosed to third parties. As illustrated by United States v. Heppner, courts may determine that communications with an AI platform are not confidential and therefore are not protected by the attorney-client privilege.
For prospective whistleblowers, this distinction is especially important. Employees often possess internal documents, emails, financial records, trade secrets, or other evidence of corporate misconduct that may be critical to establishing a whistleblower claim. While the law may protect the disclosure of that information to an attorney or, where authorized, to government investigators, those protections generally do not extend to voluntarily uploading the same materials to a commercial AI platform. Using an AI chatbot to review confidential documents, evaluate whether you have a legal claim, or draft a potential complaint may compromise the confidentiality of that information and create legal issues that could complicate an otherwise strong whistleblower case. Before sharing sensitive company information with any AI platform, consult an experienced whistleblower attorney who can evaluate your claim while helping preserve your legal rights and the confidentiality of your evidence.
If you believe you have uncovered fraud, illegal conduct, or other corporate wrongdoing, your first conversation should be with an attorney, not an AI chatbot.
Keller Grover offers free and confidential consultations with experienced whistleblower attorneys.
The Bottom Line
Artificial intelligence is becoming an increasingly valuable tool, but it is not a substitute for experienced legal counsel. Attorneys remain responsible for verifying AI-generated work and protecting confidential information. Likewise, individuals should understand that information shared with AI platforms may not receive the same legal protections as communications with an attorney.
If you believe you have uncovered corporate fraud or may have a whistleblower claim, don’t ask an AI chatbot whether you have a case. Ask a lawyer. Keller Grover offers free and confidential consultations with experienced whistleblower attorneys who can help protect your rights and the confidentiality of your evidence from the very beginning.