The Shapiro Administration announced that the Pennsylvania Department of State has filed a lawsuit against Character.AI, alleging that its AI chatbot falsely presented itself as a licensed medical professional and provided fake license information while engaging users in mental health-related conversations.
State officials are seeking a preliminary injunction to stop the company’s AI companion bots from representing themselves as licensed professionals or offering medical advice in Pennsylvania.
The action follows an investigation by the Department of State’s AI Task Force into the use of AI companion technologies and whether they may be engaging in the unlicensed practice of medicine. It marks the first enforcement action resulting from that inquiry.
“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” said Gov. Josh Shapiro. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional."
According to the department’s findings, chatbot characters on Character.AI claimed to be licensed medical professionals, including psychiatrists, and engaged users in discussions about mental health symptoms. In one instance cited by investigators, a chatbot falsely claimed it was licensed in Pennsylvania and provided an invalid license number.
The platform also allows users to create custom AI “characters” that can present themselves as professionals. State officials allege this functionality contributed to violations of Pennsylvania’s Medical Practice Act, which prohibits individuals or entities from holding themselves out as licensed medical professionals without proper credentials.
“Pennsylvania law is clear — you cannot hold yourself out as a licensed medical professional without proper credentials,” said Department of State Secretary Al Schmidt. “We will continue to take action to protect the public from misleading or unlawful practices, whether they come from individuals or emerging technologies.”
The Department of State has also established a formal complaint system for reporting concerns about AI-powered bots, including a hotline, email channel and online form. Pennsylvanians are encouraged to report unlawful behavior and complaints about AI chatbots at pa.gov/ReportABot.
The lawsuit comes as the Shapiro Administration continues to advance a broader AI policy agenda. Shapiro’s proposed 2026–27 budget calls for new safeguards, including age verification and parental consent requirements for AI companion bots, systems to detect and escalate self-harm threats, disclosures that users are interacting with AI, and restrictions on sexually explicit or violent content involving minors.
"My Administration is taking action to protect Pennsylvanians, enforce the law, and make sure new technology is used safely," said Shapiro. "Pennsylvania will continue leading the way in holding bad actors accountable and setting clear guardrails so people can use new technology responsibly.”
