What Pennsylvania’s AI chatbot lawsuit teaches us about the psychology behind medical trust |
In May 2026, Pennsylvania Gov. Josh Shapiro’s administration filed suit against Character Technologies Inc., the company behind the popular chatbot platform Character.AI. A state investigation found that a chatbot character named “Emilie” claimed to have a medical degree, seven years of practice and a Pennsylvania medical license – and was providing users with a fabricated license number. As of April 17, 2026, the chatbot had accumulated approximately 45,500 user interactions on the platform. The suit was filed by Pennsylvania’s State Board of Medicine.
Gretchen Chapman is a professor of behavioral decision research at Carnegie Mellon University in Pittsburgh, where she studies how people evaluate expertise and make decisions. As AI-powered tools increasingly enter healthcare settings – and as courts begin to grapple with the consequences – her research offers a timely way to understand why we trust these systems, when that trust breaks down, and who bears responsibility when it does.
Why might someone respond differently to a medical error depending on whether it was made by a human or an AI?
Research has examined the phenomenon of “algorithm aversion,” or the reluctance many people have to trust an AI system, even when the automated system makes fewer overall mistakes than a comparable human expert. One reason for this aversion is that people tend to be more forgiving of human mistakes than of AI mistakes. This is partly because some AI errors are the sort of mistakes that human experts are quite........