Is using AI for therapy safe? Study raises concerns

A new study from Brown University raises concerns about using AI chatbots like ChatGPT for mental health support. Presented at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society, the research identifies 15 key risks linked to AI therapy.

The study concluded that AI chatbots don't adapt to personal context, offer generic advice, or respond inadequately in personal crises. The identified risks include a lack of contextual adaptation, substandard therapeutic collaboration, deceptive empathy, unfair discrimination, and poor safety management.

The use of AI in some cases can be deceptive, such as in the use of empathetic phrases such as “I understand", which don't necessarily mean they have any real emotions. It can also perpetuate false beliefs, inadequate support in crises such as suicidal tendencies, and so on.

Another Ph.D. student at Brown University, Zainab Iftikhar, explained that the main reason is that humans are answerable to professional associations, but artificial intelligence is not. “When LLM counsellors make mistakes, there are no regulatory frameworks to hold them liable,” she said.

Another professor at Brown University, Ellie Pavlick, who teaches computer science, said that the evaluation process of artificial intelligence is not properly reviewed. “It is far easier to build and deploy systems than to understand them fully. Careful critique is essential to avoid doing more harm than good,” Pavlick said.

Experts are saying that using chatbots for therapy could be dangerous and could cause harm if proper precautions are not taken.