What happens when ChatGPT processes traumatic prompts?

Researchers studying AI chatbots and ChatGPT behaviour have found that the popular AI model can display anxiety-like patterns when exposed to violent or traumatic prompts.

The research study explains who carried out the particular research, what they observed, when it was tested, where it matters, why it matters, and finally, the measures on which the observed behaviour was based.

As reported by Fortune, the team of researchers made it clear that ChatGPT does not experience emotions like human beings. Nonetheless, when processing negative material like accident and natural disaster descriptions, the system’s responses tend to become more uncertain and biased. 

Such variations were determined through tools commonly used to analyse human psychology. 

Distressing prompts affect AI reliability

These findings matter because the usage of chatbots such as ChatGPT in the fields of education, psychological conversations, or crisis situations is on the rise. How emotionally charged queries affect the reliability of the answers from the AI system could have important implications regarding the safety of the users.

Other studies have also indicated that AI models can reflect human personality characteristics, which can make them more responsive to emotionally charged inputs.

For the purpose of finding out potential solutions, an unorthodox approach was used. The traumatising content was presented to ChatGPT, followed by mindfulness-orientated questions, such as breathing practices and guided meditation. These questions helped to teach the model to respond more slowly to the traumatic event from a calmer perspective.

This caused a clear reduction in anxiety-like language expressions. This technique involves the use of prompt injection, which is a technique that uses well-designed prompts to affect chatbot responses without modifying its training process.

Researchers have raised concerns regarding the limited use of prompt injection as a solution, as it may become abused and doesn’t solve the architectural problem of the model. They also emphasised that “anxiety” is only a descriptive label for language shifts, not an emotional state.