ChatGPT safety concerns grow over sexual, violent images created with simple prompts: Report

OpenAI’s latest ChatGPT version, GPT-5.4, has been under fire as the researchers have issued a warning that the chatbot could be manipulated into generating sexualized and graphically violent imagery.

As reported by the BBC, the British AI security startup has exposed the safety flaw in the public version of ChatGPT. Under this loophole, simple prompts, originally intended to generate humorous content, could be tricked into bypassing safety filters to produce prohibited and obscene materials, including illustrations of sexual violence and dreadful crime scenes.

OpenAI claims to have multiple layers of protection, but researchers found that existing safety protocols could easily be evaded with “small and innocent-looking instructions.”

Besides creating disturbing graphics, the researchers also explained that the latest model could be misused in creating “non-consensual sexual deepfakes of real people.”

However, in response to these claims OpenAI issued a statement regarding the fixing of such kinds of issues.

"After investigating this trend, we've introduced additional safeguards against this type of prompt," it said in a statement. The company also asserted that they put in place multiple layers of protections to prevent users from breaching the model’s policies and conditions.

Given the loopholes, the experts describe AI safety as a monumental challenge. AI chatbots lack human knowledge of intent and context, so they struggle to follow complex safety policies. As a result, with the evolution of more sophisticated techniques, the rogue actors can easily make models to bypass them.

According to Jim Nightingale, the firm's AI safety and security researcher, “I'm struck that while what I saw was generated, an artificial image, it has ties to real images, and the real world.”