xAI restricts Grok image editing after backlash from California and Europe

Elon Musk’s artificial intelligence company xAI has imposed new restrictions on its Grok chatbot, limiting image editing capabilities after the tool generated sexualised images that drew criticism from regulators in the United States and abroad.

In a post on X late Wednesday, xAI said it had blocked all users from editing images of real people in revealing clothing, including swimwear. The company said the restriction applies to everyone, including paid subscribers, and is part of new technical safeguards introduced in response to recent concerns.

The decision comes after reported cases of hyper-realistic images of females being manipulated into degrading poses and clothing and spread extensively on the X platform. In some instances, minors were allegedly manipulated to appear wearing swimwear, angering concerned parents, advocates, and elected officials.

xAI has also indicated that it no longer allows the generation of images of people wearing scant clothing based on the user’s geographic location in areas where such images are prohibited, though it has not revealed which areas this pertains to.

Pressure mounted on xAI after the California government called for an explanation from the company. The state Attorney General, Rob Bonta, demanded that the company halt the production and distribution of AI material featuring sexual content, while Governor Gavin Newsom called an urgent investigation into the issue. This is one of the most aggressive actions taken by US officials to address AI-generated illegal imagery to date.

The furor has also provoked foreign reactions. Threats of action have come from European authorities and the United Kingdom. Indonesia blocked access to Grok temporarily. Politicians and advocacy groups have called upon companies like Apple and Google to remove Grok from their app stores.

Musk, who owns xAI and X, initially appeared to downplay the issue but has since said the platform takes reports of child sexual abuse material seriously.