X Moves to Block Grok AI From Creating Sexualised Images After Global Investigation Wave

 

Elon Musk’s social media platform X has announced sweeping restrictions on its artificial intelligence chatbot Grok, preventing the tool from generating images of real people in revealing clothing, a move that follows mounting international pressure and regulatory investigations into the platform’s role in enabling the creation of non-consensual sexually explicit content.

The announcement, made by X’s safety team on Wednesday, represents a significant policy reversal for the company, which had previously allowed users of Grok’s controversial “Spicy Mode” feature to create sexualised deepfake images using straightforward text commands such as “put her in a bikini” or “remove her clothes.”

Under the new measures, X said it would “geoblock the ability” of all Grok and X users to create images depicting people in “bikinis, underwear, and similar attire” in jurisdictions where such actions violate local laws. The platform has also implemented what it described as “technological measures” to prevent the editing of images showing real people in revealing clothing.

Read Also: FG Set to Lift Ban on Twitter Operations in Nigeria

“This restriction applies to all users, including paid subscribers,” X’s safety team stated, adding that image creation and photo-editing capabilities through the Grok account would now be limited exclusively to paying subscribers as an “extra layer of protection.”

The decision comes amid a cascade of regulatory actions across multiple continents. California Attorney General Rob Bonta launched an investigation into xAI, the developer behind Grok, earlier on Wednesday, citing widespread concerns over the platform’s generation and distribution of sexually explicit material.

“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” Bonta said in a statement. “We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material.”

The California investigation seeks to determine whether xAI violated state law after explicit imagery created through the platform was allegedly “used to harass people across the internet.” California Governor Gavin Newsom weighed in on the controversy, describing xAI’s decision to permit sexually explicit deepfakes as “vile” and urging the state’s attorney general to hold the company accountable.

The regulatory pressure extended beyond the United States. Indonesia became the first country to impose a complete block on Grok access on Saturday, with neighbouring Malaysia following suit the next day. On Thursday, Malaysia’s communications minister, Fahmi Fadzil, said national regulators had determined that X’s initial steps to prevent Grok from generating indecent images were “not done in totality.”

Fahmi indicated that Malaysia would lift its temporary restriction on Grok if X could successfully demonstrate complete deactivation and prevention of harmful content generation.

India also confirmed that X had removed thousands of posts and hundreds of user accounts in response to official complaints about sexually explicit AI-generated content. Britain’s communications regulator, Ofcom, announced on Monday that it was opening a formal probe into whether X had failed to comply with UK law regarding the sexual images.

Read Also: COURT AWARDS $25,000 AGAINST META IN FALANA’S PRIVACY LAWSUIT OVER FALSE HEALTH CLAIM

The European Commission, acting as the European Union’s digital watchdog, acknowledged X’s new measures but indicated it would conduct a thorough assessment. “We have taken note of additional measures X is taking to ban Grok from generating sexualised images of women and children,” European Commission spokesperson Thomas Regnier said. “We will carefully assess these changes to make sure they effectively protect citizens in the EU.”

The scale of the problem became clearer last week when Paris-based non-profit AI Forensics published an analysis of more than 20,000 Grok-generated images. The study found that over half depicted “individuals in minimal attire,” with the vast majority being women, and approximately two per cent appearing to be minors.

Adding to the pressure on xAI, a coalition of 28 civil society organisations submitted open letters on Wednesday to the chief executives of Apple and Google, urging the technology giants to ban both Grok and X from their respective app stores in light of the surge in sexualised imagery.

The controversy highlights broader concerns about the rapid development and deployment of generative artificial intelligence tools without adequate safeguards. While AI image generators have existed for several years, the ease with which Grok’s “Spicy Mode” allowed users to create non-consensual intimate images of real individuals marked a new frontier in the misuse of such technology.

Non-consensual intimate images, commonly referred to as deepfakes when created using artificial intelligence, have become an increasingly pressing legal and ethical concern worldwide. Multiple jurisdictions have enacted or proposed legislation criminalising the creation and distribution of such content, recognising the severe psychological harm inflicted on victims and the potential for blackmail, harassment, and reputational damage.

The restrictions announced by X represent one of the most significant policy changes implemented by the platform since Musk acquired it in October 2022. The billionaire entrepreneur has previously positioned himself as a free speech absolutist and scaled back content moderation practices that existed under the platform’s previous ownership.

However, the global regulatory response to Grok’s capabilities appears to have forced a recalibration of that approach, at least in relation to AI-generated sexual content. Whether these measures will satisfy regulators and civil society groups remains to be seen, with several investigations still ongoing and countries maintaining restrictions pending further verification of X’s compliance efforts.

The incident also raises questions about the governance and oversight of artificial intelligence development more broadly. xAI, founded by Musk in 2023, positioned Grok as a rival to other leading AI chatbots such as OpenAI’s ChatGPT and Google’s Gemini, marketing it as less restricted and more willing to engage with controversial topics.

That positioning now appears to have backfired, with the company facing potential legal liability across multiple jurisdictions and reputational damage that could affect both xAI and Musk’s other ventures.