🔗 Share this article

The social media platform X has sparked widespread outrage after individuals used its AI chatbot Grok to alter photos of females by taking off their attire.

Several instances have emerged where the tool has been employed to remove clothing from female subjects to render them in swimwear without consent, as well as to insert them in sexualized scenarios.

Developer Reaction and Legal Stance

XAI, the firm responsible for Grok, did not respond to questions, except for an automatically-generated response stating "legacy media lies".

A Home Office official stated that new laws is being proposed to outlaw image stripping tools. Under a new law, anyone who distributed such technology would "receive a jail term and significant fines".

"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with impunity."

Regulator Stance and Platform Capabilities

The regulator Ofcom said platforms must "consider the potential" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in relation to AI-generated pictures.

Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a post.

It is frequently used to offer commentary or additional background to other users' comments, but people on X are also able to modify an uploaded picture through its AI image editing feature.

Background of Criticism and Policy

The tool has been faulted for enabling users to create images and clips with explicit content and sexualised material, and it was previously accused of making inappropriate material of public figures.

"X has been enabling the production and distribution of these pictures for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.

XAI's own terms of service bans "showing likenesses of individuals in a pornographic manner".

In a statement, Ofcom said it was illegal to "generate or circulate private pictures or child sexual abuse material" and confirmed this covered explicit AI images created with AI.

It said networks such as X were required to take "appropriate steps" to "minimize the danger" of users coming across illegal content on their services and to remove it swiftly when they learn of it.