The social media network X has caused major outrage after users employed its AI chatbot Grok to manipulate images of females by removing their clothing.
Several instances have been seen where the technology has been used to remove clothing from female subjects to make them appear in bikinis without permission, as well as to put them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to ban image stripping software. Under a new criminal offence, anyone who distributed such technology would "face a prison sentence and substantial fines".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which answers X users' prompts when they mention it in a post.
It is frequently used to offer commentary or more background to other users' remarks, but people on X are also capable of alter an posted image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been condemned for permitting users to create images and videos with explicit content and sexualised material, and it was earlier alleged to have been involved in making graphic material of celebrities.
"X has been permitting the production and spread of these images for a long time without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "showing images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or illegal content" and clarified this included sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "lessen the risk" of users encountering unlawful material on their services and to remove it quickly when they become aware of it.
