The social media platform X has provoked significant backlash after individuals used its AI chatbot Grok to alter photos of females by stripping their attire.
Multiple cases have been seen where the technology has been used to digitally undress women to make them appear in swimwear without consent, as well as to put them in sexualized contexts.
Developer Reaction and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to ban nudification tools. Under a proposed criminal offence, anyone who provided such tools would "face a jail term and significant penalties".
"X or Grok could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which answers X users' queries when they mention it in a post.
It is frequently used to provide commentary or more background to other users' comments, but people on X are also can alter an posted image through its AI image editing tool.
History of Controversy and Rules
The chatbot has been condemned for allowing users to produce images and videos with explicit content and sexually suggestive content, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.
"The platform has been permitting the generation and spread of these images for months without taking any steps and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was illegal to "make or share non-consensual intimate images or illegal content" and stated this covered sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate actions" to "reduce the danger" of users encountering illegal content on their services and to take down it swiftly when they learn of it.
