The social media network X has sparked major anger after users employed its AI chatbot Grok to change pictures of women by removing their garments.
Multiple examples have been seen where the tool has been used to virtually strip female subjects to show them in bikinis without consent, as well as to place them in sexualized situations.
Company Response and Regulatory Position
XAI, the company responsible for Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to ban nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a prison sentence and significant penalties".
"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people viewing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' requests when they tag it in a post.
It is frequently used to give reactions or more background to other users' comments, but people on X are also can modify an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The chatbot has been condemned for allowing users to generate images and footage with explicit content and sexually suggestive material, and it was previously accused of generating inappropriate content of celebrities.
"X has been allowing the creation and distribution of these images for months without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own acceptable use policy bans "showing images of people in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or illegal content" and clarified this included explicit AI images created with AI.
It said platforms such as X were obligated to take "appropriate actions" to "reduce the danger" of users encountering unlawful content on their services and to remove it swiftly when they become aware of it.
