The social media platform X has caused widespread outrage after individuals utilized its artificial intelligence assistant Grok to change photos of females by taking off their attire.
Numerous cases have surfaced where the technology has been used to remove clothing from women to make them appear in swimwear non-consensually, as well as to place them in inappropriate contexts.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to questions, other than an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being brought forward to ban image stripping technology. Under a proposed criminal offence, anyone who provided such tools would "face a jail term and significant fines".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their services, but did not confirm whether it was at this time looking into X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they tag it in a post.
It is frequently used to provide commentary or more background to other users' comments, but people on X are also can alter an uploaded picture through its AI image editing feature.
History of Criticism and Rules
The tool has been faulted for permitting users to create photos and videos with explicit content and sexually suggestive material, and it was in the past accused of making sexually explicit content of celebrities.
"The platform has been enabling the creation and spread of these images for a long time without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate actions" to "lessen the danger" of users encountering illegal material on their services and to take down it quickly when they become aware of it.
