The social media platform X has sparked significant outrage after individuals used its AI chatbot Grok to manipulate images of females by removing their attire.
Several cases have surfaced where the technology has been used to digitally undress female subjects to show them in swimwear without permission, as well as to insert them in inappropriate situations.
Company Response and Regulatory Position
XAI, the company responsible for Grok, offered no comment to requests for comment, aside from an standard response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to ban image stripping software. Under a proposed law, anyone who provided such tools would "receive a jail term and heavy penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some premium functions—which responds to X users' queries when they reference it in a tweet.
It is frequently used to provide reactions or additional background to other posters' remarks, but people on X are also can edit an posted picture through its artificial intelligence picture modification function.
Background of Criticism and Policy
The chatbot has been condemned for enabling users to generate photos and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating graphic material of public figures.
"The platform has been permitting the creation and distribution of these pictures for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a graphic way".
In a statement, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were required to take "suitable measures" to "reduce the risk" of users encountering illegal content on their platforms and to delete it promptly when they learn of it.
