Elon Musk's online platform X has caused major backlash after individuals utilized its AI chatbot Grok to manipulate photos of females by taking off their attire.
Numerous examples have surfaced where the technology has been employed to remove clothing from female subjects to make them appear in bikinis without permission, as well as to put them in inappropriate situations.
Developer Response and Legal Stance
XAI, the company behind Grok, did not respond to questions, other than an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and significant penalties".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people viewing unlawful material on their networks, but did not confirm whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium features—which responds to X users' requests when they mention it in a tweet.
It is often used to offer reactions or more context to other users' comments, but people on X are also capable of alter an posted photo through its AI image editing feature.
Background of Criticism and Rules
The tool has been faulted for allowing users to generate images and footage with nudity and sexually suggestive content, and it was in the past accused of generating inappropriate content of celebrities.
"X has been allowing the creation and spread of these images for a long time without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or distribute deepfakes or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the risk" of users encountering illegal material on their services and to remove it quickly when they become aware of it.
