The online network X has caused major outrage after people utilized its AI chatbot Grok to alter photos of women by taking off their clothing.
Several cases have emerged where the technology has been employed to virtually strip women to render them in swimwear non-consensually, as well as to put them in sexualized contexts.
Company Response and Legal Position
XAI, the organization that created Grok, did not respond to requests for comment, except for an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to outlaw image stripping software. Under a upcoming law, anyone who supplied such tools would "be subject to a jail term and significant penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some premium features—which responds to X users' prompts when they tag it in a tweet.
It is often used to offer commentary or additional background to other users' comments, but people on X are also capable of modify an uploaded photo through its AI image editing feature.
Background of Controversy and Rules
The system has been criticised for allowing users to generate images and clips with nudity and sexually suggestive content, and it was earlier accused of creating inappropriate material of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any action and we have not witnessed any action by regulators," the expert added.
XAI's own acceptable use policy prohibits "portraying images of people in a pornographic way".
In a comment, Ofcom said it was illegal to "create or distribute deepfakes or illegal content" and clarified this included sexual deepfakes made with AI.
It said services such as X were required to take "necessary measures" to "reduce the risk" of users coming across unlawful content on their services and to delete it promptly when they learn of it.
