The social media platform X has sparked widespread outrage after people used its artificial intelligence assistant Grok to manipulate photos of females by removing their clothing.
Multiple instances have been seen where the technology has been employed to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to put them in inappropriate situations.
Company Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit nudification technology. Under a upcoming law, anyone who supplied such tools would "face a prison sentence and significant penalties".
"The platform is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which answers X users' requests when they reference it in a message.
It is frequently used to provide reactions or additional context to other posters' remarks, but people on X are also capable of modify an posted picture through its AI image editing tool.
History of Controversy and Policy
The system has been condemned for enabling users to produce pictures and videos with explicit content and sexually suggestive material, and it was earlier accused of generating inappropriate material of public figures.
"X has been enabling the production and spread of these pictures for a long time without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said services such as X were obligated to take "necessary steps" to "lessen the risk" of users coming across unlawful material on their services and to take down it quickly when they become aware of it.
