The online platform X has provoked major backlash after users employed its AI chatbot Grok to alter photos of women by stripping their garments.
Several examples have emerged where the tool has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to place them in sexually explicit scenarios.
Company Response and Regulatory Stance
XAI, the firm that created Grok, remained silent to questions, other than an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to outlaw image stripping tools. Under a upcoming criminal offence, anyone who distributed such technology would "receive a jail term and significant fines".
"X or Grok has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for features—which responds to X users' requests when they tag it in a message.
It is commonly used to give commentary or more context to other posters' remarks, but people on X are also able to modify an posted photo through its AI image editing function.
History of Controversy and Policy
The system has been faulted for permitting users to create pictures and clips with nudity and sexualised material, and it was previously accused of making sexually explicit material of public figures.
"The platform has been allowing the generation and spread of these images for months without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of people in a graphic way".
In a statement, Ofcom said it was against the law to "create or distribute private pictures or illegal content" and clarified this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "appropriate steps" to "minimize the danger" of users coming across illegal content on their platforms and to take down it quickly when they are notified of it.
