Indonesia became the first country in the world to officially block access to the Grok chatbot, developed by Elon Musk's xAI company, due to the risk of fake pornographic content generation through artificial intelligence. Just a day later, another Southeast Asian country, Malaysia, announced that it had taken a similar measure, setting a regulatory precedent against abuses of generative AI.
"The practice of non-consensual sexual deepfakes represents a serious violation of human rights, dignity, and the safety of citizens in the digital space," stated the Indonesian Ministry of Communication and Digital Affairs in a press release. "To protect women, children, and the public from the risks of fake pornographic content generated by AI technology, we have decided to temporarily block access to the Grok application," added the agency, leaving the door open to a potential review if adequate safeguards are implemented.
Meanwhile, the Communications and Multimedia Commission of Malaysia stated that the decision was made following "repeated misuse of Grok to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images." The Malaysian regulator particularly criticized the AI integrated into the X social network - also owned by Musk - for keeping its controversial image creation function active, which currently allows paying users to sexualize photos of women and minors, even from real images.
The measures taken by Indonesia and Malaysia represent the first formal bans against the AI system developed by xAI. However, the controversy surrounding Grok has also spread to the West, especially to the United Kingdom, where authorities have stated that they are considering additional sanctions and even the extreme option of completely blocking the X platform if the use of this technology is not controlled.
The origin of the controversy dates back to late December, when social media users began exploiting Grok's image generation function to create non-consensual sexual deepfakes, often using real photos of people who had not given their consent. Within days, the tool was used to create sexualized or fake nude images of women and even minors, sparking a wave of criticism from governments, civil organizations, and digital rights experts.
The initial response from Musk's company was heavily criticized. Instead of immediately deactivating or redesigning the problematic function, xAI chose to restrict image generation and editing to paying users, a decision that many governments and experts deemed insufficient and counterproductive.
British Prime Minister Keir Starmer described the creation of non-consensual sexual images through AI as "repugnant and illegal," and instructed the country's regulator to assess whether a broader intervention is warranted, including the possibility of blocking X if effective controls on Grok are not implemented.
A UK watchdog, the Internet Watch Foundation (IWF), also reported detecting users on the dark web sharing what it described as "criminal images" created with Grok, including depictions of nearly naked underage girls. The IWF also denounced that the tool had been used to virtually "undress" women in photos where they were wearing modest religious clothing, such as hijabs, saris, or nun habits, adding a dimension of cultural and religious discrimination to the scandal.
Musk criticized the UK government's stance, rejecting any attempt at restriction as a form of "censorship." However, regulatory pressure continues to mount.
In France, a criminal complaint has been filed for the creation of non-consensual pornographic images through AI. From the United States, several Democratic senators formally requested that Apple and Google remove the Grok application and the X platform from their app stores until adequate security and moderation mechanisms are implemented.
The new controversy surrounding Grok has once again highlighted a broader issue: the speed of AI tool development surpasses existing regulatory frameworks. Digital rights organizations and academics have called for clearer international standards, better automatic moderation practices, and direct accountability from companies developing technologies that facilitate the creation of deepfakes or sexualized content based on real images.
Now, Indonesia and Malaysia's pioneering decision to block Grok could mark the beginning of a new phase of stricter regulation against abuses of generative AI.