Apple Threatened to Remove Grok AI App from App Store Over Deepfake Violations

Apple threatened to remove the Grok AI app from its App Store after determining that the app and the associated social media platform X violated guidelines prohibiting overtly sexual material. The threat emerged from complaints that the app did not adequately prevent users from generating nude or overly sexualized deepfakes.

This occurred amid international scrutiny of content created using the app.

The determination followed requests for X and Grok to restrict functions enabling sexualized deepfakes. S. senators, Apple's efforts to address the issue included evaluating the platforms' responses.

The letter, cited by NBC News, detailed that X had announced restrictions on AI use for undressing images on January 14, applying to all users including paid subscribers. Apple requested a plan from X and Grok to enhance content moderation, but found it insufficient. The letter stated that X had resolved its violations, while Grok remained non-compliant.

As a result, Apple rejected the Grok submission and notified the developer that further changes were needed or the app could be removed.

Resolution of the Dispute After the threat, the Grok developer submitted new code to Apple, leading to approval of the updated version.

Apple noted in the letter that further engagement and changes by the developer resulted in substantial improvements. The app was then deemed compliant and approved for the App Store, which connects to more than 2 billion devices. The letter was signed by Apple's senior director of government affairs, Timothy Powderly.

It responded to inquiries from Democratic senators regarding the platforms' handling of harmful content. The missive highlighted the close call for Grok's availability on the marketplace.

Broader Context and Investigations The threat came amid public backlash over sexualized AI images generated by Grok and shared on X.

, and the European Union initiated probes into X related to this content. Elon Musk described these investigations as attempts at censorship. Prior to the threat, X and xAI, the company operating Grok, filed a lawsuit against Apple in August.

The suit alleged that Apple delayed the review process for Grok updates, an accusation Apple denied. The senators had also asked Apple and Google to remove X and Grok from their marketplaces. One senator expressed disappointment to NBC News that Google did not address the matter with the same seriousness as Apple, citing the nature of the images produced.

Google reportedly engaged with the teams behind X and Grok to emphasize policy adherence and obtain commitments to address harmful content promotion. It is unclear whether Google issued a similar threat to remove the apps.

The platforms involved have faced ongoing pressure to improve moderation of AI-generated content, particularly non-consensual sexual material. This incident underscores challenges in regulating AI tools on major app stores, affecting developers, users, and regulatory bodies worldwide.