I found out about Meta’s Muse Image while scrolling on my Instagram account. A local journalist I follow was discussing the feature, explaining to her followers step by step how to opt out through their settings. I immediately went to my own settings, only to realize that because my account is private, the new feature did not affect my personal page. While my private status on Instagram shielded me, millions of other Instagram users were not so lucky. For users with a public profile, their likeness was automatically included without their consent. The eventual response highlighted a pattern we’ve seen too many times: Deploy first, ask forgiveness later.
Released in early July 2026, Muse Image is an artificial intelligence feature developed by Meta’s new AI division, Meta Superintelligence Labs. Meta originally launched the image generation feature on Instagram, which opted in all users who have public accounts and allowed users to generate images from other people’s likenesses without consent. The auto opt-in design was alarming for users of the platform. Within days, intense public backlash forced Meta to pause the feature after critics cited privacy, consent, and copyright concerns, especially because people with public accounts had to manually alter their settings to opt out of having their likeness used.
This is common for Big Tech. TikTok recently removed its experimental AI meme remixer feature after creator outrage. The feature, turned on by default, allowed anyone to take a still image of a creator’s face from a video, manipulate it with AI, and post it into the comment section.
Similarly, Google incorporated its Nano Banana image generation model into Google Earth, allowing users worldwide to generate photorealistic deepfakes of bombed hospitals and a flooded U.S. Capitol with a text prompt, resulting in Google disabling the feature within 24 hours. For 20 years, Google Earth’s credibility has made it a trusted source beyond solely map use. It has been a vital public resource for investigative journalists, human rights advocates, and open-source researchers who have relied on its imagery to document conflicts and investigate human rights abuses. Allowing anyone to create unvetted, photorealistic deepfakes could allow bad actors to exploit that trust and spread false or misleading information – which also threatens the whole platform’s credibility.
Across these three incidents it was public outrage, rather than any internal evaluation, that served as the only real safety check – and only because the changes were conspicuous enough to be noticed. People should not have to learn about a potentially harmful feature from the news and then have to go hunting through a settings menu to escape it.
Three Simple Fixes to Protect People Going Forward
Three fixes would change that: protective defaults, advance notice before high-risk AI features launch, and a digital regulator with the authority to enforce those requirements and others like them.
The companies that operate these online platforms have enormous control over our digital lives, and users who have spent years building their networks on these platforms can’t easily leave. Silently activating invasive features exploits users’ trust and the dominant market position of several of these online platforms. That is why AI features built into online platforms should be held to even higher standards than standalone AI tools or products.
Some generative AI features can cause serious and irreversible harm. For example,Public Knowledge’s research on digital replicas identifies three types of harm that result from digital replica-related features, like the ones recently deployed and then rolled back on various platforms:
- Commercial Harms: Damaging someone’s business, economic prospects, or ability to profit through unauthorized use of their likeness
- Dignitary Harms: Violating people’s privacy, safety, and well-being through false or abusive deepfakes, including nonconsensual intimate imagery
- Democratic Harms: Harms to an informed electorate and the erosion of public trust in media resulting from realistic misleading synthetic content
Even in this one category of generative AI features, it is apparent that reputational, emotional, or epistemic damage can be difficult to unwind. Once trust is lost or boundaries are violated, simply rolling back a feature is not an adequate remedy. That is why platforms should evaluate the risks from these and other AI features before they are deployed.
To prevent these harms, we should adopt risk-based classification of generative AI features. While legislation like the EU AI Act uses risk-based classification in the context of the full range of societal harms, it is useful to adapt this approach into more narrow domains as well. Public Knowledge operationalized this approach for age-related online safety proposals in our report, Getting Age Assurance Right: A Risk-Based Framework for High-Risk Online Features. Platforms and policymakers should consider how to evaluate the harms from generative AI features through this kind of feature-level, risk-based approach, assigning features with the potential to lead to serious and irreversible harms a high risk classification.
Risk-based classification enables a measured framework for regulatory requirements that targets the features and platform actions most likely to result in harm. Most importantly, a project to identify these features in advance will assure that platforms address them proactively and preemptively instead of waiting for public disapproval to act as a safety filter.
1. Start with Protective Default Settings
Before even bringing regulation into the picture, there are some basic design requirements that users should expect and demand from online platforms. The simple principle is this: Defaults should be protective for users, and platforms should make user choice easy and accessible. Our research has concluded that safety-by-design defaults are one of the best ways to protect users. Placing the burden on individuals to navigate complex and hidden settings to opt out of risky features is a structural design failure.
We have also argued that platforms should be legally held to their own terms of service. Under a consumer protection theory, if a company promises privacy controls and content safety in its terms of service, it must actually offer them meaningfully. Therefore, defaults must be set so that users have to opt in for any sensitive generative AI feature, giving users control and autonomy. And the ability to change your mind later – whether to opt in or opt out – should be understandable, accessible, and easy to navigate.
2. Require Notice Before High-Risk Features Launch
A simple, and minimally intrusive, regulatory requirement for online platforms deploying generative AI features is to mandate a user notification before launching high-risk AI features.
For example, just like platforms notify you when they update their terms of service, they should be required to give users a similar notice in advance of deploying any high-risk AI features. This will create an opportunity for users to respond to proposed features, and provide online platforms a strong signal of how they will be received. In effect, this can speed up the cycle we’ve seen of deployment to backlash to rollback, without users being harmed in the middle.
This idea is effectively an implementation of the recommendations of the National Institute of Standards and Technology (NIST) AI Risk Management Framework (RMF), which emphasizes that risk management should start during the plan and design phase, before deployment.
NIST highlights providing notification and consultation opportunities with stakeholders who are affected before a tool is launched. The RMF is a voluntary framework, but the repeated failures of platforms to meaningfully adopt these standards points toward making some components mandatory. While mandating a formal public feedback process for online platforms could raise significant First Amendment concerns, a simple advance notice schema for select features imposes a far lighter burden on platforms. Obliging companies to evaluate generative AI features in advance and notify users makes users safer and should also help companies avoid lawsuits and having to roll back flawed features later. Notice doesn’t provide the same potential level of safety as regulating feature design, but notifying users before the features are released would be a simple, constitutionally sound first step.
3. Create a Strong Digital Regulator
Ultimately, we need more comprehensive, expert oversight of online platforms because Big Tech’s financial interests often do not align with the public’s well-being. To oversee and enforce these rules, Public Knowledge advocates for a new sector-specific digital regulator. A dedicated regulator could be the entity responsible for promulgating rules to ensure that platforms follow their own terms of service, use default settings that are protective, and notify users before launching risky AI features. This type of oversight is already happening in Europe. Under the Digital Services Act, the European Commission recently found that Meta and TikTok may have violated transparency requirements.
We need AI governance that is proactive and prevents harms, not just a post-backlash PR cleanup mechanism. Protective defaults, advance notice for high-risk features, and a dedicated expert digital regulator are three concrete steps to make sure that it is Big Tech companies, and not their users, that have the responsibility to get risky AI features right.
