Staying ahead of emerging Trust & Safety risks in 2026 means shifting from reactive content removal to systems that anticipate and prevent harm. In a new WebPurify playbook, 10 Trust & Safety leaders from Roblox, Meta, Amazon, DoorDash, Zalando and others outline 12 actions spanning early detection, living policy, explainable enforcement, unified trust infrastructure and prevention-first product design.

Why this matters now

Trust & Safety (T&S), the discipline of protecting platform users from harmful content, fraud, abuse and manipulation, has historically worked like a fire brigade: wait for harm, then respond. That model is breaking under the weight of generative AI.

The scale of the challenge shows in the market itself. The content moderation market was valued at $11.63 billion in 2025, according to Mordor Intelligence, and is projected to roughly double by the end of the decade. At the same time, 60% of organizations are expected to have formalized AI governance programs by 2026 as regulatory pressure rises.

AI sits on both sides of the problem. It gives bad actors synthetic identities at scale, realistic deepfakes and floods of low-quality content. It is also the most powerful defensive tool platforms have. The defining question for 2026, according to the leaders WebPurify interviewed, is how to build systems that stop harm from taking root in the first place.

Key insights from the whitepaper

What are the biggest emerging Trust & Safety risks in 2026?

The leaders point to AI-generated “slop” (low-quality, high-volume synthetic content), synthetic identities, deepfakes, scaled fraud operations and coordinated inauthentic behavior. Emer Cassidy of Zalando also flags “model collapse,” the degradation that happens when AI systems begin training on their own synthetic output. Underneath all of it sits a fragmented regulatory landscape, especially around age requirements that vary sharply by jurisdiction.

Why is reactive moderation no longer enough?

Volume, speed and regulation have outpaced it. Pratyoosh Ranjan Maharana, Senior Risk Manager at Amazon, describes the shift directly: “The most significant shift we’re seeing in Trust and Safety this year is how organizations are moving from reactive content moderation to AI-powered systems that can actually anticipate and prevent problems before they escalate.” Human moderators alone cannot keep pace with the scale of AI-generated content or the faster response times regulators now require.

Should platforms rely on AI or human moderators?

Both, run as a single system. Laura Higgins of Roblox warns against over-relying on technology as the sole answer and puts it simply: “It must be both.” In the model the playbook describes, AI handles detection at scale while human experts train the models, oversee implementation, and focus on complex, ambiguous or safety-critical cases.

How is Trust & Safety converging with fraud and identity?

Attackers exploit the seams between siloed teams, so the leaders expect T&S, anti-fraud and identity systems to integrate. Max Pinnau of Marketplace Risk frames the shift: “Trust is shifting from asking, ‘Is this post bad?’ to ‘Is this actor real and accountable?’” This points to actor-level visibility and shared risk signals across content, identity, fraud and payments.

What is “democratized deception”?

It describes ordinary users, not sophisticated attackers, using cheap AI tools for small, situational deceptions: a fake ID to pass age verification, a recreated receipt, an embellished resume. Chris Roberts of DoorDash argues this breaks a long-held assumption that the effort someone invests in abuse signals malicious intent. Platforms need enforcement models that address rule-breaking without treating every violation as hostile.

What’s inside the full whitepaper

• All 12 actions in full, each with expert commentary from leaders at Roblox, Meta, Amazon, DoorDash, Zalando, Vinted, Handshake, Whatnot, Marketplace Risk, Image Angel and the Digital Trust & Safety Partnership
• “What good looks like” benchmarks for every action, describing the target state for mature T&S operations
• 12 implementation checklists you can assign to owners and run this quarter
• The “living policies” model for updating guidelines at the speed of viral threats, including a rapid-update pathway and edge-case library approach
• A unified trust stack blueprint connecting moderation signals to identity, fraud and payout controls
• Youth safety and victim-centered design guidance, covering parental control usability and automatic evidence preservation
• Compliance and shared-authority readiness, including audit trails, regulatory readiness mapping and interoperability with third-party oversight
• Graduated response playbooks for democratized deception
• A 30-day starting plan for turning the 12 actions into measurable change

Turn insight into a build list

The full playbook is designed to be used, with each action paired with benchmarks and a checklist your team can start on immediately. Download it to pick your first two moves and ship the first change within 30 days.