Resources

Why judgment is the real skills gap in trust and safety: notes from Trust Issues

There was a time when trust and safety training meant a room, a PowerPoint, and a trainer who’d done the research so you didn’t have to. Definitions. Policy guidance. Slide after slide of information nobody could get anywhere else.

That world is gone. Today, anyone on a trust and safety team can open ChatGPT, Gemini, or Claude and get a clean answer to almost any policy question in seconds. The knowledge that used to be the whole point of training is now free, instant, and everywhere.

So what’s left to teach?

That’s the question at the center of a recent episode of Trust Issues, the WebPurify podcast hosted by Ailís Daly, Head of Trust and Safety at WebPurify. Her guest was Kate Taylor, Principal Learning and Development Partner at Okta, who spent years building learning programs inside trust and safety organizations, including through TikTok’s pandemic-era hypergrowth. Kate’s answer comes down to one word: judgment. Everyone has the knowledge now. The scarce skill is making a sound call with it. And the broader data on how employers’ skill priorities are shifting points in the same direction.

Knowledge stopped being the differentiator

Kate’s career started in financial compliance, training accountants on frameworks that barely move year to year. Moving into trust and safety was a different world entirely.

“Now it’s thinking about how do you apply judgment to the knowledge that you’ve got,” she said. “So it’s more about how do we take knowledge and turn it into something powerful for the organization, because we all have the knowledge now.”

That reframes the job of a trust and safety learning function. The goal becomes building the ability to weigh facts against each other, under pressure, in cases nobody wrote a clean rule for.

The rest of the labor market is moving the same way. The World Economic Forum’s Future of Jobs Report 2025, which surveyed more than 1,000 employers representing over 14 million workers across 55 economies, found that employers expect 39% of workers’ existing skill sets to be transformed or become outdated between 2025 and 2030. The same report found that roughly seven in ten employers consider analytical thinking, the ability to break a problem down and reason through it, an essential core skill.

Those are economy-wide numbers. But they describe the same shift Kate saw from inside trust and safety: access to information has become table stakes, and what employers are short on is the reasoning to use it well.

What happens when hiring outpaces training

Kate’s clearest example of what breaks under pace came from TikTok. In a more traditional organization, she said, new hires arrived a few at a time; there was room to spend six months coaching one intake before the next one landed.

At TikTok, that model didn’t survive contact with the platform’s growth. “We went from three hundred to three thousand employees in the space of a few years, and you had them for a week before the next intake came,” she said. “You’re thinking, how can I rapidly upskill these people before my attention focuses on the next batch?”

Trust and safety, in her view, was still catching up to that reality. Compared to a regulated industry with standard exams and a governing board, trust and safety training leaned heavily on self-service: here are the resources, go learn it, because we’re moving too fast to hold your hand through it.

That pace is structural to the industry. Every new format, from livestreaming to generative AI content to VR, arrives as a fresh edge case long before it arrives as a finished training module. The tooling moves just as quickly. The Business Research Company estimates the content moderation AI market will grow from $3.07 billion in 2025 to $3.88 billion in 2026, roughly 27% growth in a single year. Each new classifier or AI-assisted review workflow shifts what a moderator is looking at and what they’re expected to double-check.

Teams are, by design, always a step behind the thing they’re being asked to judge.

Organizations are betting on upskilling. The format matters.

Kate isn’t alone in seeing the old training model strain. The World Economic Forum’s Future of Jobs Report 2025 found that 85% of employers plan to prioritize upskilling their workforce.

Boston Consulting Group’s AI at Work 2025 study, which surveyed more than 10,600 leaders, managers, and frontline employees in 11 countries and regions, adds a useful detail about what that upskilling has to look like. BCG found that only half of frontline employees used AI tools regularly, and that frontline employees who received at least five hours of AI training, plus access to in-person training and coaching, were more likely to be regular users.

That’s where most plans get vague. Organizations know “teach the policy” won’t hold up much longer; fewer have decided what replaces it. Kate’s answer, teach people to reason with the policy instead of reciting it, is one of the more concrete ones available. And BCG’s data suggests delivery matters too: time with a coach appears to move behavior in ways a resource library on its own does not.

Tell your AI coach to push back

If knowledge is no longer scarce, Kate’s advice was to treat AI as a working tool for developing judgment, beyond a shortcut to answers. But she was specific about how.

“You have to be really clear about the inputs you give to that coach,” she said. “Tell it to be tough on you. Tell it to be to the point. Don’t always praise me. If I’ve got it wrong, coach me to get it right. Don’t spoon-feed me the answers.”

Her concern was the flattery loop she called out by name: ask an AI model for feedback on an email, and by default it tends to tell you it’s wonderful. “No one talks like that,” she said. “No one’s praising you and patting you on the back, left, right, and center.” Left unchecked, that pattern quietly erodes judgment instead of building it.

A study published in Science in March 2026 puts numbers behind her instinct. The study found that 11 AI models affirmed users’ actions 49% more often than humans did. In three preregistered experiments with 2,405 participants, the Science researchers found that a single sycophantic exchange left people more convinced they were right and less willing to take responsibility for repairing a conflict.

The study looked at interpersonal advice, not moderation decisions. Still, the mechanism translates uncomfortably well. For a trust and safety professional using AI to pressure-test a call on an ambiguous edge case, a tool that validates by default can make a shaky decision feel solid. A job that depends on catching your own blind spots needs a training partner willing to point them out. Kate’s fix is to make disagreement an explicit instruction.

She also had a practical habit worth borrowing: once an AI-assisted piece of work finally lands after many rounds of back-and-forth, ask the model what prompt would have gotten there in one step. “It will tell you, if you’d said all of this to me, I would have produced it,” she said. “You can save that, and use it next time as your framework.”

The manager trap

Kate pointed to Multipliers by Liz Wiseman as a lens she uses constantly in manager training, and the core idea cuts against instinct: the traits that make someone a star individual contributor can make them a bad manager.

A high performer who solves problems fast, moves independently, and has a strong bias for action gets promoted to lead a team. Then their team brings them a problem, and old habits take over. “I’m going to solve that problem, because that’s what I do,” Kate said, describing the instinct. “You’re actually diminishing the ability of the team to problem-solve for themselves.”

Judgment doesn’t develop in people who never get room to exercise it. A manager who reflexively absorbs every hard call is quietly training their team out of the exact skill trust and safety now depends on most.

And managers are stretched thin. Gallup’s State of the Global Workplace 2026 report found that global manager engagement fell from 27% in 2024 to 22% in 2025. Gallup also found that in U.S. organizations beginning to implement AI, fewer than one in three employees strongly agreed that their manager actively supported their team’s use of it.

That support seems to count. BCG’s AI at Work 2025 study found that 55% of employees with strong leadership support felt positive about generative AI, compared with 15% of those without it. It’s an association, not proof that manager training causes the lift. But it lines up with Kate’s point: the people who shape how a team is allowed to think have outsized influence on whether that team builds judgment at all.

Judgment needs psychological safety to develop

The last piece Kate raised isn’t a training design question at all. It’s a culture question: do people feel safe making a judgment call that might turn out to be wrong?

“Do they feel like if I make a judgment call and it’s wrong, is it all eyes on me?” she said. “Or did I have the support of my manager when I made that call?” Without that safety net, she argued, people stop making judgment calls and default to the safest, most rigid interpretation of a policy, whether or not it fits the case in front of them.

In trust and safety, that dynamic lands on work that is already heavy. A 2025 study by Spence and colleagues of 160 content moderators at one company found that 24.4% scored in the moderate-to-severe range for psychological distress, and 33.75% had scores suggesting clinically relevant depression symptoms. These were screening results rather than diagnoses, and the researchers found that more frequent exposure to distressing content was associated with higher distress.

The study didn’t measure psychological safety directly. What it does show is the baseline strain many moderation teams already carry. Layer a blame culture on top, and people have every reason to retreat into the most defensive reading of a policy. Punish wrong calls, and you get teams that stop making calls. Burnout and retention, the parts of trust and safety work that rarely show up in a training deck, sit downstream of that same dynamic.

What happens after the call

Kate’s framework doesn’t stop at the individual judgment call. A decision made in isolation, however well-reasoned, is only as useful as what the organization does with it afterward.

“We all learn from each other’s horror stories,” she said, describing how she thinks about knowledge-sharing across a trust and safety org. The case that trips up one team today is likely to trip up another team next quarter, unless someone routes it back into shared knowledge. “If everyone can be sharing issues that are arising, challenges that they’re seeing, we can make sure that we’re nipping those in the bud… because it’s all new. It’s all new to everyone.”

Kate also borrows an idea from network science here. A judgment call rarely needs to be made from scratch. Knowing who in the organization has already seen something close to a given edge case is, in her framing, as much a skill as the call itself: “Who are we tapping into? Who might know about this that can support me?”

Practically, a hard call shouldn’t dead-end with the person who made it. The specifics need to travel: into a shared case log, into the next calibration session, into a policy update if the pattern repeats often enough. Teams that build judgment without building that loop end up re-solving the same edge case, team by team, indefinitely.

Used carefully, AI still belongs in trust and safety work. What it does well is hand a team knowledge instantly. The call on an ambiguous, high-stakes edge case, and the muscle for making it, still belongs to the humans.

As Kate put it, thinking about what trust and safety teams will look like in five years: “Without knowledge behind our judgment, it’s just sort of biased with confidence, isn’t it? It’s just your gut instinct with confidence.” The teams that hold up will be the ones deliberately trained to use information well. Access alone won’t get them there.

This piece draws on a conversation between Ailís Daly and Kate Taylor on Trust Issues, the WebPurify podcast on the front lines of digital safety.

Listen to the full episode for more on AI coaching, manager enablement, and what great trust and safety teams will need by 2030.