By Ailís Daly, Head of Trust and Safety, EMEA, at WebPurify

Brands have handed customers a text box, a photo upload, a microphone, on snacks, sneakers, plush toys, greeting cards, jerseys. Personalization has become one of the most reliable ways to turn a browser into a buyer. But underneath that convenience, the moderation problem has quietly changed shape. It used to be about deleting a bad post after the fact. Now it’s about catching a bad order before it’s printed, sewn, stamped, and mailed to someone’s door, where there’s no delete button once it ships.

“Everything the sun touches is a trust and safety problem,” says Evelyn Douek of Stanford Law School. Snacks, sneakers, plush toys, jerseys weren’t trust and safety surfaces five years ago. Now they are, and some of the teams building these features are still treating them like a printing problem instead of a moderation one.

That mismatch shows up in a specific way: most review processes are only built to catch one kind of mistake, not both of the ones that actually matter.

Two failures, one review step

Most brands assume their review process catches “the bad stuff,” full stop. In practice, even the processes that claim to check everything are usually only built to catch one kind of mistake.

“Over-blocking is the failure you can survive,” says Ailís Daly. “The miss is the one you can’t reverse.”

That distinction matters more here than it does anywhere else in trust and safety. A social feed can quietly delete a bad post. A personalized product, once it’s in someone’s hands, can’t be pulled back. Feed moderation and product moderation look similar on paper. They’re not the same problem, and treating them as one is where a lot of brands go wrong.

There’s no takedown

Every moderation system built for the internet assumes some kind of remediation path exists after the fact: delete it, edit it, downrank it. Personalization doesn’t have one.

“Once the thing is manufactured, it exists, it’s in someone’s hands, and it’s photographable forever,” Ailís says. “So the entire weight of the system has to sit on the pre-publication gate, which is historically the weakest part of content moderation.”

There’s a second layer most brands don’t think about until it’s too late. A hateful post carries a user’s name and nothing else. A hateful hoodie carries the brand’s logo, its label, its return address. The customer designed it, but the brand made it real.

There’s a third layer that gets missed almost entirely: a feed post is broadcast to nobody in particular, but a personalized product ships to a named person at a specific address. That makes it usable for targeted harassment in a way a public post simply isn’t, whether the target is an ex, a colleague, or someone the sender has a grudge against.

Some companies try to paper over that exposure with broad discretionary language rather than precise rules. One major sportswear customization program’s rules state plainly that no slang, phrases, or profanity are accepted, and that the company can reject or cancel any customization “for any reason,” without ever specifying what that means in practice. It reads like caution. It’s really an admission that writing precise, auditable rules at this scale is genuinely hard.

Nobody owns it

Part of why that work doesn’t get done traces back to something more basic than any filter or policy.

“There are usually two specific gaps,” Ailís says. “Policy that exists at marketplace or brand level but was never translated to the personalization surface. And no clear owner, so personalization never appears on a trust and safety roadmap at all. It sits with product or growth, where the incentive is conversion, not refusal.”

Ailís saw the first gap play out firsthand at a TrustCon roundtable in San Francisco, where practitioners compared notes on political content increasingly showing up in custom orders. A brand might have a policy against political content at the marketplace level, but if a personalization feature exists on top of that marketplace, the policy has to actually flow through into enforcement on that surface. Very often, it doesn’t, because nobody translated it for the new surface, and nobody owns making sure they do.

Sixteen characters is enough

To see how far the pre-publication gate actually holds up, Ailís ran an adversarial test against a major global fashion retailer’s children’s clothing line, ordering personalized kids’ apparel with embossed text capped at 16 characters.

Within 24 hours, she had a wardrobe full of hate speech.

Nothing stopped the probing along the way. No rate limit fired, no account-level signal flagged the pattern, no velocity check ever kicked in. Every new attempt was treated like a first-time, innocent customer. And the character cap itself made things worse: short fields push people toward codes and acronyms, which happens to be exactly where automated filters are weakest.

The bigger mistake may be treating each personalised order as an isolated moderation decision. A customer who tries twelve variations of a prohibited phrase is telling you something even if attempt thirteen technically passes the filter. Mature moderation needs to look beyond the content itself to behavioural signals — velocity, repeated failed attempts, account history, recipient patterns and other indicators of intent.

“Short means constrained,” Ailís says, “and constrained means users reach for codes and acronyms, which is exactly what filters are worst at catching.”

Block, delay, or let through

When something can’t be undone, the choice Ailís pushes brands toward is delay, not allow. Blocking outright is rarely the real alternative on the table.

“Latency is a cost you can pay. A shipped product is a cost you can’t refund.”

That reframing changes how the whole review tier should be built. Auto-reject can stay narrow: slurs, known extremist symbols, hash-matched illegal imagery, the handful of cases where there’s essentially no legitimate use and speed matters more than nuance. Human review should be wide instead, covering name collisions, reclaimed language, numerals, anything that reads differently depending on what it’s printed on. Auto-allow, especially for children’s products, should start tight and only loosen as evidence builds up, not as confidence does. In practice, that often means reviewing everything at launch, treating that period as evidence-gathering, and only setting automated thresholds once there’s real data on what customers actually submit.

That middle tier isn’t a luxury add-on. Some brands already lean on it heavily: one mainstream snack brand’s own FAQ states plainly that it reviews every personalized order and reserves the right to cancel anything it judges offensive or inappropriate, a rare admission from a mainstream food brand that a real person is checking each one, even for something as small as a cookie.

Audio is where this gets hardest. A personalized voice message is exactly the kind of case where a stated review policy can still let something through, because transcription plus text classification catches maybe half of what’s actually there. It fails on whispering, on accents, on deliberate mumbling, and it ignores non-speech audio entirely, screaming, sexual sounds, harassment that never produces a transcript to flag. A voice recording also carries a data dimension text doesn’t: it’s personal data, sometimes a child’s, with consent, retention, and deletion obligations attached whether or not anyone ever reviews it. Reviewing it properly means a human actually sitting and listening in real time, which is slow and carries a genuine wellbeing cost for whoever does it.

Automated rejection has its own way of going wrong, and it cuts in the opposite direction. One athletic apparel brand’s shoe customizer once flagged the word “sweatshop” as inappropriate slang and blocked it from a custom order, a legitimate, pointed word that a filter had no context to understand. The backlash that followed was worse than anything letting the word through would have caused.

The cost no dashboard shows

A miss costs more per incident, and it’s not close. It’s permanent, photographable, and carries the brand’s name into every retelling of the story. Marketplaces have learned this repeatedly: hateful, custom-printed shirts tied to the January 6th Capitol riot remained listed on several major marketplaces well after being flagged, coming down only once public pressure made the issue unavoidable. Catching something once doesn’t guarantee the fix holds elsewhere, either. One handmade goods marketplace faced a public boycott over a wave of controversial branded merchandise it had allowed to stay listed, even as comparable items remained available across several other marketplaces without triggering the same response.

Over-blocking costs less per incident, but it doesn’t land evenly. Name-based false positives fall disproportionately on non-English names, which turns what looks like a customer service complaint into something closer to a fairness problem, one that tends to show up as a pattern across hundreds of customers rather than a single bad review.

“These two costs never appear on the same dashboard,” Ailís says. “If I could get a brand to adopt a single metric, it would be appeal rate and appeal overturn rate. A high overturn rate tells you your filter is wrong, and tells you exactly where.”

The line is moving

Regulators are starting to close that gap even where brands haven’t. Personalization can quietly change a brand’s legal status: depending on the jurisdiction and how much control a customer has, customizing a product can shift a company from being treated as a neutral distributor to being treated as the manufacturer, deciding what it makes rather than moderating what users say. Under the EU’s DSA, a personalization feature with any kind of sharing layer attached can pull a retailer into minor-protection obligations most assume are a platform problem, not theirs. And if a child uploads a photo or records their voice, that’s personal data, potentially biometric data, governed by rules like the UK’s Age Appropriate Design Code and COPPA, regardless of whether the design ever gets manufactured at all.

Generative AI is going to widen this gap further still. Once a customer can prompt an image instead of typing text, a brand can no longer enumerate the output space the way it could a text field, and style transfer alone can produce IP and likeness infringement no hash list will ever catch. The brands managing that risk well aren’t trying to police an unconstrained model. They’re constraining the generator itself, restricting to an approved style library, moderating both the prompt and the output, attaching provenance to what gets generated, and treating red-teaming as a genuine launch gate rather than a formality.

Where authority actually comes from

No filter, human or automated, hits zero mistakes on a surface this varied. What separates the brands that get ahead is whether they can show their work: what got caught, what got missed, and how the threshold moved once the evidence came in.

That’s the standard WebPurify holds itself to. Building the judgment that catches what matters, at the scale personalization runs at now, is an ongoing practice: appeal tracking, threshold review, treating every miss and every over-block as data rather than an isolated incident.

Learn more about WebPurify’s approach to trust and safety.