Everything in Moderation Except Moderation
By: Michael Gaston
Tags: Business, Design, Startup, Life Lessons, Social Media

After this exchange, I started to think this is what moderation should be. Less laundry lists of offensive words to be tagged and blocked and more an always on conversation entered into in good faith by people who wanted to share in a community. Doesn’t that sound nice? Yeah, well within a couple of months I stopped replying.

The conversation didn’t stop, I just no longer chose to be a part of it.

What had felt urgent and necessary in the beginning quickly began to feel like an impossible chore. And I didn’t think I had the time or the emotional energy to engage with every viewer of our work. My table manners approach felt like small stakes. Because I couldn’t scale my responses I questioned the value of responding at all. Months would go by between me reading comments and nearly a year since leaving the company I founded, it’s still my biggest regret.

So why share this now? Because I’m just now realizing after nearly two decades of social media that moderation isn’t just a tool for major platforms to limit liability. And that maybe what I dismissed as old-fashioned unscalable table manners — listening in good faith/responding respectfully/internalizing critiques — is part of the unsexy, never-ending, inefficient, dance of human-centered moderation that protects communities from extremism.

Remember the coup? We watched an angry mob, incited by misinformation and conspiracy theories storm the Capitol Building. Clubhouse (an app that continually reveals the value of great moderators) has also sparked numerous cyberbullying scandals, embroiling people like New York Times reporter Taylor Lorenz, comedian Tiffany Haddish, and actor Jason Lee. And at in-person contract content moderation offices in the Philippines and the United States, moderators have broken their NDAs to report trauma at abusive workplaces where at least one person has died.

It’s time to rethink moderation.

The community safety policies that have become standard across platforms are inadequate: they piss everyone off (regardless of political affiliation), their interventions appear arbitrary and inconsistent, and they don’t seem to do much to prevent harassment, hate speech, misinformation, or radicalization in the first place.

Twitter justifies its rules by saying that they “ensure all people can participate in the public conversation freely and safely.” Facebook’s community standards are ostensibly designed “to create a place for expression and give people a voice.” YouTube says it works “hard to maintain a safe and vibrant community.” In all these instances, the community is invoked as the object of moderation’s protection: the community, and its vibrant free debate and expression, as a fragile resource that must be safeguarded from hatred, violence, obscenity, and abuse.

Yet, we know rightwing extremism and white nationalism flourish openly on platforms like Facebook while it flags, suspends, and bans Black users for speaking out against that very same racism. Jillian York, of the Electronic Freedom Foundation, reminds us that wrongful takedowns disproportionately target women, LGBTQ+ users, artists, journalists, racial minorities, and sex workers. Tumblr’s decision to eradicate adult content from its platform — in a move to make itself more palatable to app store monopolies — also took down queer, trans+ and gender nonconforming, and sex- and body-positivity communities along with it as collateral damage. In all these cases, platforms that evangelize the merits of diverse perspectives have paradoxically marginalized or blacklisted them altogether.

And who are the moderators? Most platforms’ moderation systems rely on some combination of community flagging, human review and intervention, and AI. Each of these components carries with it major vulnerabilities and opportunities for bias: users can mobilize en masse to wrongfully report targets, contract workers around the world must interpret large volumes of troubling and permissible content under the gun and across cultural or identity-based contexts, and it is well documented that human bias has been baked into our moderation algorithms. From most users’ perspectives, these decisions are opaque and made with few avenues for recourse or appeal.

When we cannot reconcile the difference between platforms’ claims (to protect diversity and free discussion) and the biased consequences of their moderation tactics, we are left with a more cynical observation: that moderation policies and practices are really risk management strategies that better serve the corporation and its shareholders — rather than its users, let alone the vibrancy and health of the community or democracy.

What the internet needs most:

Despite everything I wrote up above, we might still be tempted to rehabilitate big platforms’ community guidelines. We may feel forced into accepting their blunt moderation tools because the alternative is unimaginable. We might also be lulled into accepting these imperfect methods given the scale at which users generate content (Twitter publishes 6,000 tweets per second.) But these aren’t the only tactics — all over the world people are already demonstrating alternatives.

For example: open source projects — from Linux to Wikipedia — are also rich communities where people generate, edit, and vet content towards a collective goal. Open source software, in particular, is organized around the acknowledgment that users themselves generate the value in a collaboration. As such, transparency, community ownership, and collective responsibility become the values by which people engage with each other. That’s a far cry from what we’re used to with big social media platforms that regulate content through proprietary algorithms and opaque protocols, each motivated by the volume of engagements (like shares or watchtime) rather than the quality of engagement.

If we’re to make good on our promise to advance debate, promote safety, and create vibrant and meaningful conversation, our industry needs to reorganize its technologies around moderation as a design problem from the beginning and throughout the life of our platforms.

When I say moderation is a design problem, I mean that moderation needs to be a priority for every function of a platform’s business — from conception onward, not as a kind of chore that needs resources after-the-fact. Moderation shouldn’t just be prioritized because it’s the ethical, right thing to do (it is;) effective and principled moderation is exactly how we get to the goal of rich and valuable engagements.

When we incorporate moderation as a virtue into our design process, we are also asking our designers, technologists, and operations workers to anticipate the scale of this challenge in the architecture of the platform itself — instead of leaving the task to contract commercial moderators or content flagging algorithms.

And along the way, we acknowledge that — like any other aspect of design — moderation is a process that is never complete and always subject to improvement.

A couple of months after I left Cut. I received a book in the mail: