Filtering isn't moderation

Recently Bluesky launched a tool called “Ozone”, that they described in their press release as a “collaborative moderation tool”.

It’s no such thing. It’s a collaborative filtering tool, and I find it interesting that they keep choosing to obfuscate the difference between the two things.

The history of moderation #

Moderation was a response to the problem of spam and flamewars on Usenet. In a moderated newsgroup, posts were sent to the moderator, who approved or discarded them.

Gems of content would get approved and posted, spam and flamewars would get quietly deleted from the moderator’s inbox. Here’s the critical thing: the moderated-out posts didn’t end up on the Usenet servers. They were not made available for anyone to read.

Usenet also had filtering, in the form of killfiles. Like Twitter, you could specify people you didn’t want to see, topics you didn’t care about, servers you didn’t want posts from, and your Usenet client would filter those things out on the fly.

There were even experiments with shared killfiles for collaborative filtering. One thing was clear, though: killfiles were not moderation. They were two fundamentally different things. With a killfile, what you saw didn’t necessarily match what other people saw, and the crap was still there even if you didn’t see it. With moderation, everyone saw the same set of posts, with the crap filtered out.

Usenet also had curation. There was a newsgroup rec.humor.funny which started out as a curated selection of the better posts from rec.humor. (It later moved to a moderation model, because it turned out that wading through crap to find nuggets of gold is a really awful way to spend time.) A similar early newsgroup mod.ber collected interesting material from the rest of Usenet, and posted the result as a curated feed.

The walled social media era #

As social media sites like Facebook sprang up, moderation became something that mostly happened after the fact. That is, posts would go into feeds, but at some point someone on Facebook’s moderation team might delete the bad ones, ideally before anyone saw them.

Later, Facebook started implementing algorithms to catch bad posts automatically, to ease the load on their moderators and try to catch harmful material sooner.

Again, though, the moderated content didn’t remain on the servers for everyone to see.

Facebook also added limited filtering, in the form of being able to block accounts you didn’t want to see in your feed. However, they didn’t describe this as moderation, because it isn’t moderation.

Before the rise of Facebook’s engagement-maximization algorithm, curation was the way things would spread — your friends would effectively curate the material they saw and repost the good posts so they’d appear in your feed. By listing people as friends, you’d effectively be subscribing to their curated post feed.

Why moderation is necessary #

You might be wondering why moderation is necessary if we have filtering and curation. I’m going to go through a few examples. Multiple examples, because sadly I suspect that there are going to be people reading this article who won’t see a problem with some of the types of content I mention.

Here’s the first example: child abuse material. The production of child abuse material is harmful.It’s harmful even if you personally never see it and none of your friends see it either. Even assuming a perfect filtering system exists, allowing people to distribute child abuse material to those who want to see it can normalize and encourage abuse.

In one recent case, an online influencer with millions of followers posted videos encouraging severely disciplining children. She’s now in prison for child abuse, but there’s a whole subculture of parents who encourage the idea that (for example) beating toddlers with a stick is a reasonable way to get them to submit to authority. The American Psychological Association has a position statement explaining that even mild physical discipline of children is psychologically harmful as well as ineffective. But many people still believe it’s effective; I hope you’re not one of them. The pro-spanking parents write books, post videos, and encourage other parents to beat their own kids.

Another example of why moderation is needed is livestreamed mass shootings. It doesn’t matter if most people don’t see the video; the problem is that there are people who do want to see it, and some of those people will be actively encouraged to carry out a mass shooting themselves.

Video of suicides is also harmful. Even stories about suicide have been found to increase suicide rates.

Maybe you’ll be convinced by the example of the video showing terrorists beheading journalist Daniel Pearl. I didn’t see it, but even so I view it as good that it was moderated out of existence as effectively as possible, because it’s propaganda that supports the terrorists.

Maybe you’re still not convinced, so for my final example let’s consider coordinated abuse and stochastic terrorism. Imagine a group of people decide they hate you, and they start watching your movements, posting your location online and encouraging others to attack you. Maybe they engage in swatting and anonymously brag about it. Does it help if you can switch on a filter and not see what they’re doing?

I hope that at least one of those examples will have convinced you that there exists content that’s harmful, even if it’s effectively tagged and filtered so that nobody has to see it who doesn’t want to see it. Hence there’s a need for moderation as well as filtering.

Here’s a lighter way to look at it: if you find that there are a bunch of Nazis in your bar, and you make them welcome but ensure they’re all wearing swastika armbands so that anyone who doesn’t like Nazis doesn’t have to socialize with them, then you haven’t actually solved the problem — you’re still the Nazi bar.

Moderation, filtering and Bluesky #

This brings me to the big question: why does the Bluesky team keep talking about moderation and filtering as if they’re the same thing?

If you’re generally in favor of what they’re doing, you might be tempted to say that they just don’t see a difference. I don’t think that’s a tenable point of view, because Bluesky does, in fact, have a moderation team. When they delete content, they seem to describe this as “content taken down” or a “network takedown”.

But that’s what most people mean when they talk about moderation. Even the people who don’t like moderation use the term to refer to material being removed from a site.

Bluesky describe how they respond to reported material:

Some of these tools take direct action on visual content by applying labels like “gore” to violent imagery

But later:

Child sexual abuse and exploitation, including the distribution of child sexual abuse material (CSAM), is obviously unacceptable anywhere on the Internet, including Bluesky. We built AT Protocol and Bluesky to ensure we can rapidly identify and remove this content anywhere it appears on our network, even when federation becomes available in the future.

So violent imagery — presumably including the terrorist beheading videos, livestreamed mass shootings, suicides, and children getting beaten — gets labeled so you can opt out of seeing it. Child sexual abuse material, on the other hand, gets moderated and removed from anywhere on the network.

Clearly the Bluesky team understand that these are two fundamentally different processes.

Does the new “Ozone” software allow you to actually moderate and remove content from the Bluesky network? I’m going to go out on a limb and say that no, it does not. So while they are selling it as “moderation”, and the media are gullibly repeating that angle, it’s actually just collaborative filtering.

The people who support Bluesky’s distributed filtering should also think about what Bluesky are saying. If they can, as they claim, remove child sexual abuse material across the entire network, then they have centralized moderation ability. So what’s to stop them from unilaterally deciding to remove your content?

Old man yells at cloud #

I understand that linguistic drift is a thing. My point is that there’s an important distinction between actually moderating and removing content from a server, and merely giving people the option of not seeing it through filtering.

When people complain about moderation, they’re not complaining about people being able to block them, or that people aren’t seeing their posts because they used a keyword everyone blocks. No, they’re complaining about content actually being removed. Similarly, when people talk about setting up a filter, they don’t expect to delete posts from the network — they just mean they are going to set something up so they don’t see those posts. So it seems to me that in everyday speech, people understand that moderation and filtering are fundamentally different things, and they don’t use the terms interchangeably.

Systems like Facebook and Twitter also make the distinction clear when they talk about how they operate. Twitter didn’t describe its word filters or account blocking as moderation. Facebook doesn’t describe unfriending people as moderation.

Bluesky, on the other hand, seem to be actively trying to obfuscate the difference between filtering and moderation. Why is that?

One answer might be that moderation is expensive. You have to employ people to do a damaging job, and as your network gets larger the amount of moderation required increases massively. If you can avoid having to moderate anything, that’s a massive cost savings. Perhaps Bluesky want to convince people that filtering is all that’s needed, and then push the cost of maintaining the filtering onto other people.

Whatever the reason, I think if Bluesky want to be trusted, they need to start being honest about when they’re talking about actual moderation (as it exists on every other social network), and when they’re talking about filtering.

What about Mastodon? #

Mastodon has filtering. It’s under the “Filters” heading in the preferences. It’s not described as moderation.

Mastodon also has moderation. When an abusive user is reported and the server owner deletes their posts (or their entire account), the posts are gone from the server. If you go look at their account, you won’t see the posts. The larger instances, such as mastodon.social, have paid moderation teams, because it’s that big of a problem.

Conclusion #

Maybe moderation is generally prone to abuse — I’ve been on the receiving end of some questionable moderation decisions. Perhaps as a society we need to make decisions about what should be moderated on the Internet, and what should just be filtered. Maybe moderation of a distributed system is impossible, or impossible to scale effectively.

Those are all topics to discuss, but if we’re going to have those conversations, we need to be clear about what we’re discussing, and that means not setting out to obscure the important differences between how moderation and filtering work.