If you think you can create a website and allow anyone to post content on there without moderation, then bless your heart, as my imaginary southern grandma would say. Being able to post anything anonymously means, for some people, that they will do exactly this. I won’t mention the content you’ll end up with, as this is a family-friendly blog.
Reddit “solved” content moderation by outsourcing it to the communities themselves. Being a moderator on Reddit is a huge time investment. Everyone wants your attention, and Reddit gets a lot of it, so any decently successful subreddit gets flooded with ads, bots and things that are not in the interest of the subreddit. This also led to the cliché of the typical Reddit mod: unemployed, always online and hungry for power, as you need the first two things to be able to successfully moderate.
The issue then arises that these mods can shape their communities however they want, for better or worse. Often for the worse, sadly. But somebody has to do it, and the community would be waaaay worse without anyone moderating it.
With the advent of ML, our computers can understand language well enough to even understand irony, so they can say if someone genuinely hates on someone or is just doing it as a joke.
Detecting irony
Detecting irony
Small rabbit hole here: How good are computers actually at detecting irony?
The paper LLMs vs. humans in sarcasm detection on German soccer tweets created a small dataset and tested it against models. While their paper showed that humans were still better, they used older models. I ran it against the latest breed of SOTA models as well as some decision models (including Jev). SOTA models are now, on average, better at understanding your joke than your neighbour - use this information however you like. Jev is not as good yet, but clearly outperformed the other decision models that I tested. Certainly good enough, I felt.
Update, October 9: I also tested Clef 27B locally, using Q8 weights with llama.cpp on my M5 Pro. It scored 56/100: all 50 literal tweets correct, but only 6 of the 50 sarcastic ones. This is one run of the quantized local model; I haven’t compared it with Cloudflare’s hosted version.
As computers are now so good at understanding language and the intent behind it, automoderation is widely used in moderating the internet; for example, YouTube is basically fully auto-moderated by now . Every other social network most certainly is similar. Humans should not have to review the filth of the internet.
These automoderation techniques are now accessible to anyone, basically. You don't need your own huge training corpus anymore — no fancy pipelines, no huge infra costs. One very interesting one was just released.
Jev is a decision model that you don't have to train to use. It's quite good at classifying anything you throw at it. And as it is very fast and cheap in doing so, I was wondering: could you automoderate a Reddit-like website purely with Jev? I could run a decently sized community with it: 10,000 posts and comments a day would cost about $15 for a 30-day month.
For scale, Reddit reported 4.4 billion posts and comments in 2025. Running that volume through Jev at the same rate would come to roughly $18,000 a month. So actually quite attainable.
So I built Jevdit — a Reddit-like social network, where everything is moderated by Jevdit. You can post and create subreddits however you want, as long as Jev permits it. You can even change the rules of a subreddit — by creating a Jevdit post where you argue with the community about it, and Jev then decides if it thinks that enough people agree with the changes.
To make sure I don’t just blindly put my hand into the fire here while Jev looks at me with a smirk, saying “Trust me, bro,” I ran some of my own benchmarks to calm down my anxiety about putting an open website into the birthplace of 4chan.
Thankfully, there are several public datasets that you can use to see how well your automoderation technique performs. I also ran it with the latest models as well as some other interesting models. I combined the results of all datasets here for an average; below, you can see the detailed results if you care.
A moderator has two jobs: stop the bad stuff and let people talk. You want a balance — if you just block every comment, you definitely catch 100% of the bad stuff. So we have two bars here: how much of the bad stuff was caught vs. how much of the good content was allowed.
Jev gives you probabilities, so you can choose how strict you want moderation to be. The benchmark shows strict, balanced and relaxed settings for Jev, and balanced settings for the other models with adjustable gates. Changing the cutoff changes how much harmful content gets caught and how much benign content gets through.
Balanced targets at least 95% harmful content blocked; Jev relaxed targets 90%. Cutoffs were chosen and evaluated on these same 325 labeled cases. Jev strict retains the original rules. All modes reuse saved model responses.
Results by dataset
The October 6 Clef run did much better at moderation than at irony. Its balanced setting caught 165/173 harmful examples (95.4%) and allowed 132/152 benign examples (86.8%). Jev balanced also caught 165 harmful examples and allowed 129 benign ones; Jeeves balanced allowed 117. Clef’s cutoff was 0.48. Like the other balanced cutoffs, I selected it on these same 325 labeled examples, so this needs a fresh test set before treating the result as an estimate for new content. The 70 unscored probes were excluded from that selection.
All 395 Clef moderation requests completed successfully. Median latency was 5.35 seconds on my M5 Pro with 48 GiB of memory, using the same Q8 model as the tweet test. There were no provider fees; the hardware and electricity costs aren’t included.
So, with all this analysis, I feel a little bit confident I won't create another Tay. So check out jevdit.com and see if Jev can handle the moderation.