One of the issues that we encountered on Bubblews was spam posts, where users with bad intentions would either post content that was a jumble of made-up content, content that was packed full with links promoting other websites or products, dropping in content from elsewhere (like Wikipedia) and claiming it as their own, or writing in other languages than English.
The implementation we ended up going with to catch this has worked great at catching spam, so I wanted share what we did in case it is useful for people in similar situations. Our stack uses Rails and Redis, so this example uses it too, but it can be adapted for other setups.
Also, a big shout out to Kevin Kimball for his work on implementing this on the site.
Notifications for new posts that are likely spam in Slack
To fight spam, we looked into a bunch of different options. We use Simhash for catching plagiarism on the site, but we also needed a way to fight spam posts that were coming from external sources. After looking at various options, we ended up deciding to use Bayes' Theorm to calculate the likelihood of a post's content being spam. Bayes is often integrated into email software as one tool for detecting incoming spam emails.
The Wikipedia post on Naive Bayes spam filtering does a nice job of explaining the details of how it works. Algorithms are used to calculate the likelihood that content containing certain words and other factors is spam.
Naive Bayes classifiers work by correlating the use of tokens (typically words, or sometimes other things), with spam and non-spam e-mails and then using Bayesian inference to calculate a probability that an email is or is not spam.
Since the Ruby community is amazing, we found that there are various gems to help with implementing Bayes. We ended up choosing stuff-classifier because of the easy Redis integration. Add it to your Gemfile with
Next up, we need to add an initializer in
config/initializers/stuff_classifier.rb with settings for the gem. The initializer tells the library where to save the data that will be used with training the classifier. On production with Heroku, we use Redis through Open Redis, so the initializer pulls in the URL from an environment variable. Locally, the initializer defaults to the standard
127.0.0.1:6379 so it does not need to be explicitly set.
Next up we need to setup in Rails the ability to easily teach the classifier what is spam and what isn't ("ham" vs. "spam").
First, let's setup the two options "ham" and "spam" on the Post model, which is what we plan on classifying.
Next we need to add in some helper methods to make it easier to see whether the Bayes' classifier thinks the post is ham or spam.
Now in our view, we want to show what Bayes thinks the post is and add buttons to allow us to toggle and train the classifier.
First, we need to add an endpoint we can hit to toggle the labeling of a post. If you are labeling posts, this could be in your posts controller.
Second, show the current label in the view and add links to be able to toggle the labeling.
Now that we setup the ability to see and toggle labeling in the view, we will want to setup some actions that occur when a new post is likely spam. For simplicity in this tutorial, we'll use an
after_create callback that will:
- Notify us (or our moderators) that a post is likely spam, so the post can be reviewed, and if it is spam, remove the post and offending user account. We shoot this into Slack.
- Hide the post from feeds to keep posts that are likely spam from showing publicly.
Now the most important part: we need to train the classifier what's spam and what isn't with data we already know the answer to. For us, we found it easiest to load up a sample of our production data and manually go through and label posts as ham or spam. If you already know for certain what posts are spam and what posts aren't, this could automated some with a script that loops through and labels each post.
Later, as you encounter posts that are incorrectly labeled, you'll want to label them correctly to continue to teach the classifier.
And that's how you can easily integrate machine learning with Ruby on Rails to fight spam!