In this guest post, Lewis Van Winkle talks about F5Bot, a free service that emails you when selected keywords are mentioned on Reddit, Hacker News, or Lobsters. He explains in detail how F5Bot is able to process millions of comments and posts from Reddit every day on a single VPS. You can check out more of Lewis Van Winkle’s writing at codeplea.com, and his open source contributions at github.com/codeplea.
I run a free service, F5Bot. The basic premise is that you enter a few keywords to monitor, and it’ll email you whenever those keywords show up on Reddit or Hacker News. I’m hoping to add a few more sites soon.
This post is going to be about how I built the service. Specifically, we’re going to focus on scraping every post and comment from Reddit in real time....