99.9% Spam Filtering Performance With Reputation-based System

Feature Article by Steve Kirsch of Propel Software

I hate spam. I really do. I'm even more upset by scam artists that attempt to steal my wife's identity (known as "phishes") to siphon her bank, or PayPal account. This is just not what email was supposed to be all about. Can this be resolved?

The only 100% proof method I'm aware of is to stop using email. This may seem like a radical approach, but there are actually organizations that have taken this step. For my company, Propel Software, shutting down email is not an option. Email is absolutely critical to our business. A couple of years ago, spam started to become a serious problem for us. It bogged down our email server and several employees were disgusted by the offensive nature of some of the messages. Personally, after spending significant time deleting spam from my Inbox, I decided to try an anti-spam solution in the hope of finding remedy. I tried several commercial solutions as well as open-source products, but none of them would work effectively. Typically, the solutions would filter anywhere between 80% and 95% and in general, the more spam that was blocked; the more legitimate messages were caught in the filters too. This bothered me so much I decided to address the problem myself.

In testing these other solutions, I realized the vendors are approaching the problem from the wrong angle. This became particular evident as vendors began to build patchworks of anti-spam techniques. It became a "Whac-a-Mole" game. Anti-spammers developed new crude techniques such as challenge/response, Bayesian, whitelists, blacklists, "finger-printing" etc., but spammers quickly found ways around them. It seemed as if all solutions ignored the fundamental truth about spammers - their behavior. Spammers can obfuscate their messages in a million ways, but they can't hide

• The origin of a message
• How the message was sent
• How often they're sending messages
• Who they are
• What the recipient is doing with the message

These facts became the guiding principles when I started to design our anti-spam solution. I assembled a small team to build a prototype that would prove my theory and like most product development projects, we cycled through a couple of iterations until we found the right recipe.

Now, nearly two years later, the algorithm, or the brains of the solution (Propel EPG, short for Propel Email Protection Gateway) records sender behavior, and based on the information establishes a reputation for a sender. A reputation-based anti-spam system keeps track of whether a sender typically engages in good behavior (sending legitimate email), or bad behavior (sending spam).

EPG is the first purely reputation-based system to identify spam and phishes. Through a complex mesh of rules, sender reputations are determined and stored at a global level (available to all EPG sites) and at a site level (local EPG sites). Finally, reputations are automatically fine-tuned through individual users' preferences.

EPG is fully automated. There is no administration required to maintain or increase filtering performance. There are no lists to maintain, no filters to tune and no database to feed with spam or legitimate messages.

The product is currently in beta testing and I'm very excited about the results so far. The filtering performance is at an average 99.86% spam detection rate with a 99.97% filtering accuracy. Figure 1 shows actual filtering performance at four current beta sites.

Figure 1: Sample EPG spam detection and accuracy rates at four beta sites (7-day average ending April 8th, 2005).

The spam detection rate and filtering accuracy will get better as we add more users. The system is designed so that filtering performance increases with the number of global users.

The finished product will be available as both email security appliance and software and will include virus scanning and cleaning for inbound and outbound traffic.

While EPG isn't as effective as getting rid of your email system all together, I sincerely believe it's as close as you can possibly get.

If you want to know more about the Propel Email Protection Gateway, or want to evaluate it, please contact my colleague Marten Nelson.

Steve Kirsch
Founder & Chairman
Propel Software

Steve has been involved with the Internet and high-tech companies for more than 27 years. Since the early 80s, he has founded three successful technology companies. Prior to Propel, he founded Infoseek Corporation, which was acquired by Disney in November of 1999. While at Infoseek, Steve was responsible for creating many of the company's award-winning products, including the NetSearch service, Ultraseek Server, Infoseek Express, GO Guides, and GO Auction.

Steve's first start up was Mouse Systems Corporation. Afterward, he founded Frame Technology, which was acquired by Adobe.

Steve is also an active philanthropist and together with his wife Michele, started a $75 million foundation, which donates to a wide variety of charitable causes. In 1999, Steve and Michele were named Outstanding Philanthropists of the Year by the Silicon Valley chapter of the National Society of Fund Raising Executives, and recognized by Slate Magazine as the 8th largest charitable givers in America.

Steve holds both B.S. and M.S. degrees in electrical engineering and computer science from the Massachusetts Institute of Technology.