This guy really should go work for Google and figure out the problems they need to deal with running a service like Gmail. Even for just a little while.

At work we had a researcher from Yahoo Mail come in and give a presentation on the machine learning techniques they use to try and stop spammers abusing their mail servers. It was eye-opening to learn just what kind of hourly battle they face to keep spam out of their systems and the ways they are trying to combat it. It was even more enlightening when the presenter told stories about the problems that machine learning can't solve - like people within the company being bribed to whitelist spam companies based in Vegas.

On the surface it's such a simple problem, and I'm sure anyone who's tried to prevent their web application's outgoing mail being marked as spam by the evil corporations of Yahoo and Google will have had the desire to go write a blog post saying what a crock of shit the whole thing is and how they would never take part in that. But here's the thing - those systems are in place because if they weren't, email would be a completely useless form of communication at this point.

The people sending spam make _millions_ of dollars abusing a system which is popular because its open and based on trust. That kind of money combined with greed gives people all different levels of drive and incentive to get their emails about bigger penises and viagra through to your inbox. Every time they prevent one form of attack, these guys will create a new one.

To do this they do things like install mail servers on unsuspecting user's machines, specifically targeting Yahoo/Hotmail/Google users because their IP will obviously need to be trusted by those companies. They will also hack into other people's private mail servers. They will spoof email headers and pretend they're someone else. They will hire people, experts, who will find new ways of breaking in to servers they detect as having mail servers running on them. All this just to get past the spam filters and prevention that make email a useful form of communication to begin with.

And let's forget the people who couldn't set up their own mail server for just a second. I like to think I know what I'm doing. After installing Postfix and jumping through all the hoops to get my emails whitelisted by Gmail and making sure I didn't have an open relay on my mail server, you know what happened? Someone managed to hack in by brute force anyway. I only noticed because of the _millions_ of automated replies that were coming in every day from dead email accounts or people that were out of office.

Now, I could have worked hard to fight this. I could have did something other than changing my passwords and hoping they didn't get crack them again. But the point is - I only ran a mailserver to get email delivered to me on my personal domain. I didn't want to have to fight and battle and dedicate myself to solving this problem. I wanted to take this thing for granted. I just wanted to send and receive email. Instead bad people could not only sit there and read all my incoming mail - but they could use my server to spam people and get me blacklisted and blocked from so many other services I worked so hard to be trusted by. And they did all this without even specifically targeting me. I was a statistic to them, someone who simply didn't know what they know. In the end, I moved my personal mail account to Google Apps, free of charge. Problem solved.

By using Gmail or Yahoo Mail or Hotmail - you are almost definitely more secure than setting up your own mailserver. You have people paid hundreds of thousands of dollars a year working full time to make sure your data is secure. I mean if privacy is your reason not to use Gmail, then I hope for your sake your mail server is secure. Maybe you think it is. I know I did too.

And all these people complaining about advertisements based on the content of their emails. Yahoo Mail had a team of like 30 people just doing _research_ on how to stop spammers. Then all these other people working on support. How does that service get provided to us _free of charge_ without advertisements or some sort of monetisation? I know in some people's heads they think it's literally just a Bayesian classifier and some hand-coded rules, but it's so beyond that.

And of course, let's not forget the fact that a lot of people would not be able to set up their own mail server anyway. Maybe you don't need them, but Hotmail, Gmail and Yahoo Mail enable hundreds of millions of people to communicate _for free_ with other people around the world that otherwise wouldn't be technically competent enough to buy a domain name and set up a local mail server. It lets you communicate with them too, because they don't get frustrated wading through hundreds of spam emails just to read the good stuff.

And that system only works because we have good guys that are fighting the bad guys who want to ruin it for the rest of us. And this is just the one example of email. Which has all this decentralised and open properties that you desire. I am reminded of Diaspora when they released a first beta of their code and it got absolutely torn to shreds for security reasons, and we haven't heard much since.

The real world sucks.

That's why I think it might be a good idea for you to go work for Google.