 tim otoh my data subtree currently has more than NUMBER NUMBER files slobbering over NUMBER million bytes even if i had a place to put that much stuff i m not sure my isp would let me email it in one msg wink skip do you have a dialup or something more modern wink much more modern a cable modem with a small upload rate cap there s a reason the less modern uncapped home went out of business wink NUMBERmb of messages zipped would probably compress pretty well under NUMBERmb i d guess with all the similarity in the headers and such you could zip each of the NUMBER sets individually and upload them somewhere i suppose this could finish over the course of an afternoon now where s somewhere i expect we ll eventually collect several datasets sourceforge isn t a good place for it they expect projects to distribute relatively small code files and complain if even those get big how about random sampling lots of public mailing lists via gmane or something similar manually cleaning it distributing that load over a number of people and then relying on your clever code and your rebalancing script to help further cleanse it what then are we training the classifier to do graham s scoring scheme is based on an assumption that the ham vs spam task is easy and half of that is due to that the ham has a lot in common it was an experiment to apply his scheme to all the comp lang python traffic which is a lot broader than he had in mind URL has long had a generous definition of on topic wink i don t expect good things to come of making it ever broader unless your goal is to investigate just how broad it can be made before it breaks down the problem with the ham is it tends to be much more tied to one person not just intimate but unique than the spam which is a feature from graham s pov the more clues the better this smoking guns only approach should work i save all incoming email for ten days gzipped mbox format before it rolls over and disappears at any one time i think i have about NUMBER NUMBER NUMBER NUMBER messages most of it isn t terribly personal which i would cull before passing along anyway and much of it is machine generated so would be of marginal use finally it s all ham n spam mixed together do we call that an omelette or a denny s grand slam unless you re volunteering to clean it tag it package it and distribute it i d call it irrelevant wink 