 tp i m reading this now as that you trained on about NUMBER spam and tp about NUMBER ham that s less than NUMBER of the sizes of the tp training sets i ve been using please try an experiment train tp on NUMBER of each and test once against the other NUMBER of each jeremy this helps a lot possibly i checked in a change to URL overnight getting rid of mincount that gave a major improvement in the f n rate too independent of tokenization here are results with the stock tokenizer unsure what stock tokenizer means to you for example it might mean tokenizer tokenize or mboxtest mytokenizer tokenize training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total false pos NUMBER NUMBER NUMBER total false neg NUMBER NUMBER NUMBER note that those rates remain much higher than i got using just NUMBER of ham and NUMBER of spam that remains a mystery and results from the tokenizer that looks at all headers except date received and x from_ unsure what that means too for example looks at might mean you enabled anthony s count them gimmick and or that you re tokenizing them yourself and or training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on mbox home jeremy mail inbox NUMBER mbox home jeremy mail spam NUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total false pos NUMBER NUMBER NUMBER total false neg NUMBER NUMBER NUMBER is it safe to conclude that avoiding any cleverness with headers is a good thing since i don t know what you did exactly i can t guess what you seemed to show is that you did something clever with headers and that doing so helped the after numbers are better than the before numbers right assuming that what you did was override what s now tokenizer tokenizer tokenize_headers with some other routine and didn t call the base tokenizer tokenize_headers at all then you re missing carefully tested treatment of just a few header fields but adding many dozens of other header fields there s no question that adding more header fields should help tokenizer tokenizer tokenize_headers doesn t do so only because my testing corpora are such that i can t add more headers without getting major benefits for bogus reasons apart from all that you said you re skipping received by several accounts that may be the most valuable of all the header fields i m meaning tokenizer tokenizer tokenize_headers skipping them too for the reason explained above offline a week or two ago neil schemenauer reported good results from this scheme ip_re re compile r d NUMBER NUMBER d NUMBER NUMBER d NUMBER NUMBER d NUMBER NUMBER for header in msg get_all received for ip in ip_re findall header parts ip split for n in range NUMBER NUMBER yield received join parts n this makes a lot of sense to me i just checked it in but left it disabled for now 