neil trained a classifier using NUMBER sets with about NUMBER ham and spam in each we re missing half his test run results due to a URL bug since fixed the before custom fiddling figures on the NUMBER reported runs were false positive percentages NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total unique fp NUMBER false negative percentages NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total unique fn NUMBER the total unique figures counts all NUMBER runs it s just the individual run fp and fn percentages we re missing for NUMBER runs jeremy reported these before custom fiddling figures on NUMBER sets with about NUMBER ham and spam in each false positive percentages NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total unique fp NUMBER false negative percentages NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total unique fn NUMBER so things are clearly working much better for neil both reported significant improvements in both f n and f p rates by folding in more header lines neal added received analysis to the base tokenizer s header analysis and jeremy skipped the base tokenizer s header analysis completely but added base subject line like but case folded tokenization for almost all header lines excepting only received data x from_ and i suspect all those starting with x vm when i try NUMBER random pairs of NUMBER ham NUMBER spam subsets in my test data i see false positive percentages NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total unique fp NUMBER false negative percentages NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total unique fn NUMBER this is much closer to what neil saw but still looks better another run on a disjoint NUMBER random pairs looked much the same total unique fp rose to NUMBER and fn fell to NUMBER on a third run with another set of disjoint NUMBER random pairs likewise with fp NUMBER and fn NUMBER so i m pretty confident that it s not going to matter which random subsets of NUMBER i take from my data it s hard to conclude anything given jeremy s much worse results if they were in line with neil s results i d suspect that i ve over tuned the algorithm to statistical quirks in my corpora 