 anthony baxter NUMBER sets each of NUMBERham NUMBERspam just ran the once it matched all NUMBER to each other URL sez training on data ham setNUMBER data spam setNUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on data ham setNUMBER data spam setNUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on data ham setNUMBER data spam setNUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on data ham setNUMBER data spam setNUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER training on data ham setNUMBER data spam setNUMBER NUMBER hams NUMBER spams NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER total false pos NUMBER NUMBER NUMBER total false neg NUMBER NUMBER NUMBER how were these msgs broken up into the NUMBER sets setNUMBER in particular is giving the other sets severe problems and setNUMBER blows the f n rate on everything it s predicting when the rates across runs within a training set vary by as much as a factor of NUMBER it suggests there was systematic bias in the way the sets were chosen for example perhaps they were broken into sets by arrival time if that s what you did you should go back and break them into sets randomly instead if you did partition them randomly the wild variance across runs is mondo mysterious i expect hammie will do a much better job on this already than hand grepping be sure to stare at the false positives and get the spam out of there yah but there s a chicken and egg problem there i want stuff that s _known_ to be right to test this stuff then you have to look at every message by eyeball any scheme has non zero error rates of both kinds so using the spambayes code to tell me whether it s spam is not going to help trust me wink it helps a lot i expect everyone who has done any testing here has discovered spam in their ham and vice versa results improve as you improve the categorization once the gross mistakes are straightened out it s much less tedious to scan the rest by eyeball on skip tokens yep it shows up in a lot of spam but also in different forms in hams but the hams each manage to pick a different variant of or whatever so they don t end up counteracting the various bits in the spam looking further a _lot_ of the bad skip rubbish is coming from uuencoded viruses c in the spam set for whatever reason there appear to be few of those in bruceg s spam collection i added code to strip uuencoded sections and pump out uuencode summary tokens instead i ll check it in it didn t make a significant difference on my usual test run a single spam in my setNUMBER is now judged as ham by the other NUMBER sets nothing else changed it does shrink the database size here by a few percent let us know whether it helps you before and after stripping uuencoded sections false positive percentages NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied won NUMBER times tied NUMBER times lost NUMBER times total unique fp went from NUMBER to NUMBER tied false negative percentages NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER lost NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER lost NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER lost NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER tied NUMBER NUMBER NUMBER NUMBER lost NUMBER NUMBER won NUMBER times tied NUMBER times lost NUMBER times total unique fn went from NUMBER to NUMBER lost NUMBER NUMBER 