 i also looked in more detail at some f p s in my geeks traffic the first one s a doozie that s the term right it has lots of html clues that are apparently ignored the clues below are loaded with snippets unique to html like br i meant to say that they were NUMBER NUMBER clues cancelled out by NUMBER NUMBER clues but that s wrong too it looks i haven t grokked this part of your code yet this one has way more than NUMBER clues and it seems the classifier basically ended up counting way more NUMBER NUMBER than NUMBER NUMBER clues and no others made it into the list i thought it was looking for clues with values in between apparently it found none that weren t exactly NUMBER NUMBER that sure sets the record for longest list of cancelling extreme clues this happened to be the longest one but there were quite a few similar ones i wonder if there s anything we can learn from looking at the clues and the html it was heavily marked up html with ads in the sidebar but the body text was a serious discussion of oo and soft coding with lots of highly technical words as clues including zope and zeo that there are any NUMBER NUMBER clues in here means the scheme ran out of anything interesting to look at adding in more header lines should cure that are there any minable but unmined header lines in your corpus left or do we have to start with a different corpus before we can make progress there the seventh was similar i scanned a bunch more until i got bored and most of them were either of the first form brief text with url followed by quoted html from website if those were text plain the html tags should have been stripped i m still confused about this part no sorry these were all of the following structure multipart mixed text plain brief text plus url s text html long html copied from website i guess you get this when you click on mail this page in some browsers that html tags aren t getting stripped remains the biggest mystery to me still this seems confused jeremy didn t use my trained classifier pickle he trained his own classifier from scratch on his own corpora that s an entirely different kind of experiment from the one you re trying indeed you re the only one so far to report results from trying my pickle on their own email and i never expected that to work well it s a much bigger mystery to me why jeremy got such relatively worse results from training his own and he s the only one so far to report results from that experiment i think it s still corpus size guido van rossum home page URL 