 jeremy hylton the total collections are NUMBER messages i trained with NUMBER NUMBER messages i m reading this now as that you trained on about NUMBER spam and about NUMBER ham that s less than NUMBER of the sizes of the training sets i ve been using please try an experiment train on NUMBER of each and test once against the other NUMBER of each do that a few times making a random split each time it won t be long until you discover why directories of individual files are a lot easier to work e g random shuffle makes this kind of thing trivial for me 