 because i get mail through several different email addresses i frequently get duplicates or triplicates or more plicates of various spam messages in saving spam for later analysis i haven t always been careful to avoid saving such duplicates i wrote a script some time ago to try an minimize the duplicates i see by calculating a loose checksum but i still have some duplicates should i delete the duplicates before training or not would people be interested in the script i d be happy to extricate it from my local modules and check it into cvs skip 