change gitignore, script is semi-working
having some problems with encoding, maybe best to build corpus by hand
forgot to add
tidying script, about to delete some stuff
tidyied script
reran script with scikit-learn source code
reran script only 420 patterns not found: good frequencies
profiling shows that the function permutations is a large bottle neck and should be removed completly, I have removed it partly and preformance has greatly improved.