now uses the sparse arff format, avoiding all those 0s that caused the file to be 200+ megabytes. Reduced to about 1 Mb
excluded tokens seen only once. they are probably typos/unlikely to be seen again.