working with the main mehtod of tokensiser to try to get it to scan in
all the text files. unsuccessful so far. but making progress