rewrote functions so as to be easier to read for someone less familiar with programming or with python. added more functions. still need to finish tokenize_files() function