Extracts tokens from an input file for further natural language processing.