tokens: struct and array , which other answers recommend , are fine for the details of the implementation , and might be all you need if your needs are always to sequentially read all of the file or a prefix of it .
ner_tags: B-Library O B-Library O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O