put a bogus threshold number beside 'onbit' designation in filename
be more clever about splitting words into tokens
be more clever about apostrophes
avoid splitting on internal middle dots
remove mention of diagonal_projection, because this causes memory leak
use euro sign to as delimiter for spellcheck list
remove debug statement
include 'ocrx_word' spans
include metadata file on book, if available
add tools for generating training lines from HOCR output
move files