Improve preprocessing pipeline, add tracking of character offsets.