restructured pre- and post-processing

Should fix detokenization issues with escaped characters