integrate the new parser
using the all_tokens mode for tokenizer
rebuild