finish bnf tokenizer