Adding common method get_tokenizer() to be shared by classifier commands.