simplified applying the tokenizers.