Adding local prediction for the full_terms_only option of token_mode