Changes to add text analysis to predictions
Adding local prediction for the full_terms_only option of token_mode