Before revision:  Hebrew is considered a Morphological Rich Language (MRL) , with unique characteristics that pose a great challenge in developing appropriate Hebrew NLP models . Analyzing multiple specifications of the BERT architecture, we come up with a language model that outperforms all existing Hebrew alternatives on multiple language tasks.
After revision:  Hebrew is considered a Morphological Rich Language (MRL) , with unique characteristics that pose a great challenge in developing appropriate Hebrew NLP models . Analyzing multiple BERT specifications, we find that while model complexity correlates with high performance on language tasks that aim to understand terms in a sentence, a more-parsimonious model better captures the sentiment of entire sentence. Either way, out BERT-based language model outperforms all existing Hebrew alternatives on multiple language tasks.