Before revision:  HeBERT is a transformer-based model for modern Hebrew text . Hebrew is considered a Morphological Rich Language (MRL) , with unique characteristics that pose a great challenge in developing appropriate Hebrew NLP models . Analyzing multiple specifications of the BERT architecture, we come up with a language model that outperforms all existing Hebrew alternatives on multiple language tasks.
After revision:  HeBERT is a transformer-based model for modern Hebrew text , which relies on a BERT (Bidirectional Encoder Representations for Transformers) architecture. BERT has been shown to outperform alternative architectures in sentiment analysis, and is suggested to be particularly appropriate for MRLs . Analyzing multiple specifications of the BERT architecture, we come up with a language model that outperforms all existing Hebrew alternatives on multiple language tasks.