GitHub repository: marcomoldovan/hierarchical-language-modeling
Primary language: Jupyter Notebook
Description: We address the task of learning contextualized word, sentence and document representations with a hierarchical language model by stacking Transformer-based encoders on a sentence level and subsequently on a document level and performing masked token prediction.