extreme-bert/extreme-bert: ExtremeBERT is a toolkit that accelerates the pretraining of customized language models on customized datasets, described in the paper “ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT”.
Category: llm
Language: Python
Topics: bert, deep-learning, language-model, language-models, machine-learning, natural-language-processing, nlp, pytorch, transformer
Homepage: https://extreme-bert.github.io/extreme-bert-page