feat: add current_task to forward

feat: use property in LoRA parametrization

feat: added LoRA copyright notice

feat: use property instead of setter

clean up embeddings.py (#7)

feat: return from_bert for from_pretrained

support-fast-tokenizer (#6)

feat: made from_bert work

feat: choose flash attention heuristically if not set explicitly

feat: select first LoRA upon initialization

feat: formatting and type hints

support-multiple-task-ids (#5)

fix: use proper initilization for embedding layer

fix: fixed typo

feat: added LoRA

feat: assert return_dict

fix: same assertions in other models

fix: assert is None for other kwargs too

feat: added head_mask

added classifier dropout

fix: formatting

fix: formatting

feat: added further GLUE models

feat: added BertForSequenceClassification

fix: cast mask to bool

reference the flash attention GitHub

fix: move flash components into top-level

feat: try to fix import error

feat: moved flash attention code into this repository

add tokenizer

feat: added encode method

fix: try to skip initialization of task type embeddings

fix: try to skip initialization of task type embeddings

feat: added option for QK normalization

fix: removed obscure config options

feat: added small config

feat: implement task type embeddings (#1)

feat: added back option not to use flash attention

feat: support gradient checkpointing

Added additional config options

removed unused imports

removed __init__ from BertPretrainedModel

added config_class and base_model_prefix

Fixed typo

Fixed typo

Try to subclass PretrainedModel

Try to subclass PretrainedModel

strict=True for debugging

try to simplify checkpointing

changed model_type