Text2Text Generation
Transformers
Safetensors
English
encoder-decoder
Inference Endpoints
Bachstelze commited on
Commit
3b68880
1 Parent(s): 2217d4c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -34,4 +34,9 @@ datasets:
34
  license: mit
35
  ---
36
 
37
- # Model Card for the test-version of instructionBERT
 
 
 
 
 
 
34
  license: mit
35
  ---
36
 
37
+ # Model Card for the test-version of instructionBERT for Bertology
38
+
39
+ A minimalistic instruction model with an already good analysed and pretrained encoder like BERT.
40
+ So we can research the [Bertology](https://aclanthology.org/2020.tacl-1.54.pdf) with instruction-tuned models and investigate [what happens to BERT embeddings during fine-tuning](https://aclanthology.org/2020.blackboxnlp-1.4.pdf).
41
+ We used the Huggingface API for [warm-starting](https://huggingface.co/blog/warm-starting-encoder-decoder) [BertGeneration](https://huggingface.co/docs/transformers/model_doc/bert-generation) with [Encoder-Decoder-Models](https://huggingface.co/docs/transformers/v4.35.2/en/model_doc/encoder-decoder) for this purpose.
42
+