How to use this model directly from the
from transformers import AutoTokenizer, AutoModelWithLMHead tokenizer = AutoTokenizer.from_pretrained("T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb") model = AutoModelWithLMHead.from_pretrained("T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb")
The usage description above - provided by Hugging Face - is wrong! Please use this:
sentence-transformers package. See here: https://github.com/UKPLab/sentence-transformers
from sentence_transformers import models from sentence_transformers import SentenceTransformer # load BERT model from Hugging Face word_embedding_model = models.Transformer( 'T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb') # Apply mean pooling to get one fixed sized sentence vector pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension(), pooling_mode_mean_tokens=True, pooling_mode_cls_token=False, pooling_mode_max_tokens=False) # join BERT model and pooling to get the sentence transformer model = SentenceTransformer(modules=[word_embedding_model, pooling_model])
This is a German sentence embedding trained on the German STSbenchmark Dataset. It was trained from Philip May and open-sourced by T-Systems-onsite.The base language model is the dbmdz/bert-base-german-uncased from Bayerische Staatsbibliothek .
Sentence-BERT (SBERT) is a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically mean-ingful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT.
We did an automatic hyperprameter optimization with Optuna and found the following hyperprameters:
The final model was trained on the combination of all three datasets: