Add README

README.md

@@ -0,0 +1,125 @@
+---
+license: apache-2.0
+language: ti
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
+- transformers
+widget:
+- text: "ግራፋይት ኣብ መላእ ዓለም ዳርጋ ብምዕሩይ ዝርጋሐ’ዩ ዝርከብ"
+---
+
+# TiRoBERTa BiEncoder Model
+
+This is a [sentence-transformers](https://www.SBERT.net) model for the Tigrinya language based on [TiRoBERTa-base](https://huggingface.co/fgaim/tiroberta-base).
+The maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+
+## Using Sentence-Transformers
+
+Using this model becomes easy when you have sentence-transformersinstalled:
+
+```shell
+pip install -U sentence-transformers
+```
+
+Then use the model as follows:
+
+```python
+from sentence_transformers import SentenceTransformer
+
+sentences = ["ሓደ ሰብኣይ ፈረስ ይጋልብ ኣሎ።", "ሓንቲ ጓል ክራር ትጻወት ኣላ።"]
+model = SentenceTransformer('fgaim/tiroberta-bi-encoder')
+embeddings = model.encode(sentences)
+print(embeddings)
+```
+
+## Using 🤗 Transformers
+
+Use the transformers library as follows:
+Pass the input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+
+# Mean Pooling - Take attention mask into account for correct averaging
+def mean_pooling(model_output, attention_mask):
+    token_embeddings = model_output[0]  # First element of model_output contains all token embeddings
+    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
+    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)
+
+
+# Sentences we want sentence embeddings for
+sentences = ["ሓደ ሰብኣይ ፈረስ ይጋልብ ኣሎ።", "ሓንቲ ጓል ክራር ትጻወት ኣላ።"]
+
+# Load model from HuggingFace Hub
+tokenizer = AutoTokenizer.from_pretrained("fgaim/tiroberta-bi-encoder")
+model = AutoModel.from_pretrained("fgaim/tiroberta-bi-encoder")
+
+# Tokenize sentences
+encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
+
+# Compute token embeddings
+with torch.no_grad():
+    model_output = model(**encoded_input)
+
+# Perform pooling. In this case, mean pooling.
+sentence_embeddings = mean_pooling(model_output, encoded_input["attention_mask"])
+
+print("Sentence embeddings:", sentence_embeddings)
+```
+
+## Architecture
+
+### Base Model
+
+The model properties:
+| Model Size | Layers  | Attn. Heads | Hidden Size  | FFN  | Parameters  | Max. Seq  |
+|------------|----|----|-----|------|------|------|
+| BASE       | 12 | 12 | 768 | 3072 | 125M | 512  |
+
+### BiEncoder Model
+
+- Max Seq Length: `512`
+- Word embedding dimension: `768`
+
+```
+SentenceTransformer(
+    Transformer(
+        {
+            'max_seq_length': 512,
+            'do_lower_case': False
+        }
+    ) # with Transformer model: RobertaModel
+
+    Pooling(
+        {
+            'word_embedding_dimension': 768,
+            'pooling_mode_cls_token': False,
+            'pooling_mode_mean_tokens': True,
+            'pooling_mode_max_tokens': False,
+            'pooling_mode_mean_sqrt_len_tokens': False,
+            'pooling_mode_weightedmean_tokens': False,
+            'pooling_mode_lasttoken': False,
+            'include_prompt': True,
+        }
+    )
+)
+```
+
+## Cite
+
+If you use this model in your product or research, you can cite it as follows:
+
+```bibtex
+@article{Fitsum2021TiPLMs,
+  author={Fitsum Gaim and Wonsuk Yang and Jong C. Park},
+  title={Monolingual Pre-trained Language Models for Tigrinya},
+  year=2021,
+  publisher={WiNLP 2021 co-located EMNLP 2021}
+}
+```