Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,10 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+model.safetensors filter=lfs diff=lfs merge=lfs -text

2_Dense/model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a548639f4e10e8b96be6a4846f0932ca2d011d491b37489a6b4751a3c096e49d
-size 132
+oid sha256:981a518204a50b1a68e27c5a2539aad511e6bf5cfaba409deb8cea8605f776eb
+size 2362560

2_Dense/pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c4af843f3f778124777c11604c9e22c6afdca8c27764a44961099e981cf6355d
-size 132
+oid sha256:06fb85120e40adf0ab188c4f0cc7684f702cb2023532947d1b85f325b0a3645c
+size 2363431

README.md

@@ -119,14 +119,9 @@ library_name: sentence-transformers
 license: apache-2.0
 ---
 
-# AviLaBSE
-This is a port of the [LaBSE](https://tfhub.dev/google/LaBSE/1) model to PyTorch. Language-agnostic BERT Sentence Encoder (LaBSE) is a BERT-based model trained for sentence embedding for 109 languages.  It can be used to map 109 languages to a shared vector space. The pre-training process combines masked language modeling with translation language modeling. The model is useful for getting multilingual sentence embeddings and for bi-text retrieval.
+# LaBSE
+This is a port of the [LaBSE](https://tfhub.dev/google/LaBSE/1) model to PyTorch. It can be used to map 109 languages to a shared vector space.
 
-- Model: [HuggingFace's model hub](https://huggingface.co/sartifyllc/AviLaBSE).
-- Paper: [arXiv](https://arxiv.org/abs/2007.01852).
-- Original model: [TensorFlow Hub](https://tfhub.dev/google/LaBSE/2).
-- Blog post: [Google AI Blog](https://ai.googleblog.com/2020/08/language-agnostic-bert-sentence.html).
-- Conversion from TensorFlow to PyTorch: [GitHub](https://github.com/sartify).
 
 ## Usage (Sentence-Transformers)
 
@@ -142,74 +137,11 @@ Then you can use the model like this:
 from sentence_transformers import SentenceTransformer
 sentences = ["This is an example sentence", "Each sentence is converted"]
 
-model = SentenceTransformer('sartifyllc/AviLaBSE')
+model = SentenceTransformer('sentence-transformers/LaBSE')
 embeddings = model.encode(sentences)
 print(embeddings)
 ```
 
-```python
-import torch
-from transformers import BertModel, BertTokenizerFast
-
-tokenizer = BertTokenizerFast.from_pretrained("sartifyllc/AviLaBSE")
-model = BertModel.from_pretrained("sartifyllc/AviLaBSE")
-model = model.eval()
-
-english_sentences = [
-    "dog",
-    "Puppies are nice.",
-    "I enjoy taking long walks along the beach with my dog.",
-]
-english_inputs = tokenizer(english_sentences, return_tensors="pt", padding=True)
-
-with torch.no_grad():
-    english_outputs = model(**english_inputs)
-```
-
-To get the sentence embeddings, use the pooler output:
-
-```python
-english_embeddings = english_outputs.pooler_output
-```
-
-Output for other languages:
-
-```python
-italian_sentences = [
-    "cane",
-    "I cuccioli sono carini.",
-    "Mi piace fare lunghe passeggiate lungo la spiaggia con il mio cane.",
-]
-japanese_sentences = ["犬", "子犬はいいです", "私は犬と一緒にビーチを散歩するのが好きです"]
-italian_inputs = tokenizer(italian_sentences, return_tensors="pt", padding=True)
-japanese_inputs = tokenizer(japanese_sentences, return_tensors="pt", padding=True)
-
-with torch.no_grad():
-    italian_outputs = model(**italian_inputs)
-    japanese_outputs = model(**japanese_inputs)
-
-italian_embeddings = italian_outputs.pooler_output
-japanese_embeddings = japanese_outputs.pooler_output
-```
-
-For similarity between sentences, an L2-norm is recommended before calculating the similarity:
-
-```python
-import torch.nn.functional as F
-
-def similarity(embeddings_1, embeddings_2):
-    normalized_embeddings_1 = F.normalize(embeddings_1, p=2)
-    normalized_embeddings_2 = F.normalize(embeddings_2, p=2)
-    return torch.matmul(
-        normalized_embeddings_1, normalized_embeddings_2.transpose(0, 1)
-    )
-
-
-print(similarity(english_embeddings, italian_embeddings))
-print(similarity(english_embeddings, japanese_embeddings))
-print(similarity(italian_embeddings, japanese_embeddings))
-```
-
 
 
 ## Evaluation Results
@@ -232,4 +164,5 @@ SentenceTransformer(
 
 ## Citing & Authors
 
-Have a look at [LaBSE](https://tfhub.dev/google/LaBSE/2) for the respective publication that describes LaBSE.
+Have a look at [LaBSE](https://tfhub.dev/google/LaBSE/1) for the respective publication that describes LaBSE.
+

flax_model.msgpack

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:211fcbb6fed2aedfa31e7da2ecd7ac485ac8010de6e18afbd8c00f722b18c8cc
-size 135
+oid sha256:4cbe50771a6b147d2da0beb6da1d80908a706cec2e2e06a09873649ed183e884
+size 1883714625

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6566b39f08255d6abd32052675a6534fc20f5c262ce06e2ab5862dbd01cf7b7d
-size 135
+oid sha256:77d8e1f2dbab6eb5d3c261ce9d3dbf1e3c69e02938c95f934f94f42c22dfa31f
+size 1883734344

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2e6ff92c0dc1e0e18a7abf27a78921b7cd1a8c51373b44970e718efc81aada64
-size 135
+oid sha256:c9e7daf739f87c2168a6d1baffdae5782eceb03eb6de61950284a925234c6865
+size 1883785969

tf_model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4e0ebd757dea4709bb5d69d66ed94e6e46f5275d67ac5f04d791b7854106a885
-size 135
+oid sha256:e971d0404bba02ce8ab4568bc9625a74e6fbe99c7cdc927d5f3095597a70c55d
+size 1883974632