added model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -1,3 +1,135 @@
----
-license: mit
----
+---
+license: mit
+language: pt
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
+- transformers
+
+---
+
+# Serafim 335m Portuguese (PT) Sentence Encoder tuned for Information Retrieval (IR)
+
+This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+
+<!--- Describe your model here -->
+
+## Usage (Sentence-Transformers)
+
+Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
+
+```
+pip install -U sentence-transformers
+```
+
+Then you can use the model like this:
+
+```python
+from sentence_transformers import SentenceTransformer
+sentences = ["This is an example sentence", "Each sentence is converted"]
+
+model = SentenceTransformer('PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir')
+embeddings = model.encode(sentences)
+print(embeddings)
+```
+
+
+
+## Usage (HuggingFace Transformers)
+Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
+
+```python
+from transformers import AutoTokenizer, AutoModel
+import torch
+
+
+#Mean Pooling - Take attention mask into account for correct averaging
+def mean_pooling(model_output, attention_mask):
+    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
+    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
+    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)
+
+
+# Sentences we want sentence embeddings for
+sentences = ['This is an example sentence', 'Each sentence is converted']
+
+# Load model from HuggingFace Hub
+tokenizer = AutoTokenizer.from_pretrained('PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir')
+model = AutoModel.from_pretrained('PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir')
+
+# Tokenize sentences
+encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
+
+# Compute token embeddings
+with torch.no_grad():
+    model_output = model(**encoded_input)
+
+# Perform pooling. In this case, mean pooling.
+sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
+
+print("Sentence embeddings:")
+print(sentence_embeddings)
+```
+
+
+
+## Evaluation Results
+
+<!--- Describe how your model was evaluated -->
+
+For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir)
+
+
+## Training
+The model was trained with the parameters:
+
+**DataLoader**:
+
+`sentence_transformers.datasets.NoDuplicatesDataLoader.NoDuplicatesDataLoader` of length 936019 with parameters:
+```
+{'batch_size': 85}
+```
+
+**Loss**:
+
+`sentence_transformers.losses.GISTEmbedLoss.GISTEmbedLoss` with parameters:
+  ```
+  {'guide': SentenceTransformer(
+  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+), 'temperature': 0.01}
+  ```
+
+Parameters of the fit()-Method:
+```
+{
+    "epochs": 1,
+    "evaluation_steps": 9361,
+    "evaluator": "sentence_transformers.evaluation.InformationRetrievalEvaluator.InformationRetrievalEvaluator",
+    "max_grad_norm": 1,
+    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
+    "optimizer_params": {
+        "lr": 1e-05
+    },
+    "scheduler": "WarmupLinear",
+    "steps_per_epoch": 936019,
+    "warmup_steps": 93602,
+    "weight_decay": 0.01
+}
+```
+
+
+## Full Model Architecture
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Citing & Authors
+
+<!--- Describe where people can find more information -->

config.json

@@ -0,0 +1,32 @@
+{
+  "_name_or_path": "models/bertimbau-335m-europarl-eubookshop-ted2020-tatoeba-ct1-nli-gist10-sts-angle20-v3",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "directionality": "bidi",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "output_past": true,
+  "pad_token_id": 0,
+  "pooler_fc_size": 768,
+  "pooler_num_attention_heads": 12,
+  "pooler_num_fc_layers": 3,
+  "pooler_size_per_head": 128,
+  "pooler_type": "first_token_transform",
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.39.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 29794
+}

config_sentence_transformers.json

@@ -0,0 +1,9 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.6.1",
+    "transformers": "4.39.3",
+    "pytorch": "2.2.2+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null
+}

eval/Information-Retrieval_evaluation_mmarco-pt-dev-small_results.csv

@@ -0,0 +1,101 @@
+epoch,steps,cos_sim-Accuracy@1,cos_sim-Accuracy@3,cos_sim-Accuracy@5,cos_sim-Accuracy@10,cos_sim-Precision@10,cos_sim-Recall@10,cos_sim-MRR@10,cos_sim-NDCG@10,cos_sim-MAP@100
+0,9361,0.5730659025787965,0.6961318051575931,0.7481375358166189,0.801432664756447,0.0841977077363897,0.7945081184336198,0.6464870719061269,0.6792996311987693,0.6460020123746733
+0,18722,0.6239255014326648,0.7518624641833811,0.7955587392550143,0.8462750716332378,0.08909742120343841,0.8396848137535816,0.6973728225769764,0.7290034259355299,0.6962497233588836
+0,28083,0.6555873925501433,0.7719197707736389,0.8164756446991404,0.8706303724928367,0.09169054441260745,0.8650668576886341,0.7247120457543078,0.7558112640897837,0.7230198288652896
+0,37444,0.6687679083094555,0.7848137535816619,0.8302292263610315,0.880515759312321,0.09276504297994269,0.8746657115568289,0.7379656160458435,0.768451695687463,0.7363352489036437
+0,46805,0.6762177650429799,0.7941260744985673,0.8402578796561605,0.8868194842406877,0.09345272206303726,0.8809813753581662,0.7457921157956979,0.7756691626397711,0.7435814165214545
+0,56166,0.6858166189111747,0.8047277936962751,0.8436962750716333,0.8883954154727793,0.09368194842406877,0.8828438395415473,0.7536730226952282,0.782288923694148,0.7516994687680697
+0,65527,0.6979942693409742,0.8160458452722062,0.8538681948424068,0.8969914040114613,0.09469914040114613,0.8919890162368672,0.7650395119843537,0.7931775333515583,0.7626705155759607
+0,74888,0.6935530085959886,0.8078796561604584,0.8494269340974212,0.8946991404011462,0.09444126074498568,0.8895296084049665,0.7603755628325828,0.7892075067610356,0.7585232513967605
+0,84249,0.6995702005730658,0.8131805157593123,0.8548710601719197,0.8998567335243552,0.09485673352435531,0.8939469914040115,0.7663747327966516,0.7945960897874323,0.7639004958475369
+0,93610,0.6906876790830946,0.8073065902578797,0.8497134670487106,0.8936962750716332,0.09432664756446993,0.888658070678128,0.7590276072224482,0.7877977570246831,0.7569873857227565
+0,102971,0.7035816618911175,0.8157593123209169,0.8532951289398281,0.9001432664756447,0.09510028653295129,0.8954751671442216,0.7684889366443799,0.7965060694805582,0.766300031753561
+0,112332,0.7054441260744986,0.8199140401146132,0.859025787965616,0.9001432664756447,0.09498567335243553,0.8945916905444126,0.7712967890116883,0.7985958039695562,0.7692136528862108
+0,121693,0.7048710601719198,0.8164756446991404,0.8551575931232092,0.8975644699140402,0.09485673352435531,0.8927531041069724,0.7692943557556723,0.7967788394120286,0.7677141886462893
+0,131054,0.7095988538681949,0.8209169054441261,0.8593123209169055,0.9035816618911174,0.09554441260744985,0.8994269340974212,0.7741508050211472,0.8021291531314765,0.7722738144261733
+0,140415,0.7088825214899713,0.8229226361031519,0.8610315186246418,0.9020057306590258,0.09534383954154728,0.8970391595033428,0.773670634920633,0.8009179381418373,0.7711893557749459
+0,149776,0.7116045845272206,0.82621776504298,0.8654727793696275,0.904297994269341,0.09558739255014327,0.8994866284622731,0.7768016327830065,0.8036993319594585,0.7740330842079026
+0,159137,0.7015759312320917,0.8197707736389684,0.8620343839541548,0.9045845272206303,0.09558739255014327,0.8997731614135625,0.7700982398690122,0.7987584568256881,0.7672526624876573
+0,168498,0.7171919770773639,0.8257879656160458,0.864756446991404,0.9073065902578796,0.09600286532951291,0.9025787965616046,0.7804059216809927,0.8073240220490654,0.7777230609943501
+0,177859,0.7106017191977078,0.8226361031518624,0.8634670487106018,0.9024355300859599,0.09551575931232092,0.8985553963705826,0.7752533883658501,0.8027616259198136,0.773262395798072
+0,187220,0.7131805157593123,0.8265042979942694,0.8627507163323782,0.9050143266475644,0.09567335243553007,0.9005611270296083,0.7773581548187557,0.8047606454951683,0.7752390879565639
+0,196581,0.7111747851002865,0.8230659025787965,0.8623209169054441,0.9055873925501433,0.09565902578796562,0.9008476599808979,0.7760536908172992,0.8039721292146235,0.7742550951359587
+0,205942,0.710028653295129,0.8273638968481375,0.8636103151862464,0.9058739255014326,0.09584527220630372,0.901647564469914,0.7762891708736971,0.8043447486832209,0.7743369545232908
+0,215303,0.7108882521489971,0.82621776504298,0.8640401146131805,0.907163323782235,0.09580229226361033,0.9019579751671442,0.7770914631373068,0.8048264766737991,0.7747002136189551
+0,224664,0.7123209169054441,0.8244985673352435,0.8667621776504298,0.9088825214899714,0.09600286532951291,0.9040830945558739,0.7784283895028866,0.8063077141267656,0.776116171150663
+0,234025,0.7227793696275072,0.832378223495702,0.8683381088825215,0.9107449856733524,0.09626074498567334,0.9062917860553964,0.7854987606312813,0.8118999932781938,0.7826070810539785
+0,243386,0.7151862464183381,0.8329512893982808,0.8700573065902579,0.9077363896848137,0.09593123209169055,0.9034861509073544,0.7816965138490913,0.8090923050738706,0.7799911263860525
+0,252747,0.7174785100286533,0.8342406876790831,0.8760744985673352,0.9124641833810888,0.09640401146131805,0.9077363896848137,0.7847258607358869,0.8121390475929811,0.7823399645221173
+0,262108,0.7229226361031519,0.8342406876790831,0.8736389684813753,0.9087392550143266,0.09603151862464185,0.9045009551098376,0.786649099467865,0.8128883244999383,0.784595686859478
+0,271469,0.7250716332378223,0.8326647564469914,0.8727793696275071,0.9091690544412607,0.09613180515759313,0.905002387774594,0.7873863532996749,0.8134940843569215,0.7852053550756493
+0,280830,0.7253581661891118,0.833810888252149,0.8729226361031519,0.9091690544412607,0.0961461318051576,0.9048949379178605,0.7881976849956775,0.8140400886080027,0.7860671454947646
+0,290191,0.7230659025787965,0.8330945558739254,0.8722063037249284,0.9134670487106017,0.09653295128939827,0.9089541547277937,0.7872035179879009,0.8141012286427842,0.7845728716478546
+0,299552,0.7207736389684813,0.8340974212034384,0.870916905444126,0.9108882521489972,0.09620343839541547,0.9060530085959886,0.785721903852275,0.8120652498673795,0.7828507927306709
+0,308913,0.7189111747851002,0.8312320916905445,0.8673352435530086,0.9131805157593124,0.09643266475644698,0.908476599808978,0.7838021899304124,0.8113631574933068,0.7813142339335086
+0,318274,0.7196275071633238,0.8299426934097421,0.8686246418338109,0.9111747851002865,0.096189111747851,0.9062559694364852,0.7835070155091629,0.8105581050230269,0.7810128145924587
+0,327635,0.7194842406876791,0.8273638968481375,0.8666189111747851,0.9095988538681948,0.09611747851002866,0.9047755491881567,0.7825052872151721,0.8094075250967152,0.779916394870083
+0,336996,0.719054441260745,0.8302292263610315,0.869484240687679,0.9107449856733524,0.09621776504297995,0.9061485195797516,0.7838771774230222,0.8108410683646141,0.7813007220083033
+0,346357,0.7214899713467049,0.8328080229226361,0.8704871060171919,0.9124641833810888,0.09647564469914041,0.908082617000955,0.7858032587438013,0.8129065394858211,0.7833497954508688
+0,355718,0.7224928366762178,0.8336676217765043,0.8713467048710601,0.914756446991404,0.09673352435530086,0.9104703915950334,0.7868108427707272,0.8143095582160038,0.7843661423825303
+0,365079,0.7260744985673352,0.8320916905444126,0.8710601719197708,0.9117478510028654,0.09633237822349572,0.9070200573065903,0.7880374198389939,0.8141430411331358,0.7854038596096389
+0,374440,0.727650429799427,0.8365329512893983,0.8720630372492837,0.9121776504297995,0.09647564469914041,0.9078199617956064,0.7900210920089135,0.81587930519042,0.7871058712615463
+0,383801,0.7326647564469914,0.8411174785100286,0.8743553008595989,0.9148997134670487,0.09673352435530086,0.9102435530085959,0.7938697298403588,0.8194385436695182,0.7912209787102934
+0,393162,0.7282234957020057,0.8379656160458453,0.8746418338108882,0.9124641833810888,0.09643266475644699,0.9077722063037249,0.7906919429662953,0.8164482745465531,0.7880198556446836
+0,402523,0.7279369627507163,0.8391117478510028,0.8742120343839541,0.9140401146131805,0.0965759312320917,0.9092048710601719,0.791618854323007,0.8174780332080618,0.7889768359046359
+0,411884,0.7287965616045845,0.8379656160458453,0.8749283667621777,0.9121776504297995,0.09637535816618911,0.9074259789875835,0.7912563673989158,0.8167336334781011,0.7886856093402478
+0,421245,0.7297994269340974,0.8385386819484241,0.8736389684813753,0.9123209169054441,0.09640401146131806,0.9076886341929321,0.7919788397689521,0.8174461882946717,0.7896637870396483
+0,430606,0.7292263610315186,0.8353868194842407,0.8740687679083095,0.9093123209169054,0.09608882521489973,0.9046680993314232,0.7905529972256313,0.8155414288382734,0.7880399512595979
+0,439967,0.7280802292263611,0.8353868194842407,0.873352435530086,0.9113180515759313,0.09643266475644699,0.907163323782235,0.7901644153363339,0.8157851568844269,0.7874592358841824
+0,449328,0.7286532951289398,0.8385386819484241,0.8734957020057307,0.9120343839541547,0.09644699140401147,0.9073662846227315,0.7911688725155755,0.8168103308213963,0.7888434035770695
+0,458689,0.7305157593123209,0.8386819484240687,0.8767908309455588,0.9148997134670487,0.09679083094555874,0.9106136580706781,0.7928432141720094,0.8188140306103279,0.7902657908673196
+0,468050,0.7259312320916905,0.8365329512893983,0.8757879656160459,0.9138968481375358,0.09673352435530086,0.9096704871060172,0.7897753786328265,0.8163048826342679,0.7872297817170122
+0,477411,0.735243553008596,0.8412607449856734,0.8780802292263611,0.9146131805157593,0.09673352435530086,0.9101599808978031,0.7961953654432146,0.8213716014228162,0.7938935925328452
+0,486772,0.7365329512893983,0.8393982808022923,0.8779369627507163,0.9140401146131805,0.09671919770773639,0.909634670487106,0.7963315254468534,0.8211693431968687,0.7939013082932934
+0,496133,0.732378223495702,0.8412607449856734,0.8734957020057307,0.9143266475644699,0.09673352435530086,0.9100644699140401,0.7938522194933354,0.8195158908323766,0.7915802593553244
+0,505494,0.7315186246418338,0.8392550143266476,0.8777936962750716,0.9171919770773639,0.09704871060171921,0.9129417382999044,0.7938803611224806,0.8203261917826545,0.7915207369111185
+0,514855,0.7332378223495702,0.8376790830945559,0.8767908309455588,0.9138968481375358,0.09666189111747851,0.909515281757402,0.7940170327921039,0.8195598250062142,0.7917557820657057
+0,524216,0.7346704871060172,0.8383954154727794,0.8756446991404011,0.9148997134670487,0.09677650429799427,0.9104465138490926,0.795399383726746,0.8207226652724668,0.7929384689651026
+0,533577,0.7373925501432664,0.8438395415472779,0.8769340974212034,0.9128939828080229,0.09659025787965617,0.9087153772683858,0.7969286965024768,0.821572941352201,0.7946694140882096
+0,542938,0.7385386819484241,0.8445558739255015,0.8820916905444126,0.917621776504298,0.09703438395415472,0.9133476599808977,0.7994114704143336,0.8245701547617706,0.7968961427704394
+0,552299,0.7368194842406877,0.8401146131805157,0.873352435530086,0.9117478510028654,0.0964756446991404,0.907378223495702,0.7961702369582015,0.8206839428983765,0.7940863033480563
+0,561660,0.7378223495702005,0.8435530085959886,0.8759312320916905,0.9128939828080229,0.09650429799426935,0.908476599808978,0.7972768567790053,0.8218037386335515,0.7951035650039776
+0,571021,0.7329512893982808,0.8422636103151863,0.8773638968481375,0.9157593123209169,0.09676217765042981,0.911222540592168,0.7953871605948948,0.8208839805621448,0.7928034722717862
+0,580382,0.7376790830945559,0.8482808022922637,0.880515759312321,0.916189111747851,0.09691977077363897,0.9120702005730659,0.7997860099149484,0.8245255646878039,0.7971691194142224
+0,589743,0.7388252148997134,0.8418338108882522,0.879083094555874,0.9156160458452722,0.09687679083094557,0.9114613180515759,0.7982837585846162,0.8232954395988985,0.7960220102504558
+0,599104,0.7373925501432664,0.8419770773638968,0.876647564469914,0.9131805157593124,0.09654727793696276,0.9085721107927412,0.7967603015418183,0.8213454631915915,0.7943547561268592
+0,608465,0.7382521489971346,0.8425501432664756,0.8785100286532951,0.9150429799426935,0.09666189111747851,0.9104703915950334,0.798083924591803,0.8228016966978262,0.795676806157982
+0,617826,0.7428366762177651,0.8446991404011461,0.8800859598853868,0.9173352435530086,0.09707736389684814,0.913264087870105,0.8016082798926625,0.8262943077953603,0.799255778594168
+0,627187,0.7398280802292264,0.842836676217765,0.876647564469914,0.9173352435530086,0.09696275071633238,0.9130014326647564,0.7996307477145587,0.8245474034907907,0.7969817518878972
+0,636548,0.7429799426934097,0.8461318051575931,0.8796561604584527,0.9148997134670487,0.09681948424068768,0.910947946513849,0.8014528471369431,0.8255916268440928,0.7992189991575884
+0,645909,0.7405444126074499,0.845702005730659,0.8803724928366762,0.9160458452722063,0.09691977077363897,0.9123925501432665,0.8006615272661103,0.825355915299221,0.7982908505597917
+0,655270,0.7385386819484241,0.8468481375358167,0.8815186246418338,0.9163323782234957,0.09689111747851004,0.9124283667621776,0.7997496816300523,0.8247907613125782,0.7975682749345687
+0,664631,0.7392550143266475,0.8459885386819485,0.8782234957020058,0.9151862464183381,0.09670487106017192,0.91097182425979,0.7994235798426329,0.8239230127007541,0.7969482295730922
+0,673992,0.7435530085959885,0.8461318051575931,0.8793696275071633,0.919054441260745,0.09710601719197709,0.9146251193887297,0.8025065948060209,0.8270604571229533,0.7998552930623065
+0,683353,0.7406876790830945,0.8431232091690545,0.8773638968481375,0.916189111747851,0.09683381088825216,0.9117120343839542,0.7997328535043414,0.8245281753468592,0.7977108870856253
+0,692714,0.7425501432664756,0.8409742120343839,0.8757879656160459,0.9151862464183381,0.09681948424068768,0.9109957020057307,0.8001610042297724,0.8246617418612496,0.7980957914115562
+0,702075,0.7445558739255015,0.8439828080229226,0.8782234957020058,0.9177650429799427,0.09702005730659026,0.9135028653295129,0.8023401896575246,0.8268892166854257,0.8002316346650901
+0,711436,0.7393982808022923,0.8436962750716333,0.8796561604584527,0.9170487106017192,0.09694842406876791,0.9126910219675264,0.7995422863510234,0.8245704322140212,0.7973125359241324
+0,720797,0.7399713467048711,0.8465616045845272,0.8795128939828081,0.9156160458452722,0.09683381088825214,0.9115329512893983,0.7997835084368015,0.8244987744284553,0.7976360043525245
+0,730158,0.7391117478510029,0.8458452722063037,0.8816618911174785,0.9163323782234957,0.09693409742120344,0.9123089780324737,0.8002410515304493,0.8252117551106486,0.7983290210029025
+0,739519,0.7401146131805157,0.8474212034383954,0.882378223495702,0.9179083094555874,0.09716332378223497,0.9142311365807069,0.801173761768317,0.8264092180274621,0.7990904501102796
+0,748880,0.740974212034384,0.8475644699140401,0.8825214899713467,0.9200573065902579,0.09729226361031518,0.9157234957020057,0.8017270432528294,0.8268560279469828,0.799146484299888
+0,758241,0.7441260744985674,0.851432664756447,0.8822349570200573,0.9189111747851003,0.09723495702005731,0.9149474689589303,0.804182414608631,0.8286628398771211,0.8018599433063859
+0,767602,0.7434097421203438,0.8498567335243553,0.8846704871060171,0.9206303724928366,0.09732091690544412,0.9163443170964661,0.8039611474962463,0.828791398713712,0.8016158132981963
+0,776963,0.7474212034383955,0.8501432664756448,0.8833810888252149,0.9210601719197707,0.09752148997134671,0.9173949379178604,0.8056000136444252,0.8302359507334512,0.8030817374694343
+0,786324,0.7472779369627507,0.8517191977077364,0.8825214899713467,0.9234957020057306,0.09770773638968483,0.9195081184336198,0.8065450607176962,0.8314982440144579,0.8039941575320785
+0,795685,0.7472779369627507,0.8521489971346705,0.8835243553008596,0.9203438395415473,0.09737822349570201,0.9162965616045845,0.8062000272888511,0.8304104027962628,0.8037790391176213
+0,805046,0.7484240687679083,0.85,0.8830945558739255,0.9199140401146132,0.09747851002865329,0.9163801337153772,0.8069388729703905,0.8311176530583418,0.8045614799549443
+0,814407,0.7482808022922636,0.8487106017191977,0.8842406876790831,0.9222063037249284,0.09766475644699141,0.9182903533906398,0.8065610360667668,0.8312624397657431,0.8041784095199004
+0,823768,0.7468481375358166,0.8482808022922637,0.8829512893982808,0.9216332378223495,0.09755014326647565,0.917836676217765,0.8056714194751445,0.8305805417110826,0.8035784882956905
+0,833129,0.7520057306590258,0.8527220630372493,0.8865329512893982,0.9239255014326647,0.09780802292263611,0.9201528175740209,0.8098985195797505,0.8342190579038499,0.807414754989088
+0,842490,0.7469914040114614,0.8501432664756448,0.8861031518624641,0.9223495702005731,0.09770773638968482,0.9188753581661891,0.8068563810433425,0.8317886324893767,0.8046371274935551
+0,851851,0.7488538681948425,0.851432664756447,0.8835243553008596,0.9223495702005731,0.09766475644699141,0.9187798471824261,0.8074184177013681,0.8320643506702107,0.8050520173882728
+0,861212,0.75,0.8504297994269341,0.8851002865329513,0.921919770773639,0.09765042979942694,0.9182784145176697,0.8083752899440565,0.8327586775929605,0.8061562750219564
+0,870573,0.7494269340974212,0.851432664756447,0.8842406876790831,0.9224928366762177,0.09767908309455586,0.9187917860553962,0.807949981807431,0.8326036462751322,0.8058698383320236
+0,879934,0.7517191977077364,0.85,0.8858166189111748,0.9226361031518625,0.09762177650429801,0.918445558739255,0.8089740528494105,0.8331435290612825,0.8067247347937723
+0,889295,0.7525787965616045,0.8508595988538682,0.8848137535816619,0.9236389684813754,0.09776504297994269,0.919782712511939,0.8097342179469678,0.8339928371003048,0.8073454620455043
+0,898656,0.7521489971346705,0.8515759312320917,0.8858166189111748,0.9239255014326647,0.09783667621776505,0.9201766953199619,0.8097220516668934,0.8341285091559775,0.8073188588724193
+0,908017,0.7518624641833811,0.852865329512894,0.8842406876790831,0.9236389684813754,0.09785100286532951,0.9199737344794651,0.8094495611042881,0.833972346529672,0.807176808610807
+0,917378,0.7524355300859599,0.8525787965616046,0.8855300859598854,0.9243553008595988,0.09787965616045846,0.9205706781279848,0.8099035793878189,0.8343884041301907,0.8075094898340566
+0,926739,0.7508595988538682,0.852865329512894,0.8853868194842407,0.9244985673352436,0.09789398280802293,0.9207139446036295,0.8091998112521029,0.8338932068221836,0.8067875445671394
+0,-1,0.7508595988538682,0.852865329512894,0.8851002865329513,0.9244985673352436,0.09789398280802293,0.9207139446036295,0.8092095329058072,0.8339279500254407,0.806861440643805

eval/Information-Retrieval_evaluation_mmarco-pt-test2019-0-passages_results.csv

@@ -0,0 +1,2 @@
+epoch,steps,cos_sim-Accuracy@1,cos_sim-Accuracy@5,cos_sim-Accuracy@10,cos_sim-Precision@1,cos_sim-Recall@1,cos_sim-Precision@5,cos_sim-Recall@5,cos_sim-Precision@10,cos_sim-Recall@10,cos_sim-MRR@1,cos_sim-MRR@5,cos_sim-MRR@10,cos_sim-MRR@50,cos_sim-MRR@100,cos_sim-NDCG@10,cos_sim-MAP@100
+-1,-1,0.78,0.895,0.92,0.78,0.0027343407550304106,0.7190000000000001,0.011130662594110871,0.6809999999999999,0.01784198443319133,0.78,0.8242500000000001,0.8278115079365079,0.8307943605307737,0.830924746018302,0.7026030146458703,0.37132073114440317

eval/similarity_evaluation_assin-ptbr-test_results.csv

@@ -0,0 +1,2 @@
+epoch,steps,cosine_pearson,cosine_spearman,euclidean_pearson,euclidean_spearman,manhattan_pearson,manhattan_spearman,dot_pearson,dot_spearman
+-1,-1,0.7586022668309707,0.7548144804830863,0.7771219972530863,0.7584213743397085,0.7772875938474595,0.7582741957706486,0.7279757325250253,0.7206766264377411

eval/similarity_evaluation_assin-ptpt-test_results.csv

@@ -0,0 +1,2 @@
+epoch,steps,cosine_pearson,cosine_spearman,euclidean_pearson,euclidean_spearman,manhattan_pearson,manhattan_spearman,dot_pearson,dot_spearman
+-1,-1,0.7326055772425006,0.7395113006056051,0.7513215167673132,0.7398409260973287,0.7501345281308687,0.7388328760354175,0.7119101207197054,0.7136528623083449

eval/similarity_evaluation_assin2-test_results.csv

@@ -0,0 +1,2 @@
+epoch,steps,cosine_pearson,cosine_spearman,euclidean_pearson,euclidean_spearman,manhattan_pearson,manhattan_spearman,dot_pearson,dot_spearman
+-1,-1,0.8018548627864709,0.732930427298554,0.7836348205608707,0.7294645241286049,0.7840528232122587,0.7296980191019486,0.8011217169249465,0.7291028570664064

eval/similarity_evaluation_iris-sts-test_results.csv

@@ -0,0 +1,2 @@
+epoch,steps,cosine_pearson,cosine_spearman,euclidean_pearson,euclidean_spearman,manhattan_pearson,manhattan_spearman,dot_pearson,dot_spearman
+-1,-1,0.7807631325264405,0.7742420066919498,0.7678134296700626,0.7647332740486237,0.7664483911625676,0.762783577524515,0.7807392612996708,0.777478072852122

eval/similarity_evaluation_stsb-multi-mt-pt-test_results.csv

@@ -0,0 +1,2 @@
+epoch,steps,cosine_pearson,cosine_spearman,euclidean_pearson,euclidean_spearman,manhattan_pearson,manhattan_spearman,dot_pearson,dot_spearman
+-1,-1,0.8169657434446278,0.807754903167667,0.8089079154581047,0.808970246985139,0.80733029112189,0.8072647473087509,0.7962707762244833,0.7838486404187511

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e88524ffa87ddc42499780bff86ad5b0b22b626f86ec9355374cd058bc50c51
+size 1337630536

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 128,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,64 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": false,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 1000000000000000019884624838656,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

train-config.yaml

@@ -0,0 +1,28 @@
+trainer: "gist"
+model_name: "bertimbau-335m-mmarco-pairs-gist1-v1"
+base_model_name: "bertimbau-335m-europarl-eubookshop-ted2020-tatoeba-ct1-nli-gist10-sts-angle20-v3"
+guide_model_name: "bertimbau-100m-europarl-eubookshop-ted2020-tatoeba-ct1-nli-gist10-sts-cosent20-v1"
+validation_ir: True
+validation_ir_corpus_size: 50000
+# validation_ir_corpus_size: 500
+
+# see https://huggingface.co/docs/datasets/v2.18.0/en/about_dataset_load
+train_dataset_configs:
+  - alias: "mmarco"
+    path: "unicamp-dl/mmarco"
+    name: "portuguese"
+    split: "train"
+    # split: "train[1000:2000]"
+
+examples_are_triples: False
+examples_are_labelled: False
+seed: 1
+learning_rate: 1e-5
+warmup_ratio: 0.1
+weight_decay: 0.01
+# batch_size: 100  # 100 fits very tightly (40GB used), could crash on batches of longer texts 
+batch_size: 85  # 85 uses up to 37.5GB out of 40GB
+use_amp: True
+epochs: 1
+# validations_per_epoch: 1
+validations_per_epoch: 100