Hugging Face's logo

jmvcoelho
/
t5-base-marco-crop-pretrain-2048

Transformers
PyTorch
t5
text-generation-inference
Inference Endpoints
Model card Files Community
jmvcoelho commited on
Commit
8b479fa
·
verified ·
1 Parent(s): 0305331

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ Model pre-trained on MS-MARCO (contrastive).
2
+
3
+ Starting model: jmvcoelho/t5-base-marco-lm-pretrain-2048
4
+
5
+ Training script: https://github.com/cxcscmu/LongEmbeddingAnalysis/blob/main/scripts/contrastive_pretrain_dr.sh
6
+ T5ForConditionalGenerationRoPE class: https://github.com/cxcscmu/LongEmbeddingAnalysis/blob/main/OpenMatch/src/openmatch/modeling/rope_t5.py