- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 573, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: mixed_float16


### Training results


| Train Loss | Validation Loss | Epoch |
|:----------:|:---------------:|:-----:|
| 1.0930     | 0.9132          | 0     |


### Framework versions


- Transformers 4.15.0
- TensorFlow 2.7.0
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: arabert-gec-v2-2


model author: s3h


task: text2text-generation


tags: t5, text2text-generation, transformers, generated_from_keras_callback, model-index, autonlp_compatible


tags:


- conversational


# Harry Potter DialoGPT Model


model name: DialoGPT_medium_harrypotter


model author: Shike


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


yes


model name: Kellin


model author: K3LLiN


license: apache-2.0
tags:


- image-classification
- pytorch
datasets:
- imagenette


# ResNet-34 model


Pretrained on [ImageNette](https://github.com/fastai/imagenette). The ResNet architecture was introduced in [this paper](https://arxiv.org/pdf/1512.03385.pdf).


## Model description


The core idea of the author is to help the gradient propagation through numerous layers by adding a skip connection.


## Installation


### Prerequisites


Python 3.6 (or higher) and [pip](https://pip.pypa.io/en/stable/)/[conda](https://docs.conda.io/en/latest/miniconda.html) are required to install Holocron.


### Latest stable release


You can install the last stable release of the package using [pypi](https://pypi.org/project/pylocron/) as follows:


```shell
pip install pylocron
```


or using [conda](https://anaconda.org/frgfm/pylocron):


```shell
conda install -c frgfm pylocron
```


### Developer mode


Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source *(install [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) first)*:


```shell
git clone https://github.com/frgfm/Holocron.git
pip install -e Holocron/.
```


## Usage instructions


```python
from PIL import Image
from torchvision.transforms import Compose, ConvertImageDtype, Normalize, PILToTensor, Resize
from torchvision.transforms.functional import InterpolationMode
from holocron.models import model_from_hf_hub

model = model_from_hf_hub("frgfm/resnet34").eval()

img = Image.open(path_to_an_image).convert("RGB")

# Preprocessing
config = model.default_cfg
transform = Compose([
    Resize(config['input_shape'][1:], interpolation=InterpolationMode.BILINEAR),
    PILToTensor(),
    ConvertImageDtype(torch.float32),
    Normalize(config['mean'], config['std'])
])

input_tensor = transform(img).unsqueeze(0)

# Inference
with torch.inference_mode():
    output = model(input_tensor)
probs = output.squeeze(0).softmax(dim=0)
```


## Citation


Original paper


```bibtex
@article{DBLP:journals/corr/HeZRS15,
  author    = {Kaiming He and
               Xiangyu Zhang and
               Shaoqing Ren and
               Jian Sun},
  title     = {Deep Residual Learning for Image Recognition},
  journal   = {CoRR},
  volume    = {abs/1512.03385},
  year      = {2015},
  url       = {http://arxiv.org/abs/1512.03385},
  eprinttype = {arXiv},
  eprint    = {1512.03385},
  timestamp = {Wed, 17 Apr 2019 17:23:45 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/HeZRS15.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


Source of this implementation


```bibtex
@software{Fernandez_Holocron_2020,
author = {Fernandez, François-Guillaume},
month = {5},
title = {{Holocron}},
url = {https://github.com/frgfm/Holocron},
year = {2020}
}
```


model name: resnet34


model author: frgfm


task: image-classification


tags: pytorch, dataset:imagenette, arxiv:1512.03385, transformers, license:apache-2.0, image-classification


license: apache-2.0
tags:


- image-classification
- pytorch
- onnx
datasets:
- imagenette


# RepVGG-A0 model


Pretrained on [ImageNette](https://github.com/fastai/imagenette). The RepVGG architecture was introduced in [this paper](https://arxiv.org/pdf/2101.03697.pdf).


## Model description


The core idea of the author is to distinguish the training architecture (with shortcut connections), from the inference one (a pure highway network). By designing the residual block, the training architecture can be reparametrized into a simple sequence of convolutions and non-linear activations.


## Installation


### Prerequisites


### Latest stable release


```shell
pip install pylocron
```


### Developer mode


## Usage instructions


```python
from PIL import Image
from torchvision.transforms import Compose, ConvertImageDtype, Normalize, PILToTensor, Resize
from torchvision.transforms.functional import InterpolationMode
from holocron.models import model_from_hf_hub

model = model_from_hf_hub("frgfm/repvgg_a0").eval()

img = Image.open(path_to_an_image).convert("RGB")

# Preprocessing
config = model.default_cfg
transform = Compose([
    Resize(config['input_shape'][1:], interpolation=InterpolationMode.BILINEAR),
    PILToTensor(),
    ConvertImageDtype(torch.float32),
    Normalize(config['mean'], config['std'])
])

input_tensor = transform(img).unsqueeze(0)

# Inference
with torch.inference_mode():
    output = model(input_tensor)
probs = output.squeeze(0).softmax(dim=0)
```


## Citation


Original paper


```bibtex
@article{DBLP:journals/corr/abs-2101-03697,
  author    = {Xiaohan Ding and
               Xiangyu Zhang and
               Ningning Ma and
               Jungong Han and
               Guiguang Ding and
               Jian Sun},
  title     = {RepVGG: Making VGG-style ConvNets Great Again},
  journal   = {CoRR},
  volume    = {abs/2101.03697},
  year      = {2021},
  url       = {https://arxiv.org/abs/2101.03697},
  eprinttype = {arXiv},
  eprint    = {2101.03697},
  timestamp = {Tue, 09 Feb 2021 15:29:34 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2101-03697.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


Source of this implementation


model name: repvgg_a0


model author: frgfm


task: image-classification


tags: pytorch, onnx, dataset:imagenette, arxiv:2101.03697, transformers, license:apache-2.0, image-classification


license: apache-2.0
tags:


- image-classification
- pytorch
datasets:
- imagenette


# ResNet-18 model


Pretrained on [ImageNette](https://github.com/fastai/imagenette). The ResNet architecture was introduced in [this paper](https://arxiv.org/pdf/1512.03385.pdf).


## Model description


The core idea of the author is to help the gradient propagation through numerous layers by adding a skip connection.


## Installation


### Prerequisites


Python 3.6 (or higher) and [pip](https://pip.pypa.io/en/stable/)/[conda](https://docs.conda.io/en/latest/miniconda.html) are required to install Holocron.


### Latest stable release


You can install the last stable release of the package using [pypi](https://pypi.org/project/pylocron/) as follows:


```shell
pip install pylocron
```


or using [conda](https://anaconda.org/frgfm/pylocron):


```shell
conda install -c frgfm pylocron
```


### Developer mode


Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source *(install [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) first)*:


```shell
git clone https://github.com/frgfm/Holocron.git
pip install -e Holocron/.
```


## Usage instructions


```python
from PIL import Image
from torchvision.transforms import Compose, ConvertImageDtype, Normalize, PILToTensor, Resize
from torchvision.transforms.functional import InterpolationMode
from holocron.models import model_from_hf_hub

model = model_from_hf_hub("frgfm/resnet18").eval()

img = Image.open(path_to_an_image).convert("RGB")

# Preprocessing
config = model.default_cfg
transform = Compose([
    Resize(config['input_shape'][1:], interpolation=InterpolationMode.BILINEAR),
    PILToTensor(),
    ConvertImageDtype(torch.float32),
    Normalize(config['mean'], config['std'])
])

input_tensor = transform(img).unsqueeze(0)

# Inference
with torch.inference_mode():
    output = model(input_tensor)
probs = output.squeeze(0).softmax(dim=0)
```


## Citation


Original paper


```bibtex
@article{DBLP:journals/corr/HeZRS15,
  author    = {Kaiming He and
               Xiangyu Zhang and
               Shaoqing Ren and
               Jian Sun},
  title     = {Deep Residual Learning for Image Recognition},
  journal   = {CoRR},
  volume    = {abs/1512.03385},
  year      = {2015},
  url       = {http://arxiv.org/abs/1512.03385},
  eprinttype = {arXiv},
  eprint    = {1512.03385},
  timestamp = {Wed, 17 Apr 2019 17:23:45 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/HeZRS15.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


Source of this implementation


```bibtex
@software{Fernandez_Holocron_2020,
author = {Fernandez, François-Guillaume},
month = {5},
title = {{Holocron}},
url = {https://github.com/frgfm/Holocron},
year = {2020}
}
```


model name: resnet18


model author: frgfm


task: image-classification


tags: pytorch, dataset:imagenette, arxiv:1512.03385, transformers, license:apache-2.0, image-classification


license: apache-2.0
tags:


- image-classification
- pytorch
- onnx
datasets:
- imagenette


# ReXNet-1.0x model


Pretrained on [ImageNette](https://github.com/fastai/imagenette). The ReXNet architecture was introduced in [this paper](https://arxiv.org/pdf/2007.00992.pdf).


## Model description


The core idea of the author is to add a customized Squeeze-Excitation layer in the residual blocks that will prevent channel redundancy.


## Installation


### Prerequisites


### Latest stable release


```shell
pip install pylocron
```


### Developer mode


## Usage instructions


```python
from PIL import Image
from torchvision.transforms import Compose, ConvertImageDtype, Normalize, PILToTensor, Resize
from torchvision.transforms.functional import InterpolationMode
from holocron.models import model_from_hf_hub

model = model_from_hf_hub("frgfm/rexnet1_0x").eval()

img = Image.open(path_to_an_image).convert("RGB")

# Preprocessing
config = model.default_cfg
transform = Compose([
    Resize(config['input_shape'][1:], interpolation=InterpolationMode.BILINEAR),
    PILToTensor(),
    ConvertImageDtype(torch.float32),
    Normalize(config['mean'], config['std'])
])

input_tensor = transform(img).unsqueeze(0)

# Inference
with torch.inference_mode():
    output = model(input_tensor)
probs = output.squeeze(0).softmax(dim=0)
```


## Citation


Original paper


```bibtex
@article{DBLP:journals/corr/abs-2007-00992,
  author    = {Dongyoon Han and
               Sangdoo Yun and
               Byeongho Heo and
               Young Joon Yoo},
  title     = {ReXNet: Diminishing Representational Bottleneck on Convolutional Neural
               Network},
  journal   = {CoRR},
  volume    = {abs/2007.00992},
  year      = {2020},
  url       = {https://arxiv.org/abs/2007.00992},
  eprinttype = {arXiv},
  eprint    = {2007.00992},
  timestamp = {Mon, 06 Jul 2020 15:26:01 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2007-00992.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


Source of this implementation


model name: rexnet1_0x


model author: frgfm


task: image-classification


tags: pytorch, onnx, dataset:imagenette, arxiv:2007.00992, transformers, license:apache-2.0, image-classification


tags:


- conversational


# Harry Potter DialoGPT Model


model name: DialoGPT-small-harrypotter


model author: bookemdan


task: conversational


tags: pytorch, conversational


- exbert
license: apache-2.0


# ouBioBERT-Base, Uncased


Bidirectional Encoder Representations from Transformers for Biomedical Text Mining by Osaka University (ouBioBERT) is a language model based on the BERT-Base (Devlin, et al., 2019) architecture. We pre-trained ouBioBERT on PubMed abstracts from the PubMed baseline (ftp://ftp.ncbi.nlm.nih.gov/pubmed/baseline) via our method.


The details of the pre-training procedure can be found in Wada, et al. (2020).


## Evaluation


We evaluated the performance of ouBioBERT in terms of the biomedical language understanding evaluation (BLUE) benchmark (Peng, et al., 2019). The numbers are mean (standard deviation) on five different random seeds.


| Dataset         |  Task Type                   |  Score       |
|:----------------|:-----------------------------|-------------:|
| MedSTS          |  Sentence similarity         |  84.9 (0.6)  |
| BIOSSES         |  Sentence similarity         |  92.3 (0.8)  |
| BC5CDR-disease  |  Named-entity recognition    |  87.4 (0.1)  |
| BC5CDR-chemical |  Named-entity recognition    |  93.7 (0.2)  |
| ShARe/CLEFE     |  Named-entity recognition    |  80.1 (0.4)  |
| DDI             |  Relation extraction         |  81.1 (1.5)  |
| ChemProt        |  Relation extraction         |  75.0 (0.3)  |
| i2b2 2010       |  Relation extraction         |  74.0 (0.8)  |
| HoC             |  Document classification     |  86.4 (0.5)  |
| MedNLI          |  Inference                   |  83.6 (0.7)  |
| **Total**       |  Macro average of the scores |**83.8 (0.3)**|


## Code for Fine-tuning


We made the source code for fine-tuning freely available at [our repository](https://github.com/sy-wada/blue_benchmark_with_transformers).


## Citation


If you use our work in your research, please kindly cite the following paper:


```bibtex
@misc{2005.07202,
Author = {Shoya Wada and Toshihiro Takeda and Shiro Manabe and Shozo Konishi and Jun Kamohara and Yasushi Matsumura},
Title = {A pre-training technique to localize medical BERT and enhance BioBERT},
Year = {2020},
Eprint = {arXiv:2005.07202},
}
```


model name: oubiobert-base-uncased


model author: seiya


tags: pytorch, jax, bert, pretraining, arxiv:2005.07202, transformers, exbert, license:apache-2.0, infinity_compatible


language: ca
datasets:


- common_voice
- parlament_parla
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Catalan XLSR Wav2Vec2 Large
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
datasets:
    - name: Common Voice ca
type: common_voice
args: ca
    - name: ParlamentParla
url: https://www.openslr.org/59/
metrics:
    - name: Test WER
type: wer
value: 6.92
    - name: Google Crowsourced Corpus WER
type: wer
value: 12.99
    - name: Audiobook “La llegenda de Sant Jordi” WER
type: wer
value: 13.23


# Wav2Vec2-Large-XLSR-Català


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Catalan language using the [Common Voice](https://huggingface.co/datasets/common_voice) and [ParlamentParla](https://www.openslr.org/59/) datasets.


**Attention:** The split train/dev/test used does not fully map with the CommonVoice 6.1 dataset. A custom split was used combining both the CommonVoice and ParlamentParla dataset and can be found [here](https://github.com/ccoreilly/wav2vec2-catala). Evaluating on the CV test dataset will produce a biased WER as 1144 audio files of that dataset were used in training/evaluation of this model.
WER was calculated using this [test.csv](https://github.com/ccoreilly/wav2vec2-catala/blob/master/test.csv) which was not seen by the model during training/evaluation.


You can find training and evaluation scripts in the github repository [ccoreilly/wav2vec2-catala](https://github.com/ccoreilly/wav2vec2-catala)


When using this model, make sure that your speech input is sampled at 16kHz.


## Results


Word error rate was evaluated on the following datasets unseen by the model:


| Dataset | WER |
| ------- | --- |
| [Test split CV+ParlamentParla]((https://github.com/ccoreilly/wav2vec2-catala/blob/master/test.csv)) | 6.92% |
| [Google Crowsourced Corpus](https://www.openslr.org/69/) | 12.99% |
| Audiobook “La llegenda de Sant Jordi” | 13.23% |


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ca", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("ccoreilly/wav2vec2-large-xlsr-catala")
model = Wav2Vec2ForCTC.from_pretrained("ccoreilly/wav2vec2-large-xlsr-catala")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


model name: wav2vec2-large-xlsr-catala


model author: softcatala


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, ca, dataset:common_voice, dataset:parlament_parla, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: ca
datasets:


- common_voice
- parlament_parla
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- speech-to-text
license: apache-2.0
model-index:
- name: Catalan VoxPopuli Wav2Vec2 Large
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
datasets:
    - name: Common Voice ca
type: common_voice
args: ca
    - name: ParlamentParla
url: https://www.openslr.org/59/
metrics:
    - name: Test WER
type: wer
value: 5.98
    - name: Google Crowsourced Corpus WER
type: wer
value: 12.14
    - name: Audiobook “La llegenda de Sant Jordi” WER
type: wer
value: 12.02


# Wav2Vec2-Large-100k-VoxPopuli-Català


Fine-tuned [facebook/wav2vec2-large-100k-voxpopuli](https://huggingface.co/facebook/wav2vec2-large-100k-voxpopuli) on Catalan language using the [Common Voice](https://huggingface.co/datasets/common_voice) and [ParlamentParla](https://www.openslr.org/59/) datasets.


**Attention:** The split train/dev/test used does not fully map with the CommonVoice 6.1 dataset. A custom split was used combining both the CommonVoice and ParlamentParla dataset and can be found [here](https://github.com/ccoreilly/wav2vec2-catala). Evaluating on the CV test dataset will produce a biased WER as 1144 audio files of that dataset were used in training/evaluation of this model.
WER was calculated using this [test.csv](https://github.com/ccoreilly/wav2vec2-catala/blob/master/test-filtered.csv) which was not seen by the model during training/evaluation.


| Dataset | WER |
| ------- | --- |
| [Test split CV+ParlamentParla]((https://github.com/ccoreilly/wav2vec2-catala/blob/master/test-filtered.csv)) | 5.98% |
| [Google Crowsourced Corpus](https://www.openslr.org/69/) | 12.14% |
| Audiobook “La llegenda de Sant Jordi” | 12.02% |


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ca", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("ccoreilly/wav2vec2-large-100k-voxpopuli-catala")
model = Wav2Vec2ForCTC.from_pretrained("ccoreilly/wav2vec2-large-100k-voxpopuli-catala")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


model name: wav2vec2-large-100k-voxpopuli-catala


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, ca, dataset:common_voice, dataset:parlament_parla, transformers, audio, speech, speech-to-text, license:apache-2.0, model-index


## language: ca


## Introduction


Download the model here:


* Catalan Roberta model: [julibert-2020-11-10.zip](https://www.softcatala.org/pub/softcatala/julibert/julibert-2020-11-10.zip)


## What's this?


Source code: https://github.com/Softcatala/julibert


* Corpus: Oscar Catalan Corpus (3,8G)
* Model type: Roberta
* Vocabulary size: 50265
* Steps: 500000


model name: julibert


model author: softcatala


task: fill-mask


tags: pytorch, jax, roberta, fill-mask, ca, transformers, autonlp_compatible, infinity_compatible


tags:


- conversational


#house small GPT


model name: house


model author: EzioDD


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language:


- en


widget:


- text: "27 patients have been diagnosed with PKU in Iceland since 1947. Incidence 1972-2008 is 1/8400 living births."
example_title: "Named Entity Recognition Ex. 1"
- text: "A retrospective epidemiological study of MPSs in Estonia was undertaken, and live-birth prevalence of MPS patients born between 1985 and 2006 was estimated. The live-birth prevalence for all MPS subtypes was found to be 4.05 per 100,000 live births, which is consistent with most other European studies. MPS II had the highest calculated incidence, with 2.16 per 100,000 live births (4.2 per 100,000 male live births)"
example_title: "Named Entity Recognition Ex. 2"
- text: "A retrospective study conducted between January 2015 and December 2020 revealed a total of 304,086 newborns have been screened in Kuwait. Six newborns were diagnosed with classic homocystinuria with an incidence of 1:50,000, which is not as high as in Qatar but higher than the global incidence."
example_title: "Named Entity Recognition Ex. 3"


- token-classification
- ncats


model-index:


- name: EpiExtract4GARD-v2
results:
  - task:
name: NER
type: token-classification
metrics:
    - name: Token-Level Precision
type: precision
value:
    - name: Token-Level Recall
type: recall
value:
    - name: Token-Level F1 Score
type: f_score
value:
    - name: Token-Level Precision
type: precision
value:
    - name: Token-Level Recall
type: recall
value:
    - name: Token-Level F1 Score
type: f_score
value:
datasets:
- ncats/EpiSet4NER


license: other


## DOCUMENTATION UPDATES IN PROGRESS


## Model description


**EpiExtract4GARD** is a fine-tuned [BioBERT-base-cased](https://huggingface.co/dmis-lab/biobert-base-cased-v1.1) model that is ready to use for **Named Entity Recognition** of locations (LOC), epidemiologic types (EPI), and epidemiologic rates (STAT). This model was fine-tuned on [EpiSet4NER](https://huggingface.co/datasets/ncats/EpiSet4NER) for epidemiological information from rare disease abstracts. See dataset documentation for details on the weakly supervised teaching methods and dataset biases and limitations. See [EpiExtract4GARD on GitHub](https://github.com/ncats/epi4GARD/tree/master/EpiExtract4GARD#epiextract4gard) for details on the entire pipeline.


#### How to use


You can use this model with the Hosted inference API to the right with this [test sentence](https://pubmed.ncbi.nlm.nih.gov/21659675/): "27 patients have been diagnosed with PKU in Iceland since 1947. Incidence 1972-2008 is 1/8400 living births."


See code below for use with Transformers *pipeline* for NER.:


```
from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer
model = AutoModelForTokenClassification.from_pretrained("ncats/EpiExtract4GARD")
tokenizer = AutoTokenizer.from_pretrained("ncats/EpiExtract4GARD")
NER_pipeline = pipeline('ner', model=model, tokenizer=tokenizer,aggregation_strategy='simple')

sample = "The live-birth prevalence of mucopolysaccharidoses in Estonia. Previous studies on the prevalence of mucopolysaccharidoses (MPS) in different populations have shown considerable variations. There are, however, few data with regard to the prevalence of MPSs in Fenno-Ugric populations or in north-eastern Europe, except for a report about Scandinavian countries. A retrospective epidemiological study of MPSs in Estonia was undertaken, and live-birth prevalence of MPS patients born between 1985 and 2006 was estimated. The live-birth prevalence for all MPS subtypes was found to be 4.05 per 100,000 live births, which is consistent with most other European studies. MPS II had the highest calculated incidence, with 2.16 per 100,000 live births (4.2 per 100,000 male live births), forming 53% of all diagnosed MPS cases, and was twice as high as in other studied European populations. The second most common subtype was MPS IIIA, with a live-birth prevalence of 1.62 in 100,000 live births. With 0.27 out of 100,000 live births, MPS VI had the third-highest live-birth prevalence. No cases of MPS I were diagnosed in Estonia, making the prevalence of MPS I in Estonia much lower than in other European populations. MPSs are the third most frequent inborn error of metabolism in Estonia after phenylketonuria and galactosemia."
sample2 = "Early Diagnosis of Classic Homocystinuria in Kuwait through Newborn Screening: A 6-Year Experience. Kuwait is a small Arabian Gulf country with a high rate of consanguinity and where a national newborn screening program was expanded in October 2014 to include a wide range of endocrine and metabolic disorders. A retrospective study conducted between January 2015 and December 2020 revealed a total of 304,086 newborns have been screened in Kuwait. Six newborns were diagnosed with classic homocystinuria with an incidence of 1:50,000, which is not as high as in Qatar but higher than the global incidence. Molecular testing for five of them has revealed three previously reported pathogenic variants in the <i>CBS</i> gene, c.969G>A, p.(Trp323Ter); c.982G>A, p.(Asp328Asn); and the Qatari founder variant c.1006C>T, p.(Arg336Cys). This is the first study to review the screening of newborns in Kuwait for classic homocystinuria, starting with the detection of elevated blood methionine and providing a follow-up strategy for positive results, including plasma total homocysteine and amino acid analyses. Further, we have demonstrated an increase in the specificity of the current newborn screening test for classic homocystinuria by including the methionine to phenylalanine ratio along with the elevated methionine blood levels in first-tier testing. Here, we provide evidence that the newborn screening in Kuwait has led to the early detection of classic homocystinuria cases and enabled the affected individuals to lead active and productive lives."
#Sample 1 is from: Krabbi K, Joost K, Zordania R, Talvik I, Rein R, Huijmans JG, Verheijen FV, Õunap K. The live-birth prevalence of mucopolysaccharidoses in Estonia. Genet Test Mol Biomarkers. 2012 Aug;16(8):846-9. doi: 10.1089/gtmb.2011.0307. Epub 2012 Apr 5. PMID: 22480138; PMCID: PMC3422553.
#Sample 2 is from: Alsharhan H, Ahmed AA, Ali NM, Alahmad A, Albash B, Elshafie RM, Alkanderi S, Elkazzaz UM, Cyril PX, Abdelrahman RM, Elmonairy AA, Ibrahim SM, Elfeky YME, Sadik DI, Al-Enezi SD, Salloum AM, Girish Y, Al-Ali M, Ramadan DG, Alsafi R, Al-Rushood M, Bastaki L. Early Diagnosis of Classic Homocystinuria in Kuwait through Newborn Screening: A 6-Year Experience. Int J Neonatal Screen. 2021 Aug 17;7(3):56. doi: 10.3390/ijns7030056. PMID: 34449519; PMCID: PMC8395821.

NER_pipeline(sample)
NER_pipeline(sample2)
```


Or if you download [*classify_abs.py*](https://github.com/ncats/epi4GARD/blob/master/EpiExtract4GARD/classify_abs.py), [*extract_abs.py*](https://github.com/ncats/epi4GARD/blob/master/EpiExtract4GARD/extract_abs.py), and [*gard-id-name-synonyms.json*](https://github.com/ncats/epi4GARD/blob/master/EpiExtract4GARD/gard-id-name-synonyms.json) from GitHub then you can test with this [*additional* code](https://github.com/ncats/epi4GARD/blob/master/EpiExtract4GARD/Case%20Study.ipynb):


```
import pandas as pd
import extract_abs
import classify_abs
pd.set_option('display.max_colwidth', None)

NER_pipeline = extract_abs.init_NER_pipeline()
GARD_dict, max_length = extract_abs.load_GARD_diseases()
nlp, nlpSci, nlpSci2, classify_model, classify_tokenizer = classify_abs.init_classify_model()

def search(term,num_results = 50):
    return extract_abs.search_term_extraction(term, num_results, NER_pipeline, GARD_dict, max_length,nlp, nlpSci, nlpSci2, classify_model, classify_tokenizer)

a = search(7058)
a

b = search('Santos Mateus Leal syndrome')
b

c = search('Fellman syndrome')
c

d = search('GARD:0009941')
d

e = search('Homocystinuria')
e
```


#### Limitations and bias


## Training data


It was trained on [EpiSet4NER](https://huggingface.co/datasets/ncats/EpiSet4NER). See dataset documentation for details on the weakly supervised teaching methods and dataset biases and limitations. The training dataset distinguishes between the beginning and continuation of an entity so that if there are back-to-back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes:


Abbreviation|Description
---------|--------------
O        |Outside of a named entity
B-LOC    | Beginning of a location
I-LOC    | Inside of a location
B-EPI    | Beginning of an epidemiologic type (e.g. "incidence", "prevalence", "occurrence")
I-EPI    | Epidemiologic type that is not the beginning token.
B-STAT   | Beginning of an epidemiologic rate
I-STAT   | Inside of an epidemiologic rate


### EpiSet Statistics


Beyond any limitations due to the EpiSet4NER dataset, this model is limited in numeracy due to BERT-based model's use of subword embeddings, which is crucial for epidemiologic rate identification and limits the entity-level results. Additionally, more recent weakly supervised learning techniques could be used to improve the performance of the model without improving the underlying dataset.


## Training procedure


This model was trained on a [AWS EC2 p3.2xlarge](https://aws.amazon.com/ec2/instance-types/), which utilized a single Tesla V100 GPU, with these hyperparameters:
4 epochs of training (AdamW weight decay = 0.05) with a batch size of 16. Maximum sequence length = 192. Model was fed one sentence at a time. Full config [here](https://wandb.ai/wzkariampuzha/huggingface/runs/353prhts/files/config.yaml).


## Hold-out validation  results


metric| entity-level result
-|-
f1 | 83.8
precision | 83.2
recall | 84.5


## Test results


| Dataset for Model Training | Evaluation Level |       Entity       | Precision | Recall |   F1  |
|:--------------------------:|:----------------:|:------------------:|:---------:|:------:|:-----:|
|           EpiSet           |   Entity-Level   |      Overall       |   0.556   |  0.662 | 0.605 |
|                            |                  |      Location      |   0.661   |  0.696 | 0.678 |
|                            |                  | Epidemiologic Type |   0.854   |  0.911 | 0.882 |
|                            |                  | Epidemiologic Rate |   0.143   |  0.218 | 0.173 |
|                            |    Token-Level   |      Overall       |   0.811   |  0.713 | 0.759 |
|                            |                  |      Location      |   0.949   |  0.742 | 0.833 |
|                            |                  | Epidemiologic Type |    0.9    |  0.917 | 0.908 |
|                            |                  | Epidemiologic Rate |   0.724   |  0.636 | 0.677 |


Thanks to [@William Kariampuzha](https://github.com/wzkariampuzha) at Axle Informatics/NCATS for contributing this model.


model name: EpiExtract4GARD-v2


model author: ncats


task: token-classification


tags: pytorch, bert, token-classification, en, dataset:ncats/EpiSet4NER, transformers, ncats, license:other, model-index, autonlp_compatible, infinity_compatible


## license: other


model name: EpiClassify4GARD


task: text-classification


tags: pytorch, bert, text-classification, transformers, license:other, infinity_compatible


## Model description


#### How to use


#### Limitations and bias


## Training data


### EpiSet Statistics


## Training procedure


## Hold-out validation  results


## Test results


model name: EpiExtract4GARD-v1


tags: pytorch, bert, token-classification, transformers, autonlp_compatible, infinity_compatible


tags:


- conversational


# Test Bot DialoGTP Model


model name: DialoGPT-small-TestBot


model author: balta


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


crasd


model name: ew


model author: oelkrise


language:


- en
tags:
- image-to-text
license: mit
datasets:
- coco2017


# Vit2-DistilGPT2


This model takes in an image and outputs a caption. It was trained using the Coco dataset and the full training script can be found in [this kaggle kernel](https://www.kaggle.com/sachin/visionencoderdecoder-model-training)


## Usage


```python
import Image
from transformers import AutoModel, GPT2Tokenizer, ViTFeatureExtractor
model = AutoModel.from_pretrained("sachin/vit2distilgpt2")
vit_feature_extractor = ViTFeatureExtractor.from_pretrained("google/vit-base-patch16-224-in21k")
# make sure GPT2 appends EOS in begin and end
def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None):
    outputs = [self.bos_token_id] + token_ids_0 + [self.eos_token_id]
    return outputs

GPT2Tokenizer.build_inputs_with_special_tokens = build_inputs_with_special_tokens
gpt2_tokenizer = GPT2Tokenizer.from_pretrained("distilgpt2")
# set pad_token_id to unk_token_id -> be careful here as unk_token_id == eos_token_id == bos_token_id
gpt2_tokenizer.pad_token = gpt2_tokenizer.unk_token
image = (Image.open(image_path).convert("RGB"), return_tensors="pt").pixel_values
encoder_outputs = model.generate(image.unsqueeze(0))
generated_sentences = gpt2_tokenizer.batch_decode(encoder_outputs, skip_special_tokens=True)
```


Note that the output sentence may be repeated, hence a post processing step may be required.


## Bias Warning


This model may be biased due to dataset, lack of long training and the model itself. The following gender bias is an example.
![](https://i.imgur.com/9zVN022.png)


## Results


<iframe src="https://wandb.ai/sachinruk/Vit2GPT2/reports/Shared-panel-22-01-27-23-01-56--VmlldzoxNDkyMTM3?highlightShare" style="border:none;height:1024px;width:100%">


model name: vit2distilgpt2


model author: sachin


task: image-to-text


tags: pytorch, vision-encoder-decoder, en, dataset:coco2017, transformers, image-to-text, license:mit


tags:


- conversational


# My Awesome Model


model name: Discord_BOT_RICK


model author: Daymarebait


task: conversational


tags: conversational


# Twety DialoGPT Model


model name: DialoGPT-small-twety


model author: Whitez


A VQGAN-compatible model trained on screenshots of cityscapes from 90s anime. To use, direct vqgan to the model as you would vqgan_imagenet_f16_1024, faceshq, etc.


model name: AestheticCities


model author: Xibanya


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xls-r-300m-turkish-colab
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-large-xls-r-300m-turkish-colab


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 256
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 512
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu113
- Datasets 1.13.3
- Tokenizers 0.10.3


model name: wav2vec2-large-xls-r-300m-turkish-colab


model author: FarisHijazi


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, model-index


license: mit
tags:


- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: sentence-compression-roberta
results: []


# sentence-compression-roberta


This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3465
- Accuracy: 0.8473
- F1: 0.6835
- Precision: 0.6835
- Recall: 0.6835


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     | Precision | Recall |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
| 0.5312        | 1.0   | 50   | 0.5251          | 0.7591   | 0.0040 | 0.75      | 0.0020 |
| 0.4           | 2.0   | 100  | 0.4003          | 0.8200   | 0.5341 | 0.7113    | 0.4275 |
| 0.3355        | 3.0   | 150  | 0.3465          | 0.8473   | 0.6835 | 0.6835    | 0.6835 |


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0+cu113
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: sentence-compression-roberta


model author: AlexMaclean


task: token-classification


tags: pytorch, roberta, token-classification, transformers, license:mit, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: sentence-compression
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# sentence-compression


This model is a fine-tuned version of [distilbert-base-cased](https://huggingface.co/distilbert-base-cased) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2973
- Accuracy: 0.8912
- F1: 0.8367
- Precision: 0.8495
- Recall: 0.8243


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     | Precision | Recall |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:---------:|:------:|
| 0.2686        | 1.0   | 10000 | 0.2667          | 0.8894   | 0.8283 | 0.8725    | 0.7884 |
| 0.2205        | 2.0   | 20000 | 0.2704          | 0.8925   | 0.8372 | 0.8579    | 0.8175 |
| 0.1476        | 3.0   | 30000 | 0.2973          | 0.8912   | 0.8367 | 0.8495    | 0.8243 |


### Framework versions


model name: sentence-compression


model author: AlexMaclean


task: token-classification


tags: pytorch, distilbert, token-classification, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


language:


- ja
tags:
- japanese
- text-generation
- gptj
- pytorch
- transformers
- t5tokenizer
- sentencepiece
license: apache-2.0


This pre-trained model is work in progress! Model weight download will be available in the future.


A 6.8 billion parameter pre-trained model for Japanese language, based on EleutherAI's Mesh Transformer JAX, that has a similar model structure to their GPT-J-6B pre-trained model.


EleutherAIによるMesh Transformer JAXをコードベースとした、GPT-J-6Bに似たストラクチャと約68.7億パラメータを持つ日本語pre-trainedモデルです。


- We used T5Tokenizer and SentencePiece instead of GPT-2/3 tokenizer. Normalization done by SentencePiece is must for Japanese tokenizing as there are so much many more variations for common symbols than Western languages.
- Tokenizer has a vocabulary of 52,500 tokens and trained on Japanese Wikipedia dump as of 01 Aug 2021.
- The model fits within 16GB VRAM GPUs like P100 for inference up to 1688 context length. Full 2048 context length output requires 20GB VRAM or more (e.g. GTX3090/A5000).
- The model was trained with TPUv3-128 generously provided by Google TRC for about 4 weeks. We are currently formatting additional datasets and preparing for more training time.


## Specifications


| Hyperparameter    | Value  |
|-------------------|--------|
| n_parameters      | 6,876,450,080 |
| n_layers          | 32 |
| d_model           | 4,096  |
| d_ff              | 16,384 |
| n_heads           | 16     |
| d_head            | 256    |
| n_ctx             | 2,048  |
| n_vocab           | 52,512 |
| position encoding | [Rotary position encodings (RoPE)](https://arxiv.org/abs/2104.09864) |
| RoPE dimensions   | 64 |


## Instructions


We recommend to use finetuneanon's forked transformer codebase for inferencing as split checkpoint loads up a lot faster than monolithic checkpoint supported by HuggingFace Transformers repository.


The tokenizer still uses 50256 as the <|endoftext|> substitute. Therefore 50256 should be excluded when inferencing.


## Datasets


Lack of quality Japanese corpus was one of the major challenges when we trained the model. We aimed to compile well-formatted corpuses outside of Common Crawl.


The dataset is normalized and sanitized against leading and trailing spaces, excessive CR/LF repetitions.


The whole dataset is about 400GB (as of October 2021) and 106B tokens (compared to 825GB/300B tokens for The Pile).


** Common Crawl


- Jan-Dec 2018 72GB CC100-Japanese (https://metatext.io/datasets/cc100-japanese)
- November 2018 106GB OSCAR-Japanese (https://oscar-corpus.com)
- 75GB Converted 860GB Google C4 Multilingual Japanese (re-formatted)


** Books


- 140GB Web Fictions, non-fictions and blogs corpus
- 5GB Books and Aozora Bunko corpus (weighted 2x)


** News


- 1GB Scientific news, medical news and web news corpus


** Wikipedia


- Aug 2021 3GB Assorted and Deduplicated Japanese Wikipedia (weighted 2x)
- Aug 2021 Wikibooks, Wikinews, Wikiquote, Wikisource, Wiktionary, Wikiversity and Wikivoyage


** Other Corpuses


- 2018 OpenSubtitles (https://opus.nlpl.eu/OpenSubtitles-v2018.php)
- 80-90's BBS Logs
- Assorted Blogs Crawl
- QED-ja
- TED 2020-ja


model name: gpt-j-japanese-6.8b


model author: naclbit


task: text-generation


tags: gptj, text-generation, ja, arxiv:2104.09864, transformers, japanese, pytorch, t5tokenizer, sentencepiece, license:apache-2.0


Save thhis here


model name: Testmodel


model author: mmmarchio


tags:


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- rouge
model-index:
- name: mt5-small-finetuned-pnsum2
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# mt5-small-finetuned-pnsum2


This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: nan
- Rouge1: 4.3733
- Rouge2: 1.0221
- Rougel: 4.1265
- Rougelsum: 4.1372
- Gen Len: 6.2843


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| 0.0           | 1.0   | 2500 | nan             | 4.3733 | 1.0221 | 4.1265 | 4.1372    | 6.2843  |


### Framework versions


- Transformers 4.16.1
- Pytorch 1.10.0+cu111
- Datasets 1.18.2
- Tokenizers 0.11.0


model name: mt5-small-finetuned-pnsum2


model author: MM98


task: text2text-generation


tags: pytorch, tensorboard, mt5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- squad
model-index:
- name: distilbert-base-uncased-finetuned-squad
results: []


# distilbert-base-uncased-finetuned-squad


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the squad dataset.
It achieves the following results on the evaluation set:


- Loss: 1.1453


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 1.2065        | 1.0   | 5577  | 1.1289          |
| 0.9226        | 2.0   | 11154 | 1.1019          |
| 0.7411        | 3.0   | 16731 | 1.1453          |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.9.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: distilbert-base-uncased-finetuned-squad


model author: en


task: question-answering


tags: pytorch, tensorboard, distilbert, question-answering, dataset:squad, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


- You'll need to instantiate a special RoBERTa class. Though technically a "Longformer", the elongated RoBERTa model will still need to be pulled in as such.
- To do so, use the following classes:


```python
class RobertaLongSelfAttention(LongformerSelfAttention):
    def forward(
        self,
        hidden_states,
        attention_mask=None,
        head_mask=None,
        encoder_hidden_states=None,
        encoder_attention_mask=None,
        output_attentions=False,
    ):
        return super().forward(hidden_states, attention_mask=attention_mask, output_attentions=output_attentions)

class RobertaLongForMaskedLM(RobertaForMaskedLM):
    def __init__(self, config):
        super().__init__(config)
        for i, layer in enumerate(self.roberta.encoder.layer):
            # replace the `modeling_bert.BertSelfAttention` object with `LongformerSelfAttention`
            layer.attention.self = RobertaLongSelfAttention(config, layer_id=i)
```


- Then, pull the model as `RobertaLongForMaskedLM.from_pretrained('simonlevine/bioclinical-roberta-long')`
- Now, it can be used as usual. Note you may get untrained weights warnings.
- Note that you can replace `RobertaForMaskedLM` with a different task-specific RoBERTa from Huggingface, such as RobertaForSequenceClassification.


model name: bioclinical-roberta-long


model author: simonlevine


task: fill-mask


tags: pytorch, jax, roberta, fill-mask, transformers, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: electra-base-discriminator-finetuned-cola
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.6824089073723449


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# electra-base-discriminator-finetuned-cola


This model is a fine-tuned version of [google/electra-base-discriminator](https://huggingface.co/google/electra-base-discriminator) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6367
- Matthews Correlation: 0.6824


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.4139        | 1.0   | 535  | 0.4137          | 0.6381               |
| 0.2452        | 2.0   | 1070 | 0.4887          | 0.6504               |
| 0.17          | 3.0   | 1605 | 0.5335          | 0.6757               |
| 0.1135        | 4.0   | 2140 | 0.6367          | 0.6824               |
| 0.0817        | 5.0   | 2675 | 0.6742          | 0.6755               |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: electra-base-discriminator-finetuned-cola


model author: D3xter1922


task: text-classification


tags: pytorch, tensorboard, electra, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-distilbert-hinglish-small


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-distilbert-hinglish-small')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-distilbert-hinglish-small')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-distilbert-hinglish-small')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-distilbert-hinglish-small)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:


```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: DistilBertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-distilbert-hinglish-small


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, distilbert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-roberta-base


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-roberta-base')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-roberta-base')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-roberta-base')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-roberta-base)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-roberta-base


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, roberta, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-indic-bert


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-indic-bert')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-indic-bert')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-indic-bert')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-indic-bert)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: AlbertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-indic-bert


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, albert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-distilbert-hinglish-big


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-distilbert-hinglish-big')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-distilbert-hinglish-big')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-distilbert-hinglish-big')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-distilbert-hinglish-big)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:


```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: DistilBertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-distilbert-hinglish-big


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, distilbert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-bert-base-cased


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-bert-base-cased')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-bert-base-cased')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-bert-base-cased')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-bert-base-cased)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-bert-base-cased


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-bert-hinglish-small


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-bert-hinglish-small')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-bert-hinglish-small')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-bert-hinglish-small')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-bert-hinglish-small)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:


```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-bert-hinglish-small


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-distilbert-base-cased


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-distilbert-base-cased')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-distilbert-base-cased')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-distilbert-base-cased')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-distilbert-base-cased)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DistilBertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-distilbert-base-cased


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, distilbert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-roberta-hinglish-small


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-roberta-hinglish-small')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-roberta-hinglish-small')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-roberta-hinglish-small')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-roberta-hinglish-small)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:


```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: RobertaModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-roberta-hinglish-small


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, roberta, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-bert-hinglish-big


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-bert-hinglish-big')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-bert-hinglish-big')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-bert-hinglish-big')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-bert-hinglish-big)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:


```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-bert-hinglish-big


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-contrastive-roberta-base


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-contrastive-roberta-base')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-contrastive-roberta-base')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-contrastive-roberta-base')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-contrastive-roberta-base)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.ContrastiveLoss.ContrastiveLoss` with parameters:


```
{'distance_metric': 'SiameseDistanceMetric.COSINE_DISTANCE', 'margin': 0.5, 'size_average': True}
```


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-contrastive-roberta-base


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, roberta, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-roberta-hinglish-big


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-roberta-hinglish-big')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-roberta-hinglish-big')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-roberta-hinglish-big')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-roberta-hinglish-big)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:


```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: RobertaModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sentencetransformer-roberta-hinglish-big


model author: aditeyabaral


task: sentence-similarity


tags: pytorch, roberta, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# aditeyabaral/sentencetransformer-xlm-roberta-base


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-xlm-roberta-base')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-xlm-roberta-base')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-xlm-roberta-base')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-xlm-roberta-base)


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters: