```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 407 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.SoftmaxLoss.SoftmaxLoss`


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 1,
    "evaluation_steps": 1000,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 41,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: bert-base-portuguese-cased-nli-assin-2


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 295 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.SoftmaxLoss.SoftmaxLoss`


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 1,
    "evaluation_steps": 1000,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 30,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: bert-base-portuguese-cased-nli-assin


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


hello


model name: BERT-pt-institutional


task: fill-mask


tags: pytorch, bert, fill-mask, transformers, autonlp_compatible, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 356 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 4,
    "evaluation_steps": 1000,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 143,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 75, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sbertimbau-base-nli-sts


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`sentence_transformers.datasets.NoDuplicatesDataLoader.NoDuplicatesDataLoader` of length 4059 with parameters:


```
{'batch_size': 64}
```


**Loss**:


`sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss` with parameters:


```
{'scale': 20.0, 'similarity_fct': 'cos_sim'}
```


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 1,
    "evaluation_steps": 405,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 406,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 75, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: bertimbau_base_snli_mnrl


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 3227 with parameters:


```
{'batch_size': 64, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss` with parameters:


```
{'scale': 20.0, 'similarity_fct': 'cos_sim'}
```


`torch.utils.data.dataloader.DataLoader` of length 4333 with parameters:


`sentence_transformers.losses.OnlineContrastiveLoss.OnlineContrastiveLoss`


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "sentence_transformers.evaluation.SequentialEvaluator.SequentialEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 1000,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 75, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sbertimbau-base-quora-multitask


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 8605 with parameters:


```
{'batch_size': 24, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss` with parameters:


```
{'scale': 20.0, 'similarity_fct': 'cos_sim'}
```


`torch.utils.data.dataloader.DataLoader` of length 11553 with parameters:


`sentence_transformers.losses.OnlineContrastiveLoss.OnlineContrastiveLoss`


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "sentence_transformers.evaluation.SequentialEvaluator.SequentialEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 1000,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 64, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sbertimbau-large-quora-multitask


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 701 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.SoftmaxLoss.SoftmaxLoss`


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 1,
    "evaluation_steps": 1000,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 71,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: bert-portuguese-cased-nli-assin-assin-2


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# {MODEL_NAME}


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


<!--- Describe your model here -->


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
model = AutoModel.from_pretrained('{MODEL_NAME}')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


<!--- Describe how your model was evaluated -->


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training


The model was trained with the parameters:


**DataLoader**:


`sentence_transformers.datasets.NoDuplicatesDataLoader.NoDuplicatesDataLoader` of length 8066 with parameters:


```
{'batch_size': 64}
```


**Loss**:


`sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss` with parameters:


```
{'scale': 20.0, 'similarity_fct': 'cos_sim'}
```


Parameters of the fit()-Method:


```
{
    "callback": null,
    "epochs": 1,
    "evaluation_steps": 806,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 807,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 75, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: sbertimbau-base-allnli-mnrl


model author: ricardo-filho


task: sentence-similarity


tags: pytorch, bert, feature-extraction, sentence-transformers, sentence-similarity, transformers, infinity_compatible


tags:


- image-classification
- pytorch
- huggingpics
metrics:
- accuracy


model-index:


- name: dog-races-v2
results:
  - task:
name: Image Classification
type: image-classification
metrics:
    - name: Accuracy
type: accuracy
value: 0.8299999833106995


# dog-races-v2


Autogenerated Model created thannks to HuggingPics🤗🖼. You can create your own image classifier for **anything** by running [the demo on Google Colab](https://colab.research.google.com/github/nateraw/huggingpics/blob/main/HuggingPics.ipynb).


This Model is an improvement to my last model, where the Chow Chow data included images of American pickles with the same name (contaminated data).
Current labels are: 1) Border Collie, 2) Chow Chow, 3) German Shepherd, 4) Golden Retriever, 5) Pug, 6) Rottweiler, 7) Shiba Inu, 8) Siberian Husky and 9) Tibetan Mastiff.


There is still room for improvement.


Model Accuracy: 82.99%


When tested with Stanford Dogs Dataset, these were the results:


- Golden Retriever: 90% (117/130 images labeled correctly)
- Chow Chow: 97.45% (191/196 images labeled correctly)
- Tibetan Mastiff: 12.5% (19/152 images labeled correctly). Probably some issue with the data (most were labeled as Chow Chow).


## Example Images


#### Border Collie


![Border Collie](images/Border_Collie.jpg)


#### Chow Chow dog


![Chow Chow dog](images/Chow_Chow_dog.jpg)


#### German Shepherd


![German Shepherd](images/German_Shepherd.jpg)


#### Golden Retriever


![Golden Retriever](images/Golden_Retriever.jpg)


#### Pug


![Pug](images/Pug.jpg)


#### Rottweiler


![Rottweiler](images/Rottweiler.jpg)


#### Shiba Inu


![Shiba Inu](images/Shiba_Inu.jpg)


#### Siberian Husky


![Siberian Husky](images/Siberian_Husky.jpg)


#### Tibetan Mastiff


![Tibetan Mastiff](images/Tibetan_Mastiff.jpg)


model name: dog-races


model author: roschmid


task: image-classification


tags: pytorch, tensorboard, vit, image-classification, transformers, huggingpics, model-index


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model_index:
- name: distilbert-base-uncased-finetuned-cola
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: cola
metric:
name: Matthews Correlation
type: matthews_correlation
value: 0.520875943143754


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilbert-base-uncased-finetuned-cola


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.8486
- Matthews Correlation: 0.5209


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5265        | 1.0   | 535  | 0.5479          | 0.4049               |
| 0.3571        | 2.0   | 1070 | 0.5002          | 0.5164               |
| 0.2432        | 3.0   | 1605 | 0.6242          | 0.5091               |
| 0.173         | 4.0   | 2140 | 0.7559          | 0.5120               |
| 0.1352        | 5.0   | 2675 | 0.8486          | 0.5209               |


### Framework versions


- Transformers 4.9.1
- Pytorch 1.9.0+cu102
- Datasets 1.10.2
- Tokenizers 0.10.3


model name: distilbert-base-uncased-finetuned-cola


model author: pranav1015


task: text-classification


tags: pytorch, tensorboard, distilbert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, infinity_compatible


tags:


- conversational


We love owen


model name: DialoGPT-medium-sexybabeycord


model author: vaughnw128


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


- We love asher


model name: DialoGPT-small-sexybabeycord


#Jake Peralta DialoGPT Model


model name: DialoGPT-small-jake


model author: 1Basco


@Harry Potter DialoGPT model


model name: DialoGPT-small-harrypotter


model author: ABBHISHEK


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model_index:
- name: portuguese-archival-finding-aids
results:
  - task:
name: Token Classification
type: token-classification
metric:
name: Accuracy
type: accuracy
value: 0.9617770479839446


# portuguese-archival-finding-aids


This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on an unkown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1812
- Precision: 0.8624
- Recall: 0.9557
- F1: 0.9067
- Accuracy: 0.9618


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| No log        | 1.0   | 192  | 0.1565          | 0.8511    | 0.9327 | 0.8900 | 0.9563   |
| 0.1849        | 2.0   | 384  | 0.1594          | 0.8634    | 0.9543 | 0.9065 | 0.9619   |
| 0.0454        | 3.0   | 576  | 0.1812          | 0.8624    | 0.9557 | 0.9067 | 0.9618   |


### Framework versions


- Transformers 4.10.0.dev0
- Pytorch 1.9.0+cu111
- Datasets 1.10.2
- Tokenizers 0.10.3


model name: bert-multilingual-pt-archive


model author: lfcc


task: token-classification


tags: pytorch, tensorboard, bert, token-classification, transformers, license:apache-2.0, generated_from_trainer, autonlp_compatible, infinity_compatible


license: mit
tags:


- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model_index:
- name: bert-large-pt-archive
results:
  - task:
name: Token Classification
type: token-classification
metric:
name: Accuracy
type: accuracy
value: 0.9766762474673703


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-large-pt-archive


This model is a fine-tuned version of [neuralmind/bert-large-portuguese-cased](https://huggingface.co/neuralmind/bert-large-portuguese-cased) on an unkown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.0869
- Precision: 0.9280
- Recall: 0.9541
- F1: 0.9409
- Accuracy: 0.9767


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0665        | 1.0   | 765  | 0.1020          | 0.8928    | 0.9566 | 0.9236 | 0.9696   |
| 0.0392        | 2.0   | 1530 | 0.0781          | 0.9229    | 0.9586 | 0.9404 | 0.9757   |
| 0.0201        | 3.0   | 2295 | 0.0809          | 0.9278    | 0.9550 | 0.9412 | 0.9767   |
| 0.0152        | 4.0   | 3060 | 0.0869          | 0.9280    | 0.9541 | 0.9409 | 0.9767   |


### Framework versions


model name: bert-large-pt-archive


model author: lfcc


task: token-classification


tags: pytorch, tensorboard, bert, token-classification, transformers, license:mit, generated_from_trainer, autonlp_compatible, infinity_compatible


license: mit
tags:


- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model_index:
- name: bert-base-pt-archive
results:
  - task:
name: Token Classification
type: token-classification
metric:
name: Accuracy
type: accuracy
value: 0.9700325118974698


# bert-base-pt-archive


This model is a fine-tuned version of [neuralmind/bert-base-portuguese-cased](https://huggingface.co/neuralmind/bert-base-portuguese-cased) on an unkown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1140
- Precision: 0.9147
- Recall: 0.9483
- F1: 0.9312
- Accuracy: 0.9700


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| No log        | 1.0   | 192  | 0.1438          | 0.8917    | 0.9392 | 0.9148 | 0.9633   |
| 0.2454        | 2.0   | 384  | 0.1222          | 0.8985    | 0.9417 | 0.9196 | 0.9671   |
| 0.0526        | 3.0   | 576  | 0.1098          | 0.9150    | 0.9481 | 0.9312 | 0.9698   |
| 0.0372        | 4.0   | 768  | 0.1140          | 0.9147    | 0.9483 | 0.9312 | 0.9700   |


### Framework versions


model name: bert-base-pt-archive


model author: lfcc


task: token-classification


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: opus-mt-en-ru-finetuned-v2-en-to-ru
results: []


# opus-mt-en-ru-finetuned-v2-en-to-ru


This model is a fine-tuned version of [Helsinki-NLP/opus-mt-en-ru](https://huggingface.co/Helsinki-NLP/opus-mt-en-ru) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 1.2182
- Bleu: 29.1013
- Gen Len: 29.0582


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 48
- eval_batch_size: 48
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
| 1.7662        | 1.0   | 33135 | 1.2395          | 28.5144 | 29.0305 |
| 1.6612        | 2.0   | 66270 | 1.2245          | 28.7764 | 29.1395 |
| 1.6275        | 3.0   | 99405 | 1.2182          | 29.1013 | 29.0582 |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3


model name: opus-mt-en-ru-finetuned-v2-en-to-ru


model author: kazandaev


task: text2text-generation


tags: pytorch, tensorboard, marian, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: mt5-small
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# mt5-small


This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 1.6823
- Bleu: 5.7578
- Gen Len: 17.9322


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 12
- eval_batch_size: 12
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1


### Training results


| Training Loss | Epoch | Step   | Validation Loss | Bleu   | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:------:|:-------:|
| 2.2139        | 1.0   | 298848 | 1.6823          | 5.7578 | 17.9322 |


### Framework versions


- Transformers 4.16.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.11.0


model name: mt5-small


model author: kazandaev


task: text2text-generation


tags: pytorch, tensorboard, mt5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: opus-mt-en-ru-finetuned-v3-en-to-ru
results: []


# opus-mt-en-ru-finetuned-v3-en-to-ru


This model is a fine-tuned version of [kazandaev/opus-mt-en-ru-finetuned-v3-en-to-ru](https://huggingface.co/kazandaev/opus-mt-en-ru-finetuned-v3-en-to-ru) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.8237
- Bleu: 37.4514
- Gen Len: 29.2779


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 50
- eval_batch_size: 50
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4


### Training results


| Training Loss | Epoch | Step   | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|
| 0.9367        | 1.0   | 42137  | 0.8649          | 36.2025 | 29.3907 |
| 0.9346        | 2.0   | 84274  | 0.8470          | 36.4315 | 29.484  |
| 0.9143        | 3.0   | 126411 | 0.8330          | 36.989  | 29.3385 |
| 0.9119        | 4.0   | 168548 | 0.8237          | 37.4514 | 29.2779 |


### Framework versions


- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


model name: opus-mt-en-ru-finetuned-v3-en-to-ru


model author: kazandaev


task: text2text-generation


tags: pytorch, tensorboard, marian, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- wmt16
metrics:
- bleu
model-index:
- name: opus-mt-en-ru-finetuned-en-to-ru
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation
dataset:
name: wmt16
type: wmt16
args: ru-en
metrics:
    - name: Bleu
type: bleu
value: 28.4589


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# opus-mt-en-ru-finetuned-en-to-ru


This model is a fine-tuned version of [Helsinki-NLP/opus-mt-en-ru](https://huggingface.co/Helsinki-NLP/opus-mt-en-ru) on the wmt16 dataset.
It achieves the following results on the evaluation set:


- Loss: 1.2852
- Bleu: 28.4589
- Gen Len: 29.5852


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
| 1.9547        | 1.0   | 94761 | 1.2852          | 28.4589 | 29.5852 |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: opus-mt-en-ru-finetuned-en-to-ru


model author: kazandaev


task: text2text-generation


tags: pytorch, tensorboard, marian, text2text-generation, dataset:wmt16, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


Swedish bert multilingual model trained on a machine translated (MS neural translation) SQUAD 1.1 dataset


model name: bert-multi-cased-squad_sv


model author: marbogusz


task: question-answering


tags: pytorch, jax, bert, question-answering, transformers, autonlp_compatible, infinity_compatible


tags:


- conversational


# My Awesome Model


model name: DialoGPT-medium-harry


model author: phozon


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language: es


- summarization


widget:


- text: "La Agencia Valenciana de la Innovación (AVI) financia el desarrollo de un software que integra diferentes modelos y tecnologías para la monitorización y análisis multilingüe de las redes sociales. A través de técnicas de 'deep learning' y procesamiento del lenguaje natural es capaz de interpretar la ironía y las emociones en los textos, incluso en aquellos escritos en idiomas menos extendidos, a menudo no contemplados por las herramientas comerciales. La iniciativa, bautizada como 'Guaita', está liderada por el Instituto Valenciano de Investigación en Inteligencia Artificial (VRAIN), adscrito a la Universidad Politécnica de Valencia (UPV), que cuenta a su vez para su desarrollo con la colaboración del Instituto Valenciano de Informática (ITI) y la Corporación Valenciana de Mitjans de Comunicación (CVMC).De este modo, y a solicitud del usuario o usuaria, monitorizará las redes sociales para obtener la información asociada a los temas objeto de interés y ofrecerá los resultados de forma gráfica, bien a través de una interfaz web, bien mediante la generación de informes. El programa será, además, capaz de determinar la reputación de una empresa o institución a partir de dichos análisis gracias a la combinación de distintas tecnologías de procesamiento e interpretación, destaca la agencia en un comunicado."


# NASca and NASes: Two Monolingual Pre-Trained Models for Abstractive Summarization in Catalan and Spanish


Most of the models proposed in the literature for abstractive summarization are generally suitable for the English language but not for other languages. Multilingual models were introduced to address that language constraint, but despite their applicability being broader than that of the monolingual models, their performance is typically lower, especially for minority languages like Catalan. In this paper, we present a monolingual model for abstractive summarization of textual content in the Catalan language. The model is a Transformer encoder-decoder which is pretrained and fine-tuned specifically for the Catalan language using a corpus of newspaper articles. In the pretraining phase, we introduced several self-supervised tasks to specialize the model on the summarization task and to increase the abstractivity of the generated summaries. To study the performance of our proposal in languages with higher resources than Catalan, we replicate the model and the experimentation for the Spanish language. The usual evaluation metrics, not only the most used ROUGE measure but also other more semantic ones such as BertScore, do not allow to correctly evaluate the abstractivity of the generated summaries. In this work, we also present a new metric, called content reordering, to evaluate one of the most common characteristics of abstractive summaries, the rearrangement of the original content. We carried out an exhaustive experimentation to compare the performance of the monolingual models proposed in this work with two of the most widely used multilingual models in text summarization, mBART and mT5. The experimentation results support the quality of our monolingual models, especially considering that the multilingual models were pretrained with many more resources than those used in our models. Likewise, it is shown that the pretraining tasks helped to increase the degree of abstractivity of the generated summaries. To our knowledge, this is the first work that explores a monolingual approach for abstractive summarization both in Catalan and Spanish.


# The NASes model


News Abstractive Summarization for Spanish (NASES) is a Transformer encoder-decoder model, with the same hyper-parameters than BART, to perform summarization of Spanish news articles. It is pre-trained on a combination of several self-supervised tasks that help to increase the abstractivity of the generated summaries. Four pre-training tasks have been combined: sentence permutation, text infilling, Gap Sentence Generation, and Next Segment Generation. Spanish newspapers, and Wikipedia articles in Spanish were used for pre-training the model (21GB of raw text -8.5 millions of documents-).


NASES is finetuned for the summarization task on 1.802.919 (document, summary) pairs from the Dataset for Automatic summarization of Catalan and Spanish newspaper Articles (DACSA).


### BibTeX entry


```bibtex
@Article{app11219872,
AUTHOR = {Ahuir, Vicent and Hurtado, Lluís-F. and González, José Ángel and Segarra, Encarna},
TITLE = {NASca and NASes: Two Monolingual Pre-Trained Models for Abstractive Summarization in Catalan and Spanish},
JOURNAL = {Applied Sciences},
VOLUME = {11},
YEAR = {2021},
NUMBER = {21},
ARTICLE-NUMBER = {9872},
URL = {https://www.mdpi.com/2076-3417/11/21/9872},
ISSN = {2076-3417},
DOI = {10.3390/app11219872}
}
```


model name: NASES


model author: ELiRF


task: summarization


tags: pytorch, bart, text2text-generation, es, transformers, summarization, autonlp_compatible


language: ca


tags:


- text: "Microsoft ha triat Barcelona com a base d’operacions per al seu nou hub de R+D especialitzat en l’aplicació de tecnologies d’Intel·ligència Artificial per a la millora de l’experiència d’usuari a la web. El centre situat a Barcelona serà un dels vuit centres d’investigació amb què compta a nivell mundial la divisió WebXT (Web Experiences Team) de Microsoft, una organització de més de 5.000 persones que presideix Mikhaïl Parakhin, centrada en el desenvolupament d’experiències d’usuari avançades basades en l’ús de tecnologies d’Intel·ligència Artificial i aprenentatge profund. El centre d’excel·lència pretén atraure talent de tots els països d’Europa i s’enquadra dins de l’equip de Search & AI que lidera Jordi Ribas, vicepresident corporatiu de Microsoft Corporation als Estats Units. Aquest equip especialitzat en aplicació d’Intel·ligència Artificial és el grup més gran de WebXT de la companyia i contribueix al desenvolupament de múltiples productes de Microsoft, incloent Windows, Azure i, per descomptat, Bing.  En aquesta primera fase, la inversió inclou l’actual procés de selecció obert que contempla la contractació d’una trentena d’enginyers de ‘software i científics especialitzats en àrees avançades d’enginyeria de ‘software’ incloent Intel·ligència Artificial, Machine Learning i Deep Learning, que podrien superar el centenar en els pròxims anys. L’objectiu de Microsoft és crear al voltant d’aquest equip d’enginyers un vector d’innovació en Intel·ligència Artificial –en col·laboració amb universitats, centres d’investigació i empreses de tecnologia–, reforçant els esforços per impulsar el talent digital a Espanya i la capacitació en tecnologies de ‘machine learning’."


# The NASca model


News Abstractive Summarization for Catalan (NASCA) is a Transformer encoder-decoder model, with the same hyper-parameters than BART, to perform summarization of Catalan news articles. It is pre-trained on a combination of several self-supervised tasks that help to increase the abstractivity of the generated summaries. Four pre-training tasks have been combined: sentence permutation, text infilling, Gap Sentence Generation, and Next Segment Generation. Catalan newspapers, the Catalan subset of the OSCAR corpus and Wikipedia articles in Catalan were used for pre-training the model (9.3GB of raw text -2.5 millions of documents-).


NASCA is finetuned for the summarization task on 636.596 (document, summary) pairs from the Dataset for Automatic summarization of Catalan and Spanish newspaper Articles (DACSA).


model name: NASCA


tags: pytorch, bart, text2text-generation, ca, transformers, summarization, autonlp_compatible


language:


- Tagalog
thumbnail:
tags:
- Tagalog
- Mang Bert
license: apache-2.0
datasets:
- OSCAR tl


# Mang Bert


## Model description


Fine-Tuned Roberta Model using RobertaForMaskedLM
Tagalog Dataset from OSCAR tl


## Training data


458206 text dataset from OSCAR


model name: unang_mang_bert


model author: petabyte


task: feature-extraction


tags: pytorch, roberta, feature-extraction, Tagalog, dataset:OSCAR tl, transformers, Mang Bert, license:apache-2.0, infinity_compatible


tags:


- conversational


# Family Guy (Peter) DialoGPT Model


model name: dialogpt-small-familyguy


model author: gfdream


task: conversational


# Harry Potter DialoGPT Model


model name: dialogpt-small-harrypotter


license: mit
tags:


- generated_from_trainer
datasets:
- amazon_reviews_multi
model-index:
- name: xlm-roberta-base-finetuned-marc-en
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# xlm-roberta-base-finetuned-marc-en


This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the amazon_reviews_multi dataset.
It achieves the following results on the evaluation set:


- Loss: 0.9302
- Mae: 0.5


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2


### Training results


| Training Loss | Epoch | Step | Validation Loss | Mae    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 1.1253        | 1.0   | 235  | 0.9756          | 0.5488 |
| 0.9465        | 2.0   | 470  | 0.9302          | 0.5    |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.9.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: xlm-roberta-base-finetuned-marc-en


model author: danwilbury


task: text-classification


tags: pytorch, tensorboard, xlm-roberta, text-classification, dataset:amazon_reviews_multi, transformers, license:mit, generated_from_trainer, model-index


tags :


- conversational


#Rick Sanchez DialoGPT Model


model name: DialoGPT-small-RickmfSanchez


model author: ssam


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language: "en"
tags:


- gpt2
- exbert
- commonsense
- semeval2020
- comve
license: "mit"
datasets:
- https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation
metrics:
- bleu
widget:
- text: "Chicken can swim in water. <|continue|>"


# ComVE-gpt2-large


## Model description


Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective.
The model is able to generate a reason why a given natural language statement is against commonsense.


## Intended uses & limitations


You can use the raw model for text generation to generate reasons why natural language statements are against commonsense.


#### How to use


You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script.


*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again.


#### Limitations and bias


The model biased to negate the entered sentence usually instead of producing a factual reason.


## Training data


The model is initialized from the [gpt2-large](https://github.com/huggingface/transformers/blob/master/model_cards/gpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons.


## Training procedure


Each natural language statement that against commonsense is concatenated with its reference reason with `<|conteniue|>` as a separator, then the model finetuned using CLM objective.
The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size.


## Eval results


The model achieved 16.5110/15.9299 BLEU scores on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset.


### BibTeX entry and citation info


```bibtex
@article{fadel2020justers,
  title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation},
  author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik},
  year={2020}
}
```


model name: ComVE-gpt2-large


model author: aliosm


task: text-generation


tags: pytorch, jax, gpt2, text-generation, en, dataset:https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation, transformers, exbert, commonsense, semeval2020, comve, license:mit


language: "en"
tags:


- gpt2
- exbert
- commonsense
- semeval2020
- comve
license: "mit"
datasets:
- ComVE
metrics:
- bleu
widget:
- text: "Chicken can swim in water. <|continue|>"


# ComVE-gpt2-medium


## Model description


## Intended uses & limitations


#### How to use


#### Limitations and bias


## Training data


The model is initialized from the [gpt2-medium](https://github.com/huggingface/transformers/blob/master/model_cards/gpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons.


## Training procedure


Each natural language statement that against commonsense is concatenated with its reference reason with `<|continue|>` as a separator, then the model finetuned using CLM objective.
The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size.


## Eval results


The model achieved fifth place with 16.7153/16.1187 BLEU scores and third place with 1.94 Human Evaluation score on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset.


These are some examples generated by the model:
|             Against Commonsense Statement             |               Generated Reason               |
|:-----------------------------------------------------:|:--------------------------------------------:|
| Chicken can swim in water.                            | Chicken can't swim.                          |
| shoes can fly                                         | Shoes are not able to fly.                   |
| Chocolate can be used to make a coffee pot            | Chocolate is not used to make coffee pots.   |
| you can also buy tickets online with an identity card | You can't buy tickets with an identity card. |
| a ball is square and can roll                         | A ball is round and cannot roll.             |
| You can use detergent to dye your hair.               | Detergent is used to wash clothes.           |
| you can eat mercury                                   | mercury is poisonous                         |
| A gardener can follow a suspect                       | gardener is not a police officer             |
| cars can float in the ocean just like a boat          | Cars are too heavy to float in the ocean.    |
| I am going to work so I can lose money.               | Working is not a way to lose money.          |


### BibTeX entry and citation info


model name: ComVE-gpt2-medium


model author: aliosm


task: feature-extraction


tags: pytorch, jax, gpt2, feature-extraction, en, dataset:ComVE, transformers, exbert, commonsense, semeval2020, comve, license:mit


language: "c++"
tags:


- exbert
- authorship-identification
- fire2020
- pan2020
- ai-soco
- classification
license: "mit"
datasets:
- ai-soco
metrics:
- accuracy


# ai-soco-c++-roberta-small-clas


## Model description


`ai-soco-c++-roberta-small` model fine-tuned on [AI-SOCO](https://sites.google.com/view/ai-soco-2020) task.


#### How to use


You can use the model directly after tokenizing the text using the provided tokenizer with the model files.


#### Limitations and bias


The model is limited to C++ programming language only.


## Training data


The model initialized from [`ai-soco-c++-roberta-small`](https://github.com/huggingface/transformers/blob/master/model_cards/aliosm/ai-soco-c++-roberta-small) model and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset to do text classification.


## Training procedure


The model trained on Google Colab platform using V100 GPU for 10 epochs, 32 batch size, 512 max sequence length (sequences larger than 512 were truncated). Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization.


## Eval results


The model achieved 93.19%/92.88% accuracy on AI-SOCO task and ranked in the 4th place.


### BibTeX entry and citation info


```bibtex
@inproceedings{ai-soco-2020-fire,
    title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}",
    author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo",
    booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)",
    year = "2020"
}
```


model name: ai-soco-cpp-roberta-small-clas


model author: aliosm


tags: c++, dataset:ai-soco, exbert, authorship-identification, fire2020, pan2020, ai-soco, classification, license:mit


language: "c++"
tags:


- exbert
- authorship-identification
- fire2020
- pan2020
- ai-soco
license: "mit"
datasets:
- ai-soco
metrics:
- perplexity


# ai-soco-c++-roberta-tiny


## Model description


From scratch pre-trained RoBERTa model with 1 layers and 12 attention heads using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which consists of C++ codes crawled from CodeForces website.


## Intended uses & limitations


The model can be used to do code classification, authorship identification and other downstream tasks on C++ programming language.


#### How to use


#### Limitations and bias


## Training data


The model initialized randomly and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which contains 100K C++ source codes.


## Training procedure


The model trained on Google Colab platform with 8 TPU cores for 200 epochs, 32\*8 batch size, 512 max sequence length and MLM objective. Other parameters were defaulted to the values mentioned in [`run_language_modelling.py`](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py) script. Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization.


### BibTeX entry and citation info


model name: ai-soco-cpp-roberta-tiny


model author: aliosm


tags: c++, dataset:ai-soco, exbert, authorship-identification, fire2020, pan2020, ai-soco, license:mit


language: "en"
tags:


- exbert
- commonsense
- semeval2020
- comve
license: "mit"
datasets:
- ComVE
metrics:
- bleu
widget:
- text: "Chicken can swim in water. <|continue|>"


# ComVE-distilgpt2


## Model description


Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective.
The model is able to generate a reason why a given natural language statement is against commonsense.


## Intended uses & limitations


You can use the raw model for text generation to generate reasons why natural language statements are against commonsense.


#### How to use


You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script.


*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again.


#### Limitations and bias


The model biased to negate the entered sentence usually instead of producing a factual reason.


## Training data


The model is initialized from the [distilgpt2](https://github.com/huggingface/transformers/blob/master/model_cards/distilgpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons.


## Training procedure


Each natural language statement that against commonsense is concatenated with its reference reason with `<|continue|>` as a separator, then the model finetuned using CLM objective.
The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 15 epochs, 128 maximum sequence length and 64 batch size.


## Eval results


The model achieved 13.7582/13.8026 BLEU scores on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset.


### BibTeX entry and citation info


```bibtex
@article{fadel2020justers,
  title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation},
  author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik},
  year={2020}
}
```


model name: ComVE-distilgpt2


model author: aliosm


task: text-generation


tags: pytorch, jax, gpt2, text-generation, en, dataset:ComVE, transformers, exbert, commonsense, semeval2020, comve, license:mit


language: "c++"
tags:


- exbert
- authorship-identification
- fire2020
- pan2020
- ai-soco
- classification
license: "mit"
datasets:
- ai-soco
metrics:
- accuracy


# ai-soco-c++-roberta-tiny-clas


## Model description


`ai-soco-c++-roberta-tiny` model fine-tuned on [AI-SOCO](https://sites.google.com/view/ai-soco-2020) task.


#### How to use


You can use the model directly after tokenizing the text using the provided tokenizer with the model files.


#### Limitations and bias


The model is limited to C++ programming language only.


## Training data


The model initialized from [`ai-soco-c++-roberta-tiny`](https://github.com/huggingface/transformers/blob/master/model_cards/aliosm/ai-soco-c++-roberta-tiny) model and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset to do text classification.


## Training procedure


The model trained on Google Colab platform using V100 GPU for 10 epochs, 32 batch size, 512 max sequence length (sequences larger than 512 were truncated). Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization.


## Eval results


The model achieved 87.66%/87.46% accuracy on AI-SOCO task and ranked in the 9th place.


### BibTeX entry and citation info


```bibtex
@inproceedings{ai-soco-2020-fire,
    title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}",
    author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo",
    booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)",
    year = "2020"
}
```


model name: ai-soco-cpp-roberta-tiny-clas


model author: aliosm


tags: c++, dataset:ai-soco, exbert, authorship-identification, fire2020, pan2020, ai-soco, classification, license:mit


language: "en"
tags:


- exbert
- commonsense
- semeval2020
- comve
license: "mit"
datasets:
- ComVE
metrics:
- bleu
widget:
- text: "Chicken can swim in water. <|continue|>"


# ComVE-gpt2


## Model description


Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective.
The model is able to generate a reason why a given natural language statement is against commonsense.


## Intended uses & limitations


You can use the raw model for text generation to generate reasons why natural language statements are against commonsense.


#### How to use


You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script.


*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again.


#### Limitations and bias


The model biased to negate the entered sentence usually instead of producing a factual reason.


## Training data


The model is initialized from the [gpt2](https://github.com/huggingface/transformers/blob/master/model_cards/gpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons.


## Training procedure


Each natural language statement that against commonsense is concatenated with its reference reason with `<|continue|>` as a separator, then the model finetuned using CLM objective.
The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size.


## Eval results


The model achieved 14.0547/13.6534 BLEU scores on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset.


### BibTeX entry and citation info


```bibtex
@article{fadel2020justers,
  title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation},
  author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik},
  year={2020}
}
```


model name: ComVE-gpt2


model author: aliosm


task: text-generation


tags: pytorch, jax, gpt2, text-generation, en, dataset:ComVE, transformers, exbert, commonsense, semeval2020, comve, license:mit


language: "c++"
tags:


- exbert
- authorship-identification
- fire2020
- pan2020
- ai-soco
license: "mit"
datasets:
- ai-soco
metrics:
- perplexity


# ai-soco-c++-roberta-tiny-96


## Model description


From scratch pre-trained RoBERTa model with 1 layers and 96 attention heads using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which consists of C++ codes crawled from CodeForces website.


## Intended uses & limitations


The model can be used to do code classification, authorship identification and other downstream tasks on C++ programming language.


#### How to use


You can use the model directly after tokenizing the text using the provided tokenizer with the model files.


#### Limitations and bias


The model is limited to C++ programming language only.


## Training data


The model initialized randomly and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which contains 100K C++ source codes.


## Training procedure


The model trained on Google Colab platform with 8 TPU cores for 200 epochs, 16\*8 batch size, 512 max sequence length and MLM objective. Other parameters were defaulted to the values mentioned in [`run_language_modelling.py`](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py) script. Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization.


### BibTeX entry and citation info


```bibtex
@inproceedings{ai-soco-2020-fire,
    title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}",
    author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo",
    booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)",
    year = "2020"
}
```


model name: ai-soco-cpp-roberta-tiny-96


model author: aliosm


tags: c++, dataset:ai-soco, exbert, authorship-identification, fire2020, pan2020, ai-soco, license:mit


language: "c++"
tags:


# ai-soco-c++-roberta-small


## Model description


From scratch pre-trained RoBERTa model with 6 layers and 12 attention heads using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which consists of C++ codes crawled from CodeForces website.


## Intended uses & limitations


#### How to use


#### Limitations and bias


model name: ai-soco-cpp-roberta-small


- exbert
- authorship-identification
- fire2020
- pan2020
- ai-soco
- classification
license: "mit"
datasets:
- ai-soco
metrics:
- accuracy


# ai-soco-c++-roberta-tiny-96-clas


`ai-soco-c++-roberta-tiny-96` model fine-tuned on [AI-SOCO](https://sites.google.com/view/ai-soco-2020) task.


## Training data


The model initialized from [`ai-soco-c++-roberta-tiny-96`](https://github.com/huggingface/transformers/blob/master/model_cards/aliosm/ai-soco-c++-roberta-tiny-96) model and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset to do text classification.


## Training procedure


The model trained on Google Colab platform using V100 GPU for 10 epochs, 16 batch size, 512 max sequence length (sequences larger than 512 were truncated). Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization.


## Eval results


The model achieved 91.12%/91.02% accuracy on AI-SOCO task and ranked in the 7th place.


### BibTeX entry and citation info


model name: ai-soco-cpp-roberta-tiny-96-clas


model author: aliosm


tags: c++, dataset:ai-soco, exbert, authorship-identification, fire2020, pan2020, ai-soco, classification, license:mit


license: mit
tags:


- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: finetuned-bert-bounti
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# finetuned-bert-bounti


This model is a fine-tuned version of [dbmdz/bert-base-turkish-128k-uncased](https://huggingface.co/dbmdz/bert-base-turkish-128k-uncased) on the BounTi Turkish Twitter sentiment dataset.
It achieves the following results on the evaluation set:


- Loss: 1.1188
- Accuracy: 0.7246
- F1: 0.6845
- Precision: 0.6892
- Recall: 0.6806


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 1e-05
- train_batch_size: 24
- eval_batch_size: 36
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 10


### Training results