README.md

---
'[object Object]': null
license: cc
language:
- en
library_name: adapter-transformers
pipeline_tag: text-classification
---

# Model Card for orYx-models/finetuned-roberta-leadership-sentiment-analysis 

- **Model Description:** This model is a finetuned version of the RoBERTa text classifier(cardiffnlp/twitter-roberta-base-sentiment-latest). It has been trained on a dataset comprising communications from corporate executives to their therapists. Its primary function is to determine whether statements from corporate executives convey a "Positive," "Negative," or "Neutral" sentiment, accompanied by a confidence level indicating the percentage of sentiment expressed in a statement. Being a prototype tool by orYx Models, all feedbacks and insights will be used to further refine the model.

## Model Details

### Model Information

- **Model Type:** Text Classifier
- **Language(s):** English
- **License:** Creative Commons license family
- **Finetuned from Model:** cardiffnlp/twitter-roberta-base-sentiment-latest

### Model Sources

- **HuggingFace Model ID:** cardiffnlp/twitter-roberta-base-2021-124m
- **Paper:** TimeLMs - [Link](https://arxiv.org/abs/2202.03829)

## Uses

- **Use case:** This sentiment analysis tool can analyze text from any user within an organization, such as executives, employees, or clients, and assign a sentiment to it.
- **Outcomes:** The tool generates a "Scored sentiment" which can be used to assess the likelihood of events occurring or vice versa. It can also facilitate the creation of a rating system based on the sentiments expressed in texts.

### Direct Use

```python
nlp = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)

nlp("The results don't match, but the effort seems to be always high")

Out[7]: [{'label': 'Positive', 'score': 0.9996090531349182}]
```

- Based on the text the outcomes can be "Positive, Negative, Neutral" along with their confidence score.
- 
### Recommendations
- **Continuous Monitoring:** Regularly monitor the model's performance on new data to ensure its effectiveness and reliability over time.
- **Error Analysis:** Conduct thorough error analysis to identify common patterns of misclassifications and areas for improvement.
- **Fine-Tuning:** Consider fine-tuning the model further based on feedback and insights from users, to enhance its domain-specific performance.
- **Model Interpretability:** Explore techniques for explaining the model's predictions, such as attention mechanisms or feature importance analysis, to increase trust and understanding of its decisions.


## Training Details
```

X_train, X_val, y_train, y_val = train_test_split(X,y, test_size = 0.2, stratify = y)

```

- **Train data:** 80% of 4396 records = 3516
- **Test data:** 20% of 4396 records = 879


### Training Procedure

- **Dataset Split:** Data divided into 80% training and 20% validation sets.
- **Preprocessing:** Input data tokenized into 'input_ids' and 'attention_mask' tensors.
- **Training Hyperparameters:** Set for training, evaluation, and optimization, including batch size, epochs, and logging strategies.
- **Training Execution:** Model trained with specified hyperparameters, monitored with metrics, and logged for evaluation.
- **Evaluation Metrics:** Model evaluated on loss, accuracy, F1 score, precision, and recall for both training and validation sets.

#### Preprocessing [optional]
```
'input_ids': tensor
'attention_mask': tensor
'label': tensor(2)
```

#### Training Hyperparameters
```
args = TrainingArguments(
    output_dir="output",
    do_train = True,
    do_eval = True,
    num_train_epochs = 1,
    per_device_train_batch_size = 4,
    per_device_eval_batch_size = 8,
    warmup_steps = 50,
    weight_decay = 0.01,
    logging_strategy= "steps",
    logging_dir= "logging",
    logging_steps = 50,
    eval_steps = 50,
    save_strategy = "steps",
    fp16 = True,
    #load_best_model_at_end = True
)
```
#### Speeds, Sizes, Times [optional]

- **TrainOutput**
```
global_step=879,
training_loss=0.1825900522650848,
```
- **Metrics**
```
'train_runtime': 101.6309,
'train_samples_per_second': 34.596,
'train_steps_per_second': 8.649,
'total_flos': 346915041274368.0,
'train_loss': 0.1825900522650848,
'epoch': 1.0
```

## Evaluation Metrics Results

```
# Assuming you have a list of evaluation results q and want to create a DataFrame with it
q = [Trainer.evaluate(eval_dataset=df) for df in [train_dataset, val_dataset]]

# Create DataFrame with index and select only the first 5 columns
result_df = pd.DataFrame(q, index=["train", "val"]).iloc[:,:5]

# Display the resulting DataFrame
print(result_df)

______________________________________________________________________
eval_loss  eval_Accuracy   eval_F1  eval_Precision  eval_Recall
train   0.049349       0.988908  0.987063        0.982160     0.992357
val     0.108378       0.976136  0.972464        0.965982     0.979861
______________________________________________________________________
```

**loss**  
- train   0.049349        
- val     0.108378 

**Accuracy** 
- train  0.988908   - **98.8%**
- val    0.976136   - **97.6%**

**F1**  
- train 0.987063    - **98.7%**   
- val   0.972464    - **97.2%**


**Precision**  
- train 0.982160    - **98.2%**
- val   0.965982    - **96.5%**

**Recall**
- train  0.992357   - **99.2%**
- val    0.979861   - **97.9%**




## Environmental Impact


Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** T4 GPU
- **Hours used:** 2 
- **Cloud Provider:** Google
- **Compute Region:** India
- **Carbon Emitted:** No Information Available


### Compute Infrastructure

Google Colab - T4 GPU


### References 
```
@inproceedings{camacho-collados-etal-2022-tweetnlp,
    title = "{T}weet{NLP}: Cutting-Edge Natural Language Processing for Social Media",
    author = "Camacho-collados, Jose  and
      Rezaee, Kiamehr  and
      Riahi, Talayeh  and
      Ushio, Asahi  and
      Loureiro, Daniel  and
      Antypas, Dimosthenis  and
      Boisson, Joanne  and
      Espinosa Anke, Luis  and
      Liu, Fangyu  and
      Mart{\'\i}nez C{\'a}mara, Eugenio" and others,
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, UAE",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.emnlp-demos.5",
    pages = "38--49"
}

```


## Model Card Authors [optional]

Vineedhar, relkino

## Model Card Contact

https://khalidalhosni.com/