---
language: id
tags:
- indonesian-roberta-base-indonli
license: mit
datasets:
- indonli
widget:
- text: Andi tersenyum karena mendapat hasil baik. </s></s> Andi sedih.
model-index:
- name: w11wo/indonesian-roberta-base-indonli
  results:
  - task:
      type: natural-language-inference
      name: Natural Language Inference
    dataset:
      name: indonli
      type: indonli
      config: indonli
      split: test_expert
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.6072386058981233
      verified: true
    - name: Precision Macro
      type: precision
      value: 0.6304330508019023
      verified: true
    - name: Precision Micro
      type: precision
      value: 0.6072386058981233
      verified: true
    - name: Precision Weighted
      type: precision
      value: 0.6320495884503851
      verified: true
    - name: Recall Macro
      type: recall
      value: 0.6127303344145852
      verified: true
    - name: Recall Micro
      type: recall
      value: 0.6072386058981233
      verified: true
    - name: Recall Weighted
      type: recall
      value: 0.6072386058981233
      verified: true
    - name: F1 Macro
      type: f1
      value: 0.6010566054103847
      verified: true
    - name: F1 Micro
      type: f1
      value: 0.6072386058981233
      verified: true
    - name: F1 Weighted
      type: f1
      value: 0.5995456855334425
      verified: true
    - name: loss
      type: loss
      value: 1.157181739807129
      verified: true
---

## Indonesian RoBERTa Base IndoNLI

Indonesian RoBERTa Base IndoNLI is a natural language inference (NLI) model based on the [RoBERTa](https://arxiv.org/abs/1907.11692) model. The model was originally the pre-trained [Indonesian RoBERTa Base](https://hf.co/flax-community/indonesian-roberta-base) model, which is then fine-tuned on [`IndoNLI`](https://github.com/ir-nlp-csui/indonli)'s dataset consisting of Indonesian Wikipedia, news, and Web articles [1].

After training, the model achieved an evaluation/dev accuracy of 77.06%. On the benchmark `test_lay` subset, the model achieved an accuracy of 74.24% and on the benchmark `test_expert` subset, the model achieved an accuracy of 61.66%.

Hugging Face's `Trainer` class from the [Transformers](https://huggingface.co/transformers) library was used to train the model. PyTorch was used as the backend framework during training, but the model remains compatible with other frameworks nonetheless.

## Model

| Model                             | #params | Arch.        | Training/Validation data (text) |
| --------------------------------- | ------- | ------------ | ------------------------------- |
| `indonesian-roberta-base-indonli` | 124M    | RoBERTa Base | `IndoNLI`                       |

## Evaluation Results

The model was trained for 5 epochs, with a batch size of 16, a learning rate of 2e-5, a weight decay of 0.1, and a warmup ratio of 0.2, with linear annealing to 0. The best model was loaded at the end.

| Epoch | Training Loss | Validation Loss | Accuracy |
| ----- | ------------- | --------------- | -------- |
| 1     | 0.989200      | 0.691663        | 0.731452 |
| 2     | 0.673000      | 0.621913        | 0.766045 |
| 3     | 0.449900      | 0.662543        | 0.770596 |
| 4     | 0.293600      | 0.777059        | 0.768320 |
| 5     | 0.194200      | 0.948068        | 0.764224 |

## How to Use

### As NLI Classifier

```python
from transformers import pipeline

pretrained_name = "w11wo/indonesian-roberta-base-indonli"

nlp = pipeline(
    "sentiment-analysis",
    model=pretrained_name,
    tokenizer=pretrained_name
)

nlp("Andi tersenyum karena mendapat hasil baik. </s></s> Andi sedih.")
```

## Disclaimer

Do consider the biases which come from both the pre-trained RoBERTa model and the `IndoNLI` dataset that may be carried over into the results of this model.

## References

[1] Mahendra, R., Aji, A. F., Louvan, S., Rahman, F., & Vania, C. (2021, November). [IndoNLI: A Natural Language Inference Dataset for Indonesian](https://arxiv.org/abs/2110.14566). _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.

## Author

Indonesian RoBERTa Base IndoNLI was trained and evaluated by [Wilson Wongso](https://w11wo.github.io/). All computation and development are done on Google Colaboratory using their free GPU access.