Edit model card


Indo-roberta-indonli is natural language inference classifier based on Indo-roberta model. It was trained on the trained on IndoNLI dataset. The model used was Indo-roberta and was transfer-learned to a natural inference classifier model. The model are tested using the validation, test_layer and test_expert dataset given in the github repository. The results are shown below.


Dataset Accuracy F1 Precision Recall
Test Lay 0.74329 0.74075 0.74283 0.74133
Test Expert 0.6115 0.60543 0.63924 0.61742


The model was trained on with 5 epochs, batch size 16, learning rate 2e-5 and weight decay 0.01. Achieved different metrics as shown below.

Epoch Training Loss Validation Loss Accuracy F1 Precision Recall
1 0.942500 0.658559 0.737369 0.735552 0.735488 0.736679
2 0.649200 0.645290 0.761493 0.759593 0.762784 0.759642
3 0.437100 0.667163 0.766045 0.763979 0.765740 0.763792
4 0.282000 0.786683 0.764679 0.761802 0.762011 0.761684
5 0.193500 0.925717 0.765134 0.763127 0.763560 0.763489

How to Use

As NLI Classifier

from transformers import pipeline
pretrained_name = "StevenLimcorn/indonesian-roberta-indonli"
nlp = pipeline(
nlp("Amir Sjarifoeddin Harahap lahir di Kota Medan, Sumatera Utara, 27 April 1907. Ia meninggal di Surakarta, Jawa Tengah, pada 19 Desember 1948 dalam usia 41 tahun. </s></s> Amir Sjarifoeddin Harahap masih hidup.")


Do consider the biases which come from both the pre-trained RoBERTa model and the INDONLI dataset that may be carried over into the results of this model.


Indonesian RoBERTa Base IndoNLI was trained and evaluated by Steven Limcorn. All computation and development are done on Google Colaboratory using their free GPU access.


The dataset we used is by IndoNLI.

    title = "IndoNLI: A Natural Language Inference Dataset for Indonesian",
    author = "Mahendra, Rahmad and Aji, Alham Fikri and Louvan, Samuel and Rahman, Fahrurrozi and Vania, Clara",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    publisher = "Association for Computational Linguistics",
Downloads last month

Dataset used to train StevenLimcorn/indo-roberta-indonli