|
--- |
|
license: mit |
|
language: id |
|
base_model: indolem/indobert-base-uncased |
|
widget: |
|
- text: Pelayanan lama, sangat tidak memuaskan. |
|
example_title: Sentiment analysis |
|
datasets: |
|
- indonlp/indonlu |
|
- sepidmnorozy/Indonesian_sentiment |
|
pipeline_tag: text-classification |
|
--- |
|
### Model Details |
|
This model is a fine-tuned version of [IndoBERT Base Uncased](https://huggingface.co/indolem/indobert-base-uncased), a BERT model pre-trained on Indonesian text data. It was fine-tuned to perform sentiment analysis on Indonesian comments and reviews. |
|
|
|
The model was trained on [indonlu](https://huggingface.co/datasets/indonlp/indonlu) (`SmSA`) and [indonesian_sentiment](https://huggingface.co/datasets/sepidmnorozy/Indonesian_sentiment) datasets. |
|
|
|
The model classifies a given Indonesian review text into one of three categories: |
|
|
|
* Negative |
|
* Neutral |
|
* Positive |
|
|
|
### Training hyperparameters |
|
* train_batch_size: 32 |
|
* eval_batch_size: 32 |
|
* learning_rate: 1e-4 |
|
* optimizer: AdamW with betas=(0.9, 0.999), eps=1e-8, and weight_decay=0.01 |
|
* epochs: 3 |
|
* learning_rate_scheduler: StepLR with step_size=592, gamma=0.1 |
|
|
|
### Training Results |
|
|
|
The following table shows the training results for the model: |
|
|
|
| Epoch | Loss | Accuracy | |
|
|---|---|---| |
|
| 1 | 0.2936 | 0.9310 | |
|
| 2 | 0.1212 | 0.9526 | |
|
| 3 | 0.0795 | 0.9569 | |
|
|
|
### How to Use |
|
|
|
You can load the model and perform inference as follows: |
|
``` |
|
import torch |
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("taufiqdp/indonesian-sentiment") |
|
model = AutoModelForSequenceClassification.from_pretrained("taufiqdp/indonesian-sentiment") |
|
|
|
class_names = ['negatif', 'netral', 'positif'] |
|
|
|
text = "Pelayanan lama dan tidak ramah" |
|
tokenized_text = tokenizer(text, return_tensors='pt') |
|
|
|
with torch.inference_mode(): |
|
logits = model(**tokenized_text)['logits'] |
|
|
|
result = class_names[logits.argmax(dim=1)] |
|
print(result) |
|
``` |
|
|
|
### Citation |
|
|
|
``` |
|
@misc{koto2020indolem, |
|
title={IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP}, |
|
author={Fajri Koto and Afshin Rahimi and Jey Han Lau and Timothy Baldwin}, |
|
year={2020}, |
|
eprint={2011.00677}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
@inproceedings{purwarianti2019improving, |
|
title={Improving Bi-LSTM Performance for Indonesian Sentiment Analysis Using Paragraph Vector}, |
|
author={Ayu Purwarianti and Ida Ayu Putu Ari Crisdayanti}, |
|
booktitle={Proceedings of the 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA)}, |
|
pages={1--5}, |
|
year={2019}, |
|
organization={IEEE} |
|
} |
|
``` |
