Junyi-Shen/Junyi-BERT-sentiment-analysis-IMBD

Task

Binary sentiment classification (positive / negative)

Source Code

• https://github.com/shjy2008/BERT

Training Data

• Stage 1: Large-scale sentiment pre-finetuning

  • Dataset Source: IMDB Dataset of 50K Movie Reviews

• Stage 2: IMDB sentiment fine-tuning

  • Dataset Source: IMDB sentiment (https://github.com/acmi-lab/counterfactually-augmented-data/blob/master/sentiment/new/train.tsv)

Training Method

• Loss function: Binary Cross-Entropy (BCE)
• Weight decay: 0
• Batch size: 8
• Dropout: 0.3
• Learning rate: 2e-5
• Learning rate scheduler: Enabled
• Frozen layers: First 9 transformer layers

Evaluation

• Test dataset: IMDB sentiment (https://github.com/acmi-lab/counterfactually-augmented-data/blob/master/sentiment/new/test.tsv)
• Test accuracy: 98.4%