5CD-AI
/

visobert-14gb-corpus

Inference Endpoints

Model card Files Files and versions Community

htdung167 commited on Apr 19

Commit

9162bbc

•

1 Parent(s): 5961ad0

Update README.md

Files changed (1) hide show

README.md +5 -2

README.md CHANGED Viewed

@@ -3,10 +3,13 @@ library_name: transformers
 tags: []
 ---
-# 5CD-AI/viso-twhin-bert-large
 ## Overview
 <!-- Provide a quick summary of what the model is/does. -->
-We reduce TwHIN-BERT's vocabulary size to 20k on the UIT dataset and continue pretraining for 10 epochs.
 Here are the results on 4 downstream tasks on Vietnamese social media texts, including Emotion Recognition(UIT-VSMEC), Hate Speech Detection(UIT-HSD), Spam Reviews Detection(ViSpamReviews), Hate Speech Spans Detection(ViHOS):
 <table>

 tags: []
 ---
+# 5CD-AI/visobert-14gb-corpus-pretrained
 ## Overview
 <!-- Provide a quick summary of what the model is/does. -->
+We continually pretrain `uitnlp/visobert` on a merged 14GB dataset for 5 epochs, the training dataset includes:
+- Internal data (100M comments and 15M posts on Facebook)
+- UIT data, which is used to pretrain `uitnlp/visobert`
+- MC4 ecommerce
 Here are the results on 4 downstream tasks on Vietnamese social media texts, including Emotion Recognition(UIT-VSMEC), Hate Speech Detection(UIT-HSD), Spam Reviews Detection(ViSpamReviews), Hate Speech Spans Detection(ViHOS):
 <table>