eladven commited on
Commit
738af9a
1 Parent(s): 827adae

Evaluation results for adit94/nlpcharade model as a base model for other tasks

Browse files

As part of a research effort to identify high quality models in Huggingface that can serve as base models for further finetuning, we evaluated this by finetuning on 36 datasets. The model ranks 1st among all tested models for the t5-base architecture as of 21/12/2022.


To share this information with others in your model card, please add the following evaluation results to your README.md page.

For more information please see https://ibm.github.io/model-recycling/ or contact me.

Best regards,
Elad Venezian
eladv@il.ibm.com
IBM Research AI

Files changed (1) hide show
  1. README.md +17 -0
README.md ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # adit94/nlpcharade model
2
+ This model is based on t5-base pretrained model.
3
+
4
+
5
+ ## Model Recycling
6
+
7
+ [Evaluation on 36 datasets](https://ibm.github.io/model-recycling/model_gain_chart?avg=2.78&mnli_lp=nan&20_newsgroup=-29.01&ag_news=2.38&amazon_reviews_multi=4.40&anli=1.58&boolq=10.84&cb=-8.92&cola=-2.62&copa=39.82&dbpedia=12.81&esnli=0.60&financial_phrasebank=1.31&imdb=-10.84&isear=26.32&mnli=8.64&mrpc=3.06&multirc=12.08&poem_sentiment=-29.04&qnli=-34.05&qqp=1.74&rotten_tomatoes=-36.72&rte=16.64&sst2=-9.88&sst_5bins=18.68&stsb=-5.99&trec_coarse=-30.77&trec_fine=-0.01&tweet_ev_emoji=47.56&tweet_ev_emotion=10.81&tweet_ev_hate=21.50&tweet_ev_irony=10.21&tweet_ev_offensive=-13.09&tweet_ev_sentiment=16.40&wic=4.61&wnli=0.99&wsc=17.17&yahoo_answers=21.01&model_name=adit94%2Fnlpcharade&base_name=t5-base) using adit94/nlpcharade as a base model yields average score of 78.23 in comparison to 75.45 by t5-base.
8
+
9
+ The model is ranked 1st among all tested models for the t5-base architecture as of 21/12/2022
10
+ Results:
11
+
12
+ | 20_newsgroup | ag_news | amazon_reviews_multi | anli | boolq | cb | cola | copa | dbpedia | esnli | financial_phrasebank | imdb | isear | mnli | mrpc | multirc | poem_sentiment | qnli | qqp | rotten_tomatoes | rte | sst2 | sst_5bins | stsb | trec_coarse | trec_fine | tweet_ev_emoji | tweet_ev_emotion | tweet_ev_hate | tweet_ev_irony | tweet_ev_offensive | tweet_ev_sentiment | wic | wnli | wsc | yahoo_answers |
13
+ |---------------:|----------:|-----------------------:|-------:|--------:|--------:|--------:|--------:|----------:|--------:|-----------------------:|--------:|--------:|--------:|--------:|----------:|-----------------:|-------:|--------:|------------------:|--------:|--------:|------------:|--------:|--------------:|------------:|-----------------:|-------------------:|----------------:|-----------------:|---------------------:|---------------------:|------:|-------:|--------:|----------------:|
14
+ | 56.1086 | 91.8 | 70.9459 | 48.625 | 87.5 | 66.6144 | 79.2905 | 89.4667 | 89.212 | 90.3196 | 86.6151 | 81.4919 | 97.6 | 92.4401 | 88.7255 | 72.3598 | 45.38 | 56.338 | 90.6752 | 51.8855 | 90.3196 | 83.9535 | 74.2347 | 79.3272 | 66.44 | 92.3165 | 92.4401 | 90.3196 | 74.2347 | 83.9535 | 70.9459 | 86.6151 | 71.8 | 56.338 | 77.1667 | 92.596 |
15
+
16
+
17
+ For more information, see: [Model Recycling](https://ibm.github.io/model-recycling/)