OSainz bpHigh commited on
Commit
dc4c3f8
β€’
1 Parent(s): 95be02e

GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (#18)

Browse files

- GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (9588afb347cd07a7645852f6c9b966c5222aa76e)
- convert arxiv pdf link to abs (f6e12a010342efd037641bd2cb947fca4b1a5374)
- Add percentage of validation data contaminated by including db ids on which gpt-3.5 achieves more than 75% DC-accuracy (2a68192028574cf246c6af3b1d053cae7fd82acc)
- Add PR number (326e3ca343d14319e6f0c0930fdd06b7507aaacb)


Co-authored-by: Bhavish Pahwa <bpHigh@users.noreply.huggingface.co>

Files changed (1) hide show
  1. contamination_report.csv +2 -0
contamination_report.csv CHANGED
@@ -664,6 +664,8 @@ wmt/wmt16;fr-en;GPT-3;;model;;;14.0;data-based;https://arxiv.org/abs/2005.14165;
664
  wmt/wmt16;ro-en;FLAN;;model;;;12.4;data-based;https://arxiv.org/abs/2109.01652;13
665
  wmt/wmt16;ro-en;GPT-3;;model;;;21.0;data-based;https://arxiv.org/abs/2005.14165;13
666
 
 
 
667
  xnli;en;EleutherAI/pile;;corpus;;;0.36;data-based;https://arxiv.org/abs/2310.20707;2
668
  xnli;en;allenai/c4;;corpus;;;0.12;data-based;https://arxiv.org/abs/2310.20707;2
669
  xnli;en;oscar-corpus/OSCAR-2301;;corpus;;;0.24;data-based;https://arxiv.org/abs/2310.20707;2
 
664
  wmt/wmt16;ro-en;FLAN;;model;;;12.4;data-based;https://arxiv.org/abs/2109.01652;13
665
  wmt/wmt16;ro-en;GPT-3;;model;;;21.0;data-based;https://arxiv.org/abs/2005.14165;13
666
 
667
+ xlangai/spider;;GPT-3.5;;model;;11.3;;model-based;https://arxiv.org/abs/2402.08100;18
668
+
669
  xnli;en;EleutherAI/pile;;corpus;;;0.36;data-based;https://arxiv.org/abs/2310.20707;2
670
  xnli;en;allenai/c4;;corpus;;;0.12;data-based;https://arxiv.org/abs/2310.20707;2
671
  xnli;en;oscar-corpus/OSCAR-2301;;corpus;;;0.24;data-based;https://arxiv.org/abs/2310.20707;2