SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 91: Timestamp, duration, and emissions per experiment with non-API models. (1/4)
model	timestamp	duration
EleutherAI-125m-0-shot	2022-09-01T21:33:14	8549.649220
EleutherAI-125m-1-shot	2022-09-02T00:10:03	640.861120
EleutherAI-125m-5-shot	2022-09-02T00:26:27	982.369876
EleutherAI-125m-10-shot	2022-09-02T00:51:24	1495.525381
EleutherAI-125m-15-shot	2022-09-02T01:29:03	2257.290708
EleutherAI-125m-30-shot	2022-09-02T09:04:03	27298.375266
EleutherAI-2.7b-0-shot	2022-09-03T00:36:14	3752.897449
EleutherAI-2.7b-1-shot	2022-09-03T02:04:16	5279.884696
EleutherAI-2.7b-5-shot	2022-09-03T04:28:19	8641.654516
EleutherAI-2.7b-10-shot	2022-09-03T08:18:13	13792.592126
EleutherAI-2.7b-15-shot	2022-09-03T13:33:25	18909.551123
EleutherAI-2.7b-30-shot	2022-09-03T22:47:06	33219.682098
EleutherAI-20b-0-shot	2022-08-25T07:40:55	1378.197924
EleutherAI-20b-1-shot	2022-08-25T08:15:23	807.702344
EleutherAI-20b-5-shot	2022-08-25T15:39:51	859.585535
EleutherAI-20b-10-shot	2022-08-25T16:18:50	1175.128651
EleutherAI-20b-15-shot	2022-08-25T16:47:30	1713.266182
EleutherAI-20b-30-shot	2022-08-25T17:45:28	3469.811664
EleutherAI-6b-0-shot	2022-08-24T22:29:30	1287.627453
EleutherAI-6b-1-shot	2022-08-24T23:22:30	1831.554774
EleutherAI-6b-5-shot	2022-08-25T00:16:57	3255.128955
EleutherAI-6b-10-shot	2022-08-25T01:23:21	3971.650578
EleutherAI-6b-15-shot	2022-08-25T02:26:23	3772.113814
EleutherAI-6b-30-shot	2022-08-25T04:18:30	6719.419030
EleutherAI-1.3b-0-shot	2022-09-02T09:54:06	3000.666020
EleutherAI-1.3b-1-shot	2022-09-02T10:46:30	3142.207699
EleutherAI-1.3b-5-shot	2022-09-02T12:25:25	5933.046596
EleutherAI-1.3b-10-shot	2022-09-02T12:39:00	8509.257493
EleutherAI-1.3b-15-shot	2022-09-02T18:00:39	11615.289366
EleutherAI-1.3b-30-shot	2022-09-02T23:33:39	19978.306457
M Compute and Emissions
Find below in Table 91 until Table 94 the timestamps, durations, and emissions per experiment (calculated with the CodeCarbon library in Python). Find below in Table 95 until Table 98 the cpu-type and count and gpu-type and count per experiment. In terms of compute the following GPU hours can be estimated if we assume each run is entirely done on the GPU (which is not true in reality, but worst case):
NVIDIA A100-SXM4-40GB used for 926.42913921515 hours.
Tesla V100-PCIE-32GB used for 29.282544113265143 hours.
Tesla V100-PCIE-16GB used for 11.462701331244574 hours.
Table 92: Timestamp, duration, and emissions per experiment with non-API models. (2/4)
model	timestamp	duration
BLOOM-3b-0-shot	2022-08-31T12:54:37	5178.369790
BLOOM-3b-1-shot	2022-08-31T14:39:32	6292.560350
BLOOM-3b-5-shot	2022-08-31T17:37:29	10675.230701
BLOOM-3b-10-shot	2022-08-31T21:59:27	15715.744792
BLOOM-3b-15-shot	2022-09-01T03:41:02	20492.823278
BLOOM-3b-30-shot	2022-09-01T15:47:21	43577.882397
BLOOM-7b1-0-shot	2022-08-25T04:56:35	625.931470
BLOOM-7b1-1-shot	2022-08-25T05:07:13	630.628939
BLOOM-7b1-5-shot	2022-08-25T05:24:22	1022.138932
BLOOM-7b1-10-shot	2022-08-25T05:49:00	1471.008220
BLOOM-7b1-15-shot	2022-08-25T06:23:26	2058.455127
BLOOM-7b1-30-shot	2022-08-25T07:29:46	3972.772039
BLOOM-560m-0-shot	2022-08-29T15:35:52	2541.248956
BLOOM-560m-1-shot	2022-08-29T18:52:16	2532.794568
BLOOM-560m-5-shot	2022-08-29T20:16:16	5038.547060
BLOOM-560m-10-shot	2022-08-29T22:17:43	7285.239875
BLOOM-560m-15-shot	2022-08-30T00:38:23	8438.096533
BLOOM-560m-30-shot	2022-08-30T04:38:44	14419.447170
BLOOM-1b1-0-shot	2022-08-30T05:18:44	2398.828856
BLOOM-1b1-1-shot	2022-08-30T06:06:45	2879.435828
BLOOM-1b1-5-shot	2022-08-30T07:35:59	5352.607075
BLOOM-1b1-10-shot	2022-08-30T10:15:02	9541.535419
BLOOM-1b1-15-shot	2022-08-30T13:22:42	11257.077128
BLOOM-1b1-30-shot	2022-08-30T18:08:15	17131.797610
BLOOM-176b-0-shot	2022-10-14T12:51:11	3015.240235
BLOOM-176b-1-shot	2022-10-14T13:57:53	3906.461752
BLOOM-176b-5-shot	2022-10-14T20:41:10	7411.725385
BLOOM-176b-10-shot	2022-10-23T21:43:21	14462.201855
BLOOM-176b-15-shot	2022-10-24T01:14:10	12609.026736
BLOOM-176b-30-shot	2022-10-14T20:47:02	33159.499966
Table 93: Timestamp, duration, and emissions per experiment with non-API models. (3/4)
model	timestamp	duration
OPT-13b-0-shot	2022-08-25T07:07:08	878.202579
OPT-13b-1-shot	2022-08-25T07:31:30	458.133617
OPT-13b-5-shot	2022-08-25T07:37:39	578.308507
OPT-13b-10-shot	2022-08-25T08:01:50	821.158826
OPT-13b-15-shot	2022-08-25T08:20:49	1131.479665
OPT-13b-30-shot	2022-08-25T16:05:27	2235.869414
OPT-350m-0-shot	2022-09-16T17:26:28	389.173905
OPT-350m-1-shot	2022-09-16T17:33:42	424.832551
OPT-350m-5-shot	2022-09-16T18:00:14	1583.824094
OPT-350m-10-shot	2022-09-16T18:32:12	1908.822462
OPT-350m-15-shot	2022-09-16T19:03:23	1863.625027
OPT-350m-30-shot	2022-09-16T19:47:29	2637.811867
OPT-125m-0-shot	2022-09-16T15:15:56	273.178967
OPT-125m-1-shot	2022-09-16T15:20:28	259.680856
OPT-125m-5-shot	2022-09-16T15:41:37	1259.801105
OPT-125m-10-shot	2022-09-16T16:09:59	1693.598805
OPT-125m-15-shot	2022-09-16T16:41:46	1899.415318
OPT-125m-30-shot	2022-09-16T17:19:51	2276.441314
OPT-6.7b-0-shot	2022-08-24T23:03:07	1140.485014
OPT-6.7b-1-shot	2022-08-24T23:17:51	872.225225
OPT-6.7b-5-shot	2022-08-24T23:34:40	995.894396
OPT-6.7b-10-shot	2022-08-24T23:55:44	1252.956499
OPT-6.7b-15-shot	2022-08-25T00:23:04	1627.749039
OPT-6.7b-30-shot	2022-08-25T01:05:49	2553.054289
OPT-2.7b-0-shot	2022-09-18T16:35:05	686.197892
OPT-2.7b-1-shot	2022-09-18T16:45:11	593.508211
OPT-2.7b-5-shot	2022-09-18T17:12:11	1613.313387
OPT-2.7b-10-shot	2022-09-18T17:44:48	1949.808232
OPT-2.7b-15-shot	2022-09-18T18:22:02	2225.927837
OPT-2.7b-30-shot	2022-09-18T19:09:05	2815.327871
OPT-30b-0-shot	2022-08-25T19:03:37	591.665447
OPT-30b-1-shot	2022-08-25T19:14:32	645.923823
OPT-30b-5-shot	2022-08-25T16:44:22	1825.821606
OPT-30b-10-shot	2022-08-25T17:07:22	1372.752916
OPT-30b-15-shot	2022-08-25T17:41:05	2015.006104
OPT-30b-30-shot	2022-08-25T18:10:39	3859.078056
OPT-1.3b-0-shot	2022-09-17T17:53:50	595.193443
OPT-1.3b-1-shot	2022-09-17T18:03:45	579.367790
OPT-1.3b-5-shot	2022-09-17T18:33:18	1759.103432
OPT-1.3b-10-shot	2022-09-17T19:12:19	2327.300123
OPT-1.3b-15-shot	2022-09-17T19:48:32	2161.637401
OPT-1.3b-30-shot	2022-09-17T20:37:00	2893.829010
OPT-175b-0-shot	2022-10-19T15:02:56	2387.104187
OPT-175b-1-shot	2022-10-19T16:34:06	1589.972279
OPT-175b-5-shot	2022-10-19T17:25:58	3072.591171
OPT-175b-10-shot	2022-10-19T17:33:15	6211.692086
OPT-175b-15-shot	2022-10-19T21:29:16	8019.585246
OPT-175b-30-shot	2022-10-19T21:36:53	19901.470347
OPT-66b-0-shot	2022-08-25T18:58:11	2834.901372
OPT-66b-1-shot	2022-08-25T19:22:09	1427.806986
OPT-66b-5-shot	2022-08-25T19:47:39	1521.168440
OPT-66b-10-shot	2022-08-25T20:24:56	2228.407874
OPT-66b-15-shot	2022-08-25T21:41:21	3370.689256
OPT-66b-30-shot	2022-08-26T00:31:36	6816.312183
Table 94: Timestamp, duration, and emissions per experiment with non-API models. (4/4)
model	timestamp	duration
BlenderBot-2.7b-0-shot	2022-09-04T08:09:56	3656.381540
BlenderBot-2.7b-1-shot	2022-09-12T15:58:01	4051.858183
BlenderBot-2.7b-5-shot	2022-09-12T17:16:20	4696.628979
BlenderBot-2.7b-10-shot	2022-09-12T18:35:53	4772.083818
BlenderBot-2.7b-15-shot	2022-09-12T19:54:13	4698.638356
BlenderBot-2.7b-30-shot	2022-09-12T21:10:34	4579.460884
BlenderBot-9.4b-0-shot	2022-10-22T04:04:24	614.201131
BlenderBot-9.4b-1-shot	2022-10-22T17:17:21	659.975971
BlenderBot-9.4b-5-shot	2022-10-22T17:31:48	839.336277
BlenderBot-9.4b-10-shot	2022-10-22T17:46:18	843.852691
BlenderBot-9.4b-15-shot	2022-10-22T17:53:41	1262.038660
BlenderBot-9.4b-30-shot	2022-10-22T18:23:25	853.334728
BlenderBot-90m-0-shot	2022-09-14T15:11:44	273.134700
BlenderBot-90m-1-shot	2022-09-14T15:17:38	351.542638
BlenderBot-90m-5-shot	2022-09-14T15:29:50	730.774348
BlenderBot-90m-10-shot	2022-09-14T15:47:22	1050.647882
BlenderBot-90m-15-shot	2022-09-14T16:07:27	1204.079804
BlenderBot-90m-30-shot	2022-09-14T16:28:55	1285.913686
T0-3b-0-shot	2022-10-21T17:33:36	348.245298
T0-3b-1-shot	2022-10-24T23:20:57	350.730799
T0-3b-5-shot	2022-10-24T23:29:21	474.378557
T0-3b-10-shot	2022-10-25T15:56:54	676.111759
T0-3b-15-shot	2022-10-25T16:12:55	928.215524
T0-3b-30-shot	2022-10-24T23:30:17	1961.897054
T0-11b-0-shot	2022-10-21T15:38:13	2289.815276
T0-11b-1-shot	2022-10-22T19:18:25	814.872760
T0-11b-5-shot	2022-10-22T19:41:45	1368.644314
T0-11b-10-shot	2022-10-22T20:17:30	2112.628515
T0-11b-15-shot	2022-10-22T21:06:30	2904.655213
T0-11b-30-shot	2022-10-22T22:41:16	5648.105648
Flan-T5-3b-0-shot	2022-10-24T11:20:36	617.820384
Flan-T5-3b-1-shot	2022-10-25T12:29:59	348.405589
Flan-T5-3b-5-shot	2022-10-25T12:38:24	474.872964
Flan-T5-3b-10-shot	2022-10-25T12:50:00	665.592482
Flan-T5-3b-15-shot	2022-10-25T13:05:34	902.197151
Flan-T5-3b-30-shot	2022-10-25T13:37:14	1864.885266
Flan-T5-780m-0-shot	2022-10-24T11:54:09	160.503411
Flan-T5-780m-1-shot	2022-10-25T14:41:28	3816.321305
Flan-T5-780m-5-shot	2022-10-25T14:46:09	251.699700
Flan-T5-780m-10-shot	2022-10-25T14:52:09	331.340966
Flan-T5-780m-15-shot	2022-10-25T14:59:00	381.107934
Flan-T5-780m-30-shot	2022-10-25T15:11:18	705.711192
Flan-T5-11b-0-shot	2022-10-24T10:25:09	1111.283857
Flan-T5-11b-1-shot	2022-10-24T10:56:52	654.411412
Flan-T5-11b-5-shot	2022-10-25T17:26:50	1403.159768
Flan-T5-11b-10-shot	2022-10-25T18:29:59	3756.529085
Flan-T5-11b-15-shot	2022-10-25T19:21:15	3042.271478
Flan-T5-11b-30-shot	2022-10-25T20:57:13	5722.244579
Table 95: Compute used per experiment with non-API models. (1/4)
model	cpus	cpu model	gpu model
EleutherAI-125m-0-shot	10	Apple M1 Max
EleutherAI-125m-1-shot	10	Apple M1 Max
EleutherAI-125m-5-shot	10	Apple M1 Max
EleutherAI-125m-10-shot	10	Apple M1 Max
EleutherAI-125m-15-shot	10	Apple M1 Max
EleutherAI-125m-30-shot	10	Apple M1 Max
EleutherAI-2.7b-0-shot	10	Apple M1 Max
EleutherAI-2.7b-1-shot	10	Apple M1 Max
EleutherAI-2.7b-5-shot	10	Apple M1 Max
EleutherAI-2.7b-10-shot	10	Apple M1 Max
EleutherAI-2.7b-15-shot	10	Apple M1 Max
EleutherAI-2.7b-30-shot	10	Apple M1 Max
EleutherAI-20b-0-shot	48	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-20b-1-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-20b-5-shot	48	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-20b-10-shot	48	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-20b-15-shot	48	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-20b-30-shot	48	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-6b-0-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-6b-1-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-6b-5-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-6b-10-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-6b-15-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-6b-30-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
EleutherAI-1.3b-0-shot	10	Apple M1 Max
EleutherAI-1.3b-1-shot	10	Apple M1 Max
EleutherAI-1.3b-5-shot	10	Apple M1 Max
EleutherAI-1.3b-10-shot	10	Apple M1 Max
EleutherAI-1.3b-15-shot	10	Apple M1 Max
EleutherAI-1.3b-30-shot	10	Apple M1 Max
Table 96: Compute used per experiment with non-API models. (2/4)
model	cpus	cpu model	gpu model
BLOOM-3b-0-shot	10	Apple M1 Max
BLOOM-3b-1-shot	10	Apple M1 Max
BLOOM-3b-5-shot	10	Apple M1 Max
BLOOM-3b-10-shot	10	Apple M1 Max
BLOOM-3b-15-shot	10	Apple M1 Max
BLOOM-3b-30-shot	10	Apple M1 Max
BLOOM-7b1-0-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
BLOOM-7b1-1-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
BLOOM-7b1-5-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
BLOOM-7b1-10-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
BLOOM-7b1-15-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
BLOOM-7b1-30-shot	1	Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz	8 x NVIDIA A100-40GB
BLOOM-560m-0-shot	10	Apple M1 Max
BLOOM-560m-1-shot	10	Apple M1 Max
BLOOM-560m-5-shot	10	Apple M1 Max
BLOOM-560m-10-shot	10	Apple M1 Max
BLOOM-560m-15-shot	10	Apple M1 Max
BLOOM-560m-30-shot	10	Apple M1 Max
BLOOM-1b1-0-shot	10	Apple M1 Max
BLOOM-1b1-1-shot	10	Apple M1 Max
BLOOM-1b1-5-shot	10	Apple M1 Max
BLOOM-1b1-10-shot	10	Apple M1 Max
BLOOM-1b1-15-shot	10	Apple M1 Max
BLOOM-1b1-30-shot	10	Apple M1 Max
BLOOM-176b-0-shot	96	Intel(R) Xeon(R) CPU @ 2.20GHz	16 x NVIDIA A100-40GB
BLOOM-176b-1-shot	96	Intel(R) Xeon(R) CPU @ 2.20GHz	16 x NVIDIA A100-40GB
BLOOM-176b-5-shot	96	Intel(R) Xeon(R) CPU @ 2.20GHz	16 x NVIDIA A100-40GB
BLOOM-176b-10-shot	96	Intel(R) Xeon(R) CPU @ 2.20GHz	16 x NVIDIA A100-40GB
BLOOM-176b-15-shot	96	Intel(R) Xeon(R) CPU @ 2.20GHz	16 x NVIDIA A100-40GB
BLOOM-176b-30-shot	96	Intel(R) Xeon(R) CPU @ 2.20GHz	16 x NVIDIA A100-40GB