jablonkagroup/ChemBench-Results-dev huggingface dataset:
ChemBench leaderboard entry
Model: Haiyue-Ch1
Description: not provided
Architecture: Transformer decoder-only
Open weights: no
Open dataset: no
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: no
Domain-specific finetuning: no
Tool use: no
Reasoning model: yes
Scores:
Overall: 0.6483158275986961
Analytical Chemistry: 0.6776315789473685
Chemical Preference: 0.56
General Chemistry: 0.8859060402684564
Inorganic Chemistry: 0.8913043478260869
Materials Science: 0.75
Organic Chemistry: 0.8275058275058275
Physical Chemistry: 0.8181818181818182
Technical Chemistry: 0.8
Toxicity and Safety: 0.48592592592592593
ChemBench leaderboard entry
Model: gemini-2.5-flash-preview-04-17
Description: gemini-2.5-flash-preview-04-17
Architecture: not provided
Open weights: no
Open dataset: no
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: no
Domain-specific finetuning: no
Tool use: no
Reasoning model: no
Scores:
Overall: 0.6411362819129809
Analytical Chemistry: 0.618421052631579
Chemical Preference: 0.593
General Chemistry: 0.8523489932885906
Inorganic Chemistry: 0.8369565217391305
Materials Science: 0.75
Organic Chemistry: 0.8251748251748252
Physical Chemistry: 0.7454545454545455
Technical Chemistry: 0.775
Toxicity and Safety: 0.47555555555555556
ChemBench leaderboard entry
Model: ChemPile-LoRA-Ensemble
Description: not provided
Architecture: not provided
Open weights: yes
Open dataset: yes
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: yes
Domain-specific finetuning: no
Tool use: no
Reasoning model: no
Scores:
Overall: 0.4662598707824838
Analytical Chemistry: 0.35526315789473684
Chemical Preference: 0.517
General Chemistry: 0.47651006711409394
Inorganic Chemistry: 0.6304347826086957
Materials Science: 0.4523809523809524
Organic Chemistry: 0.5594405594405595
Physical Chemistry: 0.43636363636363634
Technical Chemistry: 0.575
Toxicity and Safety: 0.3348148148148148
ChemBench leaderboard entry
Model: qwen2.5-instruct
Description: not provided
Architecture: not provided
Open weights: yes
Open dataset: no
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: no
Domain-specific finetuning: no
Tool use: no
Reasoning model: no
Scores:
Overall: 0.4271356783919598
Analytical Chemistry: 0.24342105263157895
Chemical Preference: 0.504
General Chemistry: 0.436241610738255
Inorganic Chemistry: 0.5652173913043478
Materials Science: 0.39285714285714285
Organic Chemistry: 0.5034965034965035
Physical Chemistry: 0.3696969696969697
Technical Chemistry: 0.45
Toxicity and Safety: 0.3022222222222222
ChemBench leaderboard entry
Model: olympiad
Description: test
Architecture: not provided
Open weights: no
Open dataset: no
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: no
Domain-specific finetuning: no
Tool use: no
Reasoning model: no
Scores:
Overall: 0.21954725116780452
Analytical Chemistry: 0.15789473684210525
Chemical Preference: 0.371
General Chemistry: 0.10738255033557047
Inorganic Chemistry: 0.21739130434782608
Materials Science: 0.19047619047619047
Organic Chemistry: 0.18181818181818182
Physical Chemistry: 0.07878787878787878
Technical Chemistry: 0.25
Toxicity and Safety: 0.09333333333333334
ChemBench leaderboard entry
Model: Llama-2-7b-chat-hf 
Description: LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. Llama was released by Meta AI.
Architecture: not provided
Open weights: yes
Open dataset: no
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: no
Domain-specific finetuning: no
Tool use: no
Reasoning model: no
Scores:
Overall: 0.040207522697795074
Analytical Chemistry: 0.006578947368421052
Chemical Preference: 0.003
General Chemistry: 0.020134228187919462
Inorganic Chemistry: 0
Materials Science: 0
Organic Chemistry: 0.02097902097902098
Physical Chemistry: 0.024242424242424242
Technical Chemistry: 0.025
Toxicity and Safety: 0.014814814814814815
ChemBench leaderboard entry
Model: gsm8k
Description: not provided
Architecture: not provided
Open weights: yes
Open dataset: no
Mixture of experts: no
Alignment method: not provided
RLHF: no
Domain-specific pretraining: no
Domain-specific finetuning: no
Tool use: no
Reasoning model: no
Scores:
Overall: 0.005196733481811433
Analytical Chemistry: 0.013157894736842105
Chemical Preference: 0
General Chemistry: 0.013422818791946308
Inorganic Chemistry: 0.043478260869565216
Materials Science: 0
Organic Chemistry: 0.006993006993006993
Physical Chemistry: 0.006060606060606061
Technical Chemistry: 0.025
Toxicity and Safety: 0.0014814814814814814