olmo-eval: An evaluation workbench for the model development loop
• 16
AI & ML interests
Building breatkthrough AI to solve the world's biggest problems.
Recent Activity
Papers
EMO: Pretraining Mixture of Experts for Emergent Modularity
MolmoAct2: Action Reasoning Models for Real-world Deployment
Articles
Organization Card
Collection of the evaluation rollouts for MolmoAct2 conducted by Cortex AI
-
MolmoAct2: Action Reasoning Models for Real-world Deployment
Paper • 2605.02881 • Published • 350 -
allenai/eval_molmoact_candy_sorting_in-distribution
Viewer • Updated • 59.6k • 462 -
allenai/eval_molmoact_cup_stacking_in-distribution
Viewer • Updated • 32k • 433 -
allenai/eval_molmoact_cup_storing_in-distribution
Viewer • Updated • 45.4k • 473
Collection of the evaluation rollouts for MolmoAct2 conducted by Cortex AI
-
MolmoAct2: Action Reasoning Models for Real-world Deployment
Paper • 2605.02881 • Published • 350 -
allenai/eval_molmoact_candy_sorting_in-distribution
Viewer • Updated • 59.6k • 462 -
allenai/eval_molmoact_cup_stacking_in-distribution
Viewer • Updated • 32k • 433 -
allenai/eval_molmoact_cup_storing_in-distribution
Viewer • Updated • 45.4k • 473
Collection of the embodied reasoning datasets for MolmoAct2
spaces 19
pinned
Running
27
AstaBench Leaderboard
🥇
View benchmark leaderboards and performance tables
pinned
Running on Zero
MCP
Featured
30
Molmo-Point Demo
🎯
MolmoPoint - Image & Video Pointing & Tracking
pinned
Running on Zero
MCP
5
MolmoPoint-GUI Demo
🖥
MolmoPoint - GUI Image Pointing
pinned
Running
Agents
432
Reward Bench Leaderboard
📐
Explore and compare model scores on RewardBench benchmarks
pinned
Runtime error
Agents
3
HREF Leaderboard
📐
Browse and search HREF leaderboard data
pinned
Running
Agents
94
Zebra Logic Bench
🦓
Display model leaderboard and explore sample puzzles
models 949
allenai/Bolmo-7B
Text Generation • 8B • Updated • 360 • 57
allenai/Bolmo-1B
Text Generation • 1B • Updated • 2.53k • 49
allenai/H_pre_LQK_32kv_8k_11k_SWA
Updated • 35 • 2
allenai/G_pre_8kv_8k_14k_SWA
Updated • 66 • 1
allenai/H_post_LQK_8kv_4k_11k_SWA
Updated • 36
allenai/G_post_LQK_8kv_4k_14k_SWA
Updated • 36
allenai/B_post_LQK_32kv_4k_11k
Updated • 37
allenai/K_post_HQK_8kv_12k
Updated • 38
allenai/H_post_LQK_32kv_8k_11k_SWA
Updated • 38
allenai/A_post_HQK_8kv_8k_13k_SWA_fp8
Updated • 49
datasets 1,258
allenai/asta-bench-results
Viewer • Updated • 88 • 157
allenai/asta-bench-submissions
Updated • 62 • 1
allenai/molmobot-data
Viewer • Updated • 324k • 37.2k • 7
allenai/asta-summary-citation-counts
Viewer • Updated • 81.8M • 1.41k • 9
allenai/olmoearth-paper-embeddings
Updated • 13.1k • 6
allenai/MolmoAct2-BimanualYAM-Dataset
Viewer • Updated • 76M • 9.24k • 2
allenai/molmospaces
Viewer • Updated • 1M • 21.1k • 47
allenai/eval_pi05_pipette_ood
Viewer • Updated • 9.27k • 437
allenai/eval_pi05_cup_stacking_ood
Viewer • Updated • 14k • 504
allenai/eval_cosmos_cup_stacking_ood
Viewer • Updated • 16.8k • 502