Zebra Logic Bench - a allenai Collection

allenai 's Collections

PixMo

OLMo 2

Tulu 3 Datasets

Molmo

OLMoE

Tulu V2.5 Suite

Paloma

SciRIFF

AI2 Safety Toolkit

Zebra Logic Bench

ACE

Zebra Logic Bench

updated Nov 27

ZebraLogic Bench: Testing the Limits of LLMs in Logical Reasoning

Running

84

🦓

Zebra Logic Bench
allenai/ZebraLogicBench

Viewer • Updated Jul 11 • 4.26k • 407 • 9

Note 1k puzzles in grid format and 3k in mcqa format. answers are hidden.
allenai/ZebraLogicBench-private

Viewer • Updated Jul 4 • 4.26k • 20.7k • 5

Note answers are included here. to prevent data leakage, plz apply for access.
Faith and Fate: Limits of Transformers on Compositionality

Paper • 2305.18654 • Published May 29, 2023 • 6

Note Our NeurIPS 2023 (spotlight) paper about the data and analysis.