Zebra Logic Bench - a allenai Collection

allenai 's Collections

OLMoE

Tulu V2.5 Suite

Paloma

SciRIFF

AI2 Safety Toolkit

Zebra Logic Bench

Zebra Logic Bench

updated about 2 hours ago

ZebraLogic Bench: Testing the Limits of LLMs in Logical Reasoning

Running

66

🦓

Zebra Logic Bench
allenai/ZebraLogicBench

Viewer • Updated Jul 11 • 4.26k • 718 • 4

Note 1k puzzles in grid format and 3k in mcqa format. answers are hidden.
allenai/ZebraLogicBench-private

Viewer • Updated Jul 4 • 4.26k • 628 • 2

Note answers are included here. to prevent data leakage, plz apply for access.
Faith and Fate: Limits of Transformers on Compositionality

Paper • 2305.18654 • Published May 29, 2023 • 6

Note Our NeurIPS 2023 (spotlight) paper about the data and analysis.