Puzzle reasoning

ARC-AGI-2

Colored-grid puzzles that test learning a new rule fast.

Tasks: 120
Released: Mar 2025
Made by: ARC Prize Foundation
Score: Percent of tasks solved within two tries, reported alongside cost per task

In plain words

Like an IQ-test puzzle: see a few before-and-after picture pairs, work out the hidden rule, then apply it to a new picture.

Example taskSource

Colored rectangles with holes act as a key: each shape gets the color whose key has the same number of holes.

A few example pairs of colored grids, then a new grid. The solver must produce the correct output grid.

The output grid must match the answer exactly. Two attempts are allowed per task.

Humans: Human panel average 60%; every task was solved by at least two people within two tries source

Scores we track

Each number is the provider's own published result. Tap one for its source.

It targets adapting to something truly new, which ordinary people do easily but AI still finds hard.

The 120 count is per evaluation set; there are public, semi-private and private sets of 120 each.
Scores should be read with cost, since spending more computing power can raise results.
The paper reports 66% of human attempts succeeded, versus 60% on the announcement page.