ARC-AGI
Picture puzzles easy for people but hard for AI, testing whether it can learn a new rule from a few examples.
Definition
The Abstraction and Reasoning Corpus for Artificial General Intelligence, introduced by François Chollet in 2019 to measure fluid intelligence as skill-acquisition efficiency on unfamiliar tasks. Each task is a small visual puzzle: a few examples show a transformation, and the model must apply it to a new input. ARC-AGI-2 (2025) is scored pass@2, every task was solved by at least two people in two attempts or fewer, and results are reported with a cost-efficiency metric. ARC-AGI-3 continues the series.
Example
Three example grids show every blue square turning red and shifting right; the model must produce the correct grid for a fourth, unseen input.