ARC Prize has unveiled its latest benchmark, ARC-AGI-2, setting a new standard for evaluating artificial general intelligence (AGI). Accompanying this launch is the announcement of a 2025 competition hosted on Kaggle, offering a total of $1 million in prizes. This initiative aims to push the boundaries of AI capabilities by identifying and addressing existing gaps in adaptive intelligence.
Since its inception in 2019, ARC Prize has consistently guided researchers toward AGI by developing robust benchmarks. The introduction of ARC-AGI-2 marks a significant advancement from its predecessor, ARC-AGI-1, which focused on measuring fluid intelligence. This new benchmark not only continues to assess AI’s ability to adapt to new tasks but also places a stronger emphasis on efficiency and resource management.
What Makes ARC-AGI-2 More Challenging?
ARC-AGI-2 presents tasks that are easily solvable by humans but remain difficult for current AI systems. This benchmark emphasizes symbolic interpretation, compositional reasoning, and contextual rule application, areas where AI still struggles. According to the ARC Prize team,
“Good AGI benchmarks act as useful progress indicators. Better AGI benchmarks clearly discern capabilities. The best AGI benchmarks do all this and actively inspire research and guide innovation.”
This approach ensures that the benchmark not only tests AI’s problem-solving abilities but also its efficiency in using resources.
How Does ARC-AGI-2 Compare to Previous Benchmarks?
Unlike many existing benchmarks that focus on superhuman capabilities and specialized skills, ARC-AGI-2 highlights the adaptability that characterizes human intelligence. While previous benchmarks often rewarded memorization and rote learning, ARC-AGI-2 challenges AI to apply multiple interacting rules and adapt to complex contexts, areas where human intelligence excels.
What are the Competition’s Incentives?
The 2025 competition on Kaggle offers substantial rewards to encourage innovation. The grand prize of $700,000 is awarded to those who achieve 85% success within specified efficiency limits. Additional prizes include $75,000 for the top-scoring submission, $50,000 for transformative research papers, and $175,000 in other categories. These incentives are designed to foster collaboration and drive significant advancements in AGI research.
ARC-AGI-2 also focuses on measuring efficiency, ensuring that AI systems not only solve tasks but do so with minimal resources. For instance, human participants solve tasks with 100% accuracy at $17 per task, whereas AI systems like OpenAI’s o3 achieve only a 4% success rate at $200 per task. This discrepancy highlights the ongoing need for more efficient AI solutions.
As ARC Prize continues to set high standards for AGI benchmarks, the organization remains committed to identifying and inspiring novel approaches in AI research. The 2025 competition is expected to attract a diverse range of participants, potentially leading to breakthroughs that current tech giants may not achieve on their own.
ARC-AGI-2 represents a crucial step towards achieving AGI, emphasizing both the capability to solve complex tasks and the efficiency of solutions. By fostering a competitive and collaborative environment, ARC Prize aims to accelerate the development of truly adaptive and resource-efficient AI systems.
