SUCCESS OF AI MATH SOLVER TOOL IN SOLVING NON-STANDARD MATHEMATICS COMPETITION PROBLEMS

Abstract

Artificial intelligence is increasingly transforming how students learn, including their approach to mathematics and problem-solving, by offering additional support and assistance—a trend that continues to attract research interest. One line of research focuses on helping students prepare for math competitions by solving more complex mathematical problems. In addition to regular national math competitions, which allow students to progress to international mathematical Olympiads, there are also competitions aimed at popularizing mathematics and developing logical thinking in students. One such competition is the international Kangaroo competition. In this paper, we examine the performance of the AI Math Solver, available on the Interactive Mathematics platform, in solving tasks from the 2024 Kangaroo competition. The selected tasks targeted three student categories: 3rd and 4th grade elementary, 7th and 8th grade elementary, and 3rd and 4th grade high school students. The problems were uploaded as images (screenshots) in both Serbian and English, since visual elements frequently appear in the problem formulations and answer choices in the Kangaroo competition. The results are presented in two sections: a qualitative analysis of selected problems that illustrate common patterns and errors, and a quantitative analysis that summarizes the tool’s overall performance. Out of a total of 84 tasks, in both Serbian and English, the solver correctly answered 24, corresponding to a success rate of just under 30% in both languages. Furthermore, some tasks solved in Serbian were not solved in English, and vice versa. Additionally, differences were observed in the distribution of correct answers across tasks of varying difficulty levels.