ChatGPT AI-Language Model ‘Excels’ In SAT Test, Comparable To High-Scoring University Applicants

New Delhi: A recent study conducted by researchers revealed that ChatGPT , an AI language model, demonstrates problem-solving skills as good as, if not better than, college students. GPT-3, the current version of ChatGPT, was put to the test with reasoning problems commonly found in intelligence assessments like the SAT . The SAT, a widely-used standardised test for college admissions in the United States, assesses a student's proficiency in reading, writing, and math, providing scores on a scale of 400 to 1600, with the average score being 1060.

The study's findings shed light on the remarkable capabilities of GPT-3, as it performed impressively well, accurately solving 80 percent of the problems presented. Surprisingly, the AI model outperformed the average scores achieved by 40 college students from UCLA , showcasing its potential as a problem-solving tool.

To challenge the AI's reasoning skills further, the researchers devised complex shape arrangement puzzles, transformed into text for GPT-3 to comprehend. Even with novel questions, GPT-3 excelled, managing to surpass human college applicants' performance on SAT analogy questions.

Despite these remarkable feats, GPT-3 faced certain limitations. When tested on matching a passage of prose with a different short storey conveying the same meaning, the AI model did not perform as well as the human students. However, the study indicated that an upgraded version, GPT-4, displayed improved results in this area.

Lead author of the study, Taylor Webb , emphasised that ChatGPT's AI falls short of achieving human-level intelligence or artificial general intelligence. It struggled with tasks involving social interactions, math reasoning, and problems requiring an understanding of physical space.

The study has sparked curiosity among researchers, as they endeavour to comprehend the inner workings of GPT-3's reasoning abilities. The distinctive training method of GPT-3 compared to how humans learn has raised intriguing questions about the nature of its cognitive processes. The pursuit of understanding whether GPT-3 represents a form of genuine artificial intelligence remains an exciting and groundbreaking quest.