Formal Reasoning

CSI 4106 Introduction to Artificial Intelligence

Marcel Turcotte

Version: Oct 14, 2025 12:01

Preamble

Quote of the Day

Learning objectives

• Clarify…
• Summarize…
• Discuss..

Julia

User: Julia has two sisters and one brother. How many sisters does her brother Martin have?

(please answer the question)

Source: Mitchell (2025)

Julia

User: Julia has two sisters and one brother. How many sisters does her brother Martin have?

GPT-4o: Martin has two sisters.

Since Julia is one of the two sisters, and Martin is her brother, the other sister is also Martin’s sister. So, Martin has two sisters.

GPT-o3: Martin has two sisters —Julia and the other sister mentioned in the statement.

OpenAI’s o-series models are specifically designed to excel in complex problem-solving tasks, including mathematics, science, logic, and programming. The o3 model is categorized as a reasoning model, optimized for these challenging domains. In contrast, the 4o model serves as a general-purpose model, with a strong emphasis on multimodal capabilities such as audio, visual, and video processing, and is not primarily focused on deep reasoning specialization.

Source: Mitchell (2025)

Julia

User: Julia has two sisters and one brother. How many sisters does her brother Martin have?

GPT-o4-mini-high: Martin has three sisters: Julia plus her two other sisters.

Note

Julia has two sisters and one brother, named Martin. So, there are four children in total: Julia, her two sisters, and her brother Martin. Since Martin is her only brother, and he has three sisters: Julia and her two sisters. The answer is three. Previous responses saying Martin has two sisters were wrong — there are actually three sisters.

Source: Mitchell (2025)

Chess

In this blog, Professor Mathieu Acher of the University of Rennes analyzes how GPT-5 generates illegal moves.

• GPT-5 and GPT-5 Thinking as Other LLMs in Chess: Illegal Move After 4th Turn

Likewise, Euny Hong from IBM recently detailed an experiment in which a 1979 Atari game managed to outperform ChatGPT at chess.

• An Atari game from 1979 “wrecked” ChatGPT in chess. Here’s why it doesn’t really matter

What is going on? Can you explain those results?

To what extent do large language models “know” the rules that govern legal chess gameplay?

Large language models, such as GPT-5, possess the capability to accurately articulate all the necessary and sufficient rules governing legal chess gameplay.

Are there intrinsic limitations within artificial intelligence models that hinder their ability to effectively play chess? It is important to note that this question is intentionally broad, addressing artificial intelligence models as a whole, rather than solely focusing on large language models.

In Silver et al. (2018), the authors demonstrate that AlphaZero achieves superhuman performance in chess using only the basic game rules, without relying on any additional domain-specific knowledge.

By contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go by reinforcement learning from self-play. In this paper, we generalize this approach into a single AlphaZero algorithm that can achieve superhuman performance in many challenging games. Starting from random play and given no domain knowledge except the game rules, AlphaZero convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.

Despite the ability of certain AI models to perform at superhuman levels in chess, language models often struggle against relatively simple computer programs, such as those from Atari 1979. Furthermore, language models frequently generate illegal moves.

Let us pause to consider a important issue. If large language models possess the capability to “know” the rules governing legal chess play yet still often produce illegal moves, can we rely on them for critical applications? Although we might establish a set of fundamental principles, similar to Asimov’s Three Laws of Robotics, that these models should adhere to, there is a high probability that they would still contravene these guidelines.

The limitations discussed here are not inherent to artificial intelligence as a whole but are specifically related to large language models. Why is this case?

Prologue

Mitchell, Melanie. 2025. “Artificial Intelligence Learns to Reason.” Science 387 (6740): eadw5211. https://doi.org/10.1126/science.adw5211.

Russell, Stuart, and Peter Norvig. 2020. Artificial Intelligence: A Modern Approach. 4th ed. Pearson. http://aima.cs.berkeley.edu/.

Silver, David, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, et al. 2018. “A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play.” Science (New York, NY) 362 (6419): 1140–44. https://doi.org/10.1126/science.aar6404.