New AI Model Achieves 48.4% Accuracy in Machine Theory of Mind, Outperforming LLMs Alone

Toronto, Canada – A new research paper introduces a hybrid artificial intelligence model, dubbed Large Language Model-Augmented Inverse Planning (LAIP), which significantly advances machine Theory of Mind (ToM) capabilities. The study, titled "Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning," was authored by Rebekah A. Gelpi, Eric Xue, and William A. Cunningham from the University of Toronto, Vector Institute, and Schwartz Reisman Institute of Technology & Society. The paper, which was recently made public on July 4, 2025, proposes a novel approach to enable AI systems to better infer the mental states of others.

The LAIP model addresses critical limitations found in existing AI approaches to Theory of Mind. While large language models (LLMs) have shown promise in ToM benchmarks, they often exhibit brittleness and can fail on reasoning tasks. Conversely, traditional Bayesian inverse planning models, though accurate in predicting human reasoning on ToM tasks, are constrained by their inability to scale to scenarios with numerous hypotheses and actions.

The researchers' hybrid solution leverages the strengths of both methodologies. LAIP utilizes LLMs to generate hypotheses and likelihood functions, which are then integrated with a Bayesian inverse planning model to compute posterior probabilities for an agent's likely mental states given its actions. This combination allows the system to overcome the scalability issues of traditional Bayesian models while mitigating the brittleness observed in LLM-only approaches.

In experimental evaluations, the LAIP model demonstrated superior performance. For instance, in a task requiring the inference of an agent's food preferences, LAIP achieved a posterior probability of 48.4% for the correct hypothesis when a specific condition was met. This significantly outpaced other methods, including zero-shot chain-of-thought prompting (11.9%), ReAct (3.7%), Reflexion (0.3%), and a zero-shot baseline (1.2%). The model's effectiveness was also observed even when using smaller LLMs that typically perform poorly on ToM tasks.

The findings suggest a promising direction for the development of more socially intelligent generative agents. By enabling AI to predict mental states in open-ended scenarios, LAIP could contribute to more robust and human-aligned AI systems capable of understanding and navigating complex social interactions. The paper's release marks a notable step forward in the ongoing quest to imbue machines with a deeper understanding of human cognition.