NVIDIA Research has published a new paper asserting that Small Language Models (SLMs) are poised to become the dominant force in agentic AI systems, challenging the current reliance on larger models. The research indicates that SLMs are sufficiently powerful and more economical for a significant portion of AI agent tasks, potentially handling 60-80% of enterprise AI agent tasks currently assigned to much larger models. This perspective was highlighted by Rohan Paul on social media, who noted, > "Small Language Models (SLMs) not LLMs are the real future of agentic AI."
The paper, titled "Small Language Models are the Future of Agentic AI," argues that SLMs can match the performance of Large Language Models (LLMs) for many routine agent tasks while offering substantial cost reductions. According to NVIDIA's findings, SLMs can deliver 10-30 times greater efficiency in terms of computational operations (FLOPs) and energy usage compared to frontier LLMs. This efficiency allows SLMs to run locally on devices like laptops, making them inherently more suitable for real-time and on-device inference.
Agentic AI systems are designed to perform specialized tasks repetitively and with little variation, often making goal-driven decisions rather than following fixed rules. NVIDIA's research suggests that because these chores rarely require the open-ended conversational abilities of LLMs, an SLM-first design is the natural default. Rohan Paul's tweet emphasized this point, stating that the paper provides > "a recipe for swapping out the large models with SLMs without breaking anything and show that 40%-70% of calls in open agents could switch today."
The shift towards SLMs in agentic AI carries significant economic and operational implications for the industry. Organizations adopting SLMs for agentic applications can expect reduced latency, lower energy consumption, and decreased infrastructure costs. This challenges the substantial investments made in LLM infrastructure, proposing a more sustainable and cost-effective approach to deploying intelligent systems.
This research marks a pivotal moment in the AI landscape, advocating for a paradigm shift from monolithic LLMs to specialized SLMs for specific agentic workloads. NVIDIA's position aims to stimulate discussion on the effective use of AI resources and advance efforts to lower the overall costs of AI deployment. The paper encourages contributions and critiques from the research community, signaling a collaborative approach to shaping the future of AI.