LangSmith's New Insights Agent Automates Analysis of Nearly 1 Billion Daily Events for Enhanced AI Agent Performance

LangSmith, the agent engineering platform developed by LangChain, has launched its new Insights Agent, designed to automatically process and analyze the vast amounts of data generated by AI agents. The platform currently handles nearly one billion events and tens of terabytes of data daily, a scale that makes manual analysis impractical for developers. This new tool aims to provide actionable insights into user interactions and identify critical failure modes, streamlining the development and continuous improvement of AI applications.

The Insights Agent leverages advanced analytics to discover common usage patterns, agent behaviors, and critical failure modes within production traces. It automatically categorizes data hierarchically, allowing users to drill down into specific patterns, subcategories, and individual traces. This automated analysis helps developers understand what users are asking, which tools agents are utilizing, and where suboptimal responses occur, all without requiring manual review of countless traces.

This feature directly addresses the inherent challenges of building reliable, non-deterministic large language model (LLM) systems by transforming raw telemetry data into meaningful intelligence. By automating the identification of critical trends and issues, the Insights Agent enables rapid iteration and focused testing. This ensures that AI agents are not only operational but also highly effective in achieving user goals, providing crucial signals for developers to prioritize necessary improvements.

The introduction of the Insights Agent underscores LangSmith's commitment to providing comprehensive observability for AI agent development. The platform, which has seen its trace volume grow 12x year-over-year, is utilized by a significant portion of the Fortune 500 for agent engineering. The Insights Agent is now generally available for LangSmith Plus and Enterprise cloud customers, integrating seamlessly into existing development workflows.

"LangSmith processes nearly 1 billion events and 10s of TB of data every day! In order to help users sift through all of the data they're sending us, we launched Insights Agent!" stated Ankush Gola, a key contributor to the project. He further elaborated, > "This agent automatically processes your traces in the background and gives you insights into how users are interacting with your agent and failure modes for your agent. Try it out today!" The development was notably supported by team members Bogattor, Anika Somaia, and Maddy Gadams.