Crosby Co-founder John Sarihan Advocates for 'Quality Tsars' in Vertical AI

John Sarihan, Co-founder and CTO of the recently launched AI-first law firm Crosby, emphasized the critical need for dedicated quality and AI environment teams within vertical AI businesses. In a recent social media post, Sarihan stated, "Every vertical AI business should have a team dedicated to Quality & AI environments. Just like open source projects thrive with a BDFL, you need a 'Quality Tsar' as the ultimate tastemaker for evals, RL environments, and beyond." He noted that this concept was explored on the "Training Data podcast."

Crosby, which recently secured $5.8 million in seed funding from Sequoia Capital and Bain Capital Ventures, operates by automating legal contract negotiations for businesses. The firm combines artificial intelligence with human legal expertise, aiming to significantly increase "deal velocity" for its clients. Crosby's model focuses on contracts like NDAs, MSAs, and DPAs, boasting a median review time of under an hour, a stark contrast to traditional legal processes.

Sarihan's advocacy for a "Quality Tsar" stems from the challenges of achieving high accuracy in AI applications. He highlighted that while foundational models can reach "90 percent for basically free," pushing accuracy to "99 or 99.99 is actually extremely difficult." Crosby addresses this by integrating domain experts—lawyers—directly with engineers, fostering a unique feedback loop that goes beyond standard evaluations. This collaboration ensures that the AI models are fine-tuned to specific customer needs and legal nuances.

The company measures its success through metrics like "Total Turnaround Time" (TTAT) and "Human Review Time" (HuRT), aiming to minimize both. This data-driven approach, combined with the "lawyers-in-the-loop" framework, allows Crosby to maintain high standards of quality and mitigate risks inherent in AI-driven legal services. Sarihan believes that having an in-house team focused solely on quality provides a significant competitive edge in the evolving landscape of vertical AI.

Crosby's operational strategy, which includes individualized model tuning for clients, positions it at the forefront of legal tech innovation. By prioritizing quality assurance and continuous improvement through dedicated teams, the firm aims to build a more efficient and reliable legal infrastructure, transforming how businesses handle critical agreements.