DSPy-Optimized Dr.Copilot System Drives 70% Surge in Romanian Patient Satisfaction

A groundbreaking multi-agent AI system named Dr.Copilot, optimized using the DSPy framework, has achieved a remarkable 70.22% increase in positive patient reviews within Romanian medical settings. The system, designed to enhance the presentation quality of doctor-patient communication rather than medical correctness, marks a significant real-world application of Large Language Models (LLMs) in a low-resource language environment.

Developed by researchers including Omar Khattab, the creator of DSPy, Dr.Copilot supports Romanian-speaking doctors by providing targeted feedback on their written responses to patients. As stated by Omar Khattab in a recent tweet, "DSPy has already widely democratized and distributed 'verifier'-based learning for downstream LLM systems, long ago :-) Including in Romanian medical settings." This highlights DSPy's role in enabling reliable and effective LLM applications.

The Dr.Copilot system operates through three distinct LLM agents—a Scorer Agent, a Recommender Agent, and a Reconciliation Agent—whose prompts are automatically optimized using dspy.SIMBA. This optimization process allows the system to provide feedback across 17 interpretable axes, including crucial aspects like Empathy, Explanations, and Grammatical Errors. The focus on communication quality addresses a key challenge in telemedicine, where patient satisfaction is often heavily influenced by how medical advice is conveyed.

Deployed in a live environment with 41 doctors, Dr.Copilot processed 212 evaluation requests and suggested 449 recommendations. The system's success is particularly noteworthy given its design with low-resource Romanian data and its use of open-weight models, ensuring data privacy by local deployment. This approach minimizes risks associated with sensitive patient information, as no data is transmitted to external servers.

The substantial improvement in patient feedback underscores the potential of advanced LLM frameworks like DSPy to drive tangible benefits in critical sectors such as healthcare. By refining how doctors communicate, Dr.Copilot not only boosts patient satisfaction but also fosters greater trust and adherence to medical advice, demonstrating a practical and ethical advancement in AI-assisted healthcare.