Nairobi, Kenya – A collaborative real-world study between OpenAI and Penda Health, a primary care provider based in Nairobi, has demonstrated a significant reduction in diagnostic and treatment errors among clinicians utilizing an AI-powered clinical copilot. The study, encompassing 39,849 live patient visits, revealed a 16% relative reduction in diagnostic errors and a 13% reduction in treatment errors for clinicians using the AI tool compared to those who did not.
Karan Singhal announced the findings on social media, stating, "Excited to share our real-world study of an LLM clinical copilot, a collab between @OpenAI and @PendaHealth." The initiative highlights the potential of large language models (LLMs) to enhance clinical decision-making in practical healthcare settings. Penda Health, established in 2012, developed the "AI Consult" tool to provide real-time recommendations during patient visits.
The AI Consult system, powered by OpenAI's GPT-4o, integrates with electronic health records and operates in the background, acting as a safety net that activates upon detecting potential errors. OpenAI attributes the positive outcomes to a capable AI model, clinically-aligned implementation, and active deployment efforts that ensured clinicians understood and effectively utilized the tool. Clinicians retain full control over all decision-making processes.
The study, conducted between January and April 2025, showed particularly strong results in cases where the AI Consult system flagged potential issues with a "red alert," leading to a 31% reduction in diagnostic errors and an 18% reduction in treatment errors in those specific instances. This real-world application across thousands of patients sets a significant precedent for the effective integration of AI in healthcare. The research is currently being submitted to a scientific journal for peer review.
Penda Health is now undertaking a randomized controlled trial in collaboration with PATH to further evaluate the copilot's effects on overall patient outcomes. The study received ethical and scientific approval from various Kenyan health authorities, including the AMREF Health Africa Ethical and Scientific Review Committee and the Kenyan Ministry of Health. This collaborative effort aims to provide a template for the safe and effective use of LLMs to support clinicians globally.