Conference Abstracts - 2026 National Summit on Hematologic Cancers
Vol. 6, Issue Supplement 2, 2026 · S1-2
From Frontline Therapy to Relapse: Mapping Generative AI Consensus Across the CLL Treatment Journey
Gayathri Pramil Menon, MD,Yan Leyfman, MD,Noam Levi , BS,Diksha Sanjana Pasnoor, MBBS,Adeena Musheer, MBBS,Zara Baloch, MBBS,Huseyin Avni Yalcin, BS,Helena Sanchez Coloma, DDS,Maduri Balasubramanian, MD,Muskan Joshi, MD,Taha Kassim Dohadwala, MD,Ahmed Hashim Azeez, MD,Shivani Shukla, MD,Arturo Loaiza-Bonilla , MD
Submission received: 2026-08-16 / Accepted: 2026-08-25 / Published: 2026-09-25
Abstract
Background
Chronic lymphocytic leukaemia (CLL) care requires repeated treatment decisions as disease biology, prior therapy, toxicity, response, and patient priorities evolve. Generative artificial intelligence (AI) is increasingly considered for clinical decision support, yet a recommendation that appears consistent at one time point may not remain aligned as the clinical context changes. We examined where consensus among AI systems and haematologists is maintained or lost across sequential CLL treatment decisions.
Methods
Ten expert-developed CLL clinical journeys comprising 30 prespecified decision stages were evaluated by ChatGPT (GPT-5.5 Instant), Claude (Sonnet 5), Gemini (3.5 Flash), and OpenEvidence. Each AI system completed every stage in three independent runs; modal recommendations were derived when responses differed. Three hematologists independently completed the same cases. Outcomes included within-model repeatability, pairwise AI agreement, complete four-system agreement, physician agreement, and concordance across complete three-stage journeys.
Results
The analysis included 360 AI responses and 90 physician responses. Within-model exact repeatability ranged from 53.3% to 80.0%. Across 29 commonly evaluable stages, pairwise AI agreement was 60.9% (95% CI, 50.6%–70.7%), while complete four-system agreement occurred in 31.0%. Complete agreement was observed in 44.4% of first-stage, 20.0% of second-stage, and 30.0% of third-stage decisions. Hematologist pairwise agreement was 50.0% (95% CI, 39.9%–60.1%), with unanimity in 26.7% of stages. No fully evaluable three-stage clinical journey achieved complete agreement across all four AI systems.
Conclusion
Discordance in CLL decision-making was not confined to AI: both AI systems and hematologists varied across clinically evolving scenarios. However, the absence of complete AI agreement across any full treatment journey highlights a limitation of evaluating clinical decision support using isolated questions alone. Sequential CLL cases may better identify decision points where AI-supported recommendations require closer hematologist review and may provide a practical framework for studying human-AI decision support in longitudinal hematologic cancer care.