1
AIAssessment
Can ChatGPT prep your CCC packet? It finds the praise and misses the critique
- 277 end-of-rotation assessments for 25 residents: de-identifying took ~60 min, and 3 LLMs labeled the comments in under 7 min each.
- Humans found 476 comments (81.5% strengths, 18.5% areas for improvement). LLM recall was 67.8% for strengths but only 23.9% for areas for improvement; 59.7% of comments were labeled accurately overall.
Why it matters: the comments a CCC most needs are the ones the models missed. Let AI compile, but have a human read every constructive comment.
Cooney CM, Slattery R, Cooney DS, Lifchez SD, Park JE. J Grad Med Educ. Aug 2026. PMID 42602997. doi:10.4300/JGME-D-25-01049.1
Full text ›