A Prospective, Cross-Sectional, Vignette-Based Observational Study Comparing Clinical Decision-Making Performance of Pediatriciansand AI Models
Trial ID: NCT07179861
Status: COMPLETED
Study type: OBSERVATIONAL
Start date: 2025-08-27
Completion date: 2025-09-11
Sponsor: Haseki Training and Research Hospital
Conditions: Artificial Intelligence (AI) in Diagnosis; Decision Support Systems, Clinical; Clinical Decision-making; Pediatrics
Interventions: AI Suggestions (Anonymized 5-tool panel); Confidence Rating Task (1-10 Likert)
Locations: SBÜ Sultangazi Haseki Training and Research Hospital
Countries: Turkey (Türkiye)
Enrollment: 30
Age range: 28 Years - 40 Years
Sex: ALL

Summary:
This study evaluates how well anonymized artificial-intelligence (AI) tools perform on standardized pediatric case vignettes and whether showing AI suggestions can improve clinicians' answers. About 30 board-certified/eligible pediatric specialists at a single hospital complete a one-time session. Participants are randomized to two groups. Group A (n≈15): physicians answer each vignette once. Group B (n≈15): physicians answer and rate confidence (1-10), then review anonymized suggestions from five different AI tools (tool names not shown) and may keep or change their answer; changes and confidence are recorded.

Primary focus: measure AI performance (diagnostic accuracy, medication-dosing accuracy, interpretation accuracy) overall and by difficulty tier, and record AI response time. Secondary focus: quantify how AI suggestions affect human performance (change in accuracy, direction of change, confidence shift, and time). No patients or biospecimens are involved; risks are minimal (time and possible discomfort with performance review). Findings may inform safe, evidence-based ways to use AI alongside clinicians in pediatrics.

Eligibility criteria:
Inclusion Criteria:

* Board-certified or board-eligible pediatric specialist (general pediatrics) (in the first 10 years of expertise)
* Actively practicing at the participating institution/network at the time of enrollment.
* Able and willing to complete all vignette items individually in a single session and to follow study instructions for the assigned cohort (direct answers or confidence rating + viewing anonymized AI suggestions).
* Fluent in Turkish and able to use a computer interface.
* Provides written informed consent.

Exclusion Criteria:

* Pediatric subspecialist practice as primary role (e.g., cardiology, infectious diseases, neurology, neonatology, etc.), to maintain a homogeneous general pediatrics cohort.
* Prior access to or participation in creating the study vignettes, answer keys, or scoring rubrics; direct involvement with the study team.
* Inability to complete the session without external help or use of non-protocol resources (internet/AI tools) during answering (outside of anonymized AI suggestions shown by the system in Group 2).
* Failure to complete ≥90% of items or major protocol deviation (e.g., discussion with others during the task).
* Any condition judged by investigators to interfere with valid participation (e.g., severe time constraints, inability to provide consent).

Outcomes:
AI Interpretation Accuracy (%); AI Diagnostic Accuracy (%); AI Medication-Dosing Accuracy (%)