Utility of Interobserver Agreement Statistics in Establishing Radiology Resident Learning Curves During Self-directed Radiologic Anatomy Training

Abstract

Rationale and Objectives: The aim of the study was to ascertain the learning curves for the radiology residents when first introduced to an anatomic structure in magnetic resonance images (MRI) to which they have not been previously exposed to. Materials and Methods: The iliolumbar ligament is a good marker for testing learning curves of radiology residents because the ligament is not part of a routine lumbar MRI reporting and has high variability in detection. Four radiologists, three residents without previous training and one mentor, studied standard axial T1- and T2-weighted images of routine lumbar MRI examinations. Radiologists had to define iliolumbar ligament while blinded to each other's findings. Interobserver agreement analyses, namely Cohen and Fleiss kappa statistics, were performed for groups of 20 cases to evaluate the self-learning curve of radiology residents. Results: Mean kappa values of resident-mentor pairs were 0.431, 0.608, 0.604, 0.826, and 0.963 in the analysis of successive groups (P <.001). The results indicate that the concordance between the experienced and inexperienced radiologists started as weak (kappa <0.5) and gradually became very acceptable (kappa >0.8). Therefore, a junior radiology resident can obtain enough experience in identifying a rather ambiguous anatomic structure in routine MRI after a brief instruction of a few minutes by a mentor and studying approximately 80 cases by oneself. Conclusions: Implementing this methodology will help radiology educators obtain more concrete ideas on the optimal time and effort required for supported self-directed visual learning processes in resident education.