Major Article
Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: Implications for training

Presented at the American Academy of Ophthalmology Annual Meeting, Chicago, IL, October 16-19, 2010 and as a poster at the 37th Annual Meeting of the American Academy for Pediatric Ophthalmology and Strabismus, San Diego, CA, March 30-April 3, 2011.
https://doi.org/10.1016/j.jaapos.2011.06.011Get rights and content

Purpose

To measure the accuracy of image-based retinopathy of prematurity (ROP) diagnosis by pediatric ophthalmology fellows.

Methods

This was a comparative case series of expert versus nonexpert clinicians in image-based ROP diagnosis. An atlas of 804 retinal images was captured from 248 eyes of 67 premature infants with a wide-angle camera (RetCam-II, Clarity Medical Systems, Pleasanton, CA). Images were uploaded to a study website from which an expert pediatric retinal specialist and five pediatric ophthalmology fellows independently provided a diagnosis (no ROP, mild ROP, type 2 ROP, or treatment-requiring ROP) for each eye. Two different retinal specialists experienced in ROP examination served as additional controls. Primary outcome measures were sensitivity and specificity of image-based ROP diagnosis by fellows compared to a reference standard of image-based interpretation by the expert pediatric retinal specialist. Secondary outcome measure was intraphysician reliability.

Results

For detection of mild or worse ROP, the mean (range) sensitivity among the five fellows was 0.850 (0.670-0.962) and specificity was 0.919 (0.832-0.964). For detection of type 2 or worse ROP by fellows, mean (range) sensitivity was 0.527 (0.356-0.709) and specificity was 0.938 (0.777-1.000). For detection of treatment-requiring ROP, mean (range) sensitivity was 0.515 (0.267-0.765) and specificity was 0.949 (0.805-1.00).

Conclusions

Pediatric ophthalmology fellows in this study demonstrated high diagnostic specificity in image-based ROP diagnosis; however, sensitivity was lower, particularly for clinically significant disease.

Section snippets

Methods

This research was conducted under the approval of the Columbia University Institutional Review Board and included a waiver of consent for use of de-identified retinal images. Informed consent was obtained from all fellows who participated in the study. All research was performed in compliance with the Health Insurance Portability and Accountability Act of 1996.

Images from consecutive infants whose parents provided informed consent were captured during routine ROP examinations at Columbia

Results

Overall, there were five residency-trained, board-eligible ophthalmologists enrolled in pediatric ophthalmology fellowship programs who met study eligibility criteria and consented to participate. All fellows reported that they had minimal or no ROP screening experience during residency training. All were within the first 6 months (mean, 4.4 months) of their fellowship training at programs where weekly ROP examinations were performed with a faculty member.

Figure 1 reports the distribution of

Discussion

The key finding of this study is that pediatric ophthalmology fellows generally demonstrated high specificity for imaged-based detection of mild levels of ROP but showed lower diagnostic sensitivity for detecting clinically significant levels of disease (ie, type 2 and treatment-requiring ROP). Among fellows in this study, mean sensitivities for detecting type 2 and treatment-requiring ROP were approximately 50%. Fellows in this study had variable diagnostic performance and a general tendency

References (25)

  • M.G. Chiang et al.

    Remote image based retinopathy of prematurity diagnosis: A receiver operating characteristic analysis of accuracy

    Br J Ophthalmol

    (2006)
  • J. Cohen

    A coefficient of agreement for nominal scales

    Educ Psychol Meas

    (1960)
  • Cited by (0)

    This study was supported by Grant EY19474 from the National Institutes of Health (MFC), by the St. Giles Foundation (RVPC), and by the Research to Prevent Blindness (JSM, RVPC, DBG, MFC). The sponsors of funding organizations had no role in the design or conduct of this research. The authors have no commercial, proprietary, or financial interest in any of the products or companies described in this article. MFC is an unpaid member of the Scientific Advisory Board for Clarity Medical Systems (Pleasanton, CA).

    View full text