Elsevier

Ophthalmology

Volume 124, Issue 7, July 2017, Pages 953-961
Ophthalmology

Original article
Assessment of a Tele-education System to Enhance Retinopathy of Prematurity Training by International Ophthalmologists-in-Training in Mexico

Presented at: the Annual Meeting of the Association for Research in Vision and Ophthalmology; May 1–5, 2016; Seattle, WA.
https://doi.org/10.1016/j.ophtha.2017.02.014Get rights and content

Purpose

To evaluate a tele-education system developed to improve diagnostic competency in retinopathy of prematurity (ROP) by ophthalmologists-in-training in Mexico.

Design

Prospective, randomized cohort study.

Participants

Fifty-eight ophthalmology residents and fellows from a training program in Mexico consented to participate. Twenty-nine of 58 trainees (50%) were randomized to the educational intervention (pretest, ROP tutorial, ROP educational chapters, and posttest), and 29 of 58 trainees (50%) were randomized to a control group (pretest and posttest only).

Methods

A secure web-based educational system was created using clinical cases (20 pretest, 20 posttest, and 25 training chapter–based) developed from a repository of over 2500 unique image sets of ROP. For each image set used, a reference standard ROP diagnosis was established by combining the clinical diagnosis by indirect ophthalmoscope examination and image-based diagnosis by multiple experts. Trainees were presented with image-based clinical cases of ROP during a pretest, posttest, and training chapters.

Main Outcome Measures

The accuracy of ROP diagnosis (e.g., plus disease, zone, stage, category) was determined using sensitivity and specificity calculations from the pretest and posttest results of the educational intervention group versus control group. The unweighted kappa statistic was used to analyze the intragrader agreement for ROP diagnosis by the ophthalmologists-in-training during the pretest and posttest for both groups.

Results

Trainees completing the tele-education system had statistically significant improvements (P < 0.01) in the accuracy of ROP diagnosis for plus disease, zone, stage, category, and aggressive posterior ROP (AP-ROP). Compared with the control group, trainees who completed the ROP tele-education system performed better on the posttest for accurately diagnosing plus disease (67% vs. 48%; P = 0.04) and the presence of ROP (96% vs. 91%; P < 0.01). The specificity for diagnosing AP-ROP (94% vs. 78%; P < 0.01), type 2 ROP or worse (92% vs. 84%; P = 0.04), and ROP requiring treatment (89% vs. 79%; P < 0.01) was better for the trainees completing the tele-education system compared with the control group. Intragrader agreement improved for identification of plus disease, zone, stage, and category of ROP after completion of the educational intervention.

Conclusions

A tele-education system for ROP education was effective in improving the diagnostic accuracy of ROP by ophthalmologists-in-training in Mexico. This system has the potential to increase competency in ROP diagnosis and management for ophthalmologists-in-training from middle-income nations.

Section snippets

Methods

The Weill Cornell Medical College Human Studies Committee approved this as a prospective study for the analysis of retinal images and approved the educational material used in this study. Administration of the tele-education system was also reviewed by the Weill Cornell Medical College Human Studies Committee and was granted an exemption because it was considered research in an established or commonly accepted educational setting involving normal educational practices such as research on the

Results

Table S2 (available at www.aaojournal.org) summarizes key characteristics of the 58 ophthalmologists-in-training who participated in the study. Twenty-nine of 58 trainees (50%) were randomized to the intervention group, and of the 26 trainees who completed the ROP tele-education system, 6 (23%) were postgraduate year (PGY)–1 residents, 8 (31%) were PGY-2 residents, 7 (27%) were PGY-3 residents, and 5 (19%) were retina fellows. Twenty-nine of 58 trainees (50%) were randomized to the control

Discussion

The key findings of this study are as follows: (1) a tele-education system for ROP can improve the diagnostic accuracy of ROP by ophthalmologists-in-training from a middle-income nation; (2) intragrader agreement, as determined by the kappa statistic, improved for identification of plus disease, zone, stage, and category of ROP after completion of the tele-education system; and (3) the trainees completing the tele-education system felt that their understanding of the diagnosis of ROP improved

References (43)

  • Screening Examination of Premature Infants for Retinopathy of Prematurity

    Pediatrics

    (2013)
  • National Eye Institute. Facts About Retinopathy of Prematurity (ROP). Available at:...
  • Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results

    Pediatrics

    (1988)
  • Revised indications for the treatment of retinopathy of prematurity: results of the Early Treatment for Retinopathy of Prematurity randomized trial

    Arch Ophthalmol

    (2003)
  • C. Gilbert et al.

    Characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs

    Pediatrics

    (2005)
  • R.V.P. Chan et al.

    Accuracy of retinopathy of prematurity diagnosis by retinal fellows

    Retina

    (2010)
  • A. Nagiel et al.

    Retinopathy of prematurity residency training

    Ophthalmology

    (2012)
  • D.A. Cook

    Web-based learning: pros, cons and controversies

    Clin Med

    (2007)
  • R.V. Chan et al.

    The Global Education Network for Retinopathy of Prematurity (Gen-Rop): development, implementation, and evaluation of a novel tele-education system (an American Ophthalmological Society thesis)

    Trans Am Ophthalmol Soc

    (2015)
  • M.P. Yip et al.

    Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: a preliminary study

    J Telemed Telecare

    (2003)
  • M.C. Ryan et al.

    Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology

    AMIA Annu Sym Proc

    (2014)
  • Cited by (26)

    • Assessment and management of retinopathy of prematurity in the era of anti-vascular endothelial growth factor (VEGF)

      2022, Progress in Retinal and Eye Research
      Citation Excerpt :

      This form of tailored feedback provides an efficient and effective method to improving diagnostic performance among trainees(CHAN; PATEL; RYAN; JONAS et al., 2015). The GEN-ROP/i-ROP tele-education system was piloted among ophthalmology trainees in different countries around the world(CAMPBELL; SWAN; JONAS; OSTMO et al., 2015; CHAN; PATEL; RYAN; JONAS et al., 2015; PATEL; MARTINEZ-CASTELLANOS; BERRONES-MEDINA; SWAN et al., 2017). Trainees in the U.S. and Canada who participated in the program demonstrated significant improvement in their diagnostic sensitivity, notably for the detection of plus disease (49% at pre-test vs. 67% at post-test, P = 0.16), stage 3 (60% vs. 78%, P < 0.001), zone I (44% vs. 74%, P < 0.001), and TR ROP (48% vs. 78%, P < 0.001)(CHAN; PATEL; RYAN; JONAS et al., 2015).

    View all citing articles on Scopus

    Supplemental material available at www.aaojournal.org.

    Financial Disclosure(s): R.S., M.C.R., S.O., K.E.J., R.V.P.C., M.F.C.: Support – National Institutes of Health (Bethesda, MD) grant no. P30 EY010572.

    J.P.C., S.O., M.F.C.: Support – National Institutes of Health (Bethesda, MD) grant no. R01 EY19474

    M.F.C.: Support – National Institutes of Health (Bethesda, MD) grant no. R21 EY022387

    M.F.C., R.S.: Support – grant no. T32 EY23211; Oregon Clinical and Translational Research Institute grant no. TL1TR000129 from the National Center for Advancing Translational Sciences at the National Institutes of Health, Bethesda, MD

    S.N.P., J.P.C., R.S., M.C.R., S.O., K.E.J., R.V.P.C., M.F.C.: unrestricted departmental funding – Research to Prevent Blindness, New York, NY

    R.V.P.C., K.E.J.: iNsight Foundation, New York, NY; The Bernadotte Foundation for Children's Eyecare, New York, NY; Novartis Excellence in Ophthalmic Vision Award.

    M.F.C.: Scientific advisory board – Clarity Medical Systems (Pleasanton, CA); Consultant – Novartis (Basel, Switzerland).

    R.V.P.C.: Scientific advisory board – Visunex Medical Systems (Fremont, CA).

    The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. No funding organizations had any role in the design or conduct of this research.

    Author Contributions:

    Conception and design: Patel, Jonas, Ostmo, Chiang, Chan

    Analysis and interpretation: Patel, Swan, Ryan, Jonas, Ostmo, Campbell, Chiang, Chan

    Data collection: Patel, Martinez-Castellanos, Berrones-Medina, Swan, Ryan, Jonas, Ostmo, Campbell, Chiang, Chan

    Obtained funding: Not applicable

    Overall responsibility: Patel, Martinez-Castellanos, Berrones-Medina, Swan, Ryan, Jonas, Ostmo, Campbell, Chiang, Chan

    Members of the GEN-ROP Research Consortium: University of Illinois at Chicago (Chicago, IL): R.V. Paul Chan, MD, Karyn E. Jonas, RN. Oregon Health & Science University (Portland, OR): Michael F. Chiang, MD, Susan Ostmo, MS, J. Peter Campbell, MD, MPH. Weill Cornell Medical College (New York, NY): Vivien Yap, MD, Alexander D. Port, MD, Samir N. Patel, MD. Hospital for Sick Children (Toronto, ON, Canada): Leslie D. Mackeen, CRA. Asociacion para Evitar la Ceguera en Mexico (APEC) (Mexico City, Mexico): Maria Ana Martinez-Castellanos, MD, Samantha Salinas-Longoria, MD, Rafael Romero, MD, Andrea Arriola, MD. Chang Gung Memorial Hospital (Taoyuan, Taiwan): Wei-Chi Wu, MD. Ospital ng Makati (Makati City, Philippines): Rachelle Go Ang Sam Anzures, MD. Altino Ventura Foundation (Recife, Brazil): Camila V. Ventura, MD.

    Members of the i-ROP Research Consortium: Oregon Health & Science University (Portland, OR): Michael F. Chiang, MD, Susan Ostmo, MS, Kemal Sonmez, PhD, Sang Jin Kim, MD, PhD, J. Peter Campbell, MD, MPH. University of Illinois at Chicago (Chicago, IL): R.V. Paul Chan, MD, Karyn Jonas, RN. Weill Cornell Medical College (New York, NY): Anton Orlin, MD. Columbia University (New York, NY): Jason Horowitz, MD, Osode Coki, RN, Cheryl-Ann Eccles, RN, Leora Sarna, RN. Bascom Palmer Eye Institute (Miami, FL): Audina Berrocal, MD, Catherin Negron, BA. William Beaumont Hospital (Royal Oak, MI): Kimberly Denser, MD, Kristi Cumming, RN, Tammy Osentoski, RN, Tammy Check, RN, Mary Zajechowski, RN. Children's Hospital Los Angeles (Los Angeles, CA): Thomas Lee, MD, Evan Kruger, BA, Kathryn McGovern, MPH. Cedars Sinai Hospital (Los Angeles, CA): Charles Simmons, MD, Raghu Murthy, MD, Sharon Galvis, NNP. LA Biomedical Research Institute (Los Angeles, CA): Jerome Rotter, MD, Ida Chen, PhD, Xiaohui Li, MD, Kent Taylor, PhD, Kaye Roll, RN. Massachusetts General Hospital (Boston, MA): Jayashree Kalpathy-Cramer, PhD. Northeastern University (Boston, MA): Deniz Erdogmus, PhD, Stratis Ionnidis, PhD. Asociacion para Evitar la Ceguera en Mexico (APEC) (Mexico City, Mexico): Maria Ana Martinez-Castellanos, MD, Samantha Salinas-Longoria, MD, Rafael Romero, MD, Andrea Arriola, MD, Francisco Olguin-Manriquez, MD, Miroslava Meraz-Gutierrez, MD, Carlos M. Dulanto-Reinoso, MD, Cristina Montero-Mendoza, MD.

    View full text