Using AI to predict retinal disease progression

Vision loss among the elderly is a major healthcare issue: about one in three people have some vision-reducing disease by the age of 65. Age-related macular degeneration (AMD) is the most common cause of blindness in the developed world. In Europe, approximately 25% of those 60 and older have AMD. The ‘dry’ form is relatively common among people over 65, and usually causes only mild sight loss. However, about 15% of patients with dry AMD go on to develop a more serious form of the disease – exudative AMD, or exAMD – which can result in rapid and permanent loss of sight. Fortunately, there are treatments that can slow further vision loss. Although there are no preventative therapies available at present, these are being explored in clinical trials. The period before the development of exAMD may therefore represent a critical window to target for therapeutic innovations: can we predict which patients will progress to exAMD, and help prevent sight loss before it even occurs?

In our latest work, published in Nature Medicine, we collaborated with Moorfields Eye Hospital and Google Health to curate a dataset of images of eye retinas, train an artificial intelligence (AI) system that could predict the development of exAMD, and conduct a study to evaluate our model compared with expert clinicians. We demonstrate that our system is able to perform as well as, or better than, clinicians at predicting whether an eye will convert to exAMD in the next 6 months. Lastly, we explore the potential clinical applicability of our system. Our contribution highlights the potential of using AI in preventative studies for diseases such as exAMD.

The Moorfields Eye Hospital AMD dataset

We used a dataset of anonymised retinal scans from Moorfields patients with exAMD in one eye, and at high-risk of developing exAMD in their other eye. This comprises 2,795 patients across seven different Moorfields sites in London, with representation across genders, age ranges, and ethnicities. These patients attend the hospital regularly to receive treatment, undergoing high-resolution three-dimensional optical coherence tomography (OCT) imaging of both eyes, at each visit. There is often a delay between when exAMD has developed and when it is diagnosed and treated. To address this, we worked with retinal experts to review all scans for each eye and specify the scan when exAMD was first evident.

Training an early warning system for AMD

Our system is composed of two deep convolutional neural networks that take as input high-dimensional volumetric eye scans, where each scan consists of 58 million three-dimensional pixels (voxels). In our previous work, now continuing in collaboration with Google Health, we developed a model capable of segmenting these eye scans into thirteen anatomical categories. The segmented data was combined with the raw scan and both were used as inputs to the prediction model, which was trained to estimate a patient’s risk of conversion to exAMD in their other eye within the next six months.

The benefit of a two stage system is that it gives the AI different views of the eye scans. Anatomical segmentation of the images helps the system learn to model risks based on signs of known anatomical indicators such as drusen (small fatty deposits), or loss of the retinal pigment epithelium (which helps to feed and protect other layers of the retina). Providing the raw eye scans allows the model to learn to spot other subtle changes that could become potential risk factors. At the end, the system combines the information it extracts from these scans to predict when and if the eye will progress to exAMD within the next 6 months. We chose this time window to enable the system to predict at least two follow-up intervals ahead of time, assuming a maximal follow-up interval of 3 months.

Clinical expert benchmark for future prediction

It’s important to establish a benchmark of expert human performance to compare how well our system performs to clinical standards. However, prediction of exAMD is not a routine task performed by clinicians, so it’s unclear whether this task is even possible. To investigate this, we conducted a study with six retinal experts - three ophthalmologists and three optometrists, each with at least ten years of experience - to predict whether an eye will convert to exAMD within the ensuing 6 months. Despite the task’s novelty, the experts performed better than chance alone - however, the task is difficult, and there was substantial variability between their assessments. Our system performed as well as, and in some cases better than, experts in predicting exAMD progression, at the same time exhibiting less variability in agreement with each expert, compared to experts with each other.

Visualising disease progression

It may not be enough for a system to simply provide a prediction: clinicians may also ideally seek information regarding the anatomic basis for predictions, which might be of significant use for further interpretation (for example, for designing studies or considering treatments). A benefit of our system is that it automatically segments each scan into known types of tissue. Extracting these anatomical and pathological features provides a systematic method to visualise the change in these tissues over time. The risk scores given by our system align with anatomical changes over time, and together give a richer picture of exAMD conversion.

Foresight over hindsight

We’re excited by the potential to support clinicians and researchers by developing systems that can help detect retinal diseases earlier and inform the clinical understanding of their progression. A prediction system such as this could be used to inform appropriate follow-up intervals to effectively manage high-risk patients. Our work builds upon promising early work to develop predictive models for exAMD based on retinal photographs and OCT scans. Since beginning our collaboration with Moorfields Eye Hospital in 2016, we’ve published two promising studies highlighting the potential of AI to transform retinal healthcare.

However, we know there’s still a lot to do – this work does not yet represent a product that could be implemented in routine clinical practice. While our model can make better predictions than clinical experts, there are many other factors to consider for such systems to be impactful in a clinical setting. While the model was trained and evaluated on a population representative of the largest eye hospital in Europe, additional work would be needed to evaluate performance in the context of very different demographics. A recent study examining the use of a different AI system in a clinical setting highlighted just some of the sociotechnical issues for such systems in practice. Another difficult point to contend with is that any prediction system will have a certain rate of false positives: that is, when a patient is found to have a condition, or predicted to develop one, that they don’t actually have. The tradeoff of adding an imprecise AI system to an early warning loop could be unnecessarily costly to patients who aren’t actually at risk, and would need to be considered carefully in clinical studies of how such systems might be used in practice. In this paper, we propose two system operating points to balance sensitivity (a measure of how well it correctly identifies the disease) and specificity (a measure of how low the false positive rate is). For example, at a specificity of 90%, a sensitivity of 34% is achieved, meaning that the system correctly identified progression in one third of scans that did go on to progress within 6 months. This could identify a number of patients at high risk with a precision that may be sufficient to inform studies of novel treatment strategies that might mitigate vision loss and improve patient outcomes.

We would like to thank Moorfields Eye Hospital and the clinicians who helped curate the data and were involved in our benchmarking study. Please see the paper for all acknowledgements and further details on the work. In addition, we’ve open-sourced the model code for future research, available here, and Moorfields will be making the dataset available through the Ryan Initiative for Macular Research.