Images of the retina are used to diagnose and monitor the progress of a variety of diseases, including such leading causes of blindness as diabetic retinopathy, age-related macular degeneration, and glaucoma. Retinal image registration has a variety of applications. Registering a set of images taken during a single session with a patient can be used to form a single composite panorama view of the entire retina. Multimodal registration can reveal the relationship between events seen on the surface of the retina and the blood flow shown in the angiography. Registering images taken weeks, months or years apart can be used to reveal changes in the retina at the level of small regions and individual blood vessels.
Retinal image registration is challenging. The images are projections of a curved surface taken from a wide range of viewpoints using an uncalibrated camera. The retina surface itself is curved significantly and any transformation process that aligns images of the retina must take that into account. The nonvascular surface of the retina is homogenous in healthy retinas and exhibits a variety of pathologies in unhealthy retinas. The pathologies are a poor choice for longitudinal registration because they may appear and disappear over time. Only the vasculature covers the entire retina and is relatively stable over time.
The solution to the retinal image registration problem is an approach driven by the vascular structure. This can include the vessels themselves and their branching and crossover points. Differences in images, especially over longitudinal studies, can interfere with automatic registration. Images may appear dark or brighter between sessions. Some blood vessels may appear dark, while other vessels nearby may appear bright. Inclusions, pathologies and other artifacts can lead to false positives. Distortions in the image capturing process can lead to registration problems. Ultimately, the best process needs to have 1) very good initialization of seed points, 2) robust minimization algorithms that reduce the image to discrete data for the registration process, and 3) elimination or minimization of local errors, such as pathologies, geometric distortions and other local imaging issues.
In retinal imaging, the stable and prominent vascular structure should drive the minimization. Feature based techniques can be used in the minimization method where the images are aligned based on correspondences between automatically detected features. Feature based methods usually minimize an objective function of the distances between vascular landmarks correspondences. To narrow down the search space to the correct domain of convergence, matched landmarks, such as the optic disk and the fovea, can be used in the initialization. Alternatively, manually indicated landmarks and other registration points inherent in the vasculature can be used on overlapped regions of images. In all landmark-based algorithms, the major problem is finding and matching a set of consistent landmarks to sufficiently constrain the transformation needed for accurate registration. This is particularly a problem for nonlinear transformations, low image overlap, poor quality images, and longitudinal changes.
Delineation of the vasculature, also known as tracing, uses algorithms to follow the vessels, along its centerline. While most fundus images in the retinal image alignment project are basically flattened 2D images, this description can be extended into a 3D description by indicating a 3D tangent direction and a location in 3 space. After the entire tree has been analyzed, a series of data structures is rendered that represent the tree via the node points or branching, and the characteristics of the branch points. Characteristic landmarks of the vasculature are described by each branch radiating out from an origin, bifurcation or branch points and lengths in between branch points. Landmarks are invariant to rotation, translation and scaling of the image and serve as excellent correspondence points in the overlap regions of the images for the registration task..
The end result is to describe the vasculature of the retina as the foundation for the registration of retinal images from a series of time-shifted images from an individual’s retina as well as the registration of single field fundus images. The following basic image processing procedures are performed on the patient data:
Fundus images are typically acquired with the use of a variety of desktop fundus cameras on the market. The images can be acquired automatically or manually such that the retinal fields of view are slightly overlapping. Images are presented as JPEG, TIFF or BMP in most cases.
Tracing of retinal images for stable features is important for a variety of both off-line and on-line (real-time) analyses. Off-line analyses, such as those executed by a practitioner using a desktop computer or an iPad, can produce corresponding landmark pairs visible in the overlapping regions of adjacent images that can then be used to create a single panorama of the fundus. On-line analyses can be used to detect changes longitudinally, which when compared to normative values could be detectors of early pathology. Thus, the ability to efficiently and accurately reconstruct and measure morphological properties of biological elements is crucial for disclosing the structural determinants of pathology-induced abnormality of the biological elements.
Retinal image registration will be driven by the vascular structure, including the vessels themselves and their branching and crossover points. The images are aligned based on correspondences between automatically detected features.
The first part of Retinal Mapper is to identify, characterize and register the images based on these features of the local vasculature. Image normalization includes both corrections for non-uniform illumination as well as for distortion introduced by the optical lens system of the fundus camera. The vasculature will be traced by using several starting points, also known as seed points, throughout the retinal images. The centerline traces of the blood vessels will then be analyzed for the location of branch points, their characterization into bifurcations, trifurcations, or crossover points, branch widths and branch vessel angles. Similarities between pairs of images are analyzed by examining these characteristics as triangulation between branch points and the branch point characteristics themselves. These similarities are used to determine how the images should be rotated, scaled and translated so that the panorama of the retina can be reconstruction.
The second part of Retinal Mapper focuses on temporal differences. Images, both single fields of a specific area of the retina, and of the entire panorama, are aligned to each other so that the images may be properly overlaid. This global registration process using the vasculature then allows for differential analysis over time to detect the onset of disease. This part also introduces the concept of differential analysis with single field analysis and with panoramas of normal and diseased eyes. Engineering efforts to produce the differential analysis and the fine-tuning of registration with diseased eyes and weakened vasculature is part of this objective. Ultimately, a single field of the retinal taken over a period of time is successfully registered . The panoramas taken over time are aligned with each other.
The third part of Retinal Mapper screens a single patient against population survey data for the potential onset of disease, and for the full multiple examination of an individual patient over time, also for the potential onset of disease.
The time-shifted series of images are analyzed for reflections of disease. The following diseases of the retina are the initial Intelligence Application.
Diabetic Retinopathy - The World Health Organization estimates that 135 million people have diabetes mellitus worldwide and the number of people with diabetes will increase to 300 million by year 2025. Almost 20 million Americans have diabetes and an additional 16 million working-age adults have pre-diabetes and at a high risk of developing diabetes. Visual disability and blindness from diabetes have a profound socioeconomic impact. Diabetic retinopathy is the leading cause of new blindness in working-age adults in the industrialized world. It has been estimated that as much as $167M and up to 85,000 sight-years could be saved annually in the US alone by improving screening methods for diabetic retinopathy. Public pressure to improve the management of diabetic retinopathy has increased substantially, compelling the U.S. Congress to mandate reimbursement for diabetes screening in the 2003 Medicare Modernization Act of 2003.
Glaucoma– Glaucoma is a group of progressive optic neuropathologies that have in common a slow progressive degeneration of retinal ganglion cells and their axons, resulting in a distinct appearance of the optic disc and a concomitant pattern of visual loss. The biological basis of the disease is not yet fully understood, and the factors contributing to its progression are not yet fully characterized. However, intraocular pressure is the only proven treatable risk factor. Without adequate treatment, glaucoma can progress to visual disability and eventual blindness.
Hypertension– Published studies have shown the value of this approach and only recently has the technology been perfected to provide significant diagnostic value compared to current methods. Retinal microvascular abnormalities, such as generalized and focal arteriolar narrowing, arteriovenous nicking and retinopathy, reflect cumulative vascular damage from hypertension, aging and other processes. Generalized arteriolar narrowing and arteriovenous nicking are irreversable long-term markers of hypertension. There are data supporting an association between retinal microvascular abnormalities and stroke, even taking into account blood pressure and other stroke risk factors. Since the present diagnosis of hypertension and eventual stroke are expensive, inconsistent and cumbersome, there is a need for more accurate and convenient diagnostic alternatives.