Patent Description:
As another example, fundus imaging can be used to screen for or monitor various diseases, such as diabetic retinopathy, hypertension, glaucoma, and papilledema. Trained medical professionals use cameras during eye examinations for disease screening. The cameras can produce images of the back of the eye and trained medical professionals use those images to diagnose and treat one or more diseases. These images are produced either with pharmacological pupil dilation, known as mydriatic fundus imaging, or without pharmacological pupil dilation, known as non-mydriatic fundus imaging. Because pupil dilation is inversely related, in part, to the amount of ambient light, non-mydriatic fundus imaging usually occurs in low lighting environments.

<CIT> provides a plurality of methods of analysis of images of eyes of a subject taken by a reflex photometer are disclosed. In these methods, a computer coupled to the reflex photometer is programmed to locate the pupil of at least one eye, crop an image array containing the pupil from the total image and perform analyses on the cropped array to determine whether conditions are present indicative of disease processes in the eye.

In applications where it is desired to determine the locations of image features, such as eye monitoring to determine the direction that a person is gazing, determining the point at which he is gazing, or measuring the motions of his eye using a camera to capture an optical image of the eye and image processing to extract information about the eye's gaze point and/or orientation, <CIT> provides a method and apparatus for precise location of image features such as edge coordinates between the pupil and iris of the eye and of the center coordinates of light reflections off the cornea of the eye.

The following drawing figures, which form a part of this application, are illustrative of described technology and are not meant to limit the scope of the disclosure in any manner.

Various embodiments of the present disclosure will be described in detail with reference to the drawings, wherein like reference numerals represent like parts and assemblies throughout the several views. Additionally, any examples set forth in this specification are not intended to be limiting and merely set forth some of the many possible embodiments.

Broadly, the present disclosure is directed to medical digital imaging. Certain types of medical digital imaging identify and/or track pupil movement. Systems and methods of this disclosure are directed towards pupil identification during medical digital imaging.

In particular, systems and methods disclosed herein involve detecting a pupil edge in a digital image. A "pupil edge," as used herein, is meant to include a boundary between the pupil and an iris. Detecting pupil edges finds application in, for example, photorefraction ocular screening and digital fundus imaging, particularly when triggered by eye tracking. Photorefraction ocular screening devices can determine refractive error by determining a slope of an intensity distribution of pixels across a pupil. The slope of the intensity distribution can be used to infer spherical error of a test subject's eye.

Slope determination typically relies on an accurate detection of the pupil edge. Accurate determination of a pupil size is needed to infer a subject's spherical error from a calculated slope of a pixel intensity distribution. Additionally, intensity distributions at the pupil edges decrease gradually and should be excluded when calculating a slope profile of the intensity distribution across the pupil.

Existing edge detection techniques are not specific to pupil edge detection. One example edge detection technique is the Canny edge detection algorithm. Typically, existing edge detection techniques are based on a gradient function or intensity histogram for calculating threshold values that can then be used for determining pixels that represent an edge of an object within an image.

Such techniques have many drawbacks for pupil edge detection. For instance, the Canny edge detection algorithm is processing intensive. An example advantage of the systems and methods contemplated herein is improved processing efficiency for pupil edge detection. Improving the processing efficiency thereby reduces processing requirements and/or speeds up processing.

Quickly processing pupil images is particularly valuable in the context of ocular refractive error correction and/or fundus imaging including pupil tracking. Typically, imaging devices capture many frames of the pupil or fundus within a short duration. For instance, during example ocular refractive error testing disclosed in <CIT>, "Photorefraction ocular screening device and methods," a total of twenty-three frames are captured and processed in seconds. For eye tracking, pupil identification is typically needed within milliseconds to capture fundus images before the subject's gaze changes.

Another example advantage of the systems and methods contemplated herein is improved accuracy of estimated pupil size and detection. Existing techniques can inaccurately determine pupil size. It has been observed that Canny edge detection techniques incorrectly estimate pupil size by <NUM> or even by <NUM>. In some instances, existing pupil edge detection techniques fail to find a pupil-iris threshold, particularly on medium or small pupils.

Generally, systems and methods contemplated herein exclude various human and pathologic factors from affecting determination of pupil edge pixel thresholds. Existing techniques, such as gradient and histogram methodologies, usually include such factors, which can explain their inaccuracy or even inability to determine pupil edges. Example factors include images where the pupil is partially obscured or includes: eye lids, eye lashes, and cataracts.

<FIG> shows example medical imaging environment <NUM>. Example medical imaging environment <NUM> includes medical digital imaging system <NUM>, subject S, and user U. In some implementations, medical digital imaging system <NUM> is in communication with server <NUM>, typically via network <NUM>. User U uses medical digital imaging system <NUM> to obtain one or more images of subject S. Other embodiments can include more or fewer components.

In some implementations, medical imaging environment <NUM> is in a traditional medical environment, such as a general practice facility, an urgent care facility, a hospital, and the like. Alternatively, medical imaging environment <NUM> is a non-traditional medical environment, such as a school. In some instances, user U is not formally medically trained.

Medical digital imaging system <NUM> obtains and processes one or more digital images of an ocular fundus or pupil of subject S. Medical digital imaging system <NUM> can be used to assist user U when screening for, monitoring, or diagnosing various eye conditions or diseases. Example eye conditions and diseases include refractive error, hypertension, diabetic retinopathy, glaucoma and papilledema. It will be appreciated that user U operating medical digital imaging system <NUM> can be different from a person evaluating the resulting images. For example, medical digital imaging system <NUM> transmits one or more images or results to server <NUM>. Then, a clinician different from user U can access server <NUM> to then analyze the results or images.

Medical digital imaging system <NUM> can have different sizes depending on the particular implementation. For example, medical digital imaging system <NUM> can be portable and sized such that it can be hand held. Portable, hand held sizing can be advantageous for off-site screening of a particular population, such as school children or nursing home occupants. In other implementations, medical digital imaging system <NUM> is configured for more stationary operations, such as within a medical facility.

In some implementations, medical digital imaging system <NUM> provides relatively immediate screening of subject S. Example screening can include capturing one or more images, displaying stimuli to subject S, capturing images of subject S's reaction to the stimuli, and an analysis of images including the subject's reaction. Based on this processing, medical digital imaging system <NUM> can display one or more different results reflecting analysis of the images.

Medical digital imaging system <NUM>, in some implementations, displays stimuli and captures images of the subject's ocular fundus or pupils. In turn, medical digital imaging system <NUM> transmits those images for later viewing and analysis by trained clinicians or digital image processing algorithms.

Medical digital imaging system <NUM> is particularly configured to capture digital images including a pupil of subject S and to identify edge pixels of the pupil. Subsequently, medical digital imaging system <NUM> can use the identified pupil edge in various ways. For instance, determining pupil edges aids in identifying pupil location and movement during refractive error screening. As another example, identifying a pupil edge can be used as part of eye tracking, where the eye tracking can be used to initiate image capture of an ocular fundus. Other uses of determining pupil edges in digital image are contemplated.

One technique for fundus imaging requires mydriasis, dilation of a subject's pupil, which can be painful and/or inconvenient to the subject S. Example medical digital imaging system <NUM> can be used in mydriatic or non-mydriatic conditions. That is, medical digital imaging system <NUM> can capture images without requiring a mydriatic drug to be administered to the subject S before imaging.

In terms of pupil dilation, medical digital imaging system <NUM> can capture images with pupil sizes smaller than <NUM>. In some instances, medical digital imaging system <NUM> can capture wide FOV images with pupil sizes no greater than <NUM> or even no greater than <NUM>. Of course, medical digital imaging system <NUM> can capture images with larger pupil sizes, such as those greater than <NUM>.

Medical digital imaging system <NUM> includes a housing that supports system components. For instance, the housing supports one or two apertures for imaging one or two eyes at a time. In some embodiments, the housing supports positional guides for the subject S, such as an adjustable chin rest. The positional guides help align the subject's eyes with the apertures. In some embodiments, the apertures are adjustable to align them with the subject's eyes. Once the subject's eyes are aligned, user U can initiate image capture sequencing.

Medical digital imaging system <NUM> is typically connected to network <NUM>. Network <NUM> can include any type of wireless network, a wired network, or any communication network known in the art. For example, wireless connections can include cellular network connections and connections made using protocols such as <NUM>. 11a, b, and/or g. In other examples, a wireless connection can be accomplished directly between medical digital imaging system <NUM> and an external display using one or more wired or wireless protocols, such as Bluetooth, Wi-Fi Direct, radio-frequency identification (RFID), or Zigbee. Other configurations are possible.

Server <NUM> communicates with medical digital imaging system <NUM> and additional devices. For example, server <NUM> receives fundus images from medical digital imaging system <NUM> and stores the images, and possible accompanying data such as patient data, in one or more databases. Clinicians can then access stored images for analysis. Server <NUM> includes one or more components of computing device <NUM> shown in <FIG>, described in more detail below.

<FIG> is a schematic diagram showing example components of medical digital imaging system <NUM>. Medical digital imaging system <NUM> includes lens <NUM>, illumination unit <NUM>, image sensor array <NUM>, infrared LED <NUM>, fixation LED <NUM>, display <NUM>, and computing device <NUM>. Each component is in communication with, at least, computing device <NUM>. Additional components of medical digital imaging system <NUM>, not shown in <FIG>, can include a speaker unit, a range finder unit, and a front window. Commercial embodiments of medical digital imaging system <NUM> include the Welch Allyn RetinaVue™ <NUM> Imager and the Welch Allyn Spot™ Vision Screener (Welch Allyn, Skaneateles Falls, NY). Other embodiments can include more or fewer components.

Lens <NUM> focuses light onto image sensor array <NUM>. Typically, lens <NUM> is adjustable. For example, lens <NUM> can be implemented as a variable focus liquid lens or a mechanically adjustable lens. A liquid lens is an optical lens whose focal length can be controlled by the application of an external force, such as a voltage. The lens includes a transparent fluid, such as water or water and oil, sealed within a cell and a transparent membrane. By applying a force to the fluid, the curvature of the fluid changes, thereby changing the focal length. This effect is known as electrowetting. A mechanically adjustable lens can change a focal length of the lens using, for example, by a stepping motor, a voice coil actuator, an ultrasonic motor, or a piezoelectric actuator.

Illumination unit <NUM> is an optional component and illuminates the eye fundus during certain image capture operations. Illumination unit <NUM> is configured to illuminate the eye fundus of the subject. Illumination of illumination unit <NUM> is coordinated with operation of image sensor array <NUM>.

As shown, illumination unit <NUM> includes LED array <NUM>. In other embodiments, illumination unit <NUM> can include one or more additional lighting units. In addition, lighting elements in illumination unit <NUM> can include non-light-emitting diode components. LED array <NUM> can be single color or multi-color. For example, LED array <NUM> is a three-channel RGB LED, where each die is capable of independent and tandem operation.

Image sensor array <NUM> receives and processes light reflected off of the subject. Image sensor array <NUM> can be a complementary metal-oxide semiconductor (CMOS) sensor array or a charge coupled device (CCD) sensor. Image sensor array <NUM> has a plurality of rows of pixels and a plurality of columns of pixels. For example, in various implementations, the image sensor array has about <NUM> by <NUM> pixels, about <NUM> by <NUM> pixels, about <NUM> by <NUM> pixels, about <NUM> by <NUM> pixels, or about <NUM> by <NUM> pixels. Other pixel sizes are possible.

Pixels in image sensor array <NUM> include photodiodes that have a light-receiving surface and have substantially uniform length and width. During exposure, the photodiodes convert the incident light to a charge. Exposure and readout of image sensor array <NUM> can be performed as rolling shutter or global shutter.

In rolling shutter exposure and readout, each row of pixels is exposed for the same time duration, however, each row of pixels is exposed at different points in time. Rolling shutter exposure begins at a top row of image sensor array <NUM> and each row below is successively exposed and then readout. Typically, exposure of the row below begins before completing exposure and readout of the row above. In this way, at any given time during image sensor array <NUM> exposure, more than one row of pixels are exposed.

In global shutter exposure, all of the photodiodes in image sensor array <NUM> are exposed simultaneously and for the same length of time. Then readout is performed for each photodiode. Because all photodiodes are subjected to readout at the same time, usually the image sensor array must wait until readout is completed before beginning the next frame's exposure. Thus, global shutter operations typically have slower frame rates than rolling shutter operations.

Infrared LED <NUM> illuminates the eye fundus with near-infrared light. Infrared light emitted by infrared LED <NUM> preferably has a central wavelength of <NUM> nanometers. In some instances, infrared LED <NUM> emits infrared light during a preview and/or eye tracking mode. Alternatively, infrared LED <NUM> emits infrared light during image capture operations part of the ocular examination.

Medical digital imaging system <NUM> optionally includes fixation LED <NUM>. Fixation LED <NUM> produces light to guide the subject's eye for alignment. Fixation LED <NUM> can be a single color or multicolor LED. For example, the fixation LED <NUM> can produce a beam of green light that appears as a green dot when subject S looks into the medical digital imaging system <NUM>. Other colors and designs, such as a cross, "x" and circle are possible.

Medical digital imaging system <NUM> can also include display <NUM>. Display <NUM> shows images and/or results produced by medical digital imaging system <NUM>. In the example embodiment, a housing supports display <NUM>. In other embodiments, display <NUM> connects to the image processor through wired or wireless connection, and can be instantiated as a smart phone, tablet computer, or external monitor.

Medical digital imaging system <NUM> also includes computing device <NUM>, which typically includes a processing unit and a computer readable storage device. In some embodiments, the computer-readable storage device stores data instructions, which when executed by the processing device, causes the processing device to perform one or more of the functions, methods, or operations, described herein. For example, computing device <NUM> includes pupil edge detection module <NUM>. Pupil edge detection module <NUM> is configured to perform the functions and operations described herein. An example computing device <NUM> is illustrated and discussed in more detail with reference to <FIG>.

<FIG> illustrates example method <NUM> for imaging a subject. Example method <NUM> includes aligning a subject (operation <NUM>), pupil identification (operation <NUM>), and imaging/screening (operation <NUM>). Example method <NUM> is typically performed using medical digital imaging system <NUM> described above. Example method <NUM> can be performed without administering mydriatic substances to the subject and, accordingly, a subject's pupil dilation is usually no greater than <NUM>. Other embodiments can include more or fewer operations.

Example method <NUM> begins by aligning a subject (operation <NUM>). Aligning a subject (operation <NUM>) can include adjusting a relative spacing between the subject and the medical digital imaging system. In some implementations, the subject is seated in a chair during examination. Alternatively, the subject may be aligned using one or more features on the medical digital imaging system, such a chin rest. In some instances, a user holds a hand-held version of medical digital imaging system and can move closer or further away from the subject while the subject is sitting or standing. Alignment of the subject and medical digital imaging system can be guided by on-screen displays that can instruct the user to move in one or more directions. Range finding units can guide this alignment.

After the subject is aligned (operation <NUM>), pupil identification (operation <NUM>) commences. Generally, pupil identification includes identifying one or more pupils of the subject in one or more digital images. Operations performed during pupil identification (operation <NUM>) are described in greater detail below.

Then, imaging and/or screening (operation <NUM>) operations are performed. In some instances, one or more images of the subject's ocular fundus are captured and subsequently analyzed. Alternatively, the subject undergoes ocular refraction screening. During either, or both, operations, one or more visual stimuli are displayed by the medical digital imaging system. In some instances, the medical digital imaging system displays one or more results of the imaging or ocular screening.

<FIG> illustrates example operations performed during pupil identification (operation <NUM>). As shown, pupil identification (operation <NUM>) includes receiving a digital image (operation <NUM>), determining a pupil edge (operation <NUM>), identifying a pupil center (operation <NUM>), and identifying pupil size (operation <NUM>).

Pupil identification (operation <NUM>) typically begins by receiving a digital image (operation <NUM>). Digital images received during operation <NUM> are captured by the image sensor array in the medical digital imaging system. These images are usually captured during near infrared light illumination. In some instances, one or more digital images are received from an external apparatus or retrieved from remote storage.

After receiving one or more digital images (operation <NUM>), a pupil edge is determined (operation <NUM>). Broadly, determining a pupil edge (operation <NUM>) includes operations resulting in generation of an image showing the pupil outline. Additional details regarding determining a pupil edge (operation <NUM>) are described below with reference to, at least, <FIG>.

A pupil center and pupil size (operations <NUM> and <NUM>) can be determined upon determining the pupil edge (operation <NUM>). Generally, identifying pupil center and pupil size (operations <NUM> and <NUM>) are used during ocular refractive error determinations and/or eye tracking operations.

<FIG> shows example operations performed during pupil edge determination (operation <NUM>). Typically, pupil edge determination (operation <NUM>) includes obtaining a pupil candidate region (operation <NUM>), determining a threshold (operation <NUM>), binarizing an image (operation <NUM>), determining pupil edges (operation <NUM>), and generating circle data (operation <NUM>).

After receiving a digital image (operation <NUM>), a pupil candidate region is obtained (operation <NUM>). One or more previous processes have determined that the pupil candidate region includes the pupil of a test subject. Determining a pupil candidate region is described in detail in <CIT>.

The pupil candidate region is square or rectangular shaped. Typically, the pupil candidate region is square and has side lengths between <NUM> and <NUM> pixels. The pupil candidate region is usually larger than <NUM> pixels by <NUM> pixels to provide additional space between the pupil edges and the edge areas of the pupil candidate region. In some implementations, the pupil candidate region is <NUM> by <NUM>. In other implementations, the pupil candidate region is <NUM> pixels by <NUM> pixels. Other sizes of pupil candidate region are contemplated.

After obtaining a pupil candidate region (operation <NUM>), a threshold is determined (operation <NUM>). <FIG> shows determining a threshold (operation <NUM>) in greater detail. As shown, determining a threshold includes obtaining a pixel grid size (operation <NUM>), generating a mean first portion pixel intensity (operation <NUM>), generating a mean second portion pixel intensity (operation <NUM>), generating a mean pixel intensity (operation <NUM>), determining a modified standard deviation (operation <NUM>), and generating a threshold value (operation <NUM>).

Generating a threshold (operation <NUM>) begins by obtaining a pixel grid size (operation <NUM>). Generally, the pixel grid is a region of interest that is evaluated during operation <NUM>. In operation <NUM>, two areas of the pupil candidate region are evaluated and the size and shape of those two areas are defined by the pixel grid. Usually, each region of interest has the same pixel grid size.

Regarding the location of the pixel grids within the pupil candidate region, each pixel grid is preferably positioned such that pupil and non-pupil objects are unlikely to be included. Example non-pupil objects include eye lids and eye lashes. The pixel grids are positioned along the left and right side edges of the pupil candidate region. In some instances, the pixel grids are centered along the left and right side edges of the pupil candidate region. The pixel grids are not positioned along the top or bottom edges of the pupil candidate region.

The pixel grid can be different polygonal shapes, but typically the pixel grid is rectangular. The pixel grid is usually sized such that it extends along an edge much more than it extends into the pupil candidate region towards the pupil. An example pixel grid size is <NUM> columns by <NUM> rows. Another example pixel grid size is <NUM> columns by <NUM> rows. Yet another pixel grid size is <NUM> columns by <NUM> rows. Other pixel grid sizes are contemplated.

Obtaining the pixel grid size (operation <NUM>) can include determining a pixel grid size based on one or more factors, such as a size of the pupil candidate region. In some instances, the pixel grid size is predetermined and obtaining the pixel grid size (operation <NUM>) includes retrieving a saved or predetermined pixel grid size.

Referring to <FIG>, an example pupil candidate region <NUM> is shown. Pupil candidate region <NUM> includes pupil <NUM>, glint area <NUM>, pixel grid <NUM> and pixel grid <NUM>. Pupil candidate region <NUM> is <NUM> pixels by <NUM> pixels.

Pixel grid <NUM> is positioned along the left edge of pupil candidate region <NUM> and vertically centered. Pixel grid <NUM> and pupil candidate region <NUM> are sized such that there is spacing S1 between pixel grid <NUM> and pupil <NUM>. Pixel grid <NUM> is positioned along the right edge of pupil candidate region <NUM> and vertically centered. Pixel grid <NUM> and pupil candidate region <NUM> are sized such that there is spacing S2 between pixel grid <NUM> and pupil <NUM>. When S1 and S2 are nonzero, there is a low likelihood that pixel grids <NUM> and <NUM> include artifacts or the pupil, which can negatively impact the subsequent determinations in operation <NUM>.

Pixel grid <NUM> and pixel grid <NUM> are similarly sized: <NUM> columns by <NUM> rows. Taking the upper left corner of pupil candidate region <NUM> as coordinate (<NUM>,<NUM>), coordinates for pixel grid <NUM> corners are: (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>). Pixel grid <NUM> is positioned along the right edge of pupil candidate region <NUM> and vertically centered. Coordinates for pixel grid <NUM> corners are: (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>).

In another implementation, coordinates for pixel grid <NUM> corners are: (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>). Pixel grid <NUM> is positioned along the right edge of pupil candidate region <NUM>. Coordinates for pixel grid <NUM> corners are: (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>).

Referring again to <FIG>, after obtaining a pixel grid size (operation <NUM>), a mean first portion pixel intensity is generated (operation <NUM>). Generating the mean first portion pixel intensity (operation <NUM>) includes evaluating a first portion of the pupil candidate region <NUM>. The first portion of the pupil candidate region is defined by the pixel grid. The first portion can be either the first pixel grid (e.g., pixel grid <NUM> in <FIG>) or the second pixel grid (e.g., pixel grid <NUM> in <FIG>); order is not important.

Generating the mean first portion pixel intensity (operation <NUM>) includes determining a pixel intensity for each pixel in the first portion. For example, in a pixel grid size of <NUM> columns by <NUM> rows, an intensity value is determined for each of the <NUM> pixels in the first portion. The intensity value can be on a variety of scales, such as <NUM>-<NUM>, <NUM>-<NUM>, etc..

Next, a summed first portion pixel intensity is determined by summing the pixel intensity for each pixel in the first portion. Then a first portion mean pixel intensity is determined by dividing the summed first portion pixel intensity by the number of pixels in the first portion. These operations can be shown as calculating the mean in equation (<NUM>), where Xi is the pixel intensity of each pixel and the first portion includes n pixels: <MAT>.

A mean second portion pixel intensity is determined (operation <NUM>) by evaluating a second portion of the pupil candidate region <NUM>. The second portion of the pupil candidate region is defined by the pixel grid. The second portion is the opposite portion from the first portion evaluated during operation <NUM>.

Generating the mean second portion pixel intensity (operation <NUM>) includes determining a pixel intensity for each pixel in the second portion. The intensity value is on the same scale as that used for the first portion, for consistency.

Next, a summed second portion pixel intensity is determined by summing the pixel intensity for each pixel in the second portion. Then a second portion mean pixel intensity is determined by dividing the summed second portion pixel intensity by the number of pixels in the second portion. These operations can be shown as calculating the mean in equation (<NUM>), where Yi is the pixel intensity of each pixel and the second portion includes n pixels: <MAT>.

After generating a mean first portion pixel intensity (operation <NUM>) and generating a mean second portion pixel intensity (operation <NUM>), a mean pixel intensity is generated (operation <NUM>). The mean pixel intensity is generated (operation <NUM>) by summing the mean first portion pixel intensity and the mean second portion pixel intensity, and dividing that sum by <NUM>.

As an alternative to operations <NUM>, <NUM>, and <NUM>, a mean pixel intensity is generated by treating the first portion pixels and the second portion pixels as a single data set. Then, the mean pixel intensity is generated by determining the mean of the pixel intensity for each pixel in the first portion and the second portion.

Next, a modified standard deviation is determined (operation <NUM>). The modified standard deviation is generated (operation <NUM>) by first calculating a standard deviation of each pixel intensity for each pixel in both the first portion and the second portion (i.e., treating the first portion and second portion as a single data set).

After calculating the standard deviation of the pixel intensity for each pixel within the first portion and the second portion, the modified standard deviation is determined (operation <NUM>) by multiplying the standard deviation by a multiplier. Typically, the multiplier is a number greater than <NUM> but no greater than <NUM>. In some implementations, the multiplier is <NUM>.

Then a threshold value is generated (operation <NUM>). The threshold value is generated (operation <NUM>) by summing the mean pixel intensity and the modified standard deviation.

Referring again to <FIG>, after determining the threshold, the image is binarized (operation <NUM>). Broadly, binarizing the image involves converting the digital pupil image into an image or grid where each pixel has one of two values, typically a <NUM> or a <NUM>. The resulting image or grid is a representation of the pupil in the digital pupil image.

Typically, the digital pupil image pixels have one or more values. For example, if the digital pupil image is in black and white, each pixel has one value, typically on a scale of <NUM>-<NUM>. In some instances, this value can be the same as a pixel intensity value. If the digital pupil image in color, each pixel can have three values (one red value, one green value, and one blue value), where each value is typically on a scale of <NUM>-<NUM>. In some implementations, an intensity value has been determined for each pixel.

Binarizing the image (operation <NUM>) includes evaluating a value of each pixel, such as the pixel intensity value, against the threshold value. If a pixel value is less than the threshold value, then that pixel is assigned a value corresponding to pixel intensities below the threshold, such as <NUM>. If the pixel value is greater than the threshold value, then that pixel is assigned a value corresponding to pixel intensities above the threshold, such as <NUM>. In an example implementation, with threshold value T, pixel values are assigned a value of <NUM> or <NUM> based on the following function: <MAT>.

After binarizing the image (operation <NUM>), pupil edges are determined (operation <NUM>). Determining pupil edges can include generating a new image or grid, where the only nonzero pixels, or pixels with values, are those considered to be on the pupil edge. Various techniques are available to convert the binarized image to a pupil edge image.

For instance, determining pupil edges (operation <NUM>) can include taking the leftmost and rightmost nonzero pixels in each row of the binarized image, and/or taking the topmost and bottommost nonzero pixels in each column of the binarized image, and assigning those pixels values of <NUM>. The other pixels are assigned values of <NUM>. Other implementations are possible.

After pupil edges are determined (operation <NUM>), circle data are generated (operation <NUM>). Circle data can include a center of the pupil, a pupil radius or diameter, and an image or grid showing circle data. In most implementations, pupil edge data generated during operation <NUM> are used during a best-fit circle determination. Results of the best-fit circle determination include a center of a circle, a circle radius, and a circle diameter. The circle represents the subject's pupil.

In some instances, portions of the pupil edge image are omitted from the best-fit circle generation (operation <NUM>). These portions can sometimes include artifacts that may cause an inaccurate determination of the best-fit circle. In some implementations, a top portion of the pupil edge image is excluded from the best fit circle generation. For example, the top third (<NUM>/<NUM>) of the image is excluded. In some implementations, a bottom portion of the pupil edge image is excluded from the best fit circle generation. For example, the bottom quarter (¼) of the image is excluded. In some implementations, both a top portion and a bottom portion are excluded from the best fit circle.

<FIG> shows exemplary image data <NUM> generated during example pupil identification method <NUM> described above. Data set <NUM> includes image data obtained while imaging a <NUM> artificial eye. Image <NUM> is the raw pupil data image. Image <NUM> is a <NUM> pixel by <NUM> pixel image and previous processing has determined that image <NUM> likely includes the subject's pupil.

Image <NUM> is a binarized image of image <NUM>. Image <NUM> was generated using the threshold value determined during evaluation of image <NUM>. Image <NUM> is a pupil edge image generated from binarized image <NUM>. Image <NUM> is a limited circle data image generated by removing a top portion and a bottom portion from image <NUM>. In this example, data from the top <NUM>/<NUM> of image <NUM> and data from the bottom ¼ of image <NUM> were removed to generate image <NUM>. In a subsequent step, a best-fit circle is generated from data in image <NUM>.

<FIG> shows exemplary image data <NUM> generated during example pupil identification method <NUM> described above. Data set <NUM> includes image data obtained while imaging a <NUM> artificial eye with an eye lid. Image <NUM> is the raw pupil data image. Image <NUM> is a <NUM> pixel by <NUM> pixel image and previous processing has determined that image <NUM> likely includes the subject's pupil.

Image <NUM> is a binarized image of image <NUM>. Image <NUM> was generated using the threshold value determined during evaluation of image <NUM>. As shown, image <NUM> includes eye lid artifacts <NUM>.

Image <NUM> is a pupil edge image generated from binarized image <NUM>. As shown, image <NUM> includes artifacts <NUM> resulting from pupil edge determination based on image <NUM>. Artifacts <NUM> are not part of the pupil edge. Image <NUM> is a limited circle data image generated by removing a top portion and a bottom portion from image <NUM>. In this example, data from the top <NUM>/<NUM> of image <NUM> and data from the bottom ¼ of image <NUM> were removed to generate image <NUM>. By removing data from the top <NUM>/<NUM> of image <NUM>, artifacts <NUM> will not be considered during best-fit circle generation based on image <NUM>. In a subsequent step, a best-fit circle is generated from data in image <NUM>.

<FIG> show example images generated from imaging a human eye. Each digital pupil image <NUM>, <NUM>, <NUM> includes one or more artifacts.

<FIG> shows exemplary image data <NUM> generated during example pupil identification method <NUM> described above. Image <NUM> is the raw pupil data image. As shown, eye lid and eye lash artifacts <NUM> are present in image <NUM>. Image <NUM> is a <NUM> pixel by <NUM> pixel image and previous processing has determined that image <NUM> likely includes the subject's pupil.

Image <NUM> is a binarized image of image <NUM>. Image <NUM> was generated using the threshold value determined during evaluation of image <NUM>. As shown, image <NUM> includes eye lid/eye lash artifacts <NUM>.

<FIG> shows exemplary image data <NUM> generated during example pupil identification method <NUM> described above. Image <NUM> is the raw pupil data image. As shown, cataract artifacts <NUM> are present in image <NUM>. Image <NUM> is a <NUM> pixel by <NUM> pixel image and previous processing has determined that image <NUM> likely includes the subject's pupil.

Image <NUM> is a binarized image of image <NUM>. Image <NUM> was generated using the threshold value determined during evaluation of image <NUM>. As shown, image <NUM> includes artifacts <NUM>.

Image <NUM> is a pupil edge image generated from binarized image <NUM>. Image <NUM> is a limited circle data image generated by removing a top portion and a bottom portion from image <NUM>. In this example, data from the top <NUM>/<NUM> of image <NUM> and data from the bottom ¼ of image <NUM> were removed to generate image <NUM>. In a subsequent step, a best-fit circle is generated from data in image <NUM>.

<FIG> shows an example computing device <NUM> of medical digital imaging system <NUM>. As illustrated, example computing device <NUM> includes at least one central processing unit ("CPU") <NUM>, memory <NUM>, and a system bus <NUM> that couples memory <NUM> to the CPU <NUM>. Memory <NUM> includes system memory <NUM> and mass storage device <NUM>. System memory <NUM> includes a random access memory ("RAM") <NUM> and a read-only memory ("ROM") <NUM>. A basic input/output system that contains the basic routines that help to transfer information between elements within the example computing device <NUM>, such as during startup, is stored in the ROM <NUM>. Memory <NUM> further includes mass storage device <NUM>. Mass storage device <NUM> is able to store software applications <NUM>, operating system <NUM>, and data.

Mass storage device <NUM> is connected to CPU <NUM> through a mass storage controller (not shown) connected to the system bus <NUM>. Mass storage device <NUM> and its associated computer-readable data storage media provide non-volatile, non-transitory storage for the example computing device <NUM>. Although the description of computer-readable data storage media contained herein refers to a mass storage device, such as a hard disk or solid state disk, it should be appreciated by those skilled in the art that computer-readable data storage media can be any available non-transitory, physical device or article of manufacture from which the central processing unit can read data and/or instructions.

Computer-readable data storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable software instructions, data structures, program modules or other data. Example types of computer-readable data storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROMs, digital versatile discs ("DVDs"), other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the example computing device <NUM>.

According to various embodiments, the example computing device <NUM> may operate in a networked environment using logical connections to remote network devices through the network <NUM>, such as a wireless network, the Internet, or another type of network. The example computing device <NUM> may connect to the network <NUM> through a network interface unit <NUM> connected to the system bus <NUM>. The network <NUM> may be a protected network. It should be appreciated that the network interface unit <NUM> may also be utilized to connect to other types of networks and remote computing systems. The example computing device <NUM> also includes an input/output controller <NUM> for receiving and processing input from a number of other devices, including a touch user interface display screen, or another type of input device. Similarly, the input/output controller <NUM> may provide output to a touch user interface display screen or other type of output device.

As mentioned briefly above, the mass storage device <NUM> and the RAM <NUM> of the example computing device <NUM> can store software instructions and data. The software instructions include an operating system <NUM> suitable for controlling the operation of the example computing device <NUM>. The mass storage device <NUM> and/or the RAM <NUM> also store software applications <NUM>, that when executed by the CPU <NUM>, cause the example computing device <NUM> to provide the functionality of the example computing device <NUM> discussed in this disclosure. For example, the mass storage device <NUM> and/or the RAM <NUM> can store software instructions that, when executed by the CPU <NUM>, cause the medical digital imaging system <NUM> to determine pupil edge pixels in digital images.

Although various embodiments are described herein, those of ordinary skill in the art will understand that many modifications may be made thereto within the scope of the present disclosure. Accordingly, it is not intended that the scope of the disclosure in any way be limited by the examples provided.

Claim 1:
A computer-implemented method of identifying a pupil edge in a digital image, the method comprising:
receiving (<NUM>) a digital pupil image;
determining (<NUM>) a pupil candidate region in the digital pupil image;
generating (<NUM>) a mean first portion pixel intensity of a first portion of the pupil candidate region (<NUM>) and generating (<NUM>) a mean second portion pixel intensity of a second portion of the pupil candidate region (<NUM>), wherein the first portion is an area positioned along the left side edge of the pupil candidate region (<NUM>) and the second portion is an area positioned along the right side edge of the pupil candidate region (<NUM>);
averaging the mean first portion pixel intensity and the mean second portion pixel intensity to generate (<NUM>) an average pixel intensity;
determining (<NUM>) a modified standard deviation, including:
calculating a standard deviation of a pixel intensity for each pixel within the first portion and the second portion; and
multiplying the standard deviation by a multiplier, wherein the multiplier is more than <NUM> but no greater than <NUM>;
generating (<NUM>) a threshold value by summing the average pixel intensity and the modified standard deviation; and
using the threshold value, identifying the pupil edge in the digital pupil image.