White eye portraiture system and method

A system for capturing an image when an amount of sclera is visible in a preview image comprises a photosensor configured to detect an image, a memory configured to store at least a sclera setting, a processor configured to determine when at least one face is present in the detected image and further configured to determine an amount of sclera present in the face so that the determined amount of sclera is compared to the sclera setting, and an actuator configured to initiate capture of the detected image such that the detected image is captured when the determined amount of sclera is at least equal to the sclera setting.

TECHNICAL FIELD

The present invention is generally related to digital camera technology and, more particularly, is related to a system and method for capturing an image when an amount of sclera is visible in a preview image.

BACKGROUND OF THE INVENTION

In digitally based image capturing devices an image or “photograph” of an object is stored in a digital data format in the memory within, or coupled to, the image capturing device. A nonlimiting example of a digital image capturing device is the digital camera that captures still images and/or video images. Unfortunately, it is often difficult to capture the digital image at the precise moment that subjects have their eyes open. This problem is particularly acute when a large number of subjects are being photographed at the same time.

SUMMARY OF THE INVENTION

The present invention provides a system and method for capturing an image when an amount of sclera is visible in a preview image. An embodiment provides for capturing an image, comprising the steps of determining that an image preview includes at least one face, determining an amount of sclera in the face, retrieving a sclera setting, and capturing the image when the determined amount of sclera is at least equal to the sclera setting.

DETAILED DESCRIPTION

The present invention provides a system and method for capturing an image when an amount of sclera is visible in a preview image. Sclera is defined herein to be the white portion of an individual's eye visible when the eye is open.

FIG. 1is a block diagram of one embodiment of a digital camera100including a memory element104storing a white eye portraiture system106. Memory element104further includes at least regions allocated for the sclera setting144, image display control logic146, and the white eye portraiture system106. External and internal components of the digital camera are separated by cut-away lines102.

Digital camera100includes at least a lens unit108, an image capture actuation button110(or other suitable actuator), a viewing lens112, a power switch114, a memory unit interface116, a sclera specifying device118, and a plug-in interface unit120. Plug-in interface unit120, in one embodiment, includes a plurality of connection pins122. A display124is used for previewing images prior to capturing or for viewing captured images. For convenience of illustration, display124is illustrated on the top of digital camera100.

Operation of the digital camera100is initiated by actuation of the power switch114or an equivalent device having the same functionality. When digital camera100is turned on, display124typically remains off so as to conserve limited battery power of the digital camera100. In one embodiment, actuation of control button126turns on display124such that the user of digital camera100may view an image currently detected through the lens unit108on the display124. This current image displayed on display124is referred to herein as a preview image. Image display control logic146is executed by camera processor128such that the preview image is displayed.

Lens unit108is used for focusing the image on photosensor136. When the operator has focused the image to be captured and is satisfied with it, the operator actuates the image capture actuation button110(also referred to as a shutter button or a shutter release button) to cause the digital camera100to record (capture) a digital image, thus “photographing” the image.

In accordance with the present invention, when the white eye portraiture system106is executed by processor128such that a determination is made that a sufficient amount of sclera is visible in the preview image, as described in greater detail below, digital camera100captures the image. Data corresponding to the captured image is stored in the camera data image region130of memory element104.

A personal computer, printer or other processing device (not shown) is typically employed with digital cameras such that digital images captured by the digital camera may be retrieved, processed, printed and/or e-mailed. The personal computer or other processing device includes a wire connector interface (not shown) for communicating with digital camera100via plug-in interface unit120. The wire connector interface couples digital camera100and the personal computer by mating a wire connector (not shown) having a suitable plug-in attachment with plug-in interface120. Other embodiments employ a wireless interface or a card-based interface. By providing suitable instructions to the personal computer, other processing device, and/or camera processor128, the captured image data is transferred from the camera image data region130, via connection132, into the personal computer or other processing device for further processing.

In another embodiment, digital image data is stored in memory module unit134. Memory module unit134is a module of memory configured to couple to digital camera100via memory unit interface116. Accordingly, digital image data is transferred to the personal computer or other processing device using memory module unit134. When the white eye portraiture system106is executed by processor128, in response to actuation of the image capture actuation button110, such that a determination is made that a sufficient amount of sclera is visible in the preview image, as described in greater detail below, digital camera100captures the image. Data corresponding to the captured image is stored in memory module unit134. In one embodiment, memory storage interface140configures the digital image data into a suitable format for transference to the memory module unit134, via connection142. In another embodiment, memory storage interface140is not included as the camera processor128directly transmits suitably formatted digital image data to the memory module unit134via connections138and142directly.

Digital image data is transferred to the personal computer by removing memory module unit134from the digital camera100and coupling memory module unit134to a personal computer. Typically, a convenient coupling port or interface is provided on the surface of personal computer or other processing device such that memory module unit134is directly coupled to the personal computer. Once memory module unit134is coupled to the personal computer memory module interface, digital image data is transferred to the personal computer or other processing device.

Cut-away lines102demark components residing on the outside surfaces of the digital camera100and components residing internally in the digital camera100. Thus, the control button126, the sclera specifying device118, lens unit108, image capture actuation button110, power switch114, memory unit interface116, plug-in interface120, and display124are recognized as components residing on the surfaces of the digital camera100.

The internal components of digital camera100are illustrated between the two cut-away lines102. Internal components of one embodiment of digital camera100includes at least a camera processor128, a photosensor136, and a memory element104. Another embodiment of digital camera100employing memory module unit134includes memory storage interface140. For convenience, digital camera100is illustrated as storing captured image data in the camera image data region130of memory element104and in the memory module unit134. In other embodiments, captured image data is stored in either camera image data region130or memory module unit134.

Photosensor136is disposed in a suitable position behind the lens unit108such that an image (not shown) may be focused onto photosensor136for capturing. Photosensor136detects an image through lens unit108and provides information corresponding to the detected image to camera processor128, via connection148. When digital camera100is operating in a mode that displays the preview image on display124, via connection154, the user of digital camera100previews the image to determine if the user wants to “photograph” the detected current image. If so, the user of digital camera100actuates image capture actuation button110to initiate capture of the image.

When the white eye portraiture system106is operating, upon actuation of image capture actuation button110, the white eye portraiture system106analyzes the preview image, described in greater detail below, to determine the amount of sclera present in the preview image. According to the present invention, the white eye portraiture system106permits the storing of the preview image into camera image data region130only when a sufficient amount of sclera is detected in the preview image. If insufficient sclera is detected, the preview image is not saved. Thus, an image is captured only when a sufficient amount of sclera is detected. That is, when the image includes people looking at the digital camera100, the image is captured when the people have their eyes open.

In the event that insufficient sclera is detected, image data is again retrieved from the photosensor136such that another preview image is analyzed. Since the preview image is the current image sensed by photosensor136on a nearly real-time basis, the current preview image analyzed each time at block208will be different. The time difference between the current preview image and the previously analyzed image is at least equal to the exposure time that the pixels in photosensor136accumulate charge. In one embodiment, a delay is added to the logic of the white eye portraiture system106such that a predefined time between analysis of the preview images occurs. Accordingly, one or more preview images are analyzed, and the image is captured when sufficient sclera is detected. Statistically, the individuals being photographed will all, at some point during a very brief period of time, have their eyes open such that a pleasing image of the people is captured (rather than an undesirable image having some, or all, of the people with closed eyes).

In one embodiment, the white eye portraiture system106is always operational. In another embodiment, the white eye portraiture system106is optional. The optional white eye portraiture system106is activated when a user selects an optional white eye portraiture mode of operating the digital camera100. This selection of the white eye portraiture mode of operating, referred to as arming, may be effected via a menu system shown on display124. In other embodiments, the user arms the white eye portraiture system106via sclera specifying device118, control button126or another specially dedicated controller device, such as a control button, a push-button, a switch or the like.

In one embodiment, a sclera specifying device118is included on the exterior of camera100. When the sclera setting144is a variable parameter, the user actuates the sclera specifying device118(FIG. 1). The sclera setting144is communicated from sclera specifying device118to processor128, via connection152.

The sclera specifying device118may be implemented as any suitable controller device, such as, but not limited to, a calibrated dial, a push-button or the like. Another embodiment employs a menu displayed on display124such that the user may specify sclera settings by interfacing with the menu selections using known menu operating devices (not shown) on camera100.

In one embodiment, the specified sclera setting144is stored in memory element104. In another embodiment, the sclera setting is directly detected from the sclera specifying device118(FIG. 1). In another embodiment, described in greater detail below, stores the sclera setting in memory module unit134.

In yet another embodiment, the sclera setting144is a default setting preprogrammed into memory element104(or another suitable memory device residing in the digital camera100, or in memory module unit134). Thus, the user does not need to understand sclera settings or need to take the effort to specify a sclera setting. Such an embodiment is advantageous in providing a white eye portraiture system106that is easy to operate.

The white eye portraiture system106is a source program, executable program (object code), script, or any other entity comprising a set of instructions to be performed. When implemented as a source program, then the program is translated via a compiler, assembler, interpreter, or the like, which may or may not be included within memory element104. Furthermore, the white eye portraiture system106can be written in (a) an object oriented programming language, which has classes of data and methods, or (b) a procedure programming language, which has routines, subroutines, and/or functions, for example but not limited to, C, C++, Pascal, Basic, Fortran, Cobol, Perl, Java, and Ada. In the currently contemplated best mode of practicing the invention, the white eye portraiture system106employs the C and/or the C++ programming language.

The white eye portraiture system106according to the present invention can be implemented in software (e.g., firmware), hardware, or a combination thereof. In one embodiment, the white eye portraiture system106is implemented in software, as an executable program, and is executed by the camera processor128. The camera processor128is a hardware device for executing software, particularly that stored in memory element104. The camera processor128can be any custom made or commercially available camera processor, or any other suitable general purpose processor.

The memory element104can include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.) and nonvolatile memory elements (e.g., ROM, hard drive, tape, CDROM, etc.). Moreover, the memory element104may incorporate electronic, magnetic, optical, and/or other types of storage media. Note that the memory element104can have a distributed architecture, where various components are situated remote from one another, but can be accessed by the camera processor128.

When the white eye portraiture system106is implemented in software, the white eye portraiture system106can be stored on any computer readable medium for use by or in connection with any computer related system or method. In the context of this document, a computer readable medium is an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in connection with a computer related system or method. The white eye portraiture system106can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “computer-readable medium” can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a nonexhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic, compact flash card, secure digital card, or the like), flash memory, a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical) or other suitable memory.

In an alternative embodiment, where the white eye portraiture system106is implemented in hardware, the white eye portraiture system106can implemented with any or a combination of the following technologies: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.

FIG. 2is a flowchart200of the white eye portraiture system106(FIG. 1). The flowchart ofFIG. 2shows the architecture, functionality, and operation of a possible implementation of the white eye portraiture system106. In this regard, each block represents a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted inFIG. 2. For example, two blocks shown in succession inFIG. 2may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved, as will be further clarified hereinbelow.

At block202, the white eye portraiture system106is armed. The white eye portraiture system106may be armed in many ways, including but not limited to, pressing the image capture actuation button110. The user may press the image capture button110when satisfied that the preview image seen through lens112and/or on display124satisfies subject criteria such as that the subject(s) of the photograph are smiling. In one embodiment, the white eye portraiture system106is triggered whenever digital camera100is turned on.

In another embodiment, the system is activated when a user selects an optional white eye portraiture mode of operating the digital camera100. This selection of the white eye portraiture mode of operating, referred to as arming, may be effected via a menu system shown on display124. In other embodiments, the user arms the white eye portraiture system106via sclera specifying device118, control button126or another specially dedicated controller device, such as a control button, a push-button, a switch or the like. After the white eye portraiture system106is armed, the process proceeds to block204.

At block204, the white eye portraiture system106determines whether the preview image includes one or more faces. Exemplary systems for determining whether a preview image includes one or more faces, include, but are not limited to, the systems described in U.S. Pat. Nos. 5,642,431, 5,987,154, and 5,835,616, 5,642,431, 5,987,154, and 5,835,616 are entirely incorporated herein by reference.

U.S. Pat. No. 5,642,431 discloses an exemplary face detection system and method for detecting human faces includes a pattern prototype synthesizer having a preprocessor, an image database, and a prototype generator. The preprocessor normalizes and filters input images. The image database stores face and non-face example patterns. The prototype generator produces face prototypes and non-face prototypes. The face protection system of U.S. Pat. No. 5,642,431 also includes an image classifier having a distance metric processor and a network. The distance metric processor produces a two-value distance measure between the applied image and each of the prototypes. The network processes this distance information to produce a detection signal indicative of whether a face has been detected in the applied image. In one embodiment, the method above is inherently included within block204.

U.S. Pat. No. 5,987,154 discloses a system and method for detecting people in images. The system and method of U.S. Pat. No. 5,987,154 includes obtaining the difference between one image and a previous image to extract regions of motion; comparing local curvature extremes at the boundary of motion regions with a stored model of a human head; and identifying the local boundary corresponding to the model of the human head. In one embodiment, the method above is inherently included within block204.

U.S. Pat. No. 5,835,616 describes a two-step process for automatically finding a human face in an electronically digitized image, and for confirming the existence of the face by examining facial features. Step1is to detect the human face and is accomplished in stages that include enhancing the digital image with a blurring filter and edge enhancer in order to better set forth the unique facial features such as wrinkles, and curved shapes of a facial image. After prefiltering, preselected curves sometimes referred to as snakelets are dropped on the image where they become aligned to the natural wrinkles and curves of a facial image. Step2is to confirm the existence of the human face in seven stages by finding facial features of the digital image encompassing the chin, sides of the face, virtual top of the head, eyes, mouth and nose of the image. Ratios of the distances between these found facial features can be compared to previously stored reference ratios for recognition. In one embodiment, these steps are inherently included within block204of the white eye portraiture system106and are not described in detail inFIG. 2for convenience.

Thus, at block204, if the white eye portraiture system106determines that the preview image includes one or more faces, the process proceeds to block206. If the white eye portraiture system106determines the preview image does not include one or more faces, the process proceeds to block210. At block210, digital camera100captures the preview image. The captured image data corresponding to the captured image may be saved in camera image data region130or in memory module unit134, depending upon the embodiment of digital camera100.

At block206, a sclera setting144is retrieved. As described above, the sclera setting144in one embodiment is a variable setting indicating an amount or a percentage of sclera in the faces of the preview image that should be present before the image is captured. In a simple non-limiting example where the preview image includes one face and the sclera setting144is specified at 80 percent, the white eye portraiture system106allows the image to be captured when the white eye portraiture system106determines that the preview image includes at least 80 percent of the sclera that the white eye portraiture system106expects to be present in the preview image if the subject(s) in the preview image all have sufficiently wide-open eyes.

At block208, the white eye portraiture system106determines whether the preview image has sufficient sclera present based on the sclera setting of block206. Systems and methods for determining whether a preview image includes sclera, include, but are not limited to the systems described in U.S. Pat. No. 5,432,866, entirely incorporated herein by reference. Accordingly, in one embodiment, such a method or an equivalent method is inherently included within block208. When the preview image includes sufficient sclera (the YES condition), the process proceeds to block210. The process then ends at block212.

U.S. Pat. No. 5,432,866 discloses a method for detecting eye structure and its apparatus. The method examines an image having an eye and its vicinity of a human face photographed from the vertex to the jaws to detect an area positioned between a position where brightness changes from lightness to darkness and a position where brightness changes from lightness to darkness and then to lightness, and detects from the above area an area whose width is larger than the pupil's width and where an average of brightness values in the above area is darkest as the upper eyelid area.

And, the inner canthus, outer canthus and their adjacent upper eyelid line are detected by finding the darkest point on the above image to detect the inner canthus and outer canthus points and their adjacent dot groups which become candidates for the upper eyelid line, and by carrying out the logical OR between the above upper eyelid area and the above dot groups. Accordingly, in one embodiment, such a method or an equivalent method is inherently included within block208.

However, if at block208, sufficient sclera is not present (the NO condition), the process returns to block204such that the current preview image is again analyzed. Since the preview image is the current image sensed by photosensor136on a nearly real-time basis, the current preview image analyzed each time at block208will be different. The time difference between the current preview image and the previously analyzed image is at least equal to the exposure time that the pixels in photosensor136accumulate charge. In one embodiment, a delay is added to the logic of the white eye portraiture system106such that a predefined time between analysis of the preview images occurs.

Thus, if a face is detected, but sufficient sclera is not indicated, the image will not be captured by the camera100. When sufficient sclera is detected, the digital camera100captures the image. Accordingly, the logical loop of blocks204,206and208is repeated such that one or more preview images are analyzed until a sufficient amount of sclera is detected at block208. Eventually, sufficient sclera is detected at block208or the preview image no longer includes a face at block204. Accordingly, the image is captured. In another embodiment, release of the image capture actuation button110causes the process to immediately proceed to block212and end.

FIG. 3is a block diagram of an embodiment according to the present invention storing the sclera setting302in memory module unit134. Sclera setting302corresponds to the above described sclera setting144, and accordingly, sclera setting is equally applicable to any of the above described embodiments of the present invention. Thus, when a setting is specified for the amount or the percentage of sclera that must be present before a preview image is saved, the setting is stored as sclera setting302. When the preview image is saved in accordance with the present invention, the preview image is stored into the image memory region304. Such storing of the sclera setting302and captured images in the image memory region304is permitted when the memory module unit134is coupled to digital camera100by inserting memory module unit134into the memory unit interface116, as indicated by the path of insertion306(illustrated by the dash-arrowhead line).