Fingerprint authentication during holographic object display

A display apparatus including a display; a holographic projector a fingerprint scanner to read a fingerprint from a user's finger spaced from the fingerprint scanner when placed in proximity to the holographic object; a camera to image the holographic object and the user's finger; a non-transitory storage medium that stores instructions; and a processor that executes the instructions including: responsive to the receipt of images from the camera, perform image analysis to identify a position of the user's finger with respect to the display surface and the holographic object, identify a position of the holographic object with respect to the display surface and identify the proximity of the user's finger with the holographic object; and responsive to the image analysis to identify the proximity of the user's finger with the holographic object, provide an input to the fingerprint scanner to read the fingerprint of the user's finger.

BACKGROUND

The present exemplary embodiments pertain to holographic displays in which the displayed object is a three dimensional holographic image and, more particularly, to authentication of a user so as to permit interaction of the user with the three dimensional holographic image.

Next generation computer systems may use holographic objects for various user interactions. The user may have the feeling of interacting with digital objects while performing any activity with the physical objects.

BRIEF SUMMARY

The various advantages and purposes of the exemplary embodiments as described above and hereafter are achieved by providing, according to an aspect of the exemplary embodiments, a display apparatus for fingerprint authentication during holographic object display comprising: a display having a display surface; a holographic projector to display a holographic object above and spaced from the display surface; a fingerprint scanner to read a fingerprint from a user's finger spaced from the fingerprint scanner when placed in proximity to the holographic object; a camera to image the holographic object and the user's finger; a non-transitory storage medium; and a processor that executes the instructions. The processor executes the instructions to: responsive to the receipt of images from the camera, perform image analysis to: identify a position of the user's finger with respect to the display surface and the holographic object, identify a position of the holographic object with respect to the display surface and identify the proximity of the user's finger with the holographic object; and responsive to the image analysis to identify the proximity of the user's finger with the holographic object, provide an input to the fingerprint scanner to read the fingerprint of the user's finger.

According to another aspect of the exemplary embodiments, there is provided a computer-implemented method for fingerprint authentication during holographic object display comprising: displaying by a holographic projector a holographic object above and spaced from a display surface of a display; reading by a fingerprint scanner a fingerprint from a user's finger spaced from the fingerprint scanner when placed in proximity to the holographic object; performing by a camera an image analysis identifying: a position of the user's finger with respect to the display surface and the holographic object, a position of the holographic object with respect to the display surface and the proximity of the user's finger with the holographic object; responsive to identifying by the camera the proximity of the user's finger with the holographic object, providing an input to the fingerprint scanner to read the fingerprint of the user's finger; reading by the fingerprint scanner the fingerprint of the user's finger in proximity with the holographic object; and a processor to perform the method.

According to a further aspect of the exemplary embodiments, there is provided a computer program product for fingerprint authentication during holographic object display, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising: displaying by a holographic projector a holographic object above and spaced from a display surface of a display; reading by a fingerprint scanner a fingerprint from a user's finger spaced from the fingerprint scanner when placed in proximity to the holographic object; performing by a camera an image analysis identifying: a position of the user's finger with respect to the display surface and the holographic object, a position of the holographic object with respect to the display surface and a proximity of the user's finger with the holographic object; responsive to identifying by the camera the proximity of the user's finger with the holographic object, providing an input to the fingerprint scanner to read the fingerprint of the user's finger; and reading by the fingerprint scanner the fingerprint of the user's finger in proximity with the holographic object.

DETAILED DESCRIPTION

Next generation computing systems may use holographic objects for various user interactions. Through the holographic objects, the user may unlock the computing system, share information, interact with various other participating users etc. It would thus be important to have an authentication system while interacting with the holographic object. Some users may not be authorized to interact with one or more holographic objects or to execute a specific function, whereas other users may be authorized to interact with the same holographic object and/or execute the specific function. As the user may be interacting with the holographic object with the user's fingers, there is a need for a fingerprint authentication system while interacting with the holographic object.

The exemplary embodiments propose an apparatus, a method and a computer program product.

An electronic device may have a projected fingerprint scanner installed along with a holographic projector. While projecting any holographic object above a display surface of the electronic device, the electronic device may also have a capability to scan a user's fingerprint at or near the point where the user's finger touches the holographic object (i.e., touchpoint) by sharing the spatial coordinates of the holographic object with the fingerprint scanner.

A camera, and optionally additional sensors such as an ultrasonic sensor, installed in the electronic device may perform image analysis to identify the relative position of the user's fingers, holographic objects and the touch point of a user's finger on a holographic object. The fingerprint scanning function may be enabled if the holographic objects require authentication.

While the fingerprint scanner may be scanning the user's finger, the holographic object to be touched may be positioned by the holographic projector in such a way that the fingerprint scanner covers the entire scanning area of the user's finger and correspondingly the position of the fingerprint scanner may be auto adjusted to scan the user's finger at whatever may be the spatial position of the user's finger in the three dimensional (3D) space.

During the authentication process, software may validate the following information and, if authentication is successful, the user may interact with the holographic object:Software may ensure the user's finger has touched a holographic object either by sensing the position of the user's finger in the 3D space or by using non-contact tactile feedback of holographic shapes;Software may identify which holographic object is touched as there may be a plurality of holographic objects displayed;Software may identify which side of the holographic object is touched; andAuthentication may be used to execute any action on the holographic object, such as changing position of the holographic object, rotating the holographic object, etc or application access such as accessing emails, folders and so on. The software may check if the user has sufficient right to execute the required interaction with the holographic object.

If the user is not authorized to perform the interaction on the selected holographic object, then audio and/or visual feedback may be provided, such as vibrating the holographic object, changing the color or brightness or emitting an audio based alert.

Referring to the Figures in more detail, and particularly referring toFIG. 1, there is illustrated an exemplary electronic device10in which there may be a display12which may display typical digital images but may also display holographic images such as 3D holographic object30. The holographic object30is a simple shape in the form of a cube but it should be understood that more complex images may be formed. Further, while holographic object30is displayed inFIG. 1, there may be a plurality of holographic objects30formed. Such holographic objects30may be, for example, a keypad or a set of file folders.

The display12may have a display surface14. For purposes of illustration and not limitation, the electronic device10is a smartphone but could also be any electronic device having a display such as a tablet or a laptop.

The electronic device10may include one or more cameras16for taking pictures but also having the capability to image objects and through software to perform image analysis of the objects. The electronic device10may further include a plurality of holographic projectors18which may project a holographic object30above the display surface14. Some of the holographic projectors18may be on one side of the display12while other of the holographic projectors18may be on the other side of the display12.

The electronic device10may further include one or more projection fingerprint scanners20, hereafter just fingerprint scanners20, which may read the fingerprint of a user's finger that is in proximity to the holographic object30. That is, the fingerprint scanners are noncontact fingerprint scanners as they may be able to read a user's fingerprint spaced above the display surface14.

The fingerprint scanners20may be able to read the user's fingerprint at any location in range of the fingerprint scanners20, which may be several feet away from the fingerprint scanners20. For purposes of the exemplary embodiments, it is only necessary to be able to read a user's fingerprint when the user's finger32has actually touched the holographic object30at34, known as a touchpoint, or at least in very close proximity to the holographic objects30, say within a couple of millimeters.

The threshold for when the user's fingerprint is read may be configured by the user. A threshold of, for example, 2 millimeters may be a reasonable threshold so that when the user's finger32is 2 millimeters or less from the holographic objects30the user's fingerprint is read but when the user's finger32is more than 2 millimeters from the holographic objects30, the user's fingerprint is not read.

The fingerprint scanners20may work by a photographic process in which an image of a fingerprint may be photographically captured using polarized light and then the image is processed to sharpen the image.

FIG. 2is an elevation view conceptually illustrating the components of display12which may have the capability to display holographic objects. The display12may have a body22on which is supported a light source24. Preferably the light source24is a coherent light source such as a laser. Located in line with the light source24is an image hologram26which may display a three dimensional holographic object30when illuminated by the light source24. “Three-dimensional holographic object”30refers to a complete three-dimensional image in which, when a displayed image is viewed from a variety of directions, the image appears to be three-dimensional.

The image hologram26of the 3D holographic object30may be projected onto lens28which then focuses the 3D holographic object30(hereafter just “holographic object”) to be displayed above display surface14by holographic projectors18. Also shown on display surface14are cameras16.

As indicated earlier, it may be desirable or necessary to control access to the holographic object30since the holographic object30may control access to certain functions or applications in the electronic device10. Further, the holographic object30may be subject to manipulation by the user so access to the holographic object30may be desirable. Accordingly, authentication of a user's fingerprint indicating that the user has sufficient right to execute the required interaction with the holographic object30may be required before the user is able to access the holographic object30.

Authentication of a user's fingerprint may be accomplished by a combination of the cameras16, the fingerprint scanners20and image processing of the user's fingerprint.

The cameras16image the holographic object30and the user's finger32. Image analysis is performed on the images of the holographic object30and the user's finger to identify a position of the user's finger32with respect to the display surface14, a position of the user's finger32with respect to the holographic object30and also a position of the holographic object30with respect to the display surface14. If there is more than one holographic object30, image analysis will also indicate which holographic object30the user's finger32is about to touch.

Since reading of the user's fingerprint may be triggered when the user's finger32is in proximity to the holographic object30, image analysis may also identify a touchpoint of the user's finger on the holographic object. In one embodiment, reading of the user's fingerprint may also be triggered when the user's finger32is in close proximity to the holographic object30, say 2 millimeters or less.

Instead of using the cameras16and imaging analysis, the proximity of the user's finger to the holographic object30may be based on ultrasound to avoid interaction of light from the holographic image and the photons from the user's finger. In this case, the electronic device10may contain one or more ultrasound sensors35to sense the proximity of the user's finger32to the holographic object30.

Once the image analysis has indicated that the user's finger32touched the holographic object30, or in one embodiment is in close proximity to the holographic object30, an input may be provided to the fingerprint scanner20to read the fingerprint of the user's finger32.

Subsequent to reading of the user's fingerprint, the user's fingerprint may be converted to a user's identity and then checked against an authorization database. If the user is authorized, authentication is successful, the holographic object30may be “unlocked” and the user may be permitted to access the holographic object30to the extent authorized. Authorization may be full or partial as determined by the user's permission access in the authorization database.

If the user is not authorized, authentication has failed, the holographic object30may remain “locked” and the user will not be permitted to access the holographic object30in any way.

Referring now toFIG. 3, the user has been successfully authenticated and authorized to interact with the holographic object30. Holographic object30has been moved to a new location consistent with an interaction with the user. When interacting with the holographic object30, the user may continue to use the user's finger32or may use another object such as a stylus36to interact with the holographic object30.

FIG. 4illustrates details of the electronic device10. Included within the electronic device10may be a processor such as central processing unit (CPU)38having memory40. Also included within electronic device10may be storage42having computer programs44. Storage may be resident within the electronic device10or may be remotely located such as in the cloud. Among the computer programs44may be an object recognition program56to receive input from the camera16and process the images received from the camera16.

The electronic device10may further include camera instructions46to operate the camera16. Further included within computer programs44may be additional computer programs for operating the camera16including a distance measuring program58to measure the distance between the user's finger32and the display surface14, to measure the distance between the user's finger32and the holographic object30and to measure the distance between the holographic object30and the display surface14.

Also included within electronic device10may be fingerprint scanner instructions48to operate the fingerprint scanners20, holographic projector instructions50to operate the holographic projectors18, ultrasound sensor instructions52to operate the ultrasound sensor, if present, audio/visual feedback instructions54to provide audio and/or visual feedback if authentication of the user's fingerprint is unsuccessful and display instructions56to operate the display12.

Regarding authentication, computer programs44may further include authentication programs60to provide the necessary functions for authentication, authorization programs62to check a database (not shown) for a user's level of authorization to access the holographic object30locking/unlocking programs64to unlock a holographic object30if authorization to access the holographic object30is permitted and to lock the holographic object30after the user is done interacting with the holographic object or after a time-out period has been passed.