Personalized optics-free vision correction

A user profile associated with a first user is received. A user prescription associated with the first user is received. A historical interaction of the first user with a display is received. A global vision model is received. One or more display sets to be used on the display is determined based on at least the user profile, the user prescription, the global vision model, and the historical interaction.

BACKGROUND

The present invention relates generally to the field of displays, and more particularly to correcting vision by image processing on a display.

SUMMARY

Embodiments of the present invention include a method, computer program product, and system for modifying a display. In one embodiment, a user profile associated with a first user is received. A user prescription associated with the first user is received. A historical interaction of the first user with a display is received. A global vision model is received. One or more display setts to be used on the display is determined based on at least the user profile, the user prescription, the global vision model, and the historical interaction.

DETAILED DESCRIPTION

A large number of adults in the world use some sort of vision correction. This vision correction can be glasses, contacts lenses, or both. Therefore, using glasses or contact lenses is an inseparable part of the lives of people worldwide. Driven by rapid technological changes, people are now spending a significant part of their day interacting with a device via some sort of screen (e.g., mobile devices, computer monitors, televisions, and video game consoles among many others).

Embodiments of the present invention provide for an optics-free vision correction system for digital displays. Embodiments of the present invention provide for vision correction that is performed by distorting the presented content on the display so that the target image is perceived by the user without the need for eyewear (e.g., glasses, contacts, etc.). Embodiments of the present invention provide for a system and model that adjusts display preferences based on the physiological and behavioral responses received from the user.

Embodiments of the present invention provide for improvements to the display experience by providing convenience and personalized settings. Embodiments of the present invention provide for improvements to the display experience by not requiring any peripheral hardware equipment (i.e., masks sensors) and therefore the system is economical along with easy to develop and utilize. Embodiments of the present invention provide for improvements to the display experience by utilizing an eye tracker which is found in common webcams built into computers that allow for applying the preferred distortion to the displayed content based on the eye gaze locations of the user.

Referring now to various embodiments of the invention in more detail,FIG. 1is a functional block diagram of a network computing environment, generally designated100, suitable for operation of vision program116in accordance with at least one embodiment of the invention.FIG. 1provides only an illustration of one implementation and does not imply any limitation with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made by those skilled in the art without departing from the scope of the invention as recited by the claims.

Network computing environment100includes computing device110interconnected over network120. In embodiments of the present invention, network120can be a telecommunications network, a local area network (LAN), a wide area network (WAN), such as the Internet, or a combination of the three, and can include wired, wireless, or fiber optic connections. Network120may include one or more wired and/or wireless networks that are capable of receiving and transmitting data, voice, and/or video signals, including multimedia signals that include voice, data, and video formation. In general, network120may be any combination of connections and protocols that will support communications between computing device110and other computing devices (not shown) within network computing environment100.

Computing device110is a computing device that can be a laptop computer, tablet computer, netbook computer, personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smartphone, smartwatch, or any programmable electronic device capable of receiving, sending, and processing data. In general, computing device110represents any programmable electronic devices or combination of programmable electronic devices capable of executing machine readable program instructions and communicating with other computing devices (not shown) within computing environment100via a network, such as network120. In an embodiment, computing device110may include a microphone for receiving audio.

In various embodiments of the invention, computing device110may be a computing device that can be a standalone device, a management server, a web server, a media server, a mobile computing device, or any other programmable electronic device or computing system capable of receiving, sending, and processing data. In other embodiments, computing device110represents a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In an embodiment, computing device110represents a computing system utilizing clustered computers and components (e.g. database server computers, application server computers, web servers, and media servers) that act as a single pool of seamless resources when accessed within network computing environment100.

In various embodiments of the invention, computing device110includes user interface112, information repository114, vision program116, and video device118.

In an embodiment, computing device110includes user interface112. User interface112is a program that provides an interface between a user and an application. User interface112refers to the information (such as graphic, text, and sound) a program presents to a user and the control sequences the user employs to control the program. There are many types of user interfaces. In one embodiment, user interface112may be a graphical user interface (GUI). A GUI is a type of user interface that allows users to interact with electronic devices, such as a keyboard and mouse, through graphical icons and visual indicators, such as secondary notations, as opposed to text-based interfaces, typed command labels, or text navigation. In computers, GUIs were introduced in reaction to the perceived steep learning curve of command-line interfaces, which required commands to be typed on the keyboard. The actions in GUIs are often performed through direct manipulation of the graphics elements.

In an embodiment, computing device110includes information repository114. In an embodiment, information repository114may be managed by vision program116. In an alternative embodiment, information repository114may be managed by the operating system of computing device110, another program (not shown), alone, or together with, vision program116. Information repository114is a data repository that can store, gather, and/or analyze information. In some embodiments, information repository114is located externally to computing device110and accessed through a communication network, such as network120. In some embodiments, information repository114is stored on computing device110. In some embodiments, information repository114may reside on another computing device (not shown), provided information repository114is accessible by computing device110. Information repository114includes, but is not limited to, user profile information, the prescription information for users, historic interaction with displays for users, personalized vision models associated with individual users and the global vision model. In an embodiment, the historic interaction with displays for each user may be integrated into the personalize vision model associated with the user.

In an embodiment, a personalized vision model and a global vision model may be machine learning models that model relationships between the eye prescription of the user and display settings for a display that allow for the user to optimally view the display based on the eye prescription of the user. A machine learning model includes the construction and implementation of algorithms that can learn from and make predictions on data. The algorithms operate by building a model from example inputs in order to make data-driven predictions or decisions, rather than following strictly static program instructions. In an embodiment, the model is a system which explains the behavior of some system, generally at the level where some alteration of the model predicts some alteration of the real-world system. In an embodiment, a machine learning model may be used in a case where the data becomes available in a sequential fashion, in order to determine a mapping from the dataset to corresponding labels. In an embodiment, the goal of the machine learning model is to minimize some performance criteria using a loss function. In an embodiment, the goal of the machine learning model is to minimize the number of mistakes when dealing with classification problems. In yet another embodiment, the machine learning model may be any other model known in the art. In an embodiment, the machine learning model may be a SVM “Support Vector Machine”. In an alternative embodiment, the machine learning model may be any supervised learning regression algorithm. In yet another embodiment, the machine learning model may be a neural network.

Information repository114may be implemented using any volatile or non-volatile storage media for storing information, as known in the art. For example, information repository114may be implemented with a tape library, optical library, one or more independent hard disk drives, multiple hard disk drives in a redundant array of independent disks (RAID), solid-state drives (SSD), or random-access memory (RAM). Similarly, information repository114may be implemented with any suitable storage architecture known in the art, such as a relational database, an object-oriented database, or one or more tables.

In an embodiment, computing device110includes vision program116. Embodiments of the present invention provide for a vision program116that determines a user profile. In embodiments of the present invention, vision program116determines the prescription of the user associated with the user profile. In embodiments of the present invention, vision program116determines historic interaction of the user. In embodiments of the present invention, visions program116determines the display settings based on the prescription of the user, the global vision model, and the personalized vision model, if it exists. In embodiments of the present invention, vision program116modifies the display based on the display settings. In embodiments of the present invention, vision program116determines if there is user feedback. In embodiments of the present invention, vision program116receives the eye gaze of the user. In embodiments of the present invention, vision program116displays data to the user.

In an embodiment, computing device110includes video device118. In an embodiment, video device118may be integrated into computing device110. For example, video device118may be a webcam or camera that is integrated into a laptop or a display attached to a PC. In an alternative embodiment, video device118may be separate from computing device110. For example, video device118may be a digital camera, video camera, webcam, or other video device that connects to computing device110via a physical wire or via some form of wireless connection. In an embodiment, video device118records a user of computing device110in real-time. In an embodiment, video device118may include software for eye tracking or gaze tracking of the user, fatigue detection, pupil size monitoring, blinking detection, and facial express analysis. In an embodiment, video device118may include a microphone for receiving audio.

As referred to herein, all data retrieved, collected, and used, is used in an opt in manner, i.e., the data provider has given permission for the data to be used. For example, the installation procedure for a video device that would be used to capture video from a user could include an option that must be selected by the owner to allow the device to capture video and use the captured video to determine eye gaze. As another example, the system could request approval from the owner of the computing device before capturing video. Any data or information used for which the provider has not opted in is data that is publicly available.

FIG. 2is a flow chart diagram of workflow200depicting operational steps for vision program116in accordance with at least one embodiment of the invention. In an alternative embodiment, the steps of workflow200may be performed by any other program while working with vision program116. It should be appreciated that embodiments of the present invention provide at least for modifying the display of the computing device110based at least on the prescription information for users, historic interaction with displays for users, personalized vision models associated with individual users and the global vision model. However,FIG. 2provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made by those skilled in the art without departing from the scope of the invention as recited by the claims. In a preferred embodiment, a user, via user interface112, can invoke workflow200upon a user wanting vision program116to modify the display of computing device110.

Vision program116determines a user profile (step202). At step202, video vision program116receives login information from a user that is trying to utilize vision program116. In an embodiment, vision program116receives login information in the form of a user identification and an associated password. In an embodiment, the user identification may be a username, a ClientID, login credentials, or any other form of identification that identifies the user. In an embodiment, each set of login information is associated exclusively with a single user.

In step202, vision program116verifies the login information that is received. In an embodiment, vision program116compares the login information received to the login information found in the user profile information in information repository114. If the login information is incorrect, in other words the login information does not match the login information found in information repository114, vision program116notifies the user of the incorrect login information and processing of flow200ends. In this embodiment, the user may input login information again. If the login information is correct, vision program116may notify the user via the user interface on the client device of the correct login information.

Vision program116determines historic interaction (step206). At step206, vision program116determines whether any historic information for the user is available. In other words, has the use previously used vision program116, had a display modified by vision program116, and the user then interacted with the display or display settings in order to fine tune the viewing experience of the user. This fine tuning, the actual modifications of the display and the intended end viewing result, is saved in information repository114. In an embodiment, when this is the first time a user is utilizing vision program116, there may not be any historic interaction information stored in information repository114.

Vision program116modifies the display (step210). At step210, vision program116applies the determined display settings to the display of computing device110. In an embodiment, the display settings will be applied to the visual representations being shown in the user interface112. In an embodiment, the display settings will be applied to whatever is being displayed by the display being shown in user interface112. For example, a website the user was viewing or a program the user was using. In an alternative embodiment, the display settings will be applied to a test image being displayed by the display being show in user interface112. For example, the test display may be specific to visually show the optimizations that are trying to be realized by the display settings.

Vision program116determines if there is user feedback (decision step212). At decision step212, visual program116determine whether the user has provided feedback in response to the modified display via an indication. In an embodiment, feedback may be provided by video device118. In this embodiment, video device118may record information related to the eye fatigue of the user such as redness of eyes, facial expressions, blinks, pupil size, as well as voice feedback, and/or the change of position/distance of the user from the display. In an alternative embodiment, feedback or change of display settings may be provided directly by the user via user interface. In an embodiment, user feedback is received (decisions step208, yes branch), processing proceeds to step206to update the models and the display settings. In an embodiment, if user feedback is not received (decision step208, no branch), processing proceeds to step214.

Vision program116receives eye gaze (step214). At step214, vision program116receives the eye gaze of the user via video device118. In an embodiment, vision program116, based on the eye gaze of the user, determines where the user is looking on the display of computing device110.

Vision program116displays the modified display (step216). At step216, vision program116uses the determined display settings and received eye gaze to display the modified display on computing device110. In an embodiment, vision program116will modify the entire graphic that is being displayed by the user interface112on the display of computer device110. In an alternative embodiment, vision program116will modify a part or subsection of the graphic that is being displayed by the user interface112on the display of computer device110. In this embodiment, the part of the graphic that is being modified is based on where the eye gaze of the user is focused on the display.

FIG. 3is a block diagram depicting components of a computer300suitable for vision program116, in accordance with at least one embodiment of the invention.FIG. 3displays the computer300, one or more processor(s)304(including one or more computer processors), a communications fabric302, a memory306including, a RAM316, and a cache318, a persistent storage308, a communications unit312, I/O interfaces314, a display322, and external devices320. It should be appreciated thatFIG. 3provides only an illustration of one embodiment and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

As depicted, the computer300operates over the communications fabric302, which provides communications between the computer processor(s)304, memory306, persistent storage308, communications unit312, and input/output (I/O) interface(s)314. The communications fabric302may be implemented with an architecture suitable for passing data or control information between the processors304(e.g., microprocessors, communications processors, and network processors), the memory306, the external devices320, and any other hardware components within a system. For example, the communications fabric302may be implemented with one or more buses.

The memory306and persistent storage308are computer readable storage media. In the depicted embodiment, the memory306comprises a random-access memory (RAM)316and a cache318. In general, the memory306may comprise any suitable volatile or non-volatile one or more computer readable storage media.

Program instructions vision program112may be stored in the persistent storage308, or more generally, any computer readable storage media, for execution by one or more of the respective computer processors304via one or more memories of the memory306. The persistent storage308may be a magnetic hard disk drive, a solid-state disk drive, a semiconductor storage device, read only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory, or any other computer readable storage media that is capable of storing program instruction or digital information.

The communications unit312, in these examples, provides for communications with other data processing systems or devices. In these examples, the communications unit312may comprise one or more network interface cards. The communications unit312may provide communications through the use of either or both physical and wireless communications links. In the context of some embodiments of the present invention, the source of the various input data may be physically remote to the computer300such that the input data may be received, and the output similarly transmitted via the communications unit312.

The I/O interface(s)314allow for input and output of data with other devices that may operate in conjunction with the computer300. For example, the I/O interface314may provide a connection to the external devices320, which may be as a keyboard, keypad, a touch screen, or other suitable input devices. External devices320may also include portable computer readable storage media, for example thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention may be stored on such portable computer readable storage media and may be loaded onto the persistent storage308via the I/O interface(s)314. The I/O interface(s)314may similarly connect to a display322. The display322provides a mechanism to display data to a user and may be, for example, a computer monitor.