Patent Publication Number: US-2023162828-A1

Title: Personalized assistance system for user of vision correction device

Description:
The disclosure relates generally to a personalized assistance system for a user of a vision correction device and method. Humans have five basic senses: sight, hearing, smell, taste and touch. Sight gives us the ability to visualize the world around us and connects us to our surroundings. According to some scientific reports, the brain devotes more space to processing and storing visual information than the other four senses combined, underscoring the importance of sight. Many people worldwide have various issues with quality of vision, for example, due to refractive errors. At least some of these issues may be addressed with vision correction devices, such as spectacles and contact lenses. 
     SUMMARY 
     Disclosed herein is a personalized assistance system for a user of a vision correction device and method. The personalized assistance system includes a remote computing unit with a controller having a processor and tangible, non-transitory memory on which instructions are recorded. The controller is configured to selectively execute one or more machine learning models. A user device is operable by the user and configured to communicate with the remote computing unit. The user device includes an electronic diary module configured to prompt the user to answer one or more preselected questions at specific intervals. 
     The electronic diary module is configured to store respective answers entered by the user in response to the one or more preselected questions as self-reported data. The one or more preselected questions may include an inquiry into a comfort level of the user, including at least one of a dryness factor and an irritation factor. The one or more preselected questions may include an inquiry into when the user last cleaned the vision correction device. 
     The controller is configured to obtain the self-reported data from the electronic diary module and generate an analysis of the self-reported data, via the one or more machine learning models. The controller is configured to assist the user based in part on the analysis. The vision correction device may include, but is not limited to, a contact lens. For example, the contact lens may be a multi-focal lens having a first zone for distance vision, a second zone for near vision and a third zone for intermediate vision. 
     The remote computing unit may include a first cloud unit and a central server, with the controller being embedded in at least one of the first cloud unit and the central server. The user device may include a query module configured to receive at least one question generated by the user. The controller may be configured to receive the question from the query module, formulate a reply, based in part on a first one of the one or more machine learning models, and post the reply, via the query module, for consumption by the user. 
     A provider device is configured to communicate with the remote computing unit, the provider device being operable by an eye care provider associated with the user. The user device and the provider device include respective message modules. The remote computing unit may be configured to provide two-way communication between the eye care provider and the user via the respective message modules. 
     The remote computing unit may include a first database storing respective information pertaining to the user, including a type of the vision correction device. The remote computing unit may include a second database storing group data pertaining to a group of users, the group data including respective self-reported data of the group of users. The user device includes a comparative tracking module configured to enable the user to compare the self-reported data with the group data. Assisting the user may include at least one of: providing coaching on taking care of the vision correction device and/or an eye of the user; suggesting a follow-up visit with an eye care provider; and suggesting an alternative vision correction product. 
     The above features and advantages and other features and advantages of the present disclosure are readily apparent from the following detailed description of the best modes for carrying out the disclosure when taken in connection with the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    is a schematic illustration of a personalized assistance system having a remote computing unit with a controller; 
         FIG.  2    is a schematic flowchart for a method executable by the controller of  FIG.  1   ; and 
         FIG.  3    is a schematic example of a machine learning model executable by the controller of  FIG.  1   . 
     
    
    
     DETAILED DESCRIPTION 
     Referring to the drawings, wherein like reference numbers refer to like components,  FIG.  1    schematically illustrates a personalized assistance system  10  for assisting a user  12  of a vision correction device  14 . The personalized assistance system  10  may include interfacing the user  12  with an eye care provider  16  associated with the user  12 . The personalized assistance system  10  is configured to address problems leading to a user  12  no longer wearing the vision correction device  14 , i.e. mitigate a drop-off in the use of the vision correction device  14  by the user  12 . In one example, the vision correction device  14  is a contact lens having multiple zones with different respective optical powers, such as a first zone  22  for distance vision, a second zone  24  for near vision and an third zone  26  for intermediate vision. It is to be understood that the contact lens may take many different forms and include multiple and/or alternate components. Additionally, any type of vision correction device available to those skilled in the art may be employed. 
     The user  12  may employ the personalized assistance system  10  after being fitted with the vision correction device  14  by the eye care provider  16  to achieve a number of goals, including but not limited to: reporting outcomes over time so that progress can be tracked and monitored, asking questions and getting answers in real-time, and receiving personalized suggestions based on past reported outcomes and past queries. Additionally, the personalized assistance system  10  may be configured to respond to specific actions requested by the user  12 . For example, a user  12  may request setting up of a reminder to remove their vision correction device  14 . As described below, the personalized assistance system  10  leverages both self-reported data and comparative data for optimizing the experience of the user  12 . 
     Referring to  FIG.  1   , the personalized assistance system  10  includes a remote computing unit  30  having a controller C. The controller C has at least one processor P and at least one memory M (or non-transitory, tangible computer readable storage medium) on which are recorded instructions for executing a method  100 . Method  100  is shown in and described below with reference to  FIG.  2   . 
     Referring to  FIG.  1   , the remote computing unit  30  may include one or cloud units, such as a first cloud unit  32 , a second cloud unit  34  and a central server  36 . The controller C may be embedded in at least one of the cloud units and the central server  36 . The central server  36  may be a private or public source of information maintained by an organization, such as for example, a research institute, a company, a university and/or a hospital. The first cloud unit  32  and the second cloud unit  34  may include one or more servers hosted on the Internet to store, manage, and process data. 
     The controller C has access to and is specifically programmed to selectively execute one or more machine learning models  40 , such as first machine learning model  42  and second machine learning model  44 . The machine learning models  40  may be configured to find parameters, weights or a structure that minimizes a respective cost function. Each of the machine learning models  40  may be a respective regression model. In one example, the first machine learning model  42  and the second machine learning model  44  are respectively embedded in the first cloud unit  32  and the second cloud unit  34 . The remote computing unit  30  may include a first database  46  for storing respective information pertaining to the user  12 , including the type of the vision correction device. The remote computing unit  30  may include a second database  48  for storing group data pertaining to a group of users. 
     Referring to  FIG.  1   , a user device  50  is operable by the user  12  and configured to communicate with, i.e. receive and transmit wireless communication, the remote computing unit  30 , via a first network  52 . The user device  50  may include a respective processor  54  and a respective memory  56 . The user device  50  may run a first application  58 , which may be a mobile application or “app.” The circuitry and components of a server, network and mobile application (“apps”) available to those skilled in the art may be employed. 
     The user device  50  may be a smartphone, laptop, tablet, desktop or other electronic device that the user  12  may operate, for example with a touch screen interface or I/O device such as a keyboard or mouse. The plurality of modules  60  may be executed in coordination with the remote computing unit  30 . In one example, the plurality of modules  60  includes an electronic diary module  62 , a query module  64 , a first messaging module  66  and a suggestion module  68 . The plurality of modules  60  may consume the output of a common or different machine learning models  40 . 
     The electronic diary module  62  is configured to prompt the user  12  to answer one or more preselected questions at specific intervals, e.g. daily. The electronic diary module  62  is configured to store respective answers entered by the user  12  in response to the one or more preselected questions as self-reported data. The one or more preselected questions may include an inquiry into a comfort level of the user  12 , including at least one of a dryness factor and an irritation factor. The one or more preselected questions may include an inquiry into when the user  12  last cleaned the vision correction device  14 . The controller C may be configured to obtain the self-reported data from the electronic diary module  62  and generate an analysis of the self-reported data, via the one or more machine learning models  40 . The user  12  may compare the self-reported data with group data (second database  48 ) generated by other users of the same type of vision correction device  14 , via the electronic diary module  62 . 
     The controller C may be configured to assist the user  12  based in part on the analysis. Assisting the user  12  based in part on the analysis may include at least one of the following: providing coaching on taking care of the vision correction device  14  (e.g. cleaning procedures) and/or an eye of the user  12 ; comparing comfort scores and other markers at a specific time period (e.g. one week after being fitted with the vision correction device  14 ) for the user  12  relative to a group of users of the same product; suggesting a follow-up visit with the eye care provider  16 ; and suggesting an alternative vision correction product. 
     Referring to  FIG.  1   , the query module  64  in the user device  50  may be configured to receive at least one question entered by the user  12 . The controller C may be configured to receive the at least one question from the query module  64  and formulate a reply, based in part on the one or more machine learning models  40 . The reply may be posted, via the query module  64 , for consumption by the user  12 . The personalized assistance system  10  may be configured to be “adaptive” and may be updated periodically after the collection of additional data. In other words, the machine learning models  40  may be configured to be “adaptive machine learning” algorithms that are not static and that improve after additional user data is collected. 
     Referring to  FIG.  1   , a provider device  70  is operable by the eye care provider  16  associated with the user. The provider device  70  is configured to communicate with the remote computing unit  30  via a second network  72 . The provider device  70  includes a respective processor  74  and a respective memory  76 . Similar to the user device  50 , the provider device  70  may run a second application  78  (incorporating a plurality of modules  80 ) that is executed in coordination with the remote computing unit  30 . The plurality of modules  80  may include a patient database  82  (stratified by the type of vision correction device  14 ), a user progress tracker module  84  configured to track progress of the user  12  and other users associated with the eye care provider  16 , a second message module  86  and a comparative tracking module  88  configured to provide trend and comparative analyses. 
     The remote computing unit  30  may be configured to provide two-way communication between the eye care provider  16  and the user  12  via the first message module  66  and the second message modules  86 . Referring to  FIG.  1   , the personalized assistance system  10  may include a broker module  90  for routing the respective messages from the user  12  to the eye care provider  16 , and vice-versa. The broker module  90  may be configured in different ways. While  FIG.  1    shows an example implementation of the personalized assistance system  10 , it is understood that other implementations may be carried out. 
     Referring to  FIG.  1   , the first network  52  and second network  72  may be wireless or may include physical components and may be a short-range network or a long-range network. For example, the first network  52  and second network  72  may be implemented in the form of a local area network. The local area network may include, but is not limited to, a Controller Area Network (CAN), a Controller Area Network with Flexible Data Rate (CAN-FD), Ethernet, blue tooth, WIFI and other forms of data connection. The local area network may be a Bluetooth™ connection, defined as being a short-range radio technology (or wireless technology) aimed at simplifying communications among Internet devices and between devices and the Internet. Bluetooth™ is an open wireless technology standard for transmitting fixed and mobile electronic device data over short distances and creates personal networks operating within the 2.4 GHz band. The local area network may be a Wireless Local Area Network (LAN) which links multiple devices using a wireless distribution method, a Wireless Metropolitan Area Networks (MAN) which connects several wireless LANs or a Wireless Wide Area Network (WAN) which covers large areas such as neighboring towns and cities. Other types of connections may be employed. 
     The machine learning models  40  of  FIG.  1    may include a neural network algorithm. While a neural network is illustrated herein, it is understood that the machine learning models  40  may be based on different kinds or types of algorithms, including but not limited to, a neural network, support vector regression, linear or logistic regression. k-means clustering, random forest and other types. As understood by those skilled in the art, neural networks are designed to recognize patterns from real-world data (e.g. images, sound, text, time series and others), translate or convert them into numerical form and embed in vectors or matrices. The neural network may employ deep learning maps to match an input vector x to an output vectory. Stated differently, each of the plurality of machine learning models  40  learns an activation function ƒ such that ƒ(x) maps to y. The training process enables the neural network to correlate the appropriate activation function ƒ(x) for transforming the input vector x to the output vector y. In the case of a simple linear regression model, two parameters are learned: a bias and a slope. The bias is the level of the output vectory when the input vector x is 0 and the slope is the rate of predicted increase or decrease in the output vectory for each unit increase in the input vector x. Once the plurality of machine learning models  40  is respectively trained, estimated values of the output vectory may be computed with given new values of the input vector x. 
     Referring to  FIG.  3   , an example network  200  for the machine learning models  40  of  FIG.  1    is shown. The example network  200  is a feedforward artificial neural network having at least three layers of nodes N, including an input layer  202 , one or more hidden layers, such as first hidden layer  204  and second hidden layer  206 , and an output layer  208 . Each of the layers is composed of nodes N configured to perform an affine transformation of a linear sum of inputs. The nodes N are neurons characterized by a respective bias and respective weighted links. The nodes N in the input layer  202  receive the input, normalize them and forward them to nodes N in the first hidden layer  204 . Each node N in a subsequent layer computes a linear combination of the outputs of the previous layer. A network with three layers would form an activation function ƒ(x)=ƒ( 3 )(ƒ( 2 )(ƒ( 1 )(x))). The activation function ƒ may be linear for the respective nodes N in the output layer  210 . The activation function f may be a sigmoid for the first hidden layer  204  and the second hidden layer  206 . A linear combination of sigmoids is used to approximate a continuous function characterizing the output vector y. 
     The example network  200  may generate multiple outputs, such as a first output factor  212  and a second output factor  214 , with the controller C being configured to use a weighted average of the multiple outputs to obtain a final output  210 . For example, if the inputs to the input layer  202  are various factors (e.g. comfort level, visual acuity score) pertaining to a particular type of vision correction device  14 , the first output factor  212  and the second output factor  214  may be an objective satisfaction score and a subjective satisfaction score, respectively, for that particular type of vision correction device  14 . Other machine learning models available to those skilled in the art may be employed. 
     Referring now to  FIG.  2   , a flow chart of method  100  executable by the controller C of  FIG.  1    is shown. Method  100  need not be applied in the specific order recited herein, with the start and end indicated respectively by “S” and “E” in  FIG.  2   . It is understood that some blocks may be omitted. The memory M can store controller-executable instruction sets, and the processor P can execute the controller-executable instruction sets stored in the memory M. 
     Per block  110  of  FIG.  2   , the controller C is configured to determine if the electronic diary module  62  has been triggered and the user  12  has answered any of the preselected questions. If so, the method  100  proceeds to block  115 , where the controller C is configured to record the respective answers entered by the user  12 . If not, the method  100  loops back to the start S. 
     Per block  120  of  FIG.  2   , the controller C is configured to determine if the query module  64  has been triggered and the user  12  has asked a question. If so, per block  125 , the controller C is configured to formulate a reply, based in part on the one or more machine learning models  40  and post the reply, via the query module  64 , for consumption by the user  12 . For example, the controller C may extract keywords from the question and enter them as inputs into the input layer  202 . The reply may be extracted based on the output layer  208  of the example network  200 . If not, the method  100  loops back to the start S. 
     Per block  130  of  FIG.  2   , the controller C is configured to determine if one or more enabling conditions has been met. The enabling conditions may include an irritation factor and/or discomfort factor being above a predetermined threshold. If so, per block  135 , the controller C may execute one or more actions, which may include interfacing with the eye care provider  16  via a message sent to the second message module  86 . The actions may include posting a reminder via the suggestion module  68  in the user device  50 , such as “please remember to clean contact lenses daily.” 
     In summary, the personalized assistance system  10  employs a multi-prong approach utilizing one or more machine learning models  40 . The personalized assistance system  10  may be configured to recognize non-adherence to suggested guidelines, recognize when a follow up visit to the eye care provider  16  makes sense or suggest an alternative contact lens. The personalized assistance system  10  offers an effective two-way communication between the user  12  and the eye care provider  16 . 
     The controller C of  FIG.  1    includes a computer-readable medium (also referred to as a processor-readable medium), including a non-transitory (e.g., tangible) medium that participates in providing data (e.g., instructions) that may be read by a computer (e.g., by a processor of a computer). Such a medium may take many forms, including, but not limited to, non-volatile media and volatile media. Non-volatile media may include, for example, optical or magnetic disks and other persistent memory. Volatile media may include, for example, dynamic random-access memory (DRAM), which may constitute a main memory. Such instructions may be transmitted by one or more transmission media, including coaxial cables, copper wire and fiber optics, including the wires that comprise a system bus coupled to a processor of a computer. Some forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, other magnetic medium, a CD-ROM, DVD, other optical medium, punch cards, paper tape, other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, other memory chip or cartridge, or other medium from which a computer can read. 
     Look-up tables, databases, data repositories or other data stores described herein may include various kinds of mechanisms for storing, accessing, and retrieving various kinds of data, including a hierarchical database, a set of files in a file system, an application database in a proprietary format, a relational database management system (RDBMS), etc. Each such data store may be included within a computing device employing a computer operating system such as one of those mentioned above and may be accessed via a network in one or more of a variety of manners. A file system may be accessible from a computer operating system and may include files stored in various formats. An RDBMS may employ the Structured Query Language (SQL) in addition to a language for creating, storing, editing, and executing stored procedures, such as the PL/SQL language mentioned above. 
     The detailed description and the drawings or FIGS. are supportive and descriptive of the disclosure, but the scope of the disclosure is defined solely by the claims. While some of the best modes and other embodiments for carrying out the claimed disclosure have been described in detail, various alternative designs and embodiments exist for practicing the disclosure defined in the appended claims. Furthermore, the embodiments shown in the drawings or the characteristics of various embodiments mentioned in the present description are not necessarily to be understood as embodiments independent of each other. Rather, it is possible that each of the characteristics described in one of the examples of an embodiment can be combined with one or a plurality of other desired characteristics from other embodiments, resulting in other embodiments not described in words or by reference to the drawings. Accordingly, such other embodiments fall within the framework of the scope of the appended claims.