Patent ID: 12205582

DETAILED DESCRIPTION

The following detailed description illustrates embodiments of the disclosure by way of example and not by way of limitation. The description enables one skilled in the art to make and use the disclosure, describes several embodiments, adaptations, variations, alternatives, and uses of the disclosure, including what is presently believed to be the best mode of carrying out the disclosure. The disclosure is described as applied to an example embodiment, namely, systems and methods utilizing a virtual caregiver (VC) computing device. The VC computing device may be in communication with, for example, a database, one or more user computing devices, one or more sensors, one or more loudspeakers, a payment processing network, and/or other devices (e.g., wearable smart devices, smart watches, mobile phones, tablets, etc.).

The VC computing device may generate and keep track of tasks that are supposed to be performed by an individual, such as individual requiring care. The tasks tracked by the VC computing device may include, for example, taking medication, bathing, performing household chores, going to doctors, going to get a haircut, purchasing food, going out to lunch, and other tasks that some patients may have difficulty remembering. As used herein, “patient” or user or senior user may refer to individuals requiring care from a caregiver. The patient, user or senior user need not be physically located in a hospital, rather, the patient, user or senior user may be in a nursing home or assisted living facility. In addition, the patient, user or senior user may also be in their own home taking care of themselves with assistance from caregivers.

The VC computing device may generate the tasks based on received data indicating that an activity must be performed (sometimes referred to herein as “task data”). The task data may include, for example, data received via manual input at a user computing device (e.g., via a mobile “app”), audio data (e.g., speech) indicating that the activity should be performed (e.g., a verbal reminder given by a caregiver and received by a voice-controlled computing device), and/or reminders included in electronic messages (e.g., email or short messaging service (SMS) messages). The VC computing device may store the generated tasks in a task database, and collect sensor data and/or transaction data to determine whether the tasks have been completed. For example, a microphone may be used to detect audio data indicating that a medication has been taken (e.g., a sound of a pill bottle being opened and a pill being swallowed). Likewise, the VC computing device may retrieve transaction data from a payment network corresponding to a grocery purchase, indicating that the patient has remembered to purchase food. The grocery purchase may be made by the patient or by a caregiver on behalf of the patient. Based on the determination, the VC computing device may record tasks in the task database as complete or incomplete, and may generate alerts based on the record (e.g., displaying an alert to the caregiver via the mobile app).

In some embodiments, VC computing device may be additionally configured to assist patients (e.g., dementia patients) in recalling words and ideas when speaking. For example, the VC computing device may record, using the sensors (e.g., a microphone), past instances of speech (e.g., stories, remarks, questions, and/or other utterances) of the patient, and identify the past instance of speech as corresponding to a current instance of speech of the patient. If the patient has difficulty with the current instance of speech (e.g., struggling to find a word), the VC computing device may generate an audio reminder to assist the patient in remembering the intended speech and/or assist others in understanding the patient. In other words, the patient may tell a story that is received by a remote microphone, transcribed, and cataloged by the VC computing device in a memory device. Then, later when the patient is repeating the story but struggles to remember the details, the cataloged details of the story may be retrieved by the VC computing device and may be provided to the patient (e.g., an audio reminder) to help them remember the story.

The VC computing device includes a processor in communication with a memory. The VC computing device may be in communication with at least one database for storing information, such as, for example, task data (e.g., data corresponding to user tasks tracked by the VC computing device), sensor data, transaction data, and/or speech data, as described below.

The VC computing device may further be in communication with one or more sensors (e.g., microphones), speakers, and/or other remote devices (wearable smart devices, smart watches, mobile phones, tablets, etc.). For example, in some embodiments, the VC computing device may be in communication with a “smart speaker” including a microphone that receives an audio input (e.g., speech commands) and a speaker through which the smart speaker may respond to speech commands (e.g., providing information, playing music, and/or other audio responses). For example, AMAZON ECHO or ALEXA devices may be used.

The VC computing device may further be in communication with a payment processing network, from which the VC computing device may retrieve transaction data corresponding to purchases (e.g., using a payment card or payment account) made by the patient. In some embodiments, the VC computing device may be capable of monitoring transactions made via the payment processing network in real time. In some embodiments, the payment processing network may include a transaction database that stores transaction data, and the VC computing device may retrieve the stored transaction data from the transaction database. As described below, the VC computing device may use such transaction data to determine whether the patient has completed tasks when those tasks include making a purchase.

In the example embodiment, the VC computing device may be configured to receive task data. In some embodiments, the task data may be received from a user device (e.g., a wearable smart device, a personal computer (PC), tablet computer, or mobile telephone). For example, the user device may be associated with a caregiver, and the caregiver may enter tasks to be completed by a patient at the user device via a user interface (e.g., a mobile “app” and/or web page) provided by the VC computing device. The user interface may enable the caregiver, for example, to enter a particular activity that must be performed by the patient (e.g., taking a medication, bathing, paying a bill, and/or other such activities) and a corresponding due date. Such task data may be used to generate tasks (e.g., a specific data element that includes the activity to be completed and associated data), as described below.

The VC computing device will be trained to detect sounds and identify tasks being completed. For example, if the patient must take medication at noon every day, the system will be trained to identify the sounds associated with completing that task (the pills rattling in the bottle, the cap being removed, etc.).

In some embodiments, the task data may be received from a sensor (e.g., a microphone, camera, wearable smart device, smart medicine cabinet, etc.). For example, in embodiments wherein the VC computing device is in communication with a smart speaker, the VC computing device may receive direct commands via the smart speaker to create tasks, for example, from the caregiver. Additionally or alternatively, the smart speaker may detect audio data (e.g., speech data) indicating that a task should be performed (e.g., the caregiver giving verbal instructions to the patient) and transmit the audio data to the VC computing device. The VC computing device may generate tasks based on such audio data, for example, utilizing artificial intelligence (AI) and/or machine learning techniques. To train the system using machine learning, sample audio data of a sound to be recognized (such as the sound of a pill bottle being opened) are provided to the machine learning model. Several examples are used that contain the sound, as well as samples that do not contain the sound to allow the model to teach the model to differentiate and identify the identified sound. Other sounds of interest could include the sound of a faucet filling a glass of water, the sound of a microwave chime, or a refrigerator door opening or closing. Based on the sound identification, the VC computing device may create a corresponding follow up task such as placing a future order for groceries or picking up a medication.

The VC computing device may be further configured to generate, based on the task data, a task including a task identifier, a fulfillment condition (e.g., the specific activity to be performed), and a time deadline. The VC computing device may generate the task identifier, and identify the fulfillment condition and the time deadline based on the received task data. For example, the fulfillment condition and time deadline may be entered manually via the user interface, as described above. Additionally or alternatively, the fulfillment condition and time deadline may be identified from audio data utilizing, for example, AI and/or machine learning techniques, as described above. In order to determine that a specific task has been completed, sample audio of an event signaling completion will be provided to the machine learning model and further trained with sounds that do not contain the audio of the event to train the system to recognize the audio signaling completion of the event.

The VC computing device may be further configured to store the generated task in a task database. For example, the VC computing device may store each generated task identifier in the task database and store the corresponding fulfillment condition, time deadline, and/or other data in association with the task identifier.

The VC computing device may be further configured to receive sensor data from one or more sensors. The sensors may include for example, the microphone and/or smart speaker. In some embodiments, the sensors may include “smart” pill bottles or containers capable of detecting opening and closing of the pill container and/or removal of pills from the container. In some embodiments, the sensors may include smart appliances that are capable of communicating data to the VC computing device. For example, a smart refrigerator may detect when contents (e.g., groceries) are added to or removed from the refrigerator (e.g., using a door sensor, camera, or other sensor), and a smart washing machine may detect when laundry is being cleaned.

The VC computing device may further be configured to retrieve transaction data. For example, in some embodiments, the VC computing device may parse transaction data associated with a purchase which is transmitted via the payment processing network for transaction data relevant to determining whether a fulfillment condition has occurred. In such embodiments, the VC computing device may track usage of a payment card account or other bank account associated with the patient. Additionally or alternatively, in some embodiments, the VC computing device may generate and perform a query to retrieve relevant transaction data from the transaction database. In some such embodiments, the VC computing device may generate the query using AI and/or machine learning techniques. For example, data may be provided to the machine learning model that can be tied to a specific task by the vendor involved. The model may be trained to recognize a particular vendor and recognized that the patient has completed a task. Using the retrieved transaction data, the VC computing device may be able to determine that the patient has remembered to, for example, pay a bill, purchase food, purchase a medication, attend a doctor visit, or performed other activities that may be associated with a payment transaction.

The VC computing device may be further configured to compare the received sensor data and/or transaction data to the fulfillment conditions stored in the task database to determine that the fulfillment condition has been performed. For example, the VC computing device may periodically parse through each of the fulfillment conditions to determine, based on the received sensor data and/or transaction data, whether the fulfillment condition has been met. In some embodiments, the VC computing device may utilize AI and/or machine learning techniques to process the sensor data and/or transaction data so that the sensor data may be used to make such a determination. For example, the VC computing device may identify a certain audio signal (e.g., a sound) as an indicator that the patient is taking and/or has taken a certain medication. The task may be to take a particular medication at noon. The fulfillment condition may be verified by the VC computing device by detecting the sound of the cabinet being open, the pill bottle being removed and opened, and the sound of the patient taking the pill. In some examples, a sensor that is part of a smart pill bottle may indicate the pill bottle for a certain medication has been opened. In another example, transaction data may be used to confirm fulfillment of the fulfillment condition. For example, transaction data associated with paying for a haircut may be used to confirm fulfillment of that task based on an identification of the vendor. Machine learning techniques may be used to determine which bills are associated with which vendors and which tasks. For example, a bill received from vendor X may be for grass cutting services that the patient receives at their house. Payment of that bill may satisfy the condition of having the yard maintained.

The VC computing device may be further configured to record the task as fulfilled based on the determination. The VC computing device may store the record in association with the corresponding task identifier in the task database. In some embodiments, the VC computing device may parse the records and time deadlines in the task database to identify tasks that are recorded as being unfulfilled past the time deadline. For each of the identified overdue tasks, the VC computing device may generate an alert, which may be provided to, for example, the caregiver and/or the patient. For example, the VC computing device may transmit an email, SMS message, and/or another form of notification to the user computing device. Additionally or alternatively, the VC computing device may generate an audio notification (e.g., at the smart speaker) to remind the patient to perform the overdue task.

In some embodiments, VC computing device may be additionally configured to assist patients (e.g., dementia patients) in recalling words and ideas when speaking. For example, the VC computing device may record past instances of speech (e.g., stories, remarks, questions, and/or other utterances) of the patient and identify the past instance as corresponding to a current instance of speech of the patient. If the patient has difficulty with the current instance of speech (e.g., pausing unnaturally and/or struggling to find a next word), the VC computing device may generate an audio reminder to assist the patient in remembering the intended speech and/or assist others in understanding the patient. As an example, the VC computing device may record a story being told by the patient. Subsequently, the audio obtained by the VC computing device is parsed and transcribed to text. Key words may be stored in relation to the story. Later, when the patient is retelling the same story, the VC computing device may detect forgetfulness based upon an extended period of silence from the patient. Alternatively, the VC computing device may be prompted of forgetfulness by an audio prompt from the patient or the caregiver. The VC computing device can search based on the key words to identify the story and further provide details to the patient so that they remember the story and are able to complete telling the story.

In such embodiments, the VC computing device may detect, at the sensor, a first instance of speech (e.g., the past instance of speech). For example, in embodiments where the VC computing device is in communication with a microphone and/or smart speaker, the VC computing device may record speech made by the patient. The VC computing device may store the detected first instance of speech in a database (e.g., a speech database).

In such embodiments, the VC computing device may detect, at the sensor (e.g., the microphone and/or smart speaker), a second instance of speech (e.g., the current instance of speech), and in response to the detection, compare the second instance of speech to the first instance of speech. The VC computing device may determine that the second instance of speech corresponds to the first instance of speech (e.g., they correspond to the same story) and generate, at a loudspeaker (e.g., a loudspeaker included in the smart speaker), an audio reminder. The audio reminder may include, for example, key words or phrases from the first instance of speech that may assist the patient and/or other listeners in recalling the intended speech. In some embodiments, the VC computing device may utilize AI and/or machine learning techniques, for example, to determine that the second instance of speech corresponds to the first instance of speech and/or to generate the audio reminder.

The technical problems addressed by the disclosure include at least one of: (i) inefficiency in automating the tracking tasks that must be performed by an individual requiring care; (ii) inability of computing devices to automatically generate tasks based on received speech; (iii) inability of computing devices to determine that a task has been performed remotely; (iv) inability of computing devices to generate automatic alerts for a remote caregiver that a task has been performed; and/or (v) inability of a computing device to determine that a task has been completed based on transaction data corresponding to payment transactions.

The technical effects achieved by the systems and methods described herein include at least one of: (i) receiving task data from a patient or caregiver; (ii) automatically generating, based on the task data, a task including a task identifier, a fulfillment condition (e.g., what needs to be done to complete the task), and a time deadline; (iii) storing the generated task in a task database along with the task identifier, the fulfillment condition and the time deadline; (iv) receiving, from at least one sensor, sensor data; (v) retrieving, from a payment processing network, transaction data; (vi) comparing at least one of the received sensor data and the retrieved transaction data to the fulfillment condition to determine that the fulfillment condition has occurred; and/or (vii) record, in the task database, the task as fulfilled based on the determination that the fulfillment condition has occurred. In the example embodiment, sensor data may be used to confirm fulfillment without the use of transaction data. In another embodiment, the sensor data and the transaction data can be used in combination together to confirm fulfillment of the task. For example, buying groceries may be detected using transaction data received by the payment processor, loading the groceries in a smart refrigeration (Internet of Things (IoT) device) and/or smart cabinets may be detected by sensors, and eating the groceries by the patient may be detected using microphone/cameras or other sensors.

The resulting technical benefits achieved by the systems and methods of the disclosure include at least one of: (i) increased efficiency in tracking tasks that must be performed by an individual requiring care; (ii) ability for computing devices to automatically generate tasks based on received speech; (iii) inability for computing devices to determine that a task has been performed remotely; (iv) ability for computing devices to generate automatic alerts for a remote caregiver that a task has been performed, (v) ability for a computing device to determine that a task has been completed based on transaction data corresponding to payment transactions.

In one embodiment, a computer program is provided, and the program is embodied on a computer readable medium. In an example embodiment, the system is executed on a single computer system, without requiring a connection to a server computer. In a further example embodiment, the system is being run in a Windows® environment (Windows is a registered trademark of Microsoft Corporation, Redmond, Washington). In yet another embodiment, the system is run on a mainframe environment and a UNIX® server environment (UNIX is a registered trademark of X/Open Company Limited located in Reading, Berkshire, United Kingdom). In a further embodiment, the system is run on an iOS® environment (iOS is a registered trademark of Cisco Systems, Inc. located in San Jose, CA). In yet a further embodiment, the system is run on a Mac OS® environment (Mac OS is a registered trademark of Apple Inc. located in Cupertino, CA). The application is flexible and designed to run in various different environments without compromising any major functionality. In some embodiments, the system includes multiple components distributed among a plurality of computing devices. One or more components are in the form of computer-executable instructions embodied in a computer-readable medium. The systems and processes are not limited to the specific embodiments described herein. In addition, components of each system and each process can be practiced independently and separately from other components and processes described herein. Each component and process can also be used in combination with other assembly packages and processes.

In one embodiment, a computer program is provided, and the program is embodied on a computer readable medium and utilizes a Structured Query Language (SQL) with a client user interface front-end for administration and a web interface for standard user input and reports. In another embodiment, the system is web enabled and is run on a business-entity intranet. In yet another embodiment, the system is fully accessed by individuals having an authorized access outside the firewall of the business-entity through the Internet. In a further embodiment, the system is being run in a WINDOWS® environment (WINDOWS is a registered trademark of Microsoft Corporation, Redmond, Washington). The application is flexible and designed to run in various different environments without compromising any major functionality.

As used herein, an element or step recited in the singular and preceded with the word “a” or “an” should be understood as not excluding plural elements or steps, unless such exclusion is explicitly recited. Furthermore, references to “example embodiment” or “one embodiment” of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

As used herein, the term “database” may refer to either a body of data, a relational database management system (RDBMS), or to both. A database may include any collection of data including hierarchical databases, relational databases, flat file databases, object-relational databases, object oriented databases, and any other structured collection of records or data that is stored in a computer system. The above examples are for example only, and thus are not intended to limit in any way the definition and/or meaning of the term database. Examples of RDBMS's include, but are not limited to including, Oracle® Database, MySQL, IBM® DB2, Microsoft® SQL Server, Sybase®, and PostgreSQL. However, any database may be used that enables the systems and methods described herein. (Oracle is a registered trademark of Oracle Corporation, Redwood Shores, California; IBM is a registered trademark of International Business Machines Corporation, Armonk, New York; Microsoft is a registered trademark of Microsoft Corporation, Redmond, Washington; and Sybase is a registered trademark of Sybase, Dublin, California).

The term processor, as used herein, may refer to central processing units, microprocessors, microcontrollers, reduced instruction set circuits (RISC), application specific integrated circuits (ASIC), logic circuits, and any other circuit or processor capable of executing the functions described herein.

As used herein, the terms “software” and “firmware” are interchangeable, and include any computer program stored in memory for execution by a processor, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are for example only, and are thus not limiting as to the types of memory usable for storage of a computer program.

As used herein, the terms “transaction card,” “financial transaction card,” and “payment card” refer to any suitable transaction card, such as a credit card, a debit card, a prepaid card, a charge card, a membership card, a promotional card, a frequent flyer card, an identification card, a gift card, and/or any other device that may hold payment account information, such as mobile phones, Smartphones, personal digital assistants (PDAs), key fobs, and/or computers. Each type of transactions card can be used as a method of payment for performing a transaction.

FIG.1is a simplified block diagram of an exemplary virtual caregiver (VC) system100in accordance with one example embodiment of the present disclosure.FIG.2is a data flow diagram of an exemplary data flow200within VC system100. VC system100includes a VC computing device102that includes at least one database server104. VC system100may further include one or more user computing devices108(e.g., a PC, tablet computer, and/or mobile telephone), sensors110(e.g., microphones), speakers112, and/or payment processing network114.

VC computing device102may include a processor in communication with a memory. VC computing device102may be in communication with database106via database server104.

VC computing device102may further be in communication with one or more sensors110, speakers112, and/or other remote devices (e.g., microphones, cameras, IoT devices, wearables, other computing or smart devices, etc.). For example, in some embodiments, VC computing device102may be in communication with a “smart speaker” including a microphone that receives an audio input (e.g., speech commands) and speaker112through which the smart speaker may respond to speech commands (e.g., providing information, playing music, and/or other audio responses).

VC computing device102may further be in communication with payment processing network114, from which VC computing device102may retrieve transaction data corresponding to purchases (e.g., using a payment card) made by the patient. In some embodiments, VC computing device102may be capable of monitoring transactions made via payment processing network114in real time. In some embodiments, payment processing network114may include a transaction database116that stores transaction data, and VC computing device102may retrieve the stored transaction data from transaction database116. As described below, VC computing device102may use such transaction data to determine whether the patient has completed tasks. For example, payment processing network114may include an electronic transaction payment card processing system, such as a payment card processing system using the Mastercard® interchange network (Mastercard is a registered trademark of Mastercard International Incorporated located in Purchase, N.Y.). The Mastercard® interchange network is a proprietary communications standard promulgated by Mastercard International Incorporated® for the exchange of electronic transaction data between financial institutions that have registered with Mastercard International Incorporated®. For example, the messages sent over the payment processing network may include an ISO 8583 computer message.

In the example embodiment, VC computing device102may be configured to receive task data202. In some embodiments, task data202may be received from user computing device108. For example, user computing device108may be associated with a caregiver, and the caregiver may enter tasks to be completed by a patient at user computing device108via a user interface (e.g., a mobile “app” and/or web page) provided by VC computing device102. The user interface may enable the caregiver, for example, to enter a particular activity that must be performed by the patient (e.g., taking a medication, bathing, paying a bill, and/or other such activities) and a corresponding due date. Such task data202may be used to generate tasks204(e.g., a specific data element that includes the activity to be completed and associated data), as described below.

In some embodiments, task data202may additionally or alternatively be received from one or more sensors110. For example, in embodiments wherein VC computing device102is in communication with a smart speaker, VC computing device102may receive direct commands via the smart speaker to create tasks204, for example, from the caregiver. Additionally or alternatively, the smart speaker may detect audio data (e.g., speech data) indicating that an activity should be performed (e.g., the caregiver giving verbal instructions to the patient) and transmit the audio data to VC computing device102. VC computing device102may generate tasks204based on such audio data, for example, utilizing artificial intelligence (AI) and/or machine learning techniques.

VC computing device102may be further configured to generate, based on task data202, tasks204that include a task identifier, a fulfillment condition (e.g., the specific activity to be performed), and a time deadline. VC computing device102may generate the task identifier and identify the fulfillment condition and the time deadline based on the received task data202. For example, the fulfillment condition and time deadline may be entered manually via the user interface, as described above. Additionally or alternatively, the fulfillment condition and time deadline may be identified from audio data utilizing, for example, AI and/or machine learning techniques, as described above.

VC computing device102may be further configured to store the generated task204in database106(e.g., in a task database). For example, VC computing device102may store each generated task identifier in database106and store the corresponding fulfillment condition, time deadline, and/or other data in association with the task identifier.

VC computing device102may be further configured to receive sensor data206from sensors110. Sensors110may include, for example, the microphone and/or smart speaker. In some embodiments, sensors110may include “smart” pill bottles or containers capable of detecting opening and closing of the pill container and/or removal of pills from the container. In some embodiments, sensors110may include smart appliances or other IoT devices that are capable of communicating data to VC computing device102. For example, a smart refrigerator may detect when contents (e.g., groceries) are added to or removed from the refrigerator (e.g., using a door sensor, camera, or other sensor), and a smart washing machine may detect when laundry is being cleaned.

VC computing device102may further be configured to retrieve transaction data210from payment processing network114. For example, in some embodiments, VC computing device102may parse transaction data210transmitted via payment processing network114for transaction data210relevant to determining whether a fulfillment condition has occurred. In such embodiments, VC computing device102may track usage of a payment card account or other bank account associated with the patient. Additionally or alternatively, in some embodiments, VC computing device102may generate and perform a query208to retrieve relevant transaction data210from transaction database116. In some such embodiments, VC computing device102may generate the query using AI and/or machine learning techniques. Using the retrieved transaction data210, VC computing device102may be able to determine that the patient has remembered to, for example, pay a bill, purchase food, purchase a medication, attend a doctor visit, or performed other activities that may be associated with a payment transaction.

VC computing device102may be further configured to compare212the received sensor data206and/or transaction data210to the fulfillment conditions stored in database106to determine that the fulfillment condition has been performed. For example, VC computing device102may periodically parse through each of the fulfillment conditions of the stored tasks204to determine, based on the received sensor data206and/or transaction data210, whether the fulfillment condition has been met. In some embodiments, VC computing device102may utilize AI and/or machine learning techniques to process sensor data206and/or transaction data210so that the sensor data may be used to make such a determination. For example, VC computing device102may identify a certain audio signal (e.g., a sound) as an indicator that the patient is taking and/or has taken a certain medication.

VC computing device102may be further configured to record214the task as fulfilled (or unfulfilled) based on the determination. VC computing device102may store record214in association with the corresponding task identifier in database106. In some embodiments, VC computing device102may parse records214and the time deadlines in database106to identify tasks204that are recorded as being unfulfilled past the time deadline. For each of the identified overdue tasks204, VC computing device102may generate an alert216, which may be provided to, for example, the caregiver and/or the patient. For example, VC computing device102may transmit an email, SMS message, and/or another form of notification to user computing device108. Additionally or alternatively, VC computing device102may generate an audio notification (e.g., at the smart speaker) to remind the patient to perform the overdue task.

In some embodiments, VC computing device may be additionally configured to assist patients (e.g., dementia patients) in recalling words and ideas when speaking. For example, VC computing device102may record past instances of speech (e.g., stories, remarks, questions, and/or other utterances) of the patient using sensors110and identify the past instance as corresponding to a current instance of speech of the patient. If the patient has difficulty with the current instance of speech (e.g., pausing unnaturally and/or struggling to find a next word), VC computing device102may generate an audio reminder (e.g., using speaker112) to assist the patient in remembering the intended speech and/or assist others in understanding the patient.

In such embodiments, VC computing device102may detect, at sensor110, a first instance of speech218(e.g., the past instance of speech). For example, in embodiments where VC computing device102is in communication with a microphone and/or smart speaker, VC computing device102may record speech made by the patient. VC computing device102may store speech data220derived from the detected first instance of speech218in database106(e.g., in a speech database).

In such embodiments, VC computing device102may detect, at sensor110(e.g., the microphone and/or smart speaker), a second instance of speech222(e.g., the current instance of speech), and in response to the detection, compare224second instance of speech228to first instance of speech218. VC computing device102may determine that second instance of speech222corresponds to first instance of speech218(e.g., they correspond to the same story) and generate, at speaker112, an audio reminder226. Audio reminder226may include, for example, key words or phrases from the first instance of speech that may assist the patient and/or other listeners in recalling the intended speech. In some embodiments, VC computing device102may utilize AI and/or machine learning techniques, for example, to determine that second instance of speech222corresponds to first instance of speech218and/or to generate the audio reminder226.

FIG.3illustrates an example configuration of a client system300in accordance with one embodiment of the present disclosure. In the example embodiment, client system300includes at least one user computing device302, operated by a user301. User computer device302may include, but is not limited to, one or more of VC computing device102, user computing device108, sensors110, and/or speakers112(all shown inFIG.1). User computer device302includes a processor305for executing instructions, and a memory area310. In some embodiments, executable instructions are stored in memory area310. Processor305may, for example, include one or more processing units (e.g., in a multi-core configuration). Memory area310may, for example, be any device allowing information such as executable instructions and/or transaction data to be stored and retrieved. Memory area310may further include one or more computer readable media.

In the example embodiment, user computer device302further includes at least one media output component315for presenting information to user301. Media output component315may, for example, be any component capable of converting and conveying electronic information to user301. For example, media output component315may be a display component configured to display component lifecycle data in the form of reports, dashboards, communications, and the like In some embodiments, media output component315includes an output adapter (not shown), such as a video adapter and/or an audio adapter, which is operatively coupled to processor305and operatively connectable to an output device (also not shown), such as a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED) display, or “electronic ink” display) or an audio output device (e.g., a speaker or headphones).

In some embodiments, media output component315is configured to include and present a graphical user interface (not shown), such as a web browser and/or a client application, to user301. The graphical user interface may include, for example, an online store interface for viewing and/or purchasing items, and/or a wallet application for managing payment information. In some embodiments, user computer device302includes an input device320for receiving input from user301. User301may use input device320to, without limitation, select and/or enter one or more items to purchase and/or a purchase request, or to access credential information, and/or payment information. Input device320may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch sensitive panel, a touch pad, a touch screen, a gyroscope, an accelerometer, a position detector, an audio input device, a fingerprint reader/scanner, a palm print reader/scanner, a iris reader/scanner, a retina reader/scanner, a profile scanner, or the like. A single component such as a touch screen may function as both an output device of media output component315and input device320. User computing device302may also include a communication interface325, which is communicatively connectable to a remote device such as VC computing device102(shown inFIG.1). Communication interface325may include, for example, a wired or wireless network adapter or a wireless data transceiver for use with a mobile phone network (e.g., Global System for Mobile communications (GSM), 3G, 4G or Bluetooth) or other mobile data network (e.g., Worldwide Interoperability for Microwave Access (WIMAX)).

Stored in memory area310are, for example, computer readable instructions for providing a user interface to user301via media output component315and, optionally, receiving and processing input from input device320. A user interface may include, among other possibilities, a web browser, and client application. Web browsers enable users, such as user301, to display and interact with media and other information typically embedded on a web page or a website from a server system. A client application allows user301to interact with a server application from the server system. For example, instructions may be stored by a cloud service, and the output of the execution of the instructions sent to the media output component315.

Processor305executes computer-executable instructions for implementing aspects of the disclosure. In some embodiments, the processor305is transformed into a special purpose microprocessor by executing computer-executable instructions or by otherwise being programmed. For example, the processor305may be programmed with instructions such that it may execute the processes as illustrated inFIGS.6-8, below.

FIG.4illustrates an example configuration of a server system400, such as VC computing device102(shown inFIG.1). In the example embodiment, server system400includes at least one server computing device401, in electronic communication with at least one storage device417. Server computing device401may include, but is not limited to, VC computing device102. In the exemplary embodiment, server computing device401includes a processor405for executing instructions (not shown) stored in a memory area410. In an embodiment, processor405may include one or more processing units (e.g., in a multi-core configuration) for executing instructions. The instructions may be executed within various different operating systems on the server system400, such as UNIX®, LINUX® (LINUX is a registered trademark of Linus Torvalds), Microsoft Windows®, etc. More specifically, the instructions may cause various data manipulations on data stored in storage device417(e.g., create, read, update, and delete procedures. It should also be appreciated that upon initiation of a computer-based method, various instructions may be executed during initialization. Some operations may be required in order to perform one or more processes described herein, while other operations may be more general and/or specific to a particular programming language (e.g., C, C#, C++, Java, or other suitable programming languages, etc.).

In the example embodiment, processor405is operatively coupled to a communication interface415such that server system400is capable of communicating with a remote device such as a user system or another server system400. For example, communication interface415may receive requests from client system300(FIG.3) via the Internet, within the scope of the embodiment illustrated inFIG.4.

In the example embodiment, processor405is also operatively coupled to a storage device417, which may be, for example, any computer-operated hardware unit suitable for storing and/or retrieving data. In some embodiments, storage device417is integrated in server system400. For example, server system400may include one or more hard disk drives as storage device417. In certain embodiments, storage device417is external to server system400and is similar to database106(shown inFIG.1). For example, server system400may include one or more hard disk drives as storage device417. In other embodiments, storage device417is external to server system400and may be accessed by a plurality of server systems400. For example, storage device417may include multiple storage units such as hard disks or solid state disks in a redundant array of inexpensive disks (RAID) configuration. storage device417may include a storage area network (SAN) and/or a network attached storage (NAS) system.

In some embodiments, processor405is operatively coupled to storage device417via a storage interface420. Storage interface420may include, for example, a component capable of providing processor405with access to storage device417. In an exemplary embodiment, storage interface420further includes one or more of an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any similarly capable component providing processor405with access to storage device417.

In some embodiments, processor405includes one or more modules for performing specific tasks executed by processor405. For example, processor405may include a communication module430, an analytics module432, and a data management module434.

In some embodiments, communication module430may be configured for, for example, receiving task data; providing, at the user device, a user interface; receiving task data may include receiving the task data via a user input at a user device; receiving the task data from the at least one sensor; receiving, from the at least one sensor, sensor data; generating at least one of an audio alert and a text alert based on the determination that the task is overdue; detecting, at the sensor, a first instance of speech; detecting, at the sensor, a second instance of speech; and/or generating an audio reminder based on the comparison.

In some embodiments, analytics module432may be configured for, for example, generating, based on the task data, a task including a task identifier, a fulfillment condition, and a time deadline; comparing at least one of the received sensor data and the retrieved transaction data to the fulfillment condition to determine that the fulfillment condition has occurred; determining that the task is overdue based on the record and the time deadline corresponding to the task; comparing the second instance of speech to the first instance of speech in response to the detection of the second instance of speech; and/or generating, based on the generated tasks, a query to identify transactions corresponding to completion of the tasks.

In some embodiments, data management module434may be configured for, for example, storing the generated task in a task database; recording, in the task database, the task as fulfilled based on the determination; storing the detected first instance of speech in a speech database; retrieving, from the payment processing network, transaction data; parsing transactions occurring via the payment processing network; and/or performing the query in the transaction database.

Memory area410may include, but is not limited to, random-access memory (RAM) such as dynamic RAM (DRAM) or static RAM (SRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile RAM (NVRAM), and magneto-resistive random-access memory (MRAM). The above memory types are for example only, and are thus not limiting as to the types of memory usable for storage of a computer program.

FIG.5is a diagram500of components of one or more example computing devices510that may be used in VC system100shown inFIG.1. In some embodiments, computing device510is similar to VC computing device102(shown inFIG.1). Database520may be coupled with several separate components within computing device510, which perform specific tasks. In this embodiment, database520includes task data522, sensor data524, and speech data526. In some embodiments, database520is similar to database106(shown inFIG.1).

Computing device510includes database520, as well as data storage devices730for storing data within database520. Computing device510may also be in communication with one or more of a communication component540, an analytics component550, and a data management component560.

In some embodiments, communication component540may be configured for, for example, receiving task data; providing, at the user device, a user interface; receiving task data may include receiving the task data via a user input at a user device; receiving the task data from the at least one sensor; receiving, from the at least one sensor, sensor data; generating at least one of an audio alert and a text alert based on the determination that the task is overdue; detecting, at the sensor, a first instance of speech; detecting, at the sensor, a second instance of speech; and/or generating an audio reminder based on the comparison.

In some embodiments, analytics component550may be configured for, for example, generating, based on the task data, a task including a task identifier, a fulfillment condition, and a time deadline; comparing at least one of the received sensor data and the retrieved transaction data to the fulfillment condition to determine that the fulfillment condition has occurred; determining that the task is overdue based on the record and the time deadline corresponding to the task; comparing the second instance of speech to the first instance of speech in response to the detection of the second instance of speech; and/or generating, based on the generated tasks, a query to identify transactions corresponding to completion of the tasks.

In some embodiments, data management component560may be configured for, for example, storing the generated task in a task database; recording, in the task database, the task as fulfilled based on the determination; storing the detected first instance of speech in a speech database; retrieving, from the payment processing network, transaction data; parsing transactions occurring via the payment processing network; and/or performing the query in the transaction database.

FIGS.6,7, and8depict a flowchart illustrating an example computer-implemented method600for implementing a VC that assists patients and caregivers in tracking tasks, which may be implemented using VC system100(shown inFIG.1). Computer-implemented method600may be implemented by a computing device, for example, VC computing device102(shown inFIG.1), in cooperation with VC system100.

In the example embodiment, computer-implemented method600may include receiving602task data. In some embodiments, receiving602task data may be performed by VC computing device102, for example, by executing communication module430(shown inFIG.4).

In some embodiments, computer-implemented method may further include providing604, at the user device, a user interface for submitting task data. In some such embodiments, providing604the user interface may be performed by VC computing device102, for example, by executing communication module430.

In some embodiments, computer-implemented method600may further include receiving606task data which may include receiving the task data via a user input at a user device. In some such embodiments, receiving606the task data may be performed by VC computing device102, for example, by executing communication module430.

In some embodiments, computer-implemented method600may further include receiving608the task data from the at least one sensor. In such embodiments, the task data may include audio signals corresponding to speech. In some such embodiments, receiving608the task data may be performed by VC computing device102, for example, by executing communication module430.

Computer-implemented method600may further include generating610, based on the task data, a task including a task identifier, a fulfillment condition, and a time deadline. In some embodiments, generating610the task, the fulfillment condition and the time deadline may be performed by VC computing device102, for example, by executing analytics module432(shown inFIG.4).

Computer-implemented method600may further include storing612the generated task in a task database. In some embodiments, storing612the generated task may be performed by VC computing device102, for example, by executing data management module434(shown inFIG.4).

Computer-implemented method600may further include receiving614, from the at least one sensor, sensor data. In some embodiments, the sensor data may include audio signals. In some embodiments, receiving614the sensor data may be performed by VC computing device102, for example, by executing communication module430.

Computer-implemented method600may further include retrieving616, from the payment processing network, transaction data. In some embodiments, retrieving616the transaction data may be performed by VC computing device102, for example, by executing data management module434.

In some embodiments, computer-implemented method600may further include parsing618transactions occurring via the payment processing network. In some embodiments, parsing618transactions may be performed by VC computing device102, for example, by executing data management module434. Parsing the transaction data may include identifying the merchant involved in a particular transaction along with the transaction amount and time of the transaction. This parsed data may be used by the computing system to determine what product or service was purchased, and when it was purchased. This data may be further used to determine whether a task has been completed or at least whether a portion of the task has been completed. The transaction data may be used by itself or in combination with other sensor data to determine whether a task has been completed.

In some embodiments, computer-implemented method600may further include generating620, based on the generated tasks, a query to identify transactions corresponding to completion of the tasks. In some embodiments, generating620the query is performed by VC computing device102, for example, by executing analytics module432.

In such embodiments, computer-implemented method600may further include performing622the query in the transaction database. In some embodiments, performing622the query may be performed by VC computing device102, for example, by executing data management module434.

Computer-implemented method600may further include comparing624at least one of the received sensor data and the retrieved transaction data to the fulfillment condition to determine that the fulfillment condition has occurred. In some embodiments, comparing624the received sensor data to the fulfillment condition may be performed by VC computing device102, for example, by executing analytics module432.

Computer-implemented method600may further include recording626, in the task database, the task as fulfilled based on the determination that the fulfillment condition has occurred. In some embodiments, recording626the task may be performed by VC computing device102, for example, by executing data management module434.

In some embodiments, computer-implemented method600may further include determining628that the task is overdue based on the based on the fulfillment condition not having occurred by the time deadline corresponding to the task. In some such embodiments, determining628that the task is overdue may be performed by VC computing device102, for example, by executing analytics module432.

In such embodiments, computer-implemented method may further include generating630at least one of an audio alert and a text alert based on the determination that the task is overdue. In some embodiments, generating630the audio alert or the text alert may be performed by VC computing device102, for example, by executing communication module430.

FIG.9depicts a flowchart illustrating an example computer-implemented method900for implementing a VC that assists patients in speaking, which may be implemented using VC system100(shown inFIG.1). Computer-implemented method900may be implemented by a computing device, for example, VC computing device102(shown inFIG.1), in cooperation with VC system100.

In the exemplary embodiment, computer-implemented method900may include detecting902, at the sensor, a first instance of speech. In some embodiments, detecting902the first instance of speech may be performed by VC computing device102, for example, by executing communication module430(shown inFIG.4).

Computer-implemented method900may further include storing904the detected first instance of speech in a speech database. In some embodiments, storing904the first instance of speech may be performed by VC computing device102, for example, by executing data management module434(shown inFIG.4).

Computer-implemented method900may further include detecting906, at the sensor, a second instance of speech. In some embodiments, detecting906the second instance of speech may be performed by VC computing device102, for example, by executing communication module430.

Computer-implemented method900may further include comparing908the second instance of speech to the first instance of speech in response to the detection of the second instance of speech. In some embodiments, comparing908the second instance of speech to the first instance of speech may be performed by VC computing device102, for example, by executing analytics module432(shown inFIG.4).

Computer-implemented method900may further include generating910an audio reminder based on the comparison. In some embodiments, the audio reminder includes at least a portion of the first instance of speech. In some embodiments, generating910the audio reminder may be performed by VC computing device102, for example, by executing communication module430.

A processor or a processing element may employ artificial intelligence and/or be trained using supervised or unsupervised machine learning, and the machine learning program may employ a neural network, which may be a convolutional neural network, a deep learning neural network, or a combined learning module or program that learns in two or more fields or areas of interest. Machine learning may involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data. Models may be created based on example inputs in order to make valid and reliable predictions for novel inputs.

Additionally or alternatively, the machine learning programs may be trained by inputting sample data sets or certain data into the programs, such as image data, text data, report data, and/or numerical analysis. The machine learning programs may utilize deep learning algorithms that may be primarily focused on pattern recognition, and may be trained after processing multiple examples. The machine learning programs may include Bayesian program learning (BPL), voice recognition and synthesis, image or object recognition, optical character recognition, and/or natural language processing—either individually or in combination. The machine learning programs may also include natural language processing, semantic analysis, automatic reasoning, and/or machine learning.

In supervised machine learning, a processing element may be provided with example inputs and their associated outputs, and may seek to discover a general rule that maps inputs to outputs, so that when subsequent novel inputs are provided the processing element may, based on the discovered rule, accurately predict the correct output. In unsupervised machine learning, the processing element may be required to find its own structure in unlabeled example inputs. In one embodiment, machine learning techniques may be used to extract data about the computer device, the user of the computer device, the computer network hosting the computer device, services executing on the computer device, and/or other data.

Based on these analyses, the processing element may learn how to identify characteristics and patterns that may then be applied to training models, analyzing transaction and authentication data, and detecting and analyzing risk.

Having described aspects of the disclosure in detail, it will be apparent that modifications and variations are possible without departing from the scope of aspects of the disclosure as defined in the appended claims. As various changes could be made in the above constructions, products, and methods without departing from the scope of aspects of the disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.

While the disclosure has been described in terms of various specific embodiments, those skilled in the art will recognize that the disclosure can be practiced with modification within the spirit and scope of the claims.

As used herein, the term “non-transitory computer-readable media” is intended to be representative of any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as, computer-readable instructions, data structures, program modules and sub-modules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions embodied in a tangible, non-transitory, computer readable medium, including, without limitation, a storage device and/or a memory device. Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. Moreover, as used herein, the term “non-transitory computer-readable media” includes all tangible, computer-readable media, including, without limitation, non-transitory computer storage devices, including, without limitation, volatile and nonvolatile media, and removable and non-removable media such as a firmware, physical and virtual storage, CD-ROMs, DVDs, and any other digital source such as a network or the Internet, as well as yet to be developed digital means, with the sole exception being a transitory, propagating signal.

As will be appreciated based on the foregoing specification, the above-described embodiments of the disclosure may be implemented using computer programming or engineering techniques including computer software, firmware, hardware or any combination or subset thereof, wherein the technical effect is a system that automatically generates and tracks tasks to be performed by a patient. Any such resulting program, having computer-readable code means, may be embodied or provided within one or more computer-readable media, thereby making a computer program product, i.e., an article of manufacture, according to the discussed embodiments of the disclosure. The article of manufacture containing the computer code may be made and/or used by executing the code directly from one medium, by copying the code from one medium to another medium, or by transmitting the code over a network.

In addition, although various elements of the VC computing device are described herein as including general processing and memory devices, it should be understood that the VC computing device is a specialized computer configured to perform the steps described herein.

This written description uses examples to disclose the embodiments, including the best mode, and also to enable any person skilled in the art to practice the embodiments, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial locational differences from the literal language of the claims.