Patent Description:
There are provided a device, a method, a computer program product and a system as per the appended claims. According to a first aspect, a device includes one or more processers. The one or more processors receive, from a plurality of sensors, sensor data relating to a user. The plurality of sensors includes a plurality of types of sensors including a spectrometer and one or more of an accelerometer, a heart rate sensor, a blood pressure sensor, a blood sugar sensor, a perspiration sensor, a skin conductivity sensor, or an imaging sensor. The one or more processors receive a first classification model for spectroscopic analysis, that is associated with identifying a characteristic of a food item based on chemometric signature obtained for the food item, wherein the first classification model is obtained by performing spectroscopic analysis on a set of identified subjects and by utilizing an optimization technique to distinguish between respective chemometric signatures for the calibration set. The one or more processors process the sensor data, from the plurality of types of sensors, relating to the user to determine a characteristic of a food item; wherein the characteristic of the food item is one of a nutritional content of a food item, a composition of the food item and a calorie intake by a user, wherein the one or more processors, when processing the sensor data, are to: obtain a set of properties relating to the user based on sensor data from the one or more of the accelerometer, the heart rate sensor, the blood pressure sensor, the blood sugar sensor, the perspiration sensor, the skin conductivity sensor, or the imaging sensor; generate a second classification model for spectroscopy analysis based on the first classification model for spectroscopy analysis and based on the set of properties; and utilize the second classification model for spectroscopy analysis and the sensor data from the spectrometer to determine the characteristic of the food item. The one or more processors may provide, via a user interface, information relating to the determined characteristic of the food item based on processing the sensor data, from the plurality of types of sensors, relating to the user.

The one or more processors, when processing the sensor data, may: determine, based on a set of images of a food item obtained via the imaging sensor, a volume of the food item; determine, based on a chemometric signature of the food item obtained via the spectrometer and based on the volume of the food item, a nutritional content of the food item; and where the one or more processors, when providing the information identifying the health condition, may: provide information identifying the nutritional content of the food item.

The one or more processors may: receive a diet relating to the user; and where the one or more processors, when processing the sensor data, may: determine a compliance with the diet by the user based on the sensor data; and where the one or more processors, when providing information identifying the characteristic of the food item, may: provide information identifying the compliance with the diet by the user.

The one or more processors may further: generate a recommendation, relating to the determined characteristic of the food item, for the user based on processing the sensor data to determine the characteristic of the food item, the recommendation relating to the altering a user diet to cause a corresponding alteration to sensor data associated with the user diet; and provide information identifying the recommendation.

A blood glucose level may be determined based on the sensor data and the first classification model. The one or more processors may calibrate the first classification model to generate the second classification model or quantitative model based on the first classification model and the blood glucose level. The first classification model may be calibrated with an optimization technique. The second classification model may be utilized to determine another blood glucose level based on the other sensor data.

According to another aspect, a method includes receiving, at a device comprising one or more processors and from a plurality of sensors, sensor data relating to a user, the plurality of sensors including a plurality of types of sensors including a spectrometer and one or more of an accelerometer, a heart rate sensor, a blood pressure sensor, a blood sugar sensor, a perspiration sensor, a skin conductivity sensor, or an imaging sensor.

A first classification model for spectroscopic analysis is received. It is associated with identifying a characteristic of a food item based on chemometric signature obtained for the food item, wherein the first classification model is obtained by performing spectroscopic analysis on a set of identified subjects and by utilizing an optimization technique to distinguish between respective chemometric signatures for the calibration set.

The method further comprises processing the sensor data, from the plurality of types of sensors, relating to the user to determine a characteristic of a food item, wherein the characteristic of the food item is one of a nutritional content of a food item, a composition of the food item and a calorie intake by a user.

The processing comprises obtaining a set of properties relating to the user based on sensor data from the one or more of the accelerometer, the heart rate sensor, the blood pressure sensor, the blood sugar sensor, the perspiration sensor, the skin conductivity sensor, or the imaging sensor, generating a second classification model for spectroscopy analysis based on the first classification model for spectroscopy analysis and based on the set of properties , and using the second classification model for spectroscopy analysis and the sensor data from the spectrometer to determine the characteristic of the food item.

The method further comprises providing, via a user interface, information relating to the determined characteristic of the food item based on processing the sensor data, from the plurality of types of sensors, relating to the user.

According to a another aspect, a computer product includes machine executable instructions residing on a non-transitory computer readable media, which, when loaded and executed by a processor, cause the processor to perform operations according to the method described above.

There is further provided a system for determining a characteristic of a food item, the system configured to perform operations according to the method described above.

A wearable fitness tracker may utilize a set of accelerometers to obtain sensor data regarding user activity. For example, the wearable fitness tracker may determine an estimated distance traveled by a user based on sensor data identifying a movement of the wearable fitness tracker. The wearable fitness tracker may include information that may be utilized to estimate one or more metrics relating to user activity. For example, the wearable fitness tracker may estimate a calorie expenditure based on the estimated distance traveled and generic information correlating distance traveled to calorie expenditure for a generic person.

However, utilizing generic correlations to determine the one or more metrics may result in an inaccurate calculation for a particular user. Moreover, the wearable fitness tracker may fail to account for factors, other than exercise, that affect health of a user, such as nutrition, mood, illness, or the like, thereby limiting a usefulness of the wearable fitness tracker. Implementations, described herein, may utilize sensor data associated with multiple sensors to provide health information associated with a user. In this way, a single user device may determine the health information for a user, obviating the need for the user to utilize multiple different devices, thereby reducing cost, power consumption, or the like relative to utilizing the multiple different devices for health tracking. Moreover, the user device may utilize the sensor data from the multiple sensors to facilitate calibration of a model for spectroscopic analysis, thereby improving accuracy of the model relative to utilizing a model that is not calibrated based on sensor data from other sensors.

<FIG> is a diagram of an overview of an example implementation <NUM> described herein. As shown in <FIG>, example implementation <NUM> includes a user device, a health care provider application server, and a spectroscopy calibration application server.

As further shown in <FIG>, the user device receives sensor data from multiple sensors (e.g., first sensor data from a first sensor, second sensor data from a second sensor,. , and nth sensor data from an nth sensor). For example, the user device <NUM> may utilize a set of integrated sensors to obtain the sensor data, such as an integrated accelerometer sensor to obtain user activity data, an integrated heart rate sensor to obtain heart rate data, an integrated temperature sensor to obtain temperature data, or the like. Additionally, or alternatively, the user device may utilize a camera to obtain the sensor data. For example, the user device may utilize an integrated camera to capture an image, such as an image of a user's face (e.g., for a facial recognition analysis), an image of the user's skin (e.g., for a skin condition analysis), a set of images of food (e.g., a set of images for performing a volumetric analysis of the food), or the like. Additionally, the user device utilizes a spectroscopic sensor to obtain the sensor data. For example, the user device may utilize the spectroscopic sensor to determine a chemometric signature for a subject (e.g., the user or an item of food), and may classify the subject based on the chemometric signature and a classification model, wherein the classification of a food item forms part of the claimed invention. In another example, the user device may communicate with one or more sensors to obtain sensor data. For example, the user device may utilize a connection to a medical device to obtain sensor data recorded by the medical device.

As further shown in <FIG>, the user device combines the sensor data from the multiple sensors to generate health information regarding the user. For example, based on image sensor data and spectroscopic sensor data, the user device may determine, respectively, a volume of a food item and a nutritional content of the food item. Based on utilizing the spectroscopic sensor data identifying a composition of the food item and the sensor data identifying a volume of the food item, the user device may obtain a more accurate determination of calorie intake by the user than based on utilizing a user estimation of nutritional value. Based on the calorie expenditure determination and the calorie intake determination, the user device may determine a net calorie consumption for a user, and may generate a recommendation associated with the net calorie consumption, such as a nutritional recommendation, an exercise recommendation, or the like to improve user health and/or user compliance with a diet and exercise plan. In this way, the user device performs determinations relating to health of the user that are more accurate than determinations performed by a wearable fitness tracker, thereby permitting improved fitness tracking.

As another example, the user device may process spectroscopic sensor data to calibrate a classification model for spectroscopy. For example, the user device may utilize a first model to perform a first classification of first spectroscopic sensor data, such as to classify an observed chemometric signature as relating to a particular person with a particular blood pressure and after eating a particular food item. In this case, the user device may calibrate the first model to generate a second model based on the first spectroscopic sensor data and other sensor data (e.g., sensor data identifying the particular blood pressure and the particular food item), and may utilize the second model to perform another classification of second spectroscopic sensor data. For example, the user device may utilize the second classification model to distinguish between the particular person when the particular person is associated with the particular blood pressure and the particular person when the particular person is associated with another blood pressure. In this way, the user device refines a classification model to perform spectroscopy with improved accuracy relative to a classification model that is not refined based on other sensor data.

As further shown in <FIG>, the user device may provide information, such as health information relating to the user, diagnostic information relating to a health condition of the user, recommendations relating to improving user health and/or compliance with a diet and exercise plan, or the like based on processing the sensor data from the multiple sensors. Compliance with a diet and exercise plan may be associated with reducing a severity of a health condition (e.g., a heart disease condition or an obesity condition), managing symptoms of a health condition (e.g., a diabetes health condition or a stress condition), reducing a likelihood of deterioration of a health condition (e.g., a degenerative condition), achieving a desired health condition (e.g., improving diet, increasing an intake of whole grains, increasing muscle mass, or improving an athletic performance). For example, the user device may provide, for display via a user interface, a real-time health update, such as information indicating a net calorie consumption based on the calorie expenditure and the calorie intake determinations. Additionally, or alternatively, the user device may provide diagnostic information indicating that a particular condition is detected for the user, and may automatically transmit data to a specialist to establish an appointment for the user with the specialist. Additionally, or alternatively, the user device may generate a recommendation, such as a recommendation for an exercise regimen to permit the user to comply with a diet and exercise plan, and may provide updates regarding user compliance with the diet and exercise plan for display via another device (e.g., a user device utilized by a personal trainer or a nutritionist). In this way, the user device provides information customized to the user rather than generic information.

As further shown in <FIG>, the user device may provide health information to a health care provider application server for inclusion in a patient file relating to the user. In this way, the user device improves doctor patient consultations by improving data accuracy relative to a doctor relying on manual patient reporting of diet, exercise, or the like and reducing a utilization of processing resources and/or a power consumption relative to providing one or more user interfaces for receiving input of patient reporting and/or correcting the input of the patient reporting. Additionally, or alternatively, the user device may provide calibration data to spectroscopy calibration application server. For example, the user device may provide calibration information determined from a spectroscopic sensor and/or other sensors for utilization in calibrating and/or refining a classification model that is provided to one or more other user devices, thereby improving accuracy of spectroscopy for the one or more other user devices relative to the one or more other user devices receiving a model not calibrated based on the calibration information.

In this way, the user device provides health information determined based on sensor data obtained from multiple sensors, thereby reducing cost and power consumption relative to the user utilizing multiple different devices. Moreover, based on integrating the sensor data collection and processing into a single user device, the user device permits determination of health information that is not obtainable via a single sensor, such as obtaining nutrition information based on both a volumetric analysis and a spectroscopic analysis or the like. Furthermore, the user device improves calibration of a classification model for spectroscopic analysis based on performing model calibration and model refinement based on both spectroscopic data and other sensor data regarding a subject of spectroscopic analysis relative to a single initial calibration via a single device.

<FIG> is a diagram of an example environment <NUM> in which systems and/or methods, described herein, may be implemented. As shown in <FIG>, environment <NUM> may include a user device <NUM>, an application server <NUM>, and a network <NUM>. Devices of environment <NUM> may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections.

User device <NUM> includes one or more devices capable of receiving, generating, storing, processing, and/or providing health information. For example, user device <NUM> may include a communication and/or computing device, such as a mobile phone (e.g., a smart phone, a radiotelephone, etc.), a laptop computer, a tablet computer, a handheld computer, a gaming device, a wearable device (e.g., a smart wristwatch, a pair of smart eyeglasses, a smart wristband, etc.), a medical device, a spectroscopic device (e.g., a wearable spectrometer device that performs near infrared (NIR) spectroscopy, mid-infrared (mid-IR) spectroscopy, or Raman spectroscopy), or a similar type of device. In some implementations, the spectroscopic device may include a hyperspectral spectrometer (e.g., a hyperspectral imaging sensor). In some implementations, user device <NUM> may receive information from and/or transmit information to another device in environment <NUM>.

Application server <NUM> includes one or more devices capable of storing, processing, and/or routing information, such as health information, calibration information, or the like. For example, application server <NUM> may include a server that utilizes health information and/or information associated with user device <NUM>. In some implementations, application server <NUM> may include a calibration application server <NUM> that receives information associated with calibration of a spectroscopic model based on a set of measurements performed by one or more user devices <NUM>. Additionally, or alternatively, application server <NUM> may include a health care provider application server <NUM> associated with routing information for a health care provider, such as a hospital server or the like. In some implementations, application server <NUM> may receive information from and/or transmit information to another device in environment <NUM>.

Network <NUM> includes one or more wired and/or wireless networks. For example, network <NUM> may include a cellular network (e.g., a long-term evolution (LTE) network, a <NUM> network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the Public Switched Telephone Network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, or the like, and/or a combination of these or other types of networks.

<FIG> is a diagram of example components of a device <NUM>. Device <NUM> may correspond to user device <NUM> and/or application server <NUM>. In some implementations, user device <NUM> and/or application server <NUM> may include one or more devices <NUM> and/or one or more components of device <NUM>. As shown in <FIG>, device <NUM> may include a bus <NUM>, a processor <NUM>, a memory <NUM>, a storage component <NUM>, an input component <NUM>, an output component <NUM>, and a communication interface <NUM>.

Bus <NUM> includes a component that permits communication among the components of device <NUM>. Processor <NUM> is implemented in hardware, firmware, or a combination of hardware and software. Processor <NUM> may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, and/or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that interprets and/or executes instructions. In some implementations, processor <NUM> may include one or more processors capable of being programmed to perform a function. Memory <NUM> includes a random access memory (RAM), a read only memory (ROM), and/or another type of dynamic or static storage device (e.g., a flash memory, a magnetic memory, an optical memory, etc.) that stores information and/or instructions for use by processor <NUM>.

For example, storage component <NUM> may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.), a compact disc (CD), a digital versatile disc (DVD), a floppy disk, a cartridge, a magnetic tape, and/or another type of non-transitory computer-readable medium, along with a corresponding drive.

Input component <NUM> includes a component that permits device <NUM> to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally, or alternatively, input component <NUM> may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component <NUM> includes a component that provides output information from device <NUM> (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.).

Communication interface <NUM> includes a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device <NUM> to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface <NUM> may permit device <NUM> to receive information from another device and/or provide information to another device. For example, communication interface <NUM> may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, or the like.

Device <NUM> may perform one or more processes described herein. Device <NUM> may perform these processes in response to processor <NUM> executing software instructions stored by a non-transitory computer-readable medium, such as memory <NUM> and/or storage component <NUM>. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.

<FIG> is a flow chart of an example process <NUM> for providing health information based on sensor data associated with multiple sensors. In some implementations, one or more process blocks of <FIG> may be performed by user device <NUM>. In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including user device <NUM>, such as application server <NUM> or the like.

As shown in <FIG>, process <NUM> includes obtaining sensor data from a set of sensors (block <NUM>). For example, user device <NUM> may obtain sensor data from the set of sensors of user device <NUM>. In some implementations, user device <NUM> may obtain the sensor data based on communicating with the set of sensors. For example, user device <NUM> may cause a heartbeat sensor of user device <NUM> to activate and record sensor data regarding a heartbeat of a user. Similarly, user device <NUM> may cause an accelerometer sensor of user device <NUM> to activate and record accelerometer sensor data regarding movement of user device <NUM> (e.g., activity of a user using user device <NUM>).

In some implementations, user device <NUM> may obtain the sensor data based on providing a prompt via a user interface. For example, user device <NUM> may provide a prompt to cause a user to utilize user device <NUM> to capture an image of the user (e.g., a photograph of the user's face or a photograph of a portion of the user's skin). Additionally, or alternatively, user device <NUM> may provide a prompt to cause the user to utilize user device <NUM> to capture an image of an item of food. For example, user device <NUM> may be utilized to capture an image of a meal, and may process the image of the meal to identify a nutritional content of the meal. Additionally, or alternatively, user device <NUM> may capture an image automatically. For example, user device <NUM> may determine that a meal is prepared, such as based on a time of day, based on location data indicating that user device <NUM> is at a restaurant, based on social media information indicating that user device <NUM> is at a restaurant, or the like, and may automatically activate an imaging sensor to capture an image. Similarly, user device <NUM> may determine, based on an accelerometer, based on a touch sensor, or the like, that an imaging sensor of user device <NUM> is directed toward the user, and may cause the imaging sensor to capture an image of the user.

In some implementations, user device <NUM> may monitor an imaging sensor to obtain sensor data. For example, user device <NUM> may perform an object recognition technique on a set of images captured via the imaging sensor to determine whether the set of images captured include information utilizable for providing a health report, such as determining that a particular image includes a food item, determining that the particular image includes the user, or the like. In this case, user device <NUM> may select the particular image for processing (e.g., a volumetric analysis processing technique, a mood analysis processing technique, or a skin condition processing technique).

In some implementations, user device <NUM> may obtain the sensor data based on a trigger. For example, user device <NUM> may detect a user interaction with a user interface, and may be caused to obtain the sensor data. Additionally, or alternatively, user device <NUM> may obtain the sensor data periodically. For example, based on determining that a threshold period of time has elapsed (e.g., an hour, a day, or a week), user device <NUM> may obtain the sensor data. Similarly, at a particular time of day, user device <NUM> may obtain the sensor data. In some implementations, user device <NUM> may obtain the sensor data based on other sensor data. For example, user device <NUM> may detect that a user has finished an exercise regimen based on monitoring a heart rate sensor (e.g., based on heart rate sensor data), and may obtain other sensor data based on determining that the user has finished the exercise regimen. Similarly, user device <NUM> may identify the user based on chemometric signature sensor data from a spectrometer sensor, and may obtain other sensor data based on identifying the user.

As further shown in <FIG>, process <NUM> includes processing the sensor data to obtain health information (block <NUM>). For example, user device <NUM> may process the sensor data to obtain the health information. In some implementations, user device <NUM> may process the sensor data to identify one or more metrics related to user health. For example, user device <NUM> may determine a nutrition metric based on sensor data associated with a blood pressure sensor, a blood sugar sensor, a perspiration sensor, a skin conductivity sensor, or the like.

In some implementations, user device <NUM> may utilize a particular processing technique to process the sensor data. For example, user device <NUM> may utilize a classification technique associated with a classification model to identify content of a food item based on a spectroscopic measurement (e.g., a chemometric signature), and may determine health information related to food consumption based on identifying the content of the food item. In some implementations, user device <NUM> may perform the classification based on sensor data received from a hyperspectral spectrometer, which may user difficulty and errors related to distance from the subject relative to utilizing another type of sensor. Additionally, user device <NUM> may apply a pattern detection technique, a facial recognition technique, a three-dimensional depth sensing technique, or the like to an image of a user to determine a user mood, a level of fatigue, a level of stress, a migraine symptom, a skin condition, or the like. Additionally, user device <NUM> may utilize a color classification technique to determine that a color of urine in an image corresponds to a particular health condition, such as insufficient or excessive consumption of a particular vitamin (e.g., a B-vitamin deficiency) or the like.

Additionally, or alternatively, user device <NUM> may utilize a volumetric analysis technique to process the sensor data. For example, user device <NUM> may utilize a set of images of a food item (e.g., a set of images captured from different orientations and/or positions), to determine a volume of the food item. Additionally, user device <NUM> may utilize sensor data captured by a depth sensing module, a gesture recognition module, or the like to perform the volumetric analysis. In some implementations, user device <NUM> may determine a food item mass based on the volumetric analysis. For example, user device <NUM> may utilize information identifying a density of the food item (e.g., sensor data, image recognition of the food item, or a user selection of the food item and corresponding stored food item density data), and may determine a mass for the food based on the density (e.g., which may be utilized to determine a nutritional content of the food based on a spectroscopic analysis indicating a nutritional content on a per mass basis). Additionally user device <NUM> may utilize a set of images of a user to determine a volume of a portion of the user, such as a volume of a shoulder hump, a skin bump, or the like. In this way, user device <NUM> identifies a volume of a subject for comparison with the subject at another time, to determine a nutritional content of the subject (e.g., with other sensor data identifying a type of food item that is the subject), or the like.

In some implementations, user device <NUM> may process the sensor data using a comparison technique, such as comparing first sensor data recorded at a first time to second sensor data recorded at a second time. For example, user device <NUM> may compare a first three-dimensional image of a user at a first time period with a second three-dimensional image of the user at a second time period to identify changes to the user's appearance, such as growth of a mole (e.g., corresponding to tumor growth), change to a breast shape (e.g., corresponding to cyst growth), change to a body shape (e.g., corresponding to a weight gain), change to a chemometric signature (e.g., corresponding to a change in blood composition associated with a diabetes type disorder), or the like.

As further shown in <FIG>, process <NUM> includes providing the health information (block <NUM>). For example, user device <NUM> may provide the health information. In some implementations, user device <NUM> may provide, via a user interface, a health report for review by a user. For example, user device <NUM> may provide a report including health information identifying a net calorie consumption (e.g., a comparison of calorie intake and calorie expenditure) for the user. Additionally, user device <NUM> may provide information identifying a portion of sensor data processed by user device <NUM>, such as a set of vital statistics determined for the user (e.g., a blood pressure, a body temperature, a heart rate, or a perspiration level).

In some implementations, user device <NUM> may provide a recommendation based on the sensor data. For example, user device <NUM> may determine a set of health recommendations (e.g., to improve an obesity condition, to manage a diabetes condition, to prevent a degenerative condition, to improve an athletic performance, or to satisfy a nutrition goal), may select a particular health recommendation of the set of health recommendations (e.g., that the user consume a particular quantity of calories during a particular meal based on a metabolic rate of the user), and may provide health information including the particular health recommendation for display via a user interface. Additionally, or alternatively, user device <NUM> may select another health recommendation (e.g., that the user is predicted to experience improved mood levels based on consuming a particular food or engaging in a particular exercise regimen), and may provide the other health recommendation for display via another user device <NUM> (e.g., that is utilized by a physician, a life coach, a trainer, or a personal chef).

In some implementations, user device <NUM> may provide an alert based on the sensor data. For example, user device <NUM> may provide, based on sensor data identifying a health condition of the user, a particular alert identifying the health condition for display via the user interface. In this case, the alert may identify the health condition, a severity of the health condition, or the like. Additionally, or alternatively, user device <NUM> may provide an alert for display via another user device <NUM>. For example, user device <NUM> may identify a specialist doctor associated with the health condition, and may transmit an alert for display via another user device <NUM> that is utilized by the specialist doctor. Similarly, user device <NUM> may transmit an alert to cause emergency management personnel to be dispatched for the user. For example, when user device <NUM> determines that a severity of the health condition satisfies a threshold severity, user device <NUM> may utilize a location determination technique to determine a location of the user, and may transmit an alert to an ambulance dispatch, a hospital, or the like to cause emergency management personnel to be dispatched to the location of the user. In some implementations, user device <NUM> may provide a health report when providing the alert.

In some implementations, user device <NUM> may provide calibration data. For example, when user device <NUM> transmits spectroscopic data regarding a user for calibrating a model, user device <NUM> may provide health information regarding the user for calibrating the model. In this case, the health information may be utilized to calibrate the model to a particular health condition.

In some implementations, user device <NUM> may provide, for display, an augmented reality image based on the sensor data. For example, when user device <NUM> determines a caloric content of a food item based on images of the food item, user device <NUM> may determine a portion of the food item that, when consumed, corresponds to satisfying a nutrition goal. In this case, user device <NUM> may provide, for display, an image of the food item with the portion of the food item highlighted, an augmented reality display of the food item with the portion of the food item highlighted, or the like. In this way, a user may be provided information indicating how much of a food item to consume to satisfy a nutrition goal.

<FIG> are diagrams of an example implementation <NUM> relating to example process <NUM> shown in <FIG>. <FIG> show an example of providing health information based on sensor data associated with multiple sensors.

As shown in <FIG>, and by reference number <NUM>, a wearable user device <NUM> (e.g., a smart watch including a set of sensors) may store a set of diet and exercise goals received from a health care provider application server <NUM>. For example, the set of diet and exercise goals may be generated based on a user consultation with a health care provider (e.g., a doctor, a personal trainer, or a nutritionist), and may be transmitted to wearable user device <NUM> to permit wearable user device <NUM> to monitor user diet and exercise to determine and improve compliance with the set of diet and exercise goals. Assume that the set of diet and exercise goals includes a diet goal associated with the user having an intake of a threshold amount of carbohydrates in a day, an exercise goal associated with the user satisfying a threshold physical activity level during the day, and a combined diet and exercise goal associated with the user satisfying a threshold net calorie consumption (e.g., a greater calorie expenditure than calorie intake during the day).

As shown in <FIG>, and by reference number <NUM>, wearable user device <NUM> performs a spectroscopic analysis of a food item to determine a content of the food item, such as a protein content, a fat content, or a carbohydrate content using data from a spectroscopic sensor. As shown by reference number <NUM>, wearable user device <NUM> performs an image analysis of food item to determine a volume of the food item using a set of images from an imaging sensor. Assume that wearable user device <NUM> determines a carbohydrate intake and a calorie intake for the user based on the content of the food item (e.g., the spectroscopic analysis indicating the content of the food) and the volume of the food item (e.g., the volumetric analysis indicating the content of the food). Assume that based on the carbohydrate intake, wearable user device <NUM> provides information via the user interface indicating compliance with the diet goal, such as an alert indicating an amount of carbohydrates that the user is permitted to consume during the remainder of the day.

As shown in <FIG>, and by reference number <NUM>, wearable user device <NUM> may further perform an activity analysis of a user exercise regimen using data from a heart rate sensor, an accelerometer sensor, or the like. Assume that wearable user device <NUM> determines a calorie expenditure and a physical activity level based on performing the activity analysis of the user exercise regimen. Assume that based on the physical activity level, wearable user device <NUM> provides information via the user interface indicating compliance with the exercise goal, such as information indicating that the user satisfied the threshold physical activity level for the day.

As shown in <FIG>, and by reference number <NUM>, wearable user device <NUM> may further determine compliance with the combined diet and exercise goal and generates an activity plan for the user associated with causing the user to perform a particular activity (e.g., a particular quantity of steps) associated with a corresponding alteration to sensor data associated with the user (e.g., causing the sensor data to indicate that user calorie expenditure has increased for the day). Assume that wearable user device <NUM> determines that the user has failed to satisfy the combined diet and exercise goal based on the calorie intake for the user exceeding the calorie expenditure by the user. Further assume that wearable user device <NUM> determines that based on previous activity by the user, walking a particular quantity of steps by the user before the end of the day corresponds to achieving the combined diet and exercise goal. As shown by reference number <NUM>, wearable user device <NUM> provides an alert indicating that the user is to walk the particular quantity of steps before the end of the day, and continues to monitor user activity and provide alerts to cause the user to walk the particular quantity of steps. In this way, wearable user device <NUM> improves compliance with the combined diet and exercise goal relative to a step counter device that only provides information indicating steps take and estimated calorie expenditure without including other data and without comparing estimated calorie expenditure to calorie intake.

As indicated above, <FIG> are provided merely as an example.

<FIG> is a flow chart of an example process <NUM> for dynamically updating a classification model for spectroscopic analysis. In some implementations, one or more process blocks of <FIG> may be performed by user device <NUM>. In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including user device <NUM>, such as application server <NUM>.

As shown in <FIG>, process <NUM> includes obtaining a first classification model for spectroscopic analysis (block <NUM>). For example, user device <NUM> may obtain the first classification model for spectroscopic analysis. A classification model (e.g., a spectroscopic classification model) may refer to a model that may be utilized to identify a subject of spectroscopic analysis or a characteristic of the subject of spectroscopic analysis based on a chemometric signature obtained for the subject. For example, the classification model may include information associated with a set of chemometric signatures for a set of samples (e.g., a set of persons or a set of food items), and user device <NUM> may utilize the classification model to determine that a chemometric signature for a person corresponds to a particular characteristic (e.g., a blood glucose level) for the person. Similarly, user device <NUM> may utilize another classification model to determine that a chemometric signature for a food item corresponds to a particular nutritional content for the food item.

In some implementations, user device <NUM> may obtain the first classification model from a particular application server <NUM> associated with calibrating the first classification model. For example, the particular application server <NUM> may perform spectroscopic analysis on a calibration set (e.g., a set of identified subjects) using a spectrometer, and may utilize a processing technique (e.g., an optimization technique to distinguish between respective chemometric signatures for the calibration set) to generate the first classification model. In this case, user device <NUM> may receive the first classification model based on the calibration model being optimized by application server <NUM>. Additionally, user device <NUM> may obtain a particular classification model that is calibrated based on spectroscopic analysis performed on a set of persons. For example, application server <NUM> may perform a first spectroscopic analysis on a first person and a second spectroscopic analysis on a second person, and may generate the first classification model to account for differences in chemometric signatures (e.g., associated with blood glucose levels) for the first person and the second person (e.g., relating to differing body composition).

In some implementations, user device <NUM> may obtain the first classification model based on requesting the first classification model. For example, user device <NUM> may transmit a request for the first classification model, and may receive the first classification model from application server <NUM> based on transmitting the request. Additionally, or alternatively, user device <NUM> may obtain the first classification model from a data structure of user device <NUM>. For example, user device <NUM> may include the first classification model stored via a data structure. In some implementations, user device <NUM> may generate the first classification model. For example, user device <NUM> may receive a set of chemometric signatures associated with a set of subjects, and may generate the first classification model based on the set of chemometric signatures. Additionally, or alternatively, user device <NUM> may perform a set of spectroscopic measurements on a set of known subjects to obtain chemometric signatures for the set of known subjects, and may generate the first classification model based on the chemometric signatures for the set of known subjects.

As further shown in <FIG>, process <NUM> includes obtaining a set of properties of a subject for the spectroscopic analysis (block <NUM>). For example, user device <NUM> may obtain the set of properties of the subject for spectroscopic analysis. In some implementations, user device <NUM> may determine the set of properties based on sensor data from one or more sensors. For example, user device <NUM> may utilize a sensor of user device <NUM> to determine a blood glucose level, a body temperature, or the like regarding a user based on sensor data recorded by the sensor. Additionally, or alternatively, user device <NUM> may communicate (e.g., via network <NUM>) with a sensor (e.g., a Bluetooth enabled sensor), to receive sensor data associated with a property of the subject for the spectroscopic analysis (e.g., the user).

In some implementations, user device <NUM> may obtain one or more properties of the subject associated with categorizing the subject. For example, user device <NUM> may obtain information identifying a gender of the subject, an age of the subject, an ethnicity of the subject, or the like. In this case, user device <NUM> may obtain the information from a data structure stored by user device <NUM>. Additionally, or alternatively, user device <NUM> may obtain the information from application server <NUM> (e.g., an application server associated with a health care provider and storing information regarding the user). In some implementations, user device <NUM> may obtain the information via a user interface. For example, user device <NUM> may generate a user interface and provide a set of prompts for display via the user interface, and may detect user interactions with the user interface associated with providing a set of responses to the set of prompts.

In some implementations, user device <NUM> may obtain a property of the subject of the spectroscopic analysis relating to a raw absorbance spectra of the subject of the spectroscopic analysis. For example, user device <NUM> may utilize an integrated spectroscopic sensor to perform a spectroscopic measurement of a user and determine a chemometric signature (e.g., a raw absorbance spectra) of the user. Additionally, or alternatively, user device <NUM> may communicate with a spectroscopic sensor to cause the spectroscopic sensor to determine a chemometric signature associated with the user, and user device <NUM> may receive the chemometric signature from the spectroscopic sensor based on communicating with the spectroscopic sensor.

As further shown in <FIG>, process <NUM> includes generating a second classification model for the spectroscopic analysis based on the set of properties and the first classification model (block <NUM>). For example, user device <NUM> may generate the second classification model for the spectroscopic analysis based on the set of properties and the first classification model. In some implementations, user device <NUM> may utilize a model optimization technique, such as a support vector machine classifier (SVM) optimization technique, or support vector regression (SVR) optimization technique, or the like, to optimize the first classification (or quantitative) model and generate the second classification (or quantitative) model. For example, user device <NUM> may optimize the first classification model (e.g., generated based on a set of persons) to generate the second classification model for performing classification relating to a user of user device <NUM> (e.g., determining a blood glucose level for the user). In this way, user device <NUM> accounts for differences between persons (e.g., body composition differences).

Additionally, or alternatively, user device <NUM> may optimize the first classification model (e.g., generated based on a first spectrometer associated with application server <NUM>) to generate a second classification model for utilization with a second spectrometer associated with user device <NUM>. For example, user device <NUM> may generate the second classification model for classifying food items based on spectroscopic sensor data obtained via the second spectrometer. In this way, user device <NUM> accounts for differences between spectrometers, thereby improving accuracy of spectrometry relative to utilizing a single classification model generated by application server <NUM> for each user device <NUM> utilized by each user.

As further shown in <FIG>, process <NUM> includes utilizing the second classification model to perform the spectroscopic analysis (block <NUM>). For example, user device <NUM> may utilize the second classification model to perform spectroscopic analysis. In some implementations, user device <NUM> may utilize the second classification model to determine a metric associated with a user (e.g., a subject of the spectroscopic analysis).

In some implementations, user device <NUM> may utilize other sensor data regarding the user to perform the spectroscopic analysis. For example, user device <NUM> may determine, based on both a chemometric signature of a food item and a volumetric analysis of the food item, a nutritional content of the item (e.g., a calorie content, a carbohydrate content, a protein content, or a fat content). In this way, user device <NUM> may combine sensor data from multiple sensors to determine and provide health information associated with a user.

In some implementations, user device <NUM> may provide information identifying a health condition based on performing the spectroscopic analysis. For example, user device <NUM> may generate a user interface, and may provide information identifying the health condition for display via the user interface. Additionally, or alternatively, user device <NUM> may provide information identifying the health condition to application server <NUM> (e.g., for inclusion in a patient medical record associated with the user). For example, user device <NUM> may provide information identifying a nutritional content of a food item. In some implementations, user device <NUM> may provide the results of performing the spectroscopic analysis for further processing. For example, user device <NUM> may perform the spectroscopic analysis, and may include data identifying the spectroscopic analysis in a dataset that is processed to determine and provide health information as described herein with regard to <FIG>.

As further shown in <FIG>, process <NUM> may include periodically updating the second classification model (block <NUM>). For example, user device <NUM> may periodically update the second classification model. In some implementations, user device <NUM> may optimize the second model. For example, when user device <NUM> performs a spectroscopic analysis of a user, user device <NUM> may utilize the spectroscopic analysis and/or other sensor data to refine the second model using an optimization technique, such as an SVM optimization technique or the like. In this case, user device <NUM> may utilize the refined second model for performing a subsequent spectroscopic analysis. In this way, user device <NUM> improves accuracy of the second classification model relative to utilizing the second classification model without performing a refinement procedure based on spectroscopic analysis and/or other sensor data. Furthermore, over time, the model continues to become more accurate and/or more specific to the user of the model, thereby permitting user device <NUM> to perform more accurate identifications of characteristics of the user, diagnoses of conditions of the user, or the like. Moreover, based on utilizing the SVM optimization technique, user device <NUM> reduces a utilization of processor and/or memory resources as well as reducing time intensively relative to generating a new classification model. Furthermore, an incremental training approach may permit user device <NUM> to develop an accurate model as sensor data is received by user device <NUM> rather than requiring sufficient processing and/or memory resources to generate an initial model based on large quantities of sensor data related to increasing quantities of sensors related to the Internet of Things.

As further shown in <FIG>, process <NUM> may include periodically providing feedback information associated with causing an update to the first classification model (block <NUM>). For example, user device <NUM> may provide feedback information associated with causing an update to the first classification model. In some implementations, user device <NUM> may provide information identifying the second classification model to cause an update to the first classification model. For example, user device <NUM> may provide information identifying the second classification model to application server <NUM> to cause application server <NUM> to update the first classification model.

In some implementations, user device <NUM> may provide information identifying sensor data obtained by user device <NUM> to cause the update to the first classification model.

Additionallyuser device <NUM> may provide demographic information to update the first classification model. For example, when the first classification model is generated based on chemometric signatures corresponding to blood glucose levels of a group of men, user device <NUM> may determine a chemometric signature corresponding to a blood glucose level for a woman. In this case, user device <NUM> may transmit information identifying the chemometric signature for the women to cause the first classification model to be optimized, thereby performing distributed calibration of the model to improve accuracy of the model relative to utilizing a statically calibrated classification model generated based on a limited group of samples (e.g., the group of men).

<FIG> and <FIG> are diagrams of an example implementation <NUM> relating to example process <NUM> shown in <FIG>. <FIG> and <FIG> show an example of dynamically updating a classification model for spectroscopic analysis.

As shown in <FIG>, and by reference number <NUM>, calibration application server <NUM> may generate a generic model (e.g., a generic classification model) based on sample data, obtained via a spectroscopic sensor associated with calibration application server <NUM>, relating to a set of sample persons. As shown by reference number <NUM>, wearable user device <NUM> receives the generic model from calibration application server <NUM>. As shown by reference number <NUM>, wearable user device <NUM> may obtain chemometric signature data relating to a user, such as a chemometric signature of a portion of the user's body. As shown by reference number <NUM>, wearable user device <NUM> may utilize an SVM optimization technique to generate a local model (e.g., another classification model) for the user based on the chemometric signature data, other data, such as sensor data (e.g., a heart rate of the user, a skin conductivity of the user, or a blood glucose level of the user), demographic data (e.g., an age of the user, a body mass index of the user, or a gender of the user), or the like. Assume that wearable user device <NUM> stores the local classification model for utilization in classifying one or more other chemometric signatures obtained regarding the user (e.g., for diagnosing a condition of the user).

As shown in <FIG>, and by reference number <NUM>, wearable user device <NUM> provides information associated with the local model to update the generic model, such as the chemometric signature data, the one or more other chemometric signatures, the data regarding the user, or the like. As shown by reference number <NUM>, calibration application server <NUM> is caused to perform an update of the generic model based on receiving the information associated with the local model.

As indicated above, <FIG> and <FIG> are provided merely as an example. Other examples are possible and may differ from what was described with regard to <FIG> and <FIG>.

In this way, user device <NUM> utilizes sensor data from multiple sensors to provide health information regarding a user, improve calibration of one or more sensors (e.g., calibration of a classification model associated with a spectroscopic sensor), or the like. Moreover, based on utilizing sensor data form multiple sensors, user device <NUM> reduces a cost and power consumption associated with requiring a device for each sensor. Furthermore, based on utilizing an optimization technique to calibrate the spectroscopic sensor, user device <NUM> reduces a utilization of processing and/or memory resources relative to generating a new classification model each time additional sample data and reduces cost relative to generating a classification model based on sufficient sample data to perform all classifications without refinement.

As used herein, the term component is intended to be broadly construed as hardware, firmware, and/or a combination of hardware and software.

Some implementations are described herein in connection with thresholds. As used herein, satisfying a threshold may refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than the threshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, etc..

Certain user interfaces have been described herein and/or shown in the figures. A user interface may include a graphical user interface, a non-graphical user interface, a text-based user interface, etc. A user interface may provide information for display. In some implementations, a user may interact with the information, such as by providing input via an input component of a device that provides the user interface for display. In some implementations, a user interface may be configurable by a device and/or a user (e.g., a user may change the size of the user interface, information provided via the user interface, a position of information provided via the user interface, etc.). Additionally, or alternatively, a user interface may be pre-configured to a standard configuration, a specific configuration based on a type of device on which the user interface is displayed, and/or a set of configurations based on capabilities and/or specifications associated with a device on which the user interface is displayed.

Thus, the operation and behavior of the systems and/or methods were described herein without reference to specific software code-it being understood that software and hardware can be designed to implement the systems and/or methods based on the description herein.

Even though particular combinations of features disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. Although each dependent claim listed below may directly depend on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

Claim 1:
A device (<NUM>), comprising:
one or more processors to:
receive, from a plurality of sensors, sensor data relating to a user, the plurality of sensors including a plurality of types of sensors including a spectrometer and one or more of an accelerometer, a heart rate sensor, a blood pressure sensor, a blood sugar sensor, a perspiration sensor, a skin conductivity sensor, or an imaging sensor;
receive a first classification model for spectroscopic analysis, that is associated with identifying a characteristic of a food item based on chemometric signature obtained for the food item, wherein the first classification model is obtained by performing spectroscopic analysis on a set of identified subjects and by utilizing an optimization technique to distinguish between respective chemometric signatures for the calibration set;
process (<NUM>, <NUM>) the sensor data, from the plurality of types of sensors, relating to the user to determine a characteristic of a food item; wherein the characteristic of the food item is one of a nutritional content of a food item, a composition of the food item and a calorie intake by a user, wherein the one or more processors, when processing the sensor data, are to:
obtain a set of properties relating to the user based on sensor data from the one or more of the accelerometer, the heart rate sensor, the blood pressure sensor, the blood sugar sensor, the perspiration sensor, the skin conductivity sensor, or the imaging sensor;
generate a second classification model for spectroscopy analysis based on the first classification model for spectroscopy analysis and based on the set of properties;
utilize the second classification model for spectroscopy analysis and the sensor data from the spectrometer to determine the characteristic of the food item;
provide, via a user interface, information relating to the determined characteristic of the food item based on processing the sensor data, from the plurality of types of sensors, relating to the user.