System and method using two-stage neural networks for predictive health monitoring

A system and method for predictive health monitoring using neural networks, comprising a wearable device with biometric sensors, a database containing data from multiple users across many categories of health-related factors, a first set of neural networks trained on the database that makes health predictions based on a single health factor, and a second neural network that makes health predictions based on a combination of the predictions made by the first set of neural networks.

CROSS-REFERENCE TO RELATED APPLICATIONS

BACKGROUND OF THE INVENTION

Field of the Art

The disclosure relates to the field of health monitoring devices, specifically the field of wearable health monitoring devices connected to cloud-based predictive networks.

Discussion of the State of the Art

It is currently possible for an athlete of any skill or dedication to go into a gym and find many types of exercise machinery, some of which may have computer chips and various levels of software on them, and some of which may be entirely mechanical in nature. Software-ready electronics are common in stationary bikes, elliptical machines, and treadmills, and in some cases exist for more specialized uses such as measuring the force exerted by a punch for boxing and other martial arts. These electronics and the software systems running on them can measure things such as estimated burned calories in a workout, the force and speed of punches or of running, the Revolutions Per Minute (RPM) of a bike and what this means for distance based on a user's settings on a stationary bike, and in some cases treadmills, elliptical machines and stationary bikes may even allow music or TV to be streamed to the user to enhance the pleasure of working out.

However, electronics with specialized software are noticeably lacking in the area of weight training, of virtually all kinds. There exists no common system which may determine the stresses an individual is undergoing while lifting in a variety of positions and warn them of, for example, poor form, uneven stresses in muscles such as if they are bench pressing, out of bounds positions such as overextending your arms during lateral pulldowns and other exercises, and more.

As well, none of the systems even in exercise machines currently, utilize machine learning and a large number of factors and biometric data to determine and accurately predict health events relevant to a user's exercise before they occur, nor do they often consider or warn a user of improper form during exercise, or for over-training or over-exertion, or myriad other concerns when engaging in strenuous physical activity. This results in users often achieving sub-par results from athletic activity, and being at-risk for health events from improper or overly taxing exercise, with workout equipment ill-equipped to aid or even consider these possibilities or users who have already suffered health events at all.

What is needed is a system and methods for a predictive health monitoring system utilizing a smart exercise belt which may aid in exercises for users to monitor their health and exercise form and progression, and more, and communicate with users to warn them of any health-related or exercise-related issues, with the goal of preventing future incidents if possible, warning users of impending or possible incidents in the near future, and aiding users in exercising both effectively and safely.

SUMMARY OF THE INVENTION

Accordingly, the inventor has conceived and reduced to practice, a system and method for predictive health monitoring using neural networks, comprising a wearable device with biometric sensors, a database containing data from multiple users across many categories of health-related factors, a first set of neural networks trained on the database that makes health predictions based on a single health factor, and a second neural network that makes health predictions based on a combination of the predictions made by the first set of neural networks. The following non-limiting summary of the invention is provided for clarity, and should be construed consistently with embodiments described in the detailed description below.

The disclosed invention makes use of at least a plurality of sensors attached to a wearable device, including pressure sensors, oximeters, accelerometers, gyroscopes, EEG, EMG, and heart rate monitors, to learn patterns of user activity, predict health events, and assist with athletic training including assisting with preventing over-training and assisting with exercise form, as well as tracking performance over time, and to help with medical rehabilitation by measuring performance and biometric feedback during activities after a medical event, for example during physical therapy after a stroke.

A system for predictive health monitoring is disclosed, comprising: a cloud-based health prediction engine comprising: a plurality of first-stage neural networks, each configured to make a first health prediction based on a health-related factor; a second-stage neural network, configured to make a second health prediction based on a combination of the first health predictions from at least two of the plurality of first-stage neural networks; a data storage device configured to store a history of biometric data and a history of health predictions for a user of a wearable biometric monitoring and feedback device; a network-connected server comprising a memory, a processor, and a plurality of programming instructions, wherein the programming instructions, when operating on the processor, cause the network-connected server to: receive biometric data from a wearable biometric monitoring and feedback device for the user; retrieve the history of biometric data and the history of health predictions for the user from the data storage device; process the biometric data, history of biometric data, and the history of health predictions through at least two of the plurality of first-stage neural networks; receive the first health prediction from each first-stage neural network through which the biometric data was processed; process the first health predictions received through a second-stage neural network; receive the second health prediction from the second-stage neural network; send the second health prediction to the wearable biometric monitoring and feedback device, and a wearable biometric monitoring and feedback device comprising: a plurality of sensors for gathering biometric data from the user of the wearable biometric monitoring and feedback device; a network device configured to connect to the cloud-based health prediction system; a screen for providing feedback to the user; and a computing device comprising a memory, a processor, and a plurality of programming instructions, wherein the programming instructions, when operating on the processor, cause the computing device to: obtain biometric data from at least two of the plurality of sensors for the user of the wearable biometric monitoring and feedback device; send the biometric data obtained to the cloud-based health prediction engine using the network device; receive a second health prediction from a cloud-based health prediction engine; and display the second health prediction to the user.

Further, a method for predictive health monitoring is disclosed, comprising the steps of: obtaining biometric data for a user of a wearable biometric monitoring and feedback device; retrieving a history of biometric data and a history of health predictions for the user from a data storage device; processing the biometric data, history of biometric data, and the history of health predictions through at least two of a plurality of first-stage neural networks, the plurality of first-stage neural networks, each configured to make a first health prediction based on a separate health-related factor; receiving a first health prediction from each first-stage neural network through which the biometric data was processed; processing the first health predictions received through a second-stage neural network, the second-stage neural network configured to make a second health prediction based on a combination of the first health predictions from at least two of the plurality of first-stage neural networks; receiving the second health prediction from the second-stage neural network; and displaying the second health prediction to the user of the wearable biometric monitoring and feedback device.

DETAILED DESCRIPTION

The inventor has conceived, and reduced to practice, a system and methods for predictive health monitoring.

Conceptual Architecture

FIG. 1is an exemplary diagram illustrating a plurality of sensors and sensory data that may be collected as input. Sensor inputs110may comprise electro-encephalography (EEG) sensors115, electromyography (EMG) sensors120, strain or pressure gauges125, plethysmographs for detecting blood pressure130, oximeters135, GPS sensors140, altitude sensors145, temperature, humidity, and other relevant climate or weather sensors or networked devices that may obtain such information from the Internet150, gyroscopes155, accelerometers160, breathing sensors165, and heart rate monitors170. The sensor inputs110may be gathered from sensors placed on or attached to a wearable device, as shown in other drawings, which may be attached to a user during exercise or other physical activity for the purposes of both building a model of user data such as health data and exercise habits, and for predictive health monitoring as a result of this model.

FIG. 2is an exemplary diagram of various forms of gathered and monitored user data that may be monitored for changes both in an individual user and in user groups, and which may be correlated together using machine learning to find relationships between data values and changes in data values. Multiple groups of data may influence each other and be collected in relation to one another, for instance the health profile220of a user and the type of exercise they are performing or continually perform240may be analyzed together to determine if someone is performing exercises that are more likely to lead to certain injuries, in a predictive health monitoring system. Categories of monitored and acquired data includes data on comparison group size210, for instance analyzing statistics of other groups according to groups of varying granularity such as national213or individual215data, health profiles of users220, user ages230, type of exercises performed240, training goals250, and environmental data260. The comparison group size category of data210, may include, for example, classifications including global211, regional212, national213, group214, and self215, allowing data to be analyzed or corrected for comparison to varying groups of people for differing statistical analyses. The category of health profiles220may include, for example, data on user Body Mass Index (BMI)221, fitness level222, tobacco use223, alcohol use224, family history225, and medical history226, allowing for model construction to take into account health information on users. The data category of user age230, may include age groupings, for example decadal age groupings, grouping together users who are less than 20 years old231, 20-29 years old232, 30-39 years old233, 40-49 years old234, 50-59 years old235, 60-69 years old236, and 70 or more years old237, allowing prediction and model-building of health profiles and warnings where age is taken into account (for instance, users of a predictive health monitoring system who are 30-39 years old might be calculated to be at less risk of hip displacement during squats than users who are 60-69 years old, all other factors being equal). The data category relating to the type of exercise240a user or users perform may include any type of exercise, for example, running241, swimming242, cycling243, rowing244, or other types of exercise, allowing, for instance, a user's health predictions to take into account their age and the statistics of their region for individuals of similar age, but also refine the analysis to individuals and risks undertaken when a user participates in specified exercises. In this way, a user's health predictions will be different for differing exercises from another user, all other factors being equal. The data category of training goals250may include, for example, whether a user is training for strength251, endurance252, speed253, or general fitness254, which may be analyzed in the context of different intensities and practices of the exercise. For instance, an individual who is practicing weight-lifting for maximum strength gain may be assumed to be using heavier weights than an individual who is only training for endurance or general fitness, which may be taken into consideration for predicting user health risks and events ahead of time. The data category of environmental data260may include, for example, analyzing ambient and user temperature261, humidity262, altitude263, terrain264, and the rates of change of these datapoints, for instance the rate of change of the temperature as they are running outside265. With these factors being analyzed together rather than only separately, health predictions may take on a much more holistic approach and be more accurately attuned to a specific user, for heightened accuracy. For instance, a user may be recorded as being 35 years old, be an occasional smoker, desire to train for a marathon with cycling, running, and swimming, and be monitored during a swim in a warm pool. In such an instance, if their region shows individuals in similar circumstances (similar age range, health background, and exercise form and goals) have a significantly increased risk of heart attack if they undergo the exercise at a high intensity, but not if the exercise is performed at a lower intensity or in a colder pool, the system may warn them to take these precautions ahead of time, aiding to mitigate the risks of such health events. This warning may be displayed on a connected phone or computer, or with a connected audio device, to warn or inform a user.

FIG. 3is an exemplary diagram of diagram illustrating groups of related applications for the disclosed invention. One application is athletic training310, for which the system is usable and optimized for analyzing and predicting optimal exercise measurement311such as predicting and detecting the movements of a user during exercise to determine if they are exercising optimally, this being accomplished with the plurality of sensors such as EMG, gyroscopes, and accelerometers on the device, over-training prediction312such that users who train too frequently on any given exercise may be warned due to higher risk of injury and lower fitness gains from over-training the body, endurance training313such as recommending to a user optimal exercises to increase endurance based on the totality of their data with the system, strength training314such as recommending optimal exercises for strength gain for a user, long-term performance tracking and prediction315which requires longer usage of the system but may be utilized to recommend more effective, or safer, or both, training programs and activities for a user. Another application is found in preventative medicine320, including stroke prediction321, heart attack prediction322, fall prevention323, and heat exhaustion or over-exercising warnings324. These warnings are a result of the confluence of multiple factors including a user's age, nationality, fitness levels, exercise history, medical history, and more, and may be fine-tuned and made more accurate over time from both group statistical data and more refined data on a particular user. A third family of applications include medical rehabilitation330, such as pre-stroke and post-stroke analysis331, pre-heart attack and post-heart attack analysis332, and post-trauma predictive warnings333. Pre-stroke and post-stroke analysis331may take the form of monitoring a user's biometric feedback during exercise and physical activity both before and after a stroke event, and comparing them to determine patterns of behavior and user performance and health changes after a stroke event occurs. Similar methodology may be applied for analysis of a user before and after a heart attack332, while post-trauma predictive warnings333may warn a user of likely health risks and possible injuries for specific exercises or other relevant data.

FIG. 4is an exemplary diagram illustrating the progression of quality of predictions for a user over time, illustrating an initial predictive technique, a short-term improvement on predictive technique, and progression through medium-term and long-term data gathering to further improve predictive techniques for users. There is a flow of data leading from an initial data availability and predictive quality using global or national statistical data410, indicating the use of broadly available data such as national averages of hip displacement for individuals in a broad age range. This stage represents a lack of narrower regional data or personal data from a user, and represents the lowest accuracy and specificity with predictive health reporting. A second stage is short-term improvements of data availability and predictive quality, using more regional statistical data420, for instance if a small area such as a single US state or even a single city has a different rate of risks for certain health and wellness events, for instance if people in a certain area experience more bone fractures or joint displacements for some reason, or higher incidences of heart attacks. A next stage involves medium-term improvements of data availability and predictive quality, using statistical data from smaller groups with similar characteristics to each other, for example groups of people who suffer strokes between the ages of 40-59 in California430and have moderate smoking habits, with a final stage of long-term improvements of data availability and predictive quality, using an individual user's prior history, including exercise and physiological records440. As user data and global, national, regional, and local statistical and health data becomes more available, over time, the predictive quality of the system improves as shown.

FIG. 5is an exemplary diagram illustrating the overlap and confluence of various factors to achieve the greatest predictive value from overlapping data from all factors, including factors such as group size, user age, health data, and more. Comparison group size510, environmental data520, type of exercise530, training goals540, health profiles550, and age560data categories all are utilized to achieve a greatest predictive value570, rather than analyzing a user on only one analysis vector such as the type of exercise530.

FIG. 6is an exemplary diagram illustrating an exemplary structure of a system of neural networks operating on a confluence of factors related to a user's data as acquired by a system of sensors. Individual first-stage neural networks operate to analyze data based on a health-related factor such as comparison groups605, age615, health profile625, training goals635, exercise type645, and environment655. Each first-stage neural network evaluates the relevant data to find patterns of data in that category that lead to health events, utilizing descent gradient training and back propagation as shown in later drawings, and is configured to make a health prediction based on that health-related factor, for example a prediction based on the comparison group610, a prediction based on age620, a prediction based on the user's health profile630, a prediction based on training goals640, a prediction based on type of exercise650, and a prediction based on the environment in which the user is exercising660. These separate first-stage health predictions are then input into a second-stage neural network665(also called a confluence of factors neural network) that is configured to make a second health prediction670(also called a confluence of factors prediction) based on a combination of the first health predictions from at least two of the plurality of first-stage neural networks. The confluence of factors prediction670is then validated against680the aggregated results of the first-stage predictions675to ensure that the confluence neural network665is not overfitting to the data. In some embodiments, the validation680may comprise a combination or convolution of the separate neural network predictions675and the confluence factors prediction670to produce a third health prediction or to produce a confirming health prediction of either the separate neural network predictions675or the confluence factors prediction670.

FIG. 7is an exemplary diagram illustrating an exemplary series of events leading to a health event that may be predicted by training neural networks using back propagation. In this example, a series of difficult-to-correlate events leads to a heart attack. Neural networks, trained on many, many similar events using back propagation, are able to connect and identify patterns of such events and build predictive models based on such training. Here, a traumatic health event occurs780, in this case a heart attack, and the system examines the previous events leading up to the heart attack through back propagation. In this example it sees several events occur for a user before the heart attack, including a heart irregularity identification event710which can be an anomaly in the heartrate or pulse of a user, an exercise change event from running to cycling720, a group age identification event730at which point the system identified the user as part of a new group of similar users due to current trends and user events, a training goal change event, indicating a goal to train for a cycling competition740, an individual age identification event750which can occur due to an age change, or the user's age only now being input into the system, or may be the result of the system merely making note of the user's age in the back propagation even if the user's age did not change immediately prior to the heart attack, an exercise change event from cycling to rowing760, and an environmental change event, in this example a 10 degree shift in temperature of the surrounding environment770. Through examining similar series of events for large numbers of users and large amounts of data, the neural network is able to detect repeatable patterns in seemingly unrelated events which are predictive of health outcomes. Neural networks may be configured to detect such patterns within a single health-related factor. For example, a series of changes in a user's health profile that lead to a heart attack such as weight gain, followed by increased cholesterol levels, followed by high blood sugar levels. Neural networks may be used at a first stage to detect such patterns. More difficult to detect are patterns of health events across health factors. A neural network may be configured to analyze a confluence of factors to detect seemingly unrelated events that show a repeatable pattern of prediction of certain health outcomes. The example in this diagram shows, for example, a heart irregularity710followed by a couple of age-related thresholds730and750, combined with two type of exercise changes720and760, a training goal change740, and a temperature change during exercise770. It is unlikely that these events would have been correlated with the heart attack by standard health prediction models, but neural networks excel at identifying such patterns and correlations over large sets of data.

FIG. 8is an exemplary diagram of sensors being connected to a wearable device, for the gathering of data on a user during physical activities. A wearable device810as shown may be in the form of a weight belt, a wrist strap, an arm band, a chest strap, or may be formed into a vest or other wearable configuration. The wearable device810comprises core computing device components820and one or more sensors830, which may be built-in or may be in the form of removable or exchangeable attachments. Other external sensors may be attached via wires, cables, or wireless devices, for example EEG sensors could be connected via one set of connections811and EMG sensors could be connected via a second set of connections812. These core components may comprise a network adapter821, a processor822, and a digital memory storage823, such that the sensor data may be processed and the harness may connect to an auxiliary device such as a smartphone to give the user the relevant health warnings as needed. One or more additional sensor attachments on the exercise and health prediction belt830exist, including a strain gauge or gauges831, a plethysmograph832, a gyroscope833, an oximeter834, an accelerometer835, a heart rate monitor836, and a pressure sensor or plurality of pressure sensors837. These sensors are relatively common sensing and health monitoring devices, and with the EEG811and EMG812sensors, allow the wearable device to detect the plurality of biometric and positioning data about a user required for neural networks to learn to predict health-related events during exercise.

FIG. 9is an exemplary method for predictive health monitoring using neural networks. First, a predictive health monitoring harness obtains biometric data for a user of a wearable biometric monitoring and feedback device910, before retrieving a history of biometric data and a history of health predictions for the user from a data storage device920, such a data storage device either being a memory in the harness or being stored on a peripheral device such as a smartphone that may be connected to the harness. Processing the biometric data, history of biometric data, and the history of health predictions through at least two of a plurality of first-stage neural networks is then performed, the plurality of first-stage neural networks, each configured to make a first health prediction based on a separate health-related factor930, before receiving a first health prediction from each first-stage neural network through which the biometric data was processed940. The system then processes the first health predictions received through a second-stage neural network, the second-stage neural network configured to make a second health prediction based on a combination of the first health predictions from at least two of the plurality of first-stage neural networks950, receiving the second health prediction from the second-stage neural network960and then displaying the second health prediction to the user of the wearable biometric monitoring and feedback device970, through either a connected display, smartphone, or other device that can connect over a network to the monitoring harness.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented on hardware or a combination of software and hardware. For example, they may be implemented in an operating system kernel, in a separate user process, in a library package bound into network applications, on a specially constructed machine, on an application-specific integrated circuit (“ASIC”), or on a network interface card.

In addition, in some aspects, servers32may call external services37when needed to obtain additional information, or to refer to additional data concerning a particular call. Communications with external services37may take place, for example, via one or more networks31. In various aspects, external services37may comprise web-enabled services or functionality related to or installed on the hardware device itself. For example, in one aspect where client applications24are implemented on a smartphone or other electronic device, client applications24may obtain information stored in a server system32in the cloud or on an external service37deployed on one or more of a particular enterprise's or user's premises. In addition to local storage on servers32, remote storage38may be accessible through the network(s)31.