Patent Description:
Millions of people suffer from degenerative motor control (e.g., Parkinson's disease, multiple sclerosis, etc.), which inhibit one's ability to perform essential tasks such as eating, getting dressed, writing, etc. Many suffer from tremors in the upper extremities. Loss of strength, flexibility, and control occurs in upper and lower extremities as well. Shaking, tremors, and loss of muscle strength may be a result of various neurological disorders. The underlying causes of essential tremors are unclear and no effective treatment is available. As the population ages, these statistics are expected to worsen.

Previous research and solutions may include: various medications; deep brain stimulation (requiring surgery); wearable vibration device; spoons with stabilization; weighted gloves; or EMG signal filtering. Each of these techniques has one or more deficiencies. Medications may only partially address the problems, and in many cases, lack effectiveness. A person's body may build up a resistance to medications and dosages must be continuously increased. Some medications may carry significant adversarial side effects, e.g., physiological (e.g., kidney), behavioral (e.g., aggression), etc. Surgery to implant deep brain stimulation is extremely costly, and requires invasive surgery fraught with a high rate of fatal risks and severe side effects. Wearable vibration devices, such as in the Emma Project by Microsoft Corporation, have not yet established efficacy. Further, Emma has no assist mode (e.g., to give strength to the user to augment atrophied muscles), and has no exercise mode. A special spoon or fork having self-stabilization may mitigate a solution that works only for eating utensils. Weighted gloves are passive, and simply add weights in the gloves. This solution does not mitigate the tremors fully, and has a low efficacy for severe tremors and loss of strength. The gloves do not provide analysis, patient monitoring, or exercise capabilities. While early research on filtering EMG signals is being performed to find a way to suppress tremors, its efficacy has not been established. EMG filtering may also cause problems by interfering with the intended motions in attempt to filter tremors. EMG filtering does not monitor user condition, nor supplement for the partial loss of strength.

<CIT> relates to an exoskeleton device that allows patients to move a part of their body that they are unable to move by muscle power due to a paralysis. The device is configured to detect motion of another body part that the patient is able to move, infers what movement he wants to achieve with the impaired body part, and effects the movement using actuators.

<CIT> and <CIT> relate to devices for stimulating muscles in response to measuring an attempted motion and predicting the full motion.

<CIT> relates to a system for providing treatment recommendations for movement disorders. The system comprises sensors configured to measure a patient's motion and a processor to automatically determine treatment recommendations for a physician, based on the sensor data. Furthermore, a drug delivery system is disclosed that allows releasing a drug into a patient's body.

<CIT> relates to a device for suppression of muscle tremors. It comprises sensors to detect the tremors and piezoelectric actuators to generate a motion that counteracts the tremor.

In the following description, for purposes of explanation, various details are set forth in order to provide a thorough understanding of some example embodiments. It will be apparent, however, to one skilled in the art that the present subject matter may be practiced without these specific details, or with slight alterations.

An embodiment of the present subject matter is a system and method relating to improving functionality of those suffering from degenerating motor control, and extend the capacity of their independent living. Additionally, the present subject matter includes a physical therapy modality that will help to reduce muscle deterioration, in an embodiment.

The embodiments described herein provide a number of benefits to users at several stages of degenerative disease progressions, such as inferring a user's intended motion to more accurately perform that motion, mitigate tremors, appropriately supplement loss of muscle strength, or provide therapy exercises for users. Further, a user's progression may be monitored for self-customization or for reporting to medical professionals, pharmaceutical institutions, or research institutions among others. It should be noted that embodiments are not limited to human applications, but may also be implemented for animals such as dogs, horses, cats, etc..

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present subject matter. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" appearing in various places throughout the specification are not necessarily all referring to the same embodiment, or to different or mutually exclusive embodiments. Features of various embodiments may be combined in other embodiments.

For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the present subject matter. However, it will be apparent to one of ordinary skill in the art that embodiments of the subject matter described may be practiced without the specific details presented herein, or in various combinations, as described herein. Furthermore, well-known features may be omitted or simplified in order not to obscure the described embodiments. Various examples may be given throughout this description. These are merely descriptions of specific embodiments. The scope or meaning of the claims is not limited to the examples given.

Embodiments described herein differ from such technologies as virtual arms, low pass filters, or simple exoskeletons. An embodiment may include elements such as contextual awareness, intention inference, awareness of the user's physical state, selective amplification (e.g., corresponding to the amount of human strength reduction of the select muscles that would need to be involved for a given task), or continuous monitoring of a user's strength or flexibility.

An exoskeleton solution in and of itself (assuming this solution exists) might promote muscle atrophy. An embodiment helps to delay the expected muscle deterioration by limiting assistance to that needed to complete a task given a current state of muscle (e.g., the degree to which the muscle is weakened), and by promoting exercise for weakened muscles. The supplemental force is targeted (e.g., it corresponds to the weakened muscles) and modulated (e.g., the amount of force only replaces, or does less than replace, the muscle strength that has already been lost). Although the descriptions below often focus on solutions for hands, it will be understood that the techniques and methods described herein may be used for any extremity for which tremor suppression or strength augmentation is beneficial. For example, an embodiment may be integrated with leg braces to increase stability, enhance "up and go" movements, or improve posture. In another example, an embodiment may provide balance exercises and assess the state of postural dominant Parkinson's disease. In a further example, an embodiment may include a cervical or neck device to keep one's head erect if neck muscles are weak, etc..

<FIG> is an illustration <NUM> of problems <NUM> caused by neuro-muscular ailments in the hands or arms of a patient, along with strength level solutions <NUM>-e.g., modifying a user's strength-to these problems that are applicable to several applications <NUM>, according to an embodiment. When an individual experiences tremors in the hands, everyday tasks become more difficult. For instance, holding a coffee cup <NUM> without spilling the scalding coffee may be virtually impossible when one has severe tremors. Similarly, buttoning a shirt <NUM>, performing a handshake <NUM>, or typing on a keyboard <NUM> may become onerous tasks.

In an embodiment, a solution <NUM> to hand tremors includes a glove <NUM>. The glove <NUM> may include connectors 123A-E, such as cables, ribbon cables, hydraulic lines, pneumatic lines, shape changing materials (e.g., shape memory alloys, smart polymers, etc.), piezo electric lines, etc., to control individual finger joints from actuators <NUM> (e.g., motors, pumps, etc.). For instance, the glove thumb may include paired connectors 123A and 123B and corresponding actuators <NUM> for the two thumb joints 125A-B. The actuators <NUM> control the connectors 123A-E to moderate how much pressure or force is to be applied to a given joint or in which direction vector. In an example, the actuators <NUM> may initiate or stimulate muscle contraction or relaxation. The glove <NUM> may be a portion of a wearable arm augmentation equipped with sensors, actuators <NUM>, and intelligence to determine intended motion, and to modulate the arm, hand, or other extremity to correctly implement the user's intended motion. For example, this may include applying different griping pressure depending on an object being grasped (e.g., more griping pressure is applied when griping a bowling ball than when griping an orange).

Applications <NUM> of the augmentation device may include a variety of tasks. For instance, a user may be able to pick up a glass <NUM> without dropping it, or applying so much pressure that it breaks. Dexterity may be improved, such as to allow the user to button a shirt <NUM>. The user may able to pick up a slender object such as a smartphone 135A and then use the smartphone with ease 135B. Various form factors of the glove <NUM> may be implemented, such as a glove with no fingers <NUM>, or a form-fitting glove that extends from the fingertips up to the elbow <NUM>. Different form fits may be useful depending on the user's disability, e.g., tremors vs. loss of strength, etc..

As described above, the glove <NUM>, or other extremity support (e.g., skeletal muscle augmentation device), may be used to help users achieve intended motions, such as picking up a cup, suppressing tremors, etc. Further, a compute node may be used to interpret sensor information to infer the user's intended motions (e.g., hold a cup of water, button a shirt, perform a handshake, etc.).

The compute node may maintain awareness of the user's historical and current muscular strength and flexibility. Actions that correspond to where the user's strength has decayed may be selectively amplified by sending signals to the actuators controlling the device. The system, controlled by the compute node, may also provide an exercise mode for the user to a) perform effective exercise routines, or b) register changes in the user's strength, flexibility, or range of motion. In inferring user intended motion, the compute node may maintain situational and historical context awareness.

In addition to assisting the user with daily tasks and promoting focused exercise to avoid further muscular decay, historical information collected by the device (e.g., in the compute node) may be valuable in ongoing patient care. For instance, feedback data may be of great value to medical professionals (e.g., for disease progression, drug titration, etc.); caregivers (e.g., for creating or monitoring therapy regimens); or the pharmaceutical industry (e.g., for trials, research, etc.). The feedback data may be processed locally (e.g., by the device) or remotely (e.g., raw data from the device is delivered to a cloud component) to produce, for example, feature extraction, historical data, or longitudinal analysis, which may be consumed by a variety of sources (e.g., medical professionals, researchers, etc.).

<FIG> illustrates various scenarios where an assistive device may be used, according to an embodiment. It should be noted that for illustrative purposes, assistance for a hand or hand-arm combination is described. However, an assistive device may be used for various extremities, core body, or cervical support. Assistance may be provided in a home, office, or public environment, based on available training and sensor information. An environment <NUM> (e.g., a home environment) is described with various objects with which the user may interact. On the center of the illustration are objects that a user may see in the environment <NUM>. In this scenario a coffee cup <NUM> rests on a table <NUM>. In the coffee cup <NUM> may be seen a teabag <NUM> and a spoon <NUM>. An assistance system may infer or predict the user's intentions toward the objects in the environment <NUM> based on factors as described more fully below.

In an embodiment, the environment <NUM> may include camera sensors <NUM>, which may assist in context development <NUM> for the scene. Action analysis and setting/scene determination <NUM> may depend on the context. For instance, the visual aspects of the environment <NUM> as captured by camera sensors <NUM>, and context development <NUM> based on objects or situation, may assist in the analysis of the scene. In an example, the cup <NUM> (object) may indicate a setting for drinking a beverage, and a likely action is to pick up the cup <NUM>.

In an embodiment, the environment <NUM> may include audio sensors <NUM>. Audio sensors <NUM> may detect auditory cues-such as environment sounds (e.g., breaking glass), verbal cues or overrides <NUM> (e.g., an exclamation to "stop!," "whoa," etc.), or non-verbal utterances (e.g., a gasp, a moan, a yelp, etc.)-as discussed below. Intended motion inference <NUM> may use the audio cues for self-calibration (e.g., tuning) <NUM>. For example, the user may perceive that the assistive device is attempting to help move the user's hand <NUM> toward a spoon <NUM>, but the user's actual intention is to grab the cup <NUM>. In this case, for example, the user may override the device by speaking "cup, not spoon," or "I want the cup. " Overrides may be identified-e.g., via training based on the contextual aspects of the scene-by a self-calibration and tuning component.

Location <NUM> and environmental context <NUM> may be used to assist with action weighting <NUM>. For instance, it may be more likely that a user plans to drink the coffee (e.g., grasp the cup <NUM> and bring it to the lips) rather than perform an action related to opening a door. If the user is in a kitchen or dining room, some actions with the coffee cup <NUM> may be more likely than if at a retail store or in the bathroom. For instance, it may be more likely that the user intends to put an empty coffee cup <NUM> in the dishwasher when in the kitchen than if at a restaurant. In an example, a list of possible actions may be defined, such as hold cup/spoon/teabag, turn doorknob, or button clothes. A weight may be assigned to each action based on the perceived situational and operational contexts. Weights may be initially assigned manually, or may be assigned based on training criteria, and machine learning. For some contexts, the weight assigned to an action may be <NUM> or <NUM> (e.g., for a scale of percentages <NUM> to <NUM>). Thus, different levels of strength may be applied to grabbing objects; e.g., a stronger grip is used on a ceramic coffee mug and a lighter grip for a foam cup.

Device memory <NUM> with information on previous actions may be used with historical context <NUM> by a next action inference engine <NUM> to predict next actions from previous actions related to similar contexts, as discussed above. For instance, if the previous action was "stirred cup with spoon," then it may be inferred that the next action is "drink from cup. " It will be understood that when the assistance system is in a passive and reactive mode, predictive inference may be skipped in favor of simple tremor mitigation.

The environment <NUM> (e.g., employing an assistance system) may include eye and body trackers <NUM>. The system may include a gaze tracking device <NUM>. An object of interest may be identified <NUM> by user focus as determined by the eye/body tracker <NUM> or gaze tracking device <NUM>. For instance, when the user's gaze is upon the cup/spoon/teabag (<NUM>/<NUM>/<NUM>) object combination, the inference engine <NUM> may assign lower weights to actions involving the table <NUM>, home sign <NUM>, or clock <NUM>.

The environment <NUM> may also include proximity sensors <NUM>, motion detectors <NUM>, or pressure sensors <NUM>. A motion directional analysis engine <NUM> may use information from sensor(s) <NUM>, <NUM>, or <NUM> to identify user motion or object motion relative to each other. Relative X,Y,Z coordinate location or movement <NUM> may be used with spatial interpretation <NUM> to identify an object by proximity <NUM>. For instance, downward motion may be detected as the user moves their hand <NUM> down toward the cup <NUM>. When processing motion in space or relations between objects in a three-dimensional space, absolute position may be used (e.g., X, Y, Z coordinates) or relational positioning (e.g., momentum, distance between objects, or orientation such as pitch, yaw, roll, among others) to account for the six degrees of freedom generally available in three-dimensional settings.

It will be understood that not all embodiments will include all sensors as described in environment <NUM>. For instance, when a user is outside of the home, sensors that are not communicatively coupled to the assistive device (e.g., smart glove <NUM>) or on-the-go (OTG) (e.g., smartphone, tablet, wearable device, HMD, etc.) may not provide input to the assistance system. In this case, predictive or real-time inferencing analysis may be limited to available sensor information. In environment <NUM>, for example, user movement toward the cup <NUM> may predict an action of picking up the cup <NUM>, based on previous actions with these objects in the kitchen environment. The user may intend to grab the spoon <NUM> instead. For active and predictive help, for instance, the user may need to utter an override if the strength assistance help is too strong to change direction of movement unassisted. In the case of passive and reactive help to mitigate tremors, the user may be able to easily reach for the cup <NUM> without an explicit override. In an example of an environment with a gaze tracker, the gaze of the user toward the teabag <NUM> may override the historical context weighting, and correctly predict that the user intends to grab the teabag <NUM> and not the handle of the coffee cup <NUM>. In an example, when the user is in a home or known environment fit with sensors that may be integrated into the assistance system, various cameras and microphones may be fixed in the environment, as well as coupled to mobile devices worn or held by the user (e.g., wearables, HMD, smartphone, assistive device, etc.). When the user is in an environment with limited or no integrated sensors, then operation of the assistive device may be limited, with some analytics and inference engines omitted or reduced. It will be understood by those of skill in the art, upon review of the present disclosure, that specific functions may be optional or omitted depending on the available sensor and historical data.

<FIG> is a diagram illustrating a high-level method for a motion modulation apparatus <NUM> in an assistance system, according to an embodiment. Motion modulation apparatus <NUM> includes a smart glove <NUM> (e.g., assistive device) having several actuators <NUM>. Actuators <NUM> are indicated by a small circle on smart glove <NUM>. It should be noted that not all actuators <NUM> are identified so as not to complicate the figure. The actuators <NUM> may correspond to pressure or nerve points in a user's extremity (e.g., appendage, hand, etc.). When signals are sent to the actuators <NUM>, current may be applied to the user's extremity to promote or suppress movement in muscles or joints. The smart glove <NUM> may also include strong and flexible webbing to assist the user with additional strength. The webbing may accept signals to tighten, loosen, or stiffen into a desired shape or juxtaposition, based on the intended action.

In an embodiment, a consolidated sensory network <NUM> may provide an intended motion inferencer and motion modulation engine <NUM> with data from a variety of sensors on the assistive device and in the environment. Hand motion tracking <NUM> using sensors on the assistive device, and environmental modeling using data from location <NUM>, speech <NUM>, and vision <NUM> sensors, may be used to provide the intended motion inferencer and motion modulation engine <NUM> with environmental information from the sensors.

In an embodiment, the intended motion inferencer and motion modulation engine <NUM> may detect motion, and identify objects <NUM> and the user's intended actions <NUM> with respect to the objects. As discussed below, various trained machine learning models maybe used to make inferences about the user's intentions based on historical, situational, and operational contexts in a decision analysis and reporting engine <NUM>. The decision analysis and reporting engine <NUM> may assist real-time control <NUM> for the motion modulation engine <NUM> by sending actuation commands <NUM>. In an embodiment, the decision analysis and reporting engine <NUM> may use action modeling <NUM> to provide the actuation commands <NUM>. The decision analysis and reporting engine <NUM> may perform report generation <NUM> for drug titration, fine-tuning, early alerts, etc. The output of the report generation <NUM> may be used by a variety of people or institutions to better care for the user. For example, medical professionals may monitor the user's condition, care givers or relatives may be alerted to an incident (e.g., dropped cup), device maintainers may be alerted to device malfunctions, etc..

<FIG> illustrates a dynamic continuum <NUM> of the assistive device system, according to an embodiment. The glove may be passive <NUM> or active <NUM>, or a combination of both. In a completely passive mode <NUM>, the device may be considered to be disengaged <NUM>. When disengaged <NUM>, the device neither assists nor obstructs the user movements. This mode may be used when the user does not need assistance with a specific task, or this mode made be used during an emergency or system malfunction, for example initiated via explicit user command or upon a self-diagnostic indicating a failure.

The device may be in reactive mode <NUM>, where the device is reactive, e.g., in real-time, to user intended motion. The assistive device system may operate corresponding to the user's natural movements while suppressing involuntary tremors. This mode may be used when the user has natural strength but is affected by tremors. Another reactive mode may provide select amplification corresponding to muscles with diminished strength. The amplification level corresponds to the level of strength loss, which may be saved in a data store accessible by the compute node. This mode is applicable when the user has partial strength remaining.

The device may be in a predictive mode <NUM>. In a predictive mode <NUM> (e.g., autonomous mode), the assistive device may initiate motion predictively (e.g., before the user signals a motion via their muscles) based on situational context, or when the user begins a motion in a situation. This mode is applicable when the user has substantial loss of strength. For instance, if beginning a motion is difficult for a user, e.g., lifting the hand or arm away from the body, the assistive device may predict the action by identifying nearby objects, identifying a time-of-day likely situation, etc. In an example, a user may typically have breakfast at 8AM. At <NUM>:15AM the user faces a coffee cup. Based on the context, the assistive device may trigger muscle reactions to reach for and grab the cup of coffee. The device may be trained to recognize various common scenarios or learn from repeating tasks.

<FIG> is a block diagram <NUM> illustrating various components of the assistive device system, according to an embodiment. For illustrative purposes, components that may be used in existing systems are shown with dotted line borders. Components that are specific to embodiments of the assistive device as described herein are shown with dashed line borders. Integration of these components provides an improved assistive device for users with neuro-muscular problems. The illustration is divided into five functional sections, layers, or modes: sense layer <NUM>; determine what is happening layer <NUM>; judgement (or user intention) layer <NUM>; controlled motion layer <NUM>; and update layer <NUM>.

In an embodiment, the assistive device system comprises a force applicator, such as an exoskeleton-like long glove, that may be implemented with a variety of technologies-such as an electro-mechanical device using electric pumps, motors, or valves to control hydraulic or pneumatic lines, cables or ribbons, etc., or smart polymers, biologic compounds, etc. to apply forces to augment user muscle forces. The force applicator may include multiple sensors (not shown, e.g., motion, pressure, etc.) and actuators (e.g., actuators <NUM> illustrated in <FIG>). In one embodiment, the actuators <NUM> actively suppress motion that is determined to be an unintended tremor. In another embodiment, the glove <NUM> remains rigid (e.g., instead of actively counteracting tremors-giving the user a feeling that their hand is fixed in the glove <NUM>. This type of device may be comforting for people with hand tremors. Actuators move the mechanical elements of the glove (e.g., ribbons <NUM> and joints <NUM>), corresponding to the user's intended muscular movements. Actuators motion corresponds to the user's muscle strength exerted (i.e., no amplification) or actuators may selectively "amplify" motion corresponding to the amount of strength lost in particular muscles.

Referring again to <FIG>, in an embodiment, sensor data may be cumulatively aggregated to determine the user's real-time motion in space (e.g., motion detector) and from the environment for the purposes of contextual awareness. Sensor data may be aggregated from all relevant and available sensor devices. Sensors may include an array of sensors in a smart glove device <NUM> (e.g., accelerometers, gyroscopes, thermal sensor, pressure sensors, other physiological and movement sensors); sensors in an OTG device <NUM> or other wearable devices, smartphones, or mobile devices on the user; or sensors in external devices <NUM> such as cameras, proximity sensors, etc. Sensors on the OTG device <NUM> may include physical and physiological sensors (e.g., global positioning system (GPS), accelerometer, gyroscope, proximity sensors, microphone, head mounted device (HMD) camera, heart rate monitor, other physiological sensors, etc.). Sensors from external devices <NUM> such as cameras, proximity detectors, microphones, etc., may be used to provide situational contexts.

In an embodiment, sensor information from sensors in device <NUM>, OTG devices on the user <NUM>, and sensors in external devices <NUM> are provided to layer <NUM> to determine what is happening with respect to the user or what is happening in the environment (e.g., the user's situational context). A motion detector <NUM> may identify the user's movement, and movement of objects in proximity to the user. An object and gesture recognition component <NUM> identifies user gestures and objects in the user's proximity. Object recognition may assist in situation context, for instance, in the assistance with picking up and holding a coffee cup with liquid. Object recognition may identify that the user is approaching a staircase and may need assistance grasping the railing, or in the case of a leg assistive device, assistance stepping up or down. A natural language processor (NLP) <NUM> may identify speech from an audio sensor (e.g., microphone). In an embodiment, the user may provide audible (e.g., verbal or non-verbal) commands, or feedback. An embodiment may identify audible sounds other than speech. For instance, a doorbell may be identified and infer that the user is about to get up to answer the door. Or in another example, a whistling tea kettle may cause an inference that the user is about to go into the kitchen to turn off the stove. A presence location component <NUM> may identify where the user is. For instance, when the user is home, different situational contexts may be relevant as compared to when the user is at work, or shopping. In another example, a proximity sensor may sense that the user is approaching a door that has a particular kind of locking mechanism. In this example, the approach and locking mechanism are aspects of the situational context.

Once the various movement, recognition, language, and location contexts are aggregated and identified in layer <NUM>, the information may be provided to a judgement layer <NUM>. The judgment layer <NUM> identifies user intention based on movement, objects in the environment, gesture recognition, auditory cues (e.g., verbal or non-verbal utterances, environmental sounds, etc.), or location. In an embodiment, an intended motion inferencer (IMI) <NUM> may use the information from layer <NUM> to determine what motion the user intended, which may then be used, for example by a controlled motion layer <NUM>, to provide control or motion modulation information to effectuate the intended motion. In an embodiment, the IMI <NUM> determines, in real-time, what elements of the user's motion is intended, rather than motion being caused by unintended tremors. The IMI <NUM> makes determinations by correlating motion with the situational context-for example, by analyzing a motion profile-as well as optionally taking cues from the user, such as with speech and eye gaze, and the environment, for instance for sounds. In an embodiment, in the extreme case of muscle strength loss, IMI <NUM> may infer the intended motion without relying on the user's muscular motion (as discussed below with reference to <FIG>).

In an embodiment, the IMI <NUM> includes a situational context memory (SCM) component <NUM>, a situational context identifier (SCI) <NUM>, functional object profiles (FOP) <NUM>, and a learning/inference engine <NUM>. The IMI <NUM> may receive input as an explicit user interaction override component <NUM>, for instance, in the event that the user intends to do an unpredictable action, or to correct the IMI <NUM> when it did not accurately infer the user's intended motion. The IMI <NUM> may also receive self-calibration and tuning (SCT) information <NUM> to improve on the inferences made for intended motion.

In an embodiment, SCI <NUM> determines what is happening with the user and the environment based on the real-time sensor data as well as cues from the prior situational contexts (e.g., retrieved from the SCM <NUM>). SCI <NUM> provides identification of the situation correlated with historical patterns, location, time and situational context, as well as real-time sensor data. A high-probability outcome is calculated through correlation, inference and standard deviation off of norm. SCI <NUM> may receive input elements including: environmental context, situational context, and real-time sensor data. The environmental context provides information about the prior location and time indexes, in order to identify any daily patterns and associated locations and time-based events. The situational context associates the prior events, such as correlating information between the environment, location, and the associated things that occur at that location. Real-time sensor data provides additional accuracy and detail for context. SCI <NUM> uses these inputs to determine high-probability context and ensures improvements towards an increasingly accurate model.

In an embodiment, the SCM <NUM> provides historical context storage. The historical context is the history of events that occur within the environment and situation, and is used to refine the likelihood of what is going to occur given a present context. Thus, the user may historically eat breakfast every day at 8AM. The IMI <NUM> may integrate the historical context (e.g., what has happened) from the SCM <NUM> with the present context from the SCI <NUM> to more accurately infer user intent. For example, using the historical data in the SCM <NUM> corresponding to breakfast activities, and a user's movement in the hallway from the SCI <NUM>, the IMI <NUM> may infer that the user is heading toward the kitchen to begin making coffee. In another example, The IMI <NUM> may infer that the user is about to get dressed when opening the clothes closet or a dresser drawer at <NUM>:30AM. However, if it is 3PM, opening a dresser drawer may indicate that the user is about to put away clean laundry. Various possible situations may be assigned a probability based on time of day, location, movement, etc. The SCI <NUM> and the SCM <NUM> may provide high-probability situational context, which, in conjunction with motion data, is a core input to the IMI <NUM>.

In an embodiment, the SCM <NUM> assists the SCI <NUM> by defining relations among motions that have occurred to provide a continuum of motion inference. In an example, a user hand reaches in the direction of a mug with a pen lying next to it. Knowing that a short time ago that the user placed a piece of paper on the table helps to infer that the user is likely reaching for the pen. Knowing that it is breakfast time and that the user recently held a fork helps to infer that the user is likely reaching for the mug. The SCM <NUM> may assist in providing identification of the situation and events through the correlation of motion (e.g., direction, vector, rate of approach, etc.) and the refinement of the objects in order to preemptively determine the likely object to be manipulated within context. The SCM <NUM> data may be used in a variety of ways. For example, a motion inference engine, such as in the IMI <NUM>, may evaluate the object and sequence correlation in order to differentiate the response based on the object properties and sequence of events for manipulation. In another example, an approach sequence predictor may provide the specificity and procedure to narrow down the selection of which object is about to be manipulated based on the object that is most likely implicated in the upcoming action.

A FOP <NUM> may include a list of specific objects with specific corresponding attributes about how those objects may be controlled (e.g., operated). When available, the assistance system performs a lookup in the list based on such information as the user's location. When available, this profile information assists IMI <NUM> in determining the user's intended motions. For instance, the assistance system may detect the user location on the second floor at the end of the corridor. The look up in the profile list provides information that this is a door to the bathroom with the opener in a shape of a knob that needs to be turned clockwise one quarter turn to open the door. The FOP <NUM> provides identification of operational context for the objects in the list based on their function and operational modality. This may include standard objects (e.g., available to the general public), modified objects such as assisted (e.g., where the force, grasp, surface is modified to enhance or assist with targeting), and custom systems which are modified to specifically enhance a single user's limitations. FOP <NUM> may include object operation elements for standard modality, assisted modality, automatic modality, or custom modality. Here, an automatic modality includes those in which the object operates automatically, relieving the user from exerting a force to operate the object. Examples may include an automatic door opener that may be controlled through a building (e.g., home) automation network. Thus, a detected user proximity or verbal command may open or close the door.

The standard modality element or component may engage with objects for operation and predictive/proactive actions based on how everyday objects are in the situation without any enhancement. The assisted modality element may engage with objects for operation and predictive/proactive actions based on some assistive enhancements to everyday objects that allow simpler manipulation, for instance, enhancements related to the Americans with Disabilities Act (ADA) assistance. The custom modality element may engage with objects for operation and predictive/proactive actions based on custom enhancements that are specific to the user, such as home improvements and automobile/driving enhancements. In an example, a communication component (e.g., a transmitter, transceiver, etc.) may send a signal or command to an object requesting the object to assist the user in the pre-defined, custom manner. For instance, in an example, the request may be to open or close an automatic door. The FOP <NUM> may provide the high-probability operational context, which is a third core input to the IMI <NUM>.

The learning or inference engine <NUM> may include a feedback to continuously improve the decision making abilities of the assistance system. For instance, when a user verbally interferes with the predicted intent (e.g., overrides the predicted intent with an actual intent), such as when the IMI <NUM> misjudges the user's intention, a self-learning adjustment may be triggered. An inference engine within <NUM>, as discussed more below in reference to <FIG>, may comprise high-probability context elements that provide improved accuracy through learning, analytics, and a feedback loop.

In an embodiment, many of the actions shown in <FIG> may be processed on the assistive device so that the user does not need to rely on access to a cloud or Internet connection. Some processes may be performed on a local edge cloud, for instance a local server in the user's home or office, or even on a smartphone or other mobile device. To provide more independence for the user, objects, situations, and actions that are common to the user may have local profiles stored in memory on the assistive device. Training of the assistance system, or user monitoring and reporting, may require access to the Internet or the cloud, however, training, monitoring, or reporting may also occur locally. In an example, training may require use of computationally complex and intensive training models which may be better performed offline on a compute node with sufficient power. Access to the cloud or Internet server may be required to receive doctor inputs, profiles for new objects, etc. Self-calibration inputs such as overrides may be applied immediately, or be saved for a later time when the assistive device is connected to a computationally powerful device to retrain the models. It will be understood that varying combinations of processes or components may reside on the assistive device as compared to the local or remote cloud server, as necessary to provide real-time assistance to the user.

<FIG> is a block diagram illustrating an alternate inference engine of an IMI, according to an embodiment. In an embodiment, an inference engine <NUM> receives the core outputs from SCM <NUM>, SCI <NUM>, and FOP <NUM>, in the form of probabilities, including: high-probability motion/approach context <NUM>, high-probability situational context <NUM>, and high-probability operational context <NUM>, respectively. Probabilities are used because the user's motion is not deterministic. User motion is analog, not digital, and has virtually infinite possibilities. Therefore, possible situational context, operational context, and motion/approach context may be assigned a probability before correlation. These high-probability context predictions are used by learning analytics feedback loop inference engine <NUM> to receive continuous contextual feedback and train the system by correlating operational context, situational context, and motion/approach context. It will be understood that correlations may use any one of these context types as primary-e.g., as a seed, beginning, or starting point, etc.-in the training. For instance, training may focus on the situation first (e.g., the situational context is the seed) and correlate the situation with historical operations (e.g., operational context) and motion (e.g., motion or approach context). Alternatively, the training may focus on historical motion correlated with situation and movement. In an embodiment, the various contexts may be trained as peer variables. It will be understood that various machine learning models may be used to correlate the three types of context as variable input.

Once trained, machine learning models in the learning analytics and feedback loop inference engine <NUM> may provide one or more high-probability inferences for situation, movement/motion, and operation to an IMI <NUM>. It will be understood that even though the inference engine <NUM> is illustrated as a separate block or component in <FIG>, the inference engine <NUM> may be implemented as a sub-component of IMI <NUM>, or that various components of inference engine <NUM> and IMI <NUM> may be implemented as a single process or as distributed process, or a combination. The IMI <NUM> uses the high-probability inferences (e.g., for situation, movement/motion, and operation) and provides motion information to the motion control system <NUM> (e.g., layer <NUM>, or MME <NUM> illustrated in <FIG>).

Referring again to <FIG>, the IMI <NUM> may provide high-probability information as to what the user's intent (e.g., intended motion) is to the motion modulation engine (MME) <NUM>, while the MME <NUM> uses this information to perform the motion (e.g., augment the user's own muscle inputs to achieve the intended motion). In an embodiment, the MME <NUM> activates a force applicator of the assistive device (e.g., electro-mechanical apparatus, smart glove, etc.) to implement the intended motion. The MME <NUM> receives information about the user motion, for instance from the motion detector <NUM>, and the intended motion from IMI <NUM>. Based on the operating mode or other parameters, such as user strength and limitations, or predictive vs. reactive user assist modes. The MME <NUM> determines which motions to activate, via the actuators (e.g., actuators <NUM>), or through other signaling-such as a radio signal to operate an automatic door or light switch. The MME <NUM> may determine which actions will utilize the user's innate or current strength, and which actions require amplification or assistance in strength, and for which actuators, to complete the intended motion. Each actuator may provide muscle assistance or tremor suppression to a single muscle or group of muscles. Therefore, if the user has a loss of strength in only some muscles, or varying levels of strength loss, actuators may be activated at different levels of intensity. If the user has tremors in specific muscles, then the actuators corresponding to the muscles prone to tremor may be activated to suppress the tremors. In another embodiment, there is no activation of actuators for intended motion of a user, and the user may complete the motion with innate strength. However, in this case, suppression may be used for unintended motions, e.g., tremors, tics, or other motion disorders.

In an embodiment, the MME <NUM> helps to implement the three previously described operating modes: system initialization, standard user operating mode, and exercise mode. The MME <NUM> may operate differently based on receiving a system operating mode flag or identifier <NUM>. The system initialization mode may be performed by a family member, nurse, therapist or other individual to mimic the user's daily routines while operating/wearing the assistive device. Mimicking of the daily routines may be performed to train the assistance system and provide a baseline for correlation of situational, movement, and operational contexts. This training may also populate the object profiles in FOP <NUM>. The standard user operating mode is the mode for user assistance. The MME <NUM> may include machine learning to learn and fine-tune its operation. Runtime learning may enhance the initial baseline when the assistive device is in operation by the user.

In an embodiment, the assistance system includes an exercise mode. This mode provides an exercise regimen for the user (e.g., stretching and strengthening). In this mode, the system may first prompt the user before engaging in a resistance training course. The MME <NUM> may locally adjust how much resistance or stretching to use depending on the strength remaining in particular muscle(s), as well as log user progress to generate reports for the user and health providers. Exercise regimens and recommendations may include doctor inputs <NUM>. Log data may also be used to calibrate the operation of the device in standard user operating mode.

MME <NUM> may also utilize additional attributes, such as user physiological parameters (UPP) <NUM>, and doctor inputs <NUM>. The UPP <NUM> may include a database of the user's current health state, e.g., strength and flexibility, corresponding to particular muscles and joints. The UPP <NUM> may be continuously updated by the system based on the processions of the exercise mode, or received from external sources, e.g., doctors, physical therapist. Doctor inputs <NUM> may provide information regarding which muscle(s) to exercise in particular, and which exercise(s) are recommended, including frequency and duration or number of repetitions. In an embodiment, the UPP <NUM> may also include specific directives to more aggressively suppress tremors (e.g., because it is comforting to the user), or conversely to not suppress tremors (e.g., because, in some conditions, it is uncomfortable to the user).

The controlled motion layer <NUM> may also include an actuators control manager (ACM) <NUM>, a home automation controller <NUM>, and a motion or movement initiator <NUM>. ACM <NUM> may drive the network of actuators in the assistive device (e.g., smart glove) to execute a particular motion based on the direction/instructions from the MME <NUM>. The motion or movement initiator <NUM> may send signals or instructions to the individual actuators to effect the movement. A home automation controller <NUM> may communicate with a home automation type of network and devices. For instance, the IMI <NUM> may determine that the user is intending to open the door to the bathroom. As an alternative to requiring the user to open the door manually, the IMI <NUM> may send a command to the home automation controller <NUM> initiating a command to open an automatic door via the home automation network, when available with a digital door opener.

In an embodiment, the assistance system may include an emergency override <NUM> to indicate that the assistance system immediately stops and removes any suppression/amplification/interference to the user motion. This override <NUM> may be based on a particular voice command (keyword, exclamation, moaning, gasping, yelping, etc.), gesture, or activation of a particular "stop" button/switch, etc. For instance, when a user feels that device is not cooperating, or is providing an incorrect assistance/suppression, the user may deactivate the assistive device (e.g., send an immediate signal to the actuators control manager <NUM> to deactivate the actuators). Deactivation may be triggered by a physical or virtual switch or button, or by voice command. In an embodiment, deactivation (e.g., emergency override) may be triggered by someone other than the user, such as a family member, caregiver, emergency response personnel, medical professional, etc..

In an embodiment, the assistance system may include a user interaction override component <NUM> to provide corrections to the IMI <NUM> when the user perceives that the device is making incorrect predictions, but an emergency shutdown is not necessary. For instance, the IMI <NUM> may infer that user is reaching for the cup. The user may speak aloud, "I am trying to pick up the spoon. " This command or movement suggestion may be identified with NLP <NUM> and used by the IMI <NUM> to correct the action. Situational, operational, and motion contexts for this correction may be input to the learning/inference engine <NUM> for additional training. It should be noted that other cues may be used to override the IMI <NUM>. For instance, a particular motion such as a sharp jerk of the hand or pulling the hand back and forth, or other gesture, may be pre-defined to indicate to IMI <NUM> that the inference decision was inaccurate.

In an embodiment, the assistance system may include a self-calibration tuner (SCT) <NUM>. The SCT <NUM> may include a continuous self-learning platform for IMI <NUM> and MME <NUM>. In a simple form, for IMI <NUM>, the SCT <NUM> may adjust the machine learning weights for a model, when a user explicitly overrides the IMI <NUM> inference. Self-calibration of MME <NUM> may be performed when the user expressed that a particular motion was either insufficient (e.g., fingers did not enclose the door knob sufficiently to turn the knob), or that the motion created discomfort in the user (e.g., due to excessive force from the actuators).

In an embodiment, the MME <NUM> may provide historical information to an update layer <NUM>. The update layer <NUM> may include a user monitoring and reporting (UMR) <NUM> component. UMR <NUM> may provide reports or analysis to outside persons or entities (e.g., via the cloud <NUM>), such as therapists, research institutions, etc. UMR <NUM> may also provide updated information on the user's strength levels and other current operation information to UPP <NUM> for use with the MME <NUM>. During assistance mode, for instance during the user standard operating mode, and especially during the exercise mode, the assistance system may continuously assesses the user's current muscle strength, flexibility, and motor control. UMR <NUM> may update the UPP <NUM> database. UMR <NUM> may also monitor a range of physical attributes (e.g., strength, tremor intensity, biometrics, and particular activities).

UMR <NUM> may be enhanced by adding objective assessment per a pre-defined scale such as the Unified Parkinson Disease Rating Scale (UPDRS). In an example, timing tests may be performed to see how many times the patient may touch their index finger to the thumb in a specified interval, or how many times the user may pronate/supinate their hands, as measures of dexterity. A subset of tests that may be easily automated by the assistive device may be run regularly (e.g., weekly or biweekly). These strength and dexterity tests may be performed much more frequently than if performed only at an annual/semi-annual physical, which is currently typical for patients. The results of these tests may be reported up to the cloud <NUM>, for use by doctors, therapists, and other concerned parties. This data may be of great value to doctors, therapists, pharmaceuticals, research institution, etc., to provide better drug titration, physical therapy (e.g., exercises), or to help other similarly situated individuals. In an embodiment, the information is processed and particular feature set(s) provided with appropriate degrees of privacy protection, or encryption.

<FIG> comprise a block diagram further illustrating various components of the alternative assistance system as that illustrated in <FIG>, according to an embodiment. <FIG> illustrates a SCI <NUM> component (e.g., shown in <FIG>), according to an embodiment. A situational context identifier <NUM> may use high-probability context (situation) input <NUM> to identify the high-probability situational context <NUM>, which may be used by the IMI (<NUM>, <FIG>). The situational context identifier <NUM> may use several input data <NUM>, <NUM> and derived inputs such as contextual identifiers <NUM>, <NUM>, and <NUM> to identify the situational context. The high-probability context (situation) <NUM> may be derived from environmental context <NUM>, prior situational context <NUM>, and historical context <NUM>. Environmental context <NUM> may use prior location and time indexes <NUM> to identify an environment (e.g., home, office, grocery store, restaurant, hallway, kitchen, bathroom, etc.). Prior events <NUM> may be used to identify prior situational context <NUM> (e.g., in kitchen may mean eating or drinking requiring use of utensils). Historical context <NUM> may use prior historical likelihood information <NUM> (e.g., in kitchen with cup object present means picking up the cup is likely). The situation contextual identifier <NUM> may use high-probability context (situation) <NUM> and real-time sensor data <NUM> to generate the high-probability situational context <NUM>. In an embodiment, a trained machine learning model may be used at one or more points in the determination/calculation for the various contextual identifiers <NUM>, <NUM>, <NUM>, or <NUM> where the model is specific to the input <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> data used to derive the contextual identifiers <NUM>, <NUM>, <NUM>, <NUM>, or <NUM>/<NUM>.

In an example, the user enters a local coffee shop known to use paper cups. The situational context may indicate a certain pressure to be placed on the cup to hold it safely. Another coffee shop may use ceramic cups which require a different pressure to hold. If the user has frequented both coffee shops, both locations and default cup profiles may be saved as prior situational context as applied to the specific objects. Therefore, when the user enters the shop that uses ceramic cups, the location probability will correlate with the object or operational probabilities to apply the appropriate pressure. It is possible that the shop will use a different kind of cup that day, or that the coffee shop is not known. In those cases, the user might utter "I am using a paper coffee cup" to alert the assistive device to choose the correct object profile, as discussed more below.

In an embodiment, the user may indicate through audible or other means that the user intends to visit the coffee shop that uses the ceramic cups. Historical context may infer a series of actions to travel to the coffee shop, order and drink the coffee. When the user is away from home, only a subset of historical actions and functional object profiles may be available in local storage of the assistive device. For instance, the user may know that network connectivity is unreliable at the coffee shop. This situational context may be stored as a profile for the location or activity in a remote cloud server or local edge cloud server. In an embodiment, various historical actions and functional object profiles or a series of expected actions may be pre-fetched from the cloud based on the intended situational and operation context (e.g., going to the coffee shop). Therefore, the assistive device may operate at a higher level of accuracy in inferring the intended motion and actions when the data is pre-fetched than if it had to rely on the subset of information typically stored locally.

Referring now to <FIG>, there is shown a block diagram illustration of the situation and events through the correlation of motion, according to an embodiment. Identification of situation and events may be derived from the correlation of motion (direction, vector, rate of approach) and the refinement of the objects in order to preemptively determine the likely object to be manipulated within context. As discussed above, SCM <NUM> (<FIG>) may include a motion inference engine <NUM> and an approach sequence predictor <NUM>. The motion inference engine <NUM> evaluates the object and sequence correlation <NUM> in order to differentiate the response based on the object properties and sequence of events for manipulation. The approach sequence predictor <NUM> provides the specificity and procedure to narrow down the selection of which object is about to be manipulated based on the object that is most likely implicated in the upcoming action. Object implication refinement <NUM> is a technique by which refinements to object selection-e.g., which object is about to be used-are made. For example, when approaching several objects, such as a pen, notepad, newspaper, and a cup on a table, it may be difficult to determine which of the several objects the user intends to interact. However, given a pattern of user behavior to grab the cup after putting down the pen, the object selection may be refined to select the cup with a greater probability than the newspaper, for example, if the pen were recently put down. The SCM <NUM> may provide input to the high-probability context (object) <NUM>. The high-probability object context <NUM> may be used to determine the situational context memory <NUM> having both motion inference and approach sequence information regarding the object (e.g., operational context). Situational context memory <NUM> may provide the high-probability motion/approach context <NUM> to the IMI <NUM> (<FIG>).

Referring now to <FIG>, there is shown a block diagram illustrating identification of the operational context for the objects <NUM> (<FIG>), according to an embodiment. The high-probability operational context <NUM> may be identified based on the object function and operational modality. Objects may include standard objects (e.g., available to the general public, like paper or ceramic coffee cups), modified objects such as assisted objects (e.g., where the force, grasp, surface is modified to enhance or assist with targeting), and custom modality objects which are modified to specifically enhance a single user's limitations. High-probability identification (operation) <NUM> may be derived from object operation (Standard) <NUM>, object operation (Assisted) <NUM>, or object operation (Custom) <NUM>. Object operations (Standard) <NUM> may use standard modality profiles <NUM> to identify a user's likely operation for the standard object. Object operations (Assisted) <NUM> may use assisted modality profiles <NUM> to identify a user's likely operation for the assisted object. Object operations (Custom) <NUM> may use custom modality profiles <NUM> to identify a user's likely operation for the custom object. For instance, the object may be identified as a coffee cup without a handle, which may assign a higher probability to paper cup or expanded polystyrene foam cup than to a ceramic cup.

FOP <NUM> operation (e.g., shown in <FIG>) provides identification of operational context for the objects based on their function and operational modality as determined for the high-probability identification (operation) <NUM>. The functional object profiles <NUM> may provide the input to the IMI <NUM> to determine the high-probability operational context <NUM> (<FIG>). The functional object profiles <NUM> generally store information related to objects and how they are operated. For example, a functional object profile for a kitchen microwave may include: where it is located (e.g., what room, how high off of the floor, over a counter, etc.); where is the handle (e.g., on the left side, the top, etc.); where is the start button, what particular signals (e.g., sounds, lights, etc.) it may emit, among other things. In another example, the profile for a door may include the type of opening mechanism the door has, including details such as where the knob is placed, how it turns (e.g., clockwise or counter-clockwise), or whether the door has an electromechanical door opening capability. As discussed above, the IMI <NUM> provides intended movement information to the motion control layer <NUM>, which in turn controls a force applicator to effectuate the intended motion.

<FIG> comprise a flow diagram illustrating a method for assisting a user in mitigating effects of neuro-muscular ailments in various operational modes, according to an embodiment. An assistance system identifies the operational mode in block <NUM>. The assistive device may operate in system initialization or setup mode <NUM>, user assist mode <NUM>, or exercise mode <NUM>.

Referring now to <FIG>, a flow diagram is illustrated for a method of system initialization <NUM>, according to an embodiment. Sensors available for input may be identified and enrolled in block <NUM>. A method for auto-discovery may be used to identify sensors in range. Alternatively, a user may manually locate and enroll sensors. Sensors may include those on the assistive device, other OTG or wearable devices, and environmental devices. A data correlation map for the enrolled sensors may be built in block <NUM>. Sensor data correlation may establish relationships between sensors-such as where are they with respect to each other, where are they with respect to the environment (e.g., location, elevation, etc.), how their data may be correlated (e.g., in time, in overlapping field of views, etc. based on which intention inference is made. An environmental profile database <NUM> may be created in block <NUM>. A situational profile database <NUM> may be created in block <NUM>, and used to capture a situational context and, for example, inform the environmental profile database <NUM>. The environmental profile and situational profile databases <NUM>, <NUM> may collect some sensor data from the environment, as well as use training data to populate the profiles. In an example, a nurse or family member may walk through the home environment. A camera, such as coupled to a head mounted display (HMD) (e.g., glasses, visor, helmet, etc.) or mounted on a wall, may capture environmental information, e.g., located in hallway, bathroom, kitchen, family room, etc. Situational context may be entered or trained using common activities of the user in those environments.

Initial physiological parameters of the user may be pre-loaded into a user physiological parameter (UPP) database <NUM>, in block <NUM>. User strength and limitation information may be assumed, received from a health care professional, or derived from testing, and entered as a baseline. It will be understood that the UPP database <NUM> may be populated before or after the environmental profile data base <NUM> and situational profile database <NUM>.

Referring now to <FIG>, a flow diagram is illustrated for a method of user assist mode <NUM>, according to an embodiment. It will be understood that user assist mode <NUM> may have various levels of passive, reactive, or active assistance across the dynamic continuum, as illustrated in <FIG>. When the assistive device is in user assist mode <NUM>, the assistance system receives sensor data for a variety of contexts or actions, including but not limited to, movement, object identification, location, environment, and user feedback, in block <NUM>. The sensor data may be aggregated to correlate information from various sensors. Historical context may also be correlated with the sensor data to derive a real-time context. As discussed above, various machine learning models may be used to assist in the correlation of data. When in predictive mode, intended motions may be inferred in block <NUM>. In reactive mode, intended and unintended motion may be inferred, as well, to inhibit the unintended motion. In an example, a user may have very little strength to provide a predictive motion from movement. In this case, motion may be predicted based on context. Motion may be modulated in block <NUM> to alleviate tremors, unintended motion, or provide assistive strength for the intended motion. Motion modulation may be reactive instead of being predictive, modulating and executing an action. The motion modulation may be based on the user's strength and limitation parameters <NUM>. Motion modulation may include sending signals <NUM> to actuators in an assistive device such as a smart glove, flexible leg brace, cervical support, etc. It will be understood that motion modulation may be active (e.g., strength assistive) or reactive (e.g., tremor mitigation), or both, as identified by the operational mode and user physiological parameters (e.g., using the UPP database <NUM>). User monitoring and reporting, as discussed above, may be performed in block <NUM>. Calibration and tuning of the assistance may be performed in block <NUM>, as discussed above.

Referring now to <FIG>, there is shown a flow diagram illustrating a method for exercise mode <NUM> of the assistive device, not forming part of the present invention. The user, or the system, may select an exercise to practice in block <NUM>. When the system selects the exercise mode <NUM>, the system may prompt the user to start or engage in the activity. In an embodiment, the exercise may be retrieved from a list of prescribed physical therapy exercises <NUM>. The exercises may be suggested to the user on a periodic basis, or require manual selection. Various automatic reminder modes may be pre-selected by the user. Once selected, or suggested, the user may be prompted to begin the exercise in block <NUM>. Sensor data <NUM> identifying the user motion may be collected and used to infer intended motion in block <NUM>. Motion may be modulated, as necessary, in block <NUM>, and signals sent to actuators to assist intended motion or to inhibit unintended motion in block <NUM>. The user is monitored by sensors in the assistive device, in block <NUM>. Strength levels may be inferred based on movement, pressure, and other sensors in the device. Changes in user strength or limitations or other physiological parameters may be updated in the UPP database <NUM>, in block <NUM>. Reports on status of exercise regime and updated parameters may be generated, in block <NUM>. The reports may be stored for later review, or forwarded to the user or health care provider as desired.

<FIG> illustrates a block diagram of an example machine <NUM> upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform. In an embodiment, the assistive device (e.g., smart glove or leg brace, etc.) communicates with the machine <NUM> (e.g., a server machine) which may be used to execute the trained models and provide the motion controls based on inferred intended movement, according to the contextual data. The machine <NUM> may be a local or remote computer, or processing node in an OTG device such as a smartphone, tablet, or wearable device. The machine <NUM> may operate as a standalone device or may be connected (e.g., networked) to other machines. In an embodiment, the machine may be directly coupled or be integrated with the assistive device. It will be understood that when the processor <NUM> is coupled directly to the assistive device, that some components of machine <NUM> may be omitted to provide a lightweight and flexible device (e.g., display device, UI navigation device, etc.).

Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms. Circuitry is a collection of circuits implemented in tangible entities that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time and underlying hardware variability. Circuitries include members that may, alone or in combination, perform specified operations when operating. In an example, hardware of the circuitry may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuitry may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the computer readable medium is communicatively coupled to the other components of the circuitry when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuitry. For example, under operation, execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry, or by a third circuit in a second circuitry at a different time.

In an example, sensors <NUM> may include wearable, assistive device-based and environmental sensors, as described above. The machine <NUM> may include an output controller <NUM>, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

While the machine readable medium <NUM> is illustrated as a single medium, the term "machine readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) configured to store the one or more instructions <NUM>.

The term "machine readable medium" may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine <NUM> and that cause the machine <NUM> to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Non-limiting machine readable medium examples may include solid-state memories, and optical and magnetic media. In an example, a massed machine readable medium comprises a machine readable medium with a plurality of particles having invariant (e.g., rest) mass. Accordingly, massed machine-readable media are not transitory propagating signals. Specific examples of massed machine readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

The instructions <NUM> may further be transmitted or received over a communications network <NUM> using a transmission medium via the network interface device <NUM> utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) <NUM> family of standards known as Wi-Fi®, IEEE <NUM> family of standards known as WiMax®), IEEE <NUM>. <NUM> family of standards, peer-to-peer (P2P) networks, among others. In an example, the network interface device <NUM> may include one or more physical jacks (e.g., Ethernet, coaxial, or phone jacks) or one or more antennas to connect to the communications network <NUM>. In an example, the network interface device <NUM> may include a plurality of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term "transmission medium" shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine <NUM>, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

The techniques described herein are not limited to any particular hardware or software configuration; they may find applicability in any computing, consumer electronics, or processing environment. The techniques may be implemented in hardware, software, firmware or a combination, resulting in logic or circuitry which supports execution or performance of embodiments described herein.

For simulations, program code may represent hardware using a hardware description language or another functional description language which essentially provides a model of how designed hardware is expected to perform. Program code may be assembly or machine language, or data that may be compiled or interpreted. Furthermore, it is common in the art to speak of software, in one form or another as taking an action or causing a result. Such expressions are merely a shorthand way of stating execution of program code by a processing system which causes a processor to perform an action or produce a result.

Each program may be implemented in a high-level procedural, declarative, or obj ect-oriented programming language to communicate with a processing system. However, programs may be implemented in assembly or machine language, if desired. In any case, the language may be compiled or interpreted.

Program instructions may be used to cause a general-purpose or special-purpose processing system that is programmed with the instructions to perform the operations described herein. Alternatively, the operations may be performed by specific hardware components that contain hardwired logic for performing the operations, or by any combination of programmed computer components and custom hardware components. The methods described herein may be provided as a computer program product, also described as a computer or machine accessible or readable medium that may include one or more machine accessible storage media having stored thereon instructions that may be used to program a processing system or other electronic device to perform the methods.

Program code, or instructions, may be stored in, for example, volatile or non-volatile memory, such as storage devices or an associated machine readable or machine accessible medium including solid-state memory, hard-drives, floppy-disks, optical storage, tapes, flash memory, memory sticks, digital video disks, digital versatile discs (DVDs), etc., as well as more exotic mediums such as machine-accessible biological state preserving storage. A machine readable medium may include any mechanism for storing, transmitting, or receiving information in a form readable by a machine, and the medium may include a tangible medium through which electrical, optical, acoustical or other form of propagated signals or carrier wave encoding the program code may pass, such as antennas, optical fibers, communications interfaces, etc. Program code may be transmitted in the form of packets, serial data, parallel data, propagated signals, etc., and may be used in a compressed or encrypted format.

Program code may be implemented in programs executing on programmable machines such as mobile or stationary computers, personal digital assistants, smart phones, mobile Internet devices, set top boxes, cellular telephones and pagers, consumer electronics devices (including DVD players, personal video recorders, personal video players, satellite receivers, stereo receivers, cable TV receivers), and other electronic devices, each including a processor, volatile or non-volatile memory readable by the processor, at least one input device or one or more output devices. Program code may be applied to the data entered using the input device to perform the described embodiments and to generate output information. The output information may be applied to one or more output devices. One of ordinary skill in the art may appreciate that embodiments of the disclosed subject matter may be practiced with various computer system configurations, including multiprocessor or multiple-core processor systems, minicomputers, mainframe computers, as well as pervasive or miniature computers or processors that may be embedded into virtually any device. Embodiments of the disclosed subject matter may also be practiced in distributed computing environments, cloud environments, peer-to-peer or networked microservices, where tasks or portions thereof may be performed by remote processing devices that are linked through a communications network.

A processor subsystem may be used to execute the instruction on the machine-readable or machine accessible media. The processor subsystem may include one or more processors, each with one or more cores. Additionally, the processor subsystem may be disposed on one or more physical devices. The processor subsystem may include one or more specialized processors, such as a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or a fixed function processor.

Although operations may be described as a sequential process, some of the operations may in fact be performed in parallel, concurrently, or in a distributed environment, and with program code stored locally or remotely for access by single or multi-processor machines. In addition, in some embodiments the order of operations may be rearranged without departing from the disclosed subject matter. Program code may be used by or in conjunction with embedded controllers.

Examples, as described herein, may include, or may operate on, circuitry, logic or a number of components, modules, or mechanisms. Modules may be hardware, software, or firmware communicatively coupled to one or more processors in order to carry out the operations described herein. It will be understood that the modules or logic may be implemented in a hardware component or device, software or firmware running on one or more processors, or a combination. The modules may be distinct and independent components integrated by sharing or passing data, or the modules may be subcomponents of a single module, or be split among several modules. The components may be processes running on, or implemented on, a single compute node or distributed among a plurality of compute nodes running in parallel, concurrently, sequentially or a combination, as described more fully in conjunction with the flow diagrams in the figures. As such, modules may be hardware modules, and as such modules may be considered tangible entities capable of performing specified operations and may be configured or arranged in a certain manner. In an example, the software may reside on a machine-readable medium. Accordingly, the term hardware module is understood to encompass a tangible entity, be that an entity that is physically constructed, specifically configured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform part or all of any operation described herein. For example, where the modules comprise a general-purpose hardware processor configured, arranged or adapted by using software; the general-purpose hardware processor may be configured as respective different modules at different times. Modules may also be software or firmware modules, which operate to perform the methodologies described herein.

In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein. " Also, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to suggest a numerical order for their objects.

Claim 1:
A computer-implemented method for mitigating neuro-muscular ailments, the method comprising:
measuring (<NUM>), using device sensors (<NUM>) of an assistive device (<NUM>), at least one of: motion, pressure, or contraction and relaxation of muscles of an extremity of a user on which the assistive device is arranged to be worn;
processing (<NUM>) sensor data to infer an intended motion of the user, the sensor data received from the device sensors of the assistive device, and environmental sensors, wherein the environmental sensors are arranged external to the assistive device or part of the assistive device, and wherein processing the sensor data includes using context information derived from the sensor data to infer the intended motion, wherein the context information include at least one of: high-probability situational context, high-probability operational context, and high-probability motion context, wherein using the derived context information is performed with a technique configured with a plurality of accuracy levels for different operational modes, wherein an accuracy level is dependent on available sensor data in a current context related to the intended motion, and available data in a historical context arranged to be used to refine the likelihood of what is going to occur given the current context; and
controlling (<NUM>), based on the inferred intended motion, actuators (<NUM>, <NUM>) and mechanical elements of the assistive device to thereby enable augmentation of the muscle strength of the extremity to assist performance of the intended motion, which includes applying electrical current to the actuators to control the mechanical elements coupled thereto to at least one of initiate, promote and suppress muscle movement of the extremity to assist performance of the intended motion.