Brevet FR3159312A1 — publié le 20250822

This is a monitoring approach that respects privacy ( no routine biometric recognition ) to continuously extract multimodal markers from individuals/patients. The system can use various modules, including cameras, microphones, connected objects, wearables, sensors, etc . . .. This allows for the early detection of ANY pathology whatsoever that can be detected in the continuous extraction of multimodal markers from individuals/patients and the rapid transmission of alerts to emergency medical services. The multimodal monitoring supported by AI algorithms will not only detect the QUALIFICATION of the onset of a pathology, but especially the QUANTIFICATION of this onset of pathology, and even more so to provide a KINETIC MEASURE OF THIS QUANTIFICATION over a significant period of time. ONLY in cases where a detected pathology requires urgent and absolute medical intervention, and if the precision levels of the methods employed have been validated by medical professionals, the approval of the protocol's performance by the medical authority can trigger the use of biometric recognition. The use of biometric recognition allows for access to the patient's medical history, which can strengthen the diagnosis to send an alarm AND/OR accelerate patient care by already having knowledge of their medical files upon arrival in the emergency room. THIS IS A CASE OF FORCE MAJEURE THAT REQUIRES URGENT TREATMENT, GOVERNED BY MEDICAL ETHICS , AND THEREFORE EXEMPT FROM ANY PRIOR AUTHORISATION. The use of biometric recognition may also be employed without prior approval from an authority ( MEDICAL OR OTHERWISE ) , if and only if the state reliably detected by multimodal monitoring falls within the framework of the " OBLIGATION TO ASSIST A PERSON IN DANGER " according to the LAW , biometric recognition can then be used without prior authorization if it provides assistance in the diagnosis. Outside of the two cases mentioned above, "FORCE MAJEURE" AND/OR "OBLIGATION TO ASSIST A PERSON IN DANGER", the "Life Tracking Medicine", (LTM) which gives citizens the ability to create their own data heritage, allows for the CONSIDERATION OF THE PERSON’S PRIOR CONSENT at the time of the device's activation , ONLY IN CASE OF SUSPICION OF THE ARRIVAL/DETECTION OF ANY PATHLOGY, and without prior approval from an authority (MEDICAL OR OTHERWISE), for the use of biometric recognition. This case also includes persons who, being minors or lacking the capacity to make decisions, are represented by a Parent or legal guardian. In this context, the initial consent of the legal guardian will be taken into account, only in case of suspicion of the arrival/detection of any pathology. LIFE TRACKING MEDICINE - EARLY DETECTION OF BEHAVIORAL PATHOLOGIES BY SMART ALGORITHM-GUIDED MONITORING WITH PRIVACY PRESERVATION Summary Many pathologies that are on the rise today, such as Parkinson's disease, Alzheimer's and other dementias, stroke, heart failure, heart attack and other coronary diseases, epilepsy, sepsis, Asthma attack, etc., could be detected early in order to take preventive action that could save lives. With the expansion of artificial intelligence, intelligent monitoring of people's behavioral tracking can detect the occurrence of disorders in gestures or language, allowing to predict a suspicion of the onset of a targeted pathology in the future. This intelligent monitoring must remain discreet and must respect the privacy of individuals. Existing artificial intelligence algorithms can be used to monitor individuals, but in order to be accurate enough, these algorithms must identify the individuals, typically through biometrics. This allows for fine-grained detection of changes in the gestures or language of the targeted individual. Each individual has their own unique gestures and facial/vocal expressions. Without integrating these personal characteristics into monitoring algorithms for early detection of a targeted pathology, this lack of information will result in a higher number of false positives. Let’s take the example of the onset of a stroke. A significant proportion of people who experience a stroke are characterized by paralysis on one side of the face, which can be detected by measuring facial asymmetry. However, it is scientifically recognized that 95% of people who are considered normal have a slight facial asymmetry, which can be more or less pronounced. Multi-neuronal classifiers in artificial intelligence, through deep learning between two populations, one without stroke and the other with stroke, will be able to predict if a person has a stroke. However, this approach has at least two flaws: We are not sure that a person who is detected as having a stroke with this method is not actually a person without a stroke who has a natural facial asymmetry. Therefore, there will be an inevitable number of false positives. In the deep learning of the multi-neuronal classifier, between the two populations, one said to be normal, and the other with stroke, it is a matter of photos or videos of people POST-Stroke , because it is impossible to have a large population of people AT THE TIME WHERE THE STROKE HAPPENS . And in early detection, facial asymmetry can evolve. The approach of identifying people by biometric recognition is relevant for personalizing the evolution of measurements over time for the targeted person , but goes against the many rules of government institutions that regulate the protection of people's privacy . Our procedure, which we claim to be innovative, has a different approach: Multimodal Monitoring, which can be done with different modules, cameras, microphones, connected objects, wearables, sensors, etc., does not identify people for comparison with their previous behavior but is based on deep learning between two populations, as indicated above, OR on mathematical equations measuring gestural or vocal expressions to obtain values to dissociate the two populations (one with the targeted pathology and the other without). In the context where, as described above, this monitoring is based on mathematical and/or logical equations (and, or, xor, etc.) and/or other QUALITATIVE/QUANTITATIVE processes, if there is no detection of anomaly over a significant period of time, the values obtained on the monitored person during this period by the equations/processes targeting a pathology will be taken into account. These customized values will allow for a more individualized monitoring of the person for a targeted pathology, in order to have a more accurate and efficient detection. In this case, there is no biometric detection to identify the individual. In the context where, as described above, multimodal monitoring supported by AI algorithms will not only detect the QUALIFICATION of the onset of a pathology, but especially the QUANTIFICATION of this onset of pathology (to provide a scale in the measurement), and even more so to provide a KINETIC MEASURE OF THIS QUANTIFICATION (evolution of the pathology measurement, in the first minutes or over a significant period of time). The three types of measures we claim apply to ANY pathology , regardless of its nature, detectable through the continuous extraction of multimodal markers from individuals/patients. In the context where, as described above, the multimodal monitoring supported by AI algorithms to detect the qualification/quantification/kinetic measurement of the onset of a pathology has precision levels validated by medical professionals, the approval of the protocol's performance by the medical authority can trigger the use of biometric recognition. The use of biometric recognition allows for access to the patient's medical history, which can strengthen the diagnosis to send an alarm AND/OR accelerate patient care by already having knowledge of their medical files upon arrival in the emergency room. The use of biometric recognition may also be employed without prior approval from an authority ( MEDICAL OR OTHERWISE ) , if and only if the methods used have already proven effective in detecting a person's agonizing state, or a prolonged fainting spell without regaining consciousness, an epileptic state unresponsive to any external stimuli, or a serious and imminent danger that threatens the person's bodily integrity or moral well-being (distress). The use of biometric recognition may also be employed in considering the person's prior consent at the time of the device's activation , ONLY IN CASE OF SUSPICION OF THE ARRIVAL/DETECTION OF ANY PATHLOGY, and without prior approval from any authority (MEDICAL OR OTHERWISE). Biometric recognition will never be used in routine Multimodal Monitoring. Biometric recognition will only be used in the event of reliable detection of the arrival of a targeted pathology according to procedures 4) 5) and 6) described above. This particular procedure allows for greater compliance with current regulations in order to protect people's privacy. Short description of the drawings Exemplar embodiments of the invention are disclosed in the description and illustrated by the drawings in which: FIG. 1 illustrates the movements of both ankle joints of a person walking normally. FIG. 2 illustrates the movements of both ankle joints on a post-stroke individual. FIG. 3 shows the landmarks of the Mouth on a Face without Facial Deviation. FIG. 4 shows an example of recalculating the positions of the landmarks in FIG. 3 to simulate an asymmetry of the mouth. FIG. 5 shows a face without characterized asymmetry making it possible to use this template in order to diagnose with personal parameters the arrival of a stroke. FIG. 6 shows a morphological transformation of the face of the FIG. 5 making it possible to simulate the arrival of a stroke, to quantify it and evaluate the kinetic evolution of the pathology. Examples of embodiments of the present invention A computerized method for estimating a likelihood of a stroke condition of a user is performed by a computerized monitoring system comprising at least one processor and at least one monitoring device including an image capture device, having a field-of-view (FOV), and an audio capture device. To illustrate an example of facial characteristics of a user, the facial characteristics comprise a plurality of landmark points LP (coordinates) corresponding to characteristics of the user's face. The feature F can be calculated from the landmark points LP in order to assess facial asymmetry of at least a portion of the user's face. In one particular example, identifying at least 486 landmark points LP and calculating at between 15 and 36 features F. For example, a linear equation can be performed on the features F. In that case, each feature F can be multiplied by a weighting factor. The weighting factor allows for each feature F to have the substantially same representation value. In particular, assuming a linear equation as follows: W1*F1 + W2*F2 + W3*F3 + … WN-1*FN-1 + WN*FN Eq.1 where Wi is the weighting factor multiplying the feature Fi, in the case of i=N features. The features F1 to FN may take different values. For example, 20 if the feature F1 corresponds to the ratio of between the ends of the mouth and the pupils, then F1 can be equal to 0.05. If the feature F2 corresponds to the ratio of the distance between the middle of the mouth and lip ends, then F2 can be equal to 0.23, etc. In this example, the feature F2 is about four times greater than the feature F1 and will dominate in equation 1. 25 Equation 1 would then be unbalance yielding a misleading result. The weighting factor Wi multiplying the feature Fi thus allows the features to have about the same weight in the linear equation. For example, if F1 is equal to 0.05 and F12 is equal to 0.23, W1 can take the value of 4.6 and W1 can take the value of 1. It should be noted that, the features F calculated with the positive stroke dataset 20 and the negative stroke dataset 21 can determine the threshold value Tv . If, over a significant period of time, the features F calculated on the monitored person are always below the threshold value Tv, this value may be lowered in order to have a more precise and efficient personalized detection. The features F calculated on the monitored person will also allow for the measurement a QUANTIFICATION of the onset of a stroke (a scale in the measurement of facial asymmetry), and a KINETIC MEASURE of this QUANTIFICATION (example: evolution of facial asymmetry, in the first minutes or over a significant period of time). For example, in our case of a positive and negative stroke dataset comprising 104 videos, an accuracy higher than 92% was obtained . However, this QUANTITATIVE/QUALITATIVE/KINETIC measurement approach can be applied to all types of pathologies that can be monitored in real time. Another example: detecting gait balance and its deterioration. This detection can be done using various means: Camera with motion analysis of the skeleton (MediaPipe Holistic to extract human pose landmarks), Wearables to detect lower limb movements, Sensors positioned at key movement points (ankles, knees, etc.) . . . . . The FIG. 1 illustrates the movements of both ankle joints of a person walking normally, as captured using the Mediapipe Holistic library and a camera. This graph represents 2 plots, one corresponding to the kinetic readings of the right ankle and the other of the left ankle. The graph clearly demonstrates the synchronized movement between the left ankle and the right ankle. The FIG. 2 now depicts the same type of measurement of the FIG. 1 but on a post-stroke individual. The graph clearly shows a marked asymmetry between the left and the right ankle joints. To quantify this asymmetry, we can calculate the standard deviation of the intersection of the two curves representing the movement kinetics on the left and right sides. The experiment demonstrated that this standard deviation has a constant and low level in normal walking, while it is significantly higher in unbalanced walking. This assessment therefore provides a means of quantifying gait quality on a scale. In addition to assessing gait quality at a single point in time, it allows us to monitor changes in gait quality over time (by measuring the kinetic evolution of the pathology). This can be used, for example, to intervene before a person falls. This monitoring capability has the potential to improve patient outcomes by enabling timely interventions. As these measures demonstrate a satisfactory level of accuracy in the early detection of stroke, which is an absolute medical emergency, the approval of the protocol's performance by the medical authority can trigger the use of biometric recognition. The use of biometric recognition allows for access to the patient's medical history, which can strengthen the diagnosis AND/OR accelerate patient care by already having knowledge of their medical files upon arrival in the emergency room. THIS IS A CASE OF FORCE MAJEURE THAT REQUIRES URGENT TREATMENT, GOVERNED BY MEDICAL ETHICS , AND THEREFORE EXEMPT FROM ANY PRIOR AUTHORISATION. Innovative method for obtaining a dataset of people with simulated the onset of a pathology The methods for learning a multi-neuronal classifier or obtaining a threshold value for distinguishing a population with or without a pathology will be more effective if the volume of these two populations is high. This type of population is even more difficult to obtain because datasets are generally collected post-pathology, rather than with the progressive onset of a targeted pathology. However, the symptoms that identify a targeted pathology are known, such as in the occurrence of a stroke, a progressive asymmetry of the mouth, a disproportion between the left and right eye, and an imbalance in walking. Thanks to the evolution of algorithms related to artificial intelligence, we are able to detect facial keypoints as well as the full body skeleton, which allows us to monitor people's gestures, with examples of libraries such as MEDIAPIPE, DLIB, etc. Based on these 3D coordinates of points, we have, knowing the targeted pathological traits, been able to recalculate these coordinates to simulate the onset of a pathology in the video of a moving person. This innovative motion decomposition is jointly implemented by morphing algorithms to achieve a realistic rendering. This modification on each frame of a video can be decomposed into 8 steps: Find Landmark Points of the Image in each frame, from example with DLIB or MEDIAPIPE libraries. Recalculate the targeted points on the pathology concerned (e.g. stroke: mouth deviation, eye orbit disproportion, ...). For example, the FIG. 3 shows the landmarks of the Mouth on a Face without Facial Deviation. The FIG. 4 shows an example of recalculating the positions of the landmarks in FIG. 3 to simulate an asymmetry of the mouth. Triangulation Source Image (Delaunay Triangulation) Triangulation of the Destination Image with the recalculated points Extract And Warp Triangles Link The Warped Triangles Together Replace The Face On The Destination Image Seamless Cloning This is an example that we have successfully tested. Other morphing methodologies can be used, but the key is step 2, the recalculation of the targeted points on a pathology concerned, which is an innovation. Result example: The FIG. 5 shows a face without characterized asymmetry making it possible to use this template in order to diagnose with personal parameters the arrival of a stroke. FIG. 6 shows a morphological transformation of the face of the FIG. 5 making it possible to simulate the arrival of a stroke, to quantify it and evaluate the kinetic evolution of the pathology. Conversely, it is possible to have videos of people with a stroke recalculate the keypoints in order to have a transition with a symmetrical face. A computerized method for estimating a likelihood of a pathology disease of a user, such as Parkinson's disease, Alzheimer's and other dementias, stroke, heart failure, heart attack, and other coronary diseases, epilepsy, sepsis, Asthma attack, etc., the computerized method being performed by a computerized monitoring system comprising at least one processor and at least one monitoring device, such as cameras, microphones, connected objects, wearables, sensors, etc. The list of pathologies mentioned above is not exhaustive: ANY pathology whatsoever that can be detected in the continuous extraction of multimodal markers from individuals/patients falls within the scope of the claims cited in this publication. The computerized method according to claim 1, will be able to detect behavioral abnormalities associated with an early suspicion of a targeted pathology, whether neuronal or motor, based on multi-neuronal classifiers in artificial intelligence, through deep learning between two populations, OR/AND on mathematical/logical equations measuring gestural or vocal expressions to obtain values to dissociate the two populations (one with the targeted pathology and the other without). The computerized method according to claim 2, in order to improve the early detection of a nascent pathology, in knowledge of the gestural weaknesses that constitute this pathology, innovative algorithms have been created to reconstruct this occurrence of pathology by morphing simulation. The computerized method according to claim 2 and 3, the monitoring based on mathematical and/or logical equations (and, or, xor, etc.) and/or other QUALITATIVE / QUANTITATIVE processes, if there is no detection of anomaly over a significant period of time, the values obtained on the monitored person during this period by the equations/processes targeting a pathology will be taken into account. These customized values will allow for a more individualized monitoring of the person for a targeted pathology, in order to have a more accurate and efficient detection. In this case, there is no biometric detection to identify the individual. The computerized method according to claim 2, 3 and 4, multimodal monitoring supported by AI algorithms will not only detect the QUALIFICATION of the onset of a pathology, but especially the QUANTIFICATION of this onset of pathology (to provide a scale in the measurement), and even more so to provide a KINETIC MEASURE OF THIS QUANTIFICATION (evolution of the pathology measurement, in the first minutes or over a significant period of time). The three types of measures we claim apply to ANY pathology, regardless of its nature, detectable through the continuous extraction of multimodal markers from individuals/patients. The computerized method according to any one claim 1 to 5, if the multimodal monitoring supported by AI algorithms to detect the qualification of the onset of a pathology has precision levels validated by medical professionals, the approval of the protocol's performance by the medical authority can trigger the use of biometric recognition. The use of biometric recognition allows for access to the patient's medical history, which can strengthen the diagnosis to send an alarm AND/OR accelerate patient care by already having knowledge of their medical files upon arrival in the emergency room. The computerized method according to any one claims 1 to 5, the use of biometric recognition may also be employed without prior approval from an authority ( MEDICAL OR OTHERWISE ) , if and only if the methods used have already proven effective in detecting a person's agonizing state, or a prolonged fainting spell without regaining consciousness, an epileptic state unresponsive to any external stimuli, or a serious and imminent danger that threatens the person's bodily integrity or moral well-being (distress). The computerized method according to any one claims 1 to 5, outside of the two cases mentioned above, claim 5 "FORCE MAJEURE" AND/OR claim 6 "OBLIGATION TO ASSIST A PERSON IN DANGER", the "Life Tracking Medicine", (LTM) which gives citizens the ability to create their own data heritage, allows for the consideration of the person's prior consent at the time of the device's activation , ONLY IN CASE OF SUSPICION OF THE ARRIVAL/DETECTION OF ANY PATHLOGY , and without prior approval from an authority (MEDICAL OR OTHERWISE), for the use of biometric recognition. This case also includes persons who, being minors or lacking the capacity to make decisions, are represented by a Parent or legal guardian. In this context, the initial consent of the legal guardian will be taken into account, only in case of suspicion of the arrival/detection of any pathology. The computerized method according to claim 6, 7 and 8, Biometric recognition will never be used in routine Multimodal Monitoring . Biometric recognition will only be used in the event of reliable detection of the arrival of a targeted pathology according to claim 6, 7 and 8 described above