Patent Description:
Sleep apnea refers to the symptoms of involuntary weakening or even cessation of breathing during sleep. The cessation of breathing is often unnoticed until the body is severely deprived of oxygen and wakes up due to discomfort. However, hypoxia causes damage to the body, and patients can even die suddenly due to cardiovascular disease after a long period of time. It's worth noting that the effects of sleep apnea go beyond daytime inattention and tiredness. Since the heart rate slows down and blood pressure drops during normal sleep, normal sleep gives the cardiovascular system a chance to rest and recover. However, sleep in apnea patients can be interrupted several times a night, depriving them of cardiovascular rest. In addition, repeated apnea can cause blood oxygen levels to drop, cause a certain amount of stress on the body and lead to a systemic inflammatory response, and can also increase the risk of cardiovascular disease. People with sleep apnea are often unaware of symptoms. Symptoms can only be discovered when the patient goes to the hospital for detection and diagnosis with special equipment. Researchers estimate that about <NUM> percent of people with moderate to severe sleep apnea are undiagnosed. In addition, sleep apnea is involved in a wide range of levels, such as traffic accidents, industrial safety accidents, cardiovascular diseases, metabolic diseases, etc..

Moreover, events such as falls, car accidents, etc., are accidents that humans can encounter. If these accidents are detected early, the damage can be minimized.

<CIT>, <CIT> and <CIT> define systems for event detection and data compression.

Accordingly, the embodiments of the disclosure provide a detection method of an event, an event detection system, and an inference server that can detect sleep quality and an unexpected event.

An event detection system of an embodiment includes (but not limited to) one or more terminal devices and an inference server. The terminal device generates first compressed data. The first compressed data is related to a sensing result of a physiological state or a motion state. The inference server decodes the first compressed data into reconstructed data via a decoder in an anomaly detection model, encodes the reconstructed data into second compressed data via an encoder, and determines an event of the physiological state or the motion state by an error between the first compressed data and the second compressed data. The anomaly detection model includes the encoder and the decoder.

A detection method of an event of an embodiment includes (but not limited to) the following steps. First compressed data is received. The first compressed data is related to a sensing result of a physiological state or a motion state. The first compressed data is decoded into reconstructed data via a decoder in an anomaly detection model. The anomaly detection model includes the decoder and an encoder. The reconstructed data is encoded into second compressed data via the encoder. An event of the physiological state or the motion state is determined by an error between the first compressed data and the second compressed data.

An inference server of an embodiment includes (but not limited to) a communication transceiver, a memory, and a processor. The communication transceiver transmits or receives data. The memory stores a program code. The processor loads the program code to execute: receiving first compressed data via the communication transceiver, decoding first compressed data into reconstructed data via a decoder in an anomaly detection model, encoding the reconstructed data into second compressed data via an encoder in the anomaly detection model, and determining an event of a physiological state or a motion state by an error between the first compressed data and the second compressed data. The first compressed data is related to a sensing result of the physiological state or the motion state. The anomaly detection model is based on an autoencoder, and the anomaly detection model includes the decoder and the encoder.

Based on the above, according to the detection method and the event detection system and the inference server of the embodiments , the terminal device only transmits compressed data, and the inference server determines events based on the compressed data from the terminal device and the compressed data converted by the decoder and the compressor of the anomaly detection model. In this way, bandwidth can be effectively utilized, power can be saved, and event detection with data privacy can be provided.

In order to make the aforementioned features and advantages of the disclosure more comprehensible, embodiments accompanied with figures are described in detail below.

<FIG> is a block diagram of elements of an event detection system <NUM> according to an embodiment. Referring to <FIG>, the event detection system <NUM> includes (but not limited to) one or more terminal devices <NUM>, an inference server <NUM>, a training device <NUM>, and a warning system <NUM>.

The terminal device <NUM> can be an IoT device, a wearable device, a medical testing instrument, a smart phone, a tablet computer, or a sensing device.

The terminal device <NUM> includes (but not limited to) a communication transceiver <NUM>, a memory <NUM>, a sensor <NUM>, and a processor <NUM>.

The communication transceiver <NUM> can be a communication transceiver circuit supporting low power wide area network (LPWAN) techniques (LPWAN communication techniques such as long distance (LoRa) techniques, Narrow Band Internet of Things (NB-IoT), Sigfox, LTE Advanced for Machine Type Communications (LTE-MTC)), and it can also be a communication transceiver circuit or a transmission interface card supporting Wi-Fi, Bluetooth, mobile communication, USB, or Ethernet. In an embodiment, the communication transceiver <NUM> transmits or receives data with an external device (e.g., the inference server <NUM> or the training device <NUM>).

The memory <NUM> can be any form of a fixed or movable random-access memory (RAM), read-only memory (ROM), flash memory, traditional hard disk drive (HDD), solid-state drive (SSD), or similar devices. In an embodiment, the memory <NUM> stores a program code, a software module, a configuration, data, or a file (for example, compress data, a sensing result, or a feature), and is described in detail in subsequent embodiments.

The sensor <NUM> can be a heartbeat sensor, a physiological sensor, an image sensor, a motion sensor, or other types of sensors. In an embodiment, the sensor <NUM> senses a physiological state (e.g., heartbeat, respiration rate, or blood oxygen level) or a motion state (e.g., inertial attitude, acceleration, or moving direction) to obtain a sensing result. In an embodiment, the sensing result is time-dependent data. That is, data recorded with time sequence, continuous time, or multiple points in time. For example, the sensing result is an electrocardiography (ECG), respiratory airflow in a polysomnography (PSG) of a night's sleep, chest movement, abdominal muscle behavior, or an electroencephalogram.

The processor <NUM> is coupled to the communication transceiver <NUM>, the memory <NUM>, and the sensor <NUM>. The processor <NUM> can be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), neural network accelerators, or other similar devices or a combination of the above devices. In an embodiment, the processor <NUM> performs all or part of the operations of the terminal device <NUM>, and can load and execute each of the program codes, software modules, files, and data stored in the memory <NUM>.

The inference server <NUM> can be a cloud/edge computing server, a workstation, or a computer.

The inference server <NUM> includes (but not limited to) a communication transceiver <NUM>, a memory <NUM>, and a processor <NUM>.

The implementation and functions of the communication transceiver <NUM> are as provided in the description of the communication transceiver <NUM> and are not repeated herein. In an embodiment, the communication transceiver <NUM> transmits or receives data with an external device (e.g., the warning system <NUM> or the training device <NUM>).

The implementation and functions of the memory <NUM> are as provided in the description of the memory <NUM> and are not repeated herein. In an embodiment, the memory <NUM> stores data such as compressed data, reconstructed data, errors, or determination results, which is described in detail in subsequent embodiments.

The processor <NUM> is coupled to the communication transceiver <NUM> and the memory <NUM>. The implementation and functions of the processor <NUM> are as provided in the description of the processor <NUM> and is not repeated herein. In an embodiment, the processor <NUM> performs all or part of the operations of the inference server <NUM>, and can load and execute each of the program codes, software modules, files, and data stored in the memory <NUM>. In some embodiments, some operations in a method of an embodiment can be implemented by different or the same processor <NUM>.

The training device <NUM> is communicatively connected to the terminal device <NUM> and the inference server <NUM>. The training device <NUM> can be a smart phone, a tablet, a computer, a server, or a workstation. In an embodiment, the training device <NUM> trains a model based on a machine learning algorithm (e.g., autoencoder, neural network, decision tree, or random forest). The machine learning algorithm can analyze training samples to obtain patterns therefrom, so as to predict unknown data via the patterns. For example, the machine learning model establishes the association between the nodes in the hidden layer between feature data (i.e., the input of the model) and a respiratory event (i.e., the output of the model) by the labeled samples (e.g., feature data of known hypopnea/apnea events, or feature data of known normal breathing events). The machine learning model is a model constructed after learning, and can accordingly infer data to be evaluated (for example, feature data or compressed data).

The warning system <NUM> is communicatively connected to the inference server <NUM>. The warning system <NUM> can be a display, a speaker, or a communication transceiver. In an embodiment, the warning system <NUM> sends out a warning message.

In some embodiments, the functions of the training device <NUM> and/or the warning system <NUM> can also be implemented via the inference server <NUM>, or these devices are integrated into a single device.

Hereinafter, the method described in an embodiment is described in conjunction with various devices and elements in the event detection system <NUM>. Each of the processes of the present method can be adjusted according to embodiment conditions and is not limited thereto.

<FIG> is a flowchart of a detection method of an event according to an embodiment. Referring to <FIG>, the processor <NUM> of the inference server <NUM> receives first compressed data via the communication transceiver <NUM> (step S210). Specifically, the processor <NUM> of the terminal device <NUM> generates the first compressed data by the sensing result (i.e., raw data) of the sensor <NUM> for a physiological device or a motion device. In other words, the first compressed data is related to the sensing result of the physiological state or the motion state.

In an embodiment, the physiological state is a heartbeat. For example, the sensing result of the heartbeat is the R-R interval of a heartbeat waveform in a time sequence. It is worth noting that each person's heart rate is different, and under normal circumstances, the normal adult heart rate is <NUM> to <NUM> beats per minute. A normal person's heart rate is affected by many factors. The heart beats faster when exercising, slows down when resting or sleeping, and slows down some more when exhaling. In addition, conditions such as fever, nervousness, stress, and pain can also affect heart rate. Heart rate is affected by breathing rate. Under normal circumstances, the heart rate of women is faster than that of men. A normal adult breathes about <NUM> to <NUM> times per minute. The ratio of the number of breaths to heart rate is <NUM>:<NUM>, that is, for every breath, the heart beats four times. Therefore, heartbeat can be used as an indicator of a person's mood, exercise, and breathing conditions. According to the literature (https://www. gov/pmc/articles/PMC8590683/pdf/11325_2020_Article_22 <NUM>. pdf), the human respiratory condition can be detected by observing heart rate variability. Heart rate variability is as the acceleration and deceleration of the heart rhythm. Via such variation of the heart rhythm, it is also possible to understand the health condition of an individual. The terminal device <NUM> can collect the R-R interval of the heartbeat via ECG.

For example, <FIG> is a timing diagram illustrating an example of a heartbeat. Please refer to <FIG>, generally speaking, a series of wave bands in the ECG include a P wave, a QRS complex, and a T wave. The QRS complex consists of a series of <NUM> deviations. The negative wave of the first deviation in the complex is called Q wave, the first positive deviation in the complex is called R wave, and the negative deviation after R wave is called S wave. The distance between two consecutive R waves can be called R-R interval RRI.

In an embodiment, the processor <NUM> of the terminal device <NUM> can encode the sensing result of the heartbeat into the first compressed data via an encoder. The anomaly detection model includes the decoder and the encoder. The abnormal detecting model is based on an autoencoder architecture, a machine learning architecture composed of the encoder and the decoder, or other neural network architectures reconstructing a sample or other specified samples.

In an embodiment, the training device <NUM> trains the anomaly detection model by the plurality of samples labeled as events. For example, the anomaly detection model for heartbeats can determine a sleep apnea event, wherein the waveform of the heartbeat can have significant rapid fluctuations when a sleep apnea event occurs. Medical testing equipment can mark R-R interval segments having a sleep apnea event. Assuming that <MAT> (i is one of <NUM> to I) is the training sample of the R-R interval in the time sequence, then <MAT> is the t-th R-R interval value of the i-th patient having a sleep apnea symptom (that is, a sleep apnea event occurs) at the moment, and I represents the number of all apnea patients in the training sample. In an embodiment, in order to remove the variation of individual data, R{i} can be standardized to R̃{i}, so that these training samples have the characteristics of the average value of <NUM> and the variation of <NUM>: <MAT>.

For the anomaly detection model of the autoencoder, the training device <NUM> can divide the time sequence data of the R-R interval labeled as the sleep apnea event into a plurality of RRI time sequence segments having the same samples. For example, if <MAT>, and the number of samples is N, then <MAT> <MAT>, and <MAT>. That is, an RRI sample with a sample number of #(R̃{i}) is cut into M segments, and each segment has N RRI samples. K represents the K-th segment RRI sample after segmentation, and the symbol # is the number of RRI samples.

<FIG> is a schematic diagram of an autoencoder according to an embodiment. Please refer to <FIG>, <MAT> is input data X1 in an anomaly detection model ADM. Reconstructed data X1' is data compressed by an encoder ECR of the anomaly detection model ADM and decompressed by a decoder DCR. The number of samples of the reconstructed data X1' is the same as that of the input data X1. That is, if the number of samples of the input data X1 is N, the reconstructed data X1' also has N samples.

In the training of the anomaly detection model, the anomaly detection model ADM can be split into two neural networks, the encoder ECR and the decoder DCR. The input data is input to the encoder ECR, and is compressed into relatively low-dimensional compressed data Z1 (for example, a one-dimensional vector) by the encoder ECR. The compressed data Z1 is input to the decoder DCR, and the reconstructed data X1' having the same size as the input data X1 is restored by the decoder DCR. In order to make the reconstructed data X1' similar to or the same as the input data X1, mean-square error (MSE), mean absolute error (MAE), cross entropy, or focal loss can be used as the loss function. In addition, the training device <NUM> can update the weights in the neural network via backpropagation to minimize the loss function, and then train the abnormality detection model ADM.

The training device <NUM> can transmit the encoder ECR in the anomaly detection model ADM to the terminal device <NUM>, and transmit the encoder ECR and decoder DCR in the anomaly detection model ADM to the inference server <NUM>.

In another embodiment, the motion state is an inertial attitude. For example, motion index (e.g., velocity, angular velocity, or acceleration), orientation, or displacement on three or six axes. The processor <NUM> of the terminal device <NUM> can encode the sensing result of the inertial attitude into the first compressed data via an encoder. That is, the terminal device <NUM> can encode the sensing result into the first compressed data via the trained encoder ECR. The anomaly detection model for inertial attitude can determine a fall event or a car accident event, wherein when a fall event or a car accident event occurs, the intensity of the acceleration and the orientation of the attitude can be changed rapidly.

It should be noted that the types of events can still be changed according to the actual needs of the user.

Then, the processor <NUM> of the terminal device <NUM> can transmit the first compressed data to the inference server <NUM> via the communication transceiver <NUM>.

In an embodiment, the processor <NUM> of the inference server <NUM> can receive the first compressed data from one or more terminal devices <NUM> via a low power wide area network (LPWAN) via the communication transceiver <NUM>. LPWAN has the characteristics of long-distance communication and power saving, and LPWAN can solve the transmission issue of the Internet of Things. It is worth noting that, compared with mobile network standards that pursue larger bandwidth, higher speed, or lower latency but consume more power, there is a wider deployment range for machine-to-machine (M2M) communication, and the need for frequent battery replacement is avoided, and LPWAN techniques provide the characteristics of less data volume, long-distance transmission, and power saving, and therefore are suitable for the application field of Internet of Things. For example, for environmental monitoring or asset tracking, LPWAN techniques with a transmission distance of up to <NUM> kilometers can significantly reduce deployment costs, and only a few stations are needed to cover a large area. LPWAN techniques are, for example, LoRa, Sigfox, and NB-IoT. However, LPWAN emphasizes coverage and power saving, but has relatively limited bandwidth. For example, LoRA is only <NUM> bps, Sigfox is <NUM> bps to <NUM> kbps, and NB-IOT is <NUM> kbps. Under the limitation of limited bandwidth, it should be difficult to load a large number of terminal devices <NUM> for simultaneous transmission, and it is even more difficult to meet instant communication requirements such as health and industrial safety monitoring. However, in the embodiments the sensing result can be compressed, thus not only reducing the amount of data to effectively utilize bandwidth, but can also achieve the effect of data privacy.

In other embodiments, the network between the terminal device <NUM> and the inference server <NUM> can also be other communication techniques. For example, Wi-Fi or Bluetooth.

Referring to <FIG>, the processor <NUM> of the inference server <NUM> decodes the first compressed data into reconstructed data via the decoder in the anomaly detection model (step S220). Specifically, <FIG> is a schematic diagram of an anomaly detection model ADM according to an embodiment. Please refer to <FIG>, which is an architecture diagram of an autoencoder. One of the objects of the autoencoder is to find the functions gø and fθ such that X2'= fθ (gø (X2)), and X2≈X2'. gø is the mathematical function of the encoder ECR, and fθ is the mathematical function of the decoder DCR. Compressed data Z2 (hereinafter referred to as the first compressed data) is the data obtained by compressing the input data X2 (i.e., the above sensing result) via the function gø. In an embodiment the compression operation corresponding to the function gø can be implemented via the terminal device <NUM>, so that the first compressed data can be transmitted via the network.

Taking heartbeat time sequence data as an example, the input data X2 is a sequence of heartbeat time data (the number of samples can exceed a certain number (for example, <NUM>, <NUM>, or <NUM>) or the sensing time can exceed a certain time (for example, <NUM> minutes, <NUM> minutes, or <NUM> hour)), and the first compressed data Z2 is the data after the input data X2 is compressed via the encoder ECR. The reconstruction data X2' is data decoded by the decoder DCR. To detect abnormal conditions (for example, the above events), the models of the functions gø and fθ are trained using the heartbeat time sequence data of sleep apnea as described above. After training, the models of the encoder ECR and the decoder DCR can be used to detect whether there is a sleep apnea event within a period of time. If the input data X2 of the heartbeat time sequence is sequentially reconstructed by the encoder ECR and the decoder DCR in the anomaly detection model ADM, and the reconstructed data X2' is equal to or approximate to the input data X2, it is considered normal (i.e., a sleep apnea event occurs). Otherwise, it is considered abnormal (i.e., a sleep apnea event does not occur or a normal event occurs).

In an embodiment, the processor <NUM> can determine whether the error between the input data X2 and the reconstructed data X2' (for example, |X2-X2'|) is less than a first threshold. The first threshold can be obtained by data occupying a specific quantile (e.g., <NUM>, <NUM>, or <NUM>) in the data distribution of all training samples in which a sleep apnea event occurs. In response to the error between the input data X2 and the reconstructed data X2' being less than the first threshold, the processor <NUM> can determine that an event (such as a sleep apnea event) occurs. In response to that the error between the input data X2 and the reconstructed data X2' is not less than the first threshold, the processor <NUM> can determine that an event does not occur (for example, a sleep apnea event does not occur or a normal event occurs).

In an embodiment the decoding operation corresponding to the function fθ can be implemented via the inference server <NUM>. It should be noted that, in the architecture of <FIG>, the terminal device <NUM> still needs to transmit the input data X2 for the inference server <NUM> to compare the reconstructed data X2'. In other words, the terminal device <NUM> needs to transmit both the input data X2 and the compressed data Z2, thus affecting bandwidth usage and cannot achieve the effect of data protection.

Referring to <FIG>, the processor <NUM> of the inference server <NUM> encodes the reconstructed data into second compressed data via the encoder of the anomaly detection model (step S230). Specifically, <FIG> is a schematic diagram of the anomaly detection model ADM according to another embodiment. Please refer to <FIG>, compared with the architecture in <FIG>, the inference server <NUM> also implements the compression operation corresponding to the function gø. That is, the encoder ECR encodes the reconstructed data X2' into compressed data Z2' (hereinafter referred to as second compressed data). If the second compressed data Z2' of the input data X2 sequentially processed by the encoder ECR, the decoder DCR, and the encoder ECR in the anomaly detection model ADM is equal to or approximate to the first compressed data Z2, it is considered normal (for example, a sleep apnea event occurs, a fall event occurs, or a car accident event occurs). Otherwise, it is considered abnormal (e.g., a sleep apnea event does not occur or a normal event occurs). It can be seen from this that the terminal device <NUM> only needs to transmit the first compressed data Z2 to the inference server <NUM>.

Referring to <FIG>, the processor <NUM> determines the event of the physiological state or the motion state by the error between the first compressed data and the second compressed data (step S240). In an embodiment, the event determined by the processor <NUM> by the error between the first compressed data and the second compressed data refers to a predicted event output by the inference server <NUM>. That is, the prediction result of the anomaly detection model is an event of the physiological state or the motion state.

Specifically, <FIG> is a flowchart of event determination according to an embodiment of the invention. Referring to <FIG>, the processor <NUM> can determine whether the error between the first compressed data Z2 and the second compressed data Z2' (for example, |Z2-Z2'|) is less than a second threshold (step S710). The second threshold can be obtained by data occupying a specific quantile (e.g., <NUM>, <NUM>, or <NUM>) in the data distribution of all training samples in which a sleep apnea event occurs. In response to the error between the first compressed data Z2 and the second compressed data Z2' being less than the second threshold, the processor <NUM> can determine that an event occurs (such as a sleep apnea event) (step S720). In response to the error between the first compressed data Z2 and the second compressed data Z2' being not less than the second threshold, the processor <NUM> can determine that an event does not occur (such as a sleep apnea event does not occur or a normal event occurs) (step S730).

Similarly, for the determination of a fall, a car accident, or other events, it is also possible to determine whether the event occurs by the error between the first compressed data Z2 and the second compressed data Z2', and details are not repeated herein.

In other embodiments, if the training phase utilizes training samples labeled as normal events (e.g., a sleep apnea event does not occur, a normal breathing event does not occur, a fall event does not occur, or a car accident event does not occur), then it can also be that an abnormal event occurs when the error between the first compressed data and the second compressed data is not less than the corresponding threshold (for example, a sleep apnea event occurs, a fall event occurs, or a car accident occurs). When the error between the first compressed data and the second compressed data is less than the corresponding threshold, there is an abnormal event (for example, a sleep apnea event does not occur, a normal breathing event occurs, a fall event does not occur, or a car accident event does not occur).

For example, normal events refer to time sequence segments marked as free of anomalies. The features of these time sequence segments are usually distributed within a certain range. It is assumed that there is a time sequence data with a Gaussian distribution. If the data to be evaluated is detected to be located within <NUM>% of the standard deviation of the mean value of the Gaussian distribution, the data to be evaluated is usually determined to come from the Gaussian distribution (regarded as normal). On the contrary, if located outside <NUM>% of the standard deviation, the data to be evaluated is identified as abnormal. From the concept of distance, normal data is closer to the data in the data distribution of the normal data. Conversely, abnormal data is compared to data in a data distribution farther than the normal data. If the model is trained on the normal data, this model means that the prediction thereof of the normal data is relatively more accurate. That is, when the normal data goes through the anomaly detection model, the compressed and restored value (that is, the reconstructed data) is closer to the original input value. If the abnormal data is brought into this abnormal detection model, it is difficult to restore the abnormal data via compression. Therefore, the distance between the abnormal data and the restored data is farther. The first threshold and second threshold are based on this concept.

It should be noted that, for different terminal devices, the inference server <NUM> can deploy corresponding anomaly detection models respectively. In other words, for the first compressed data from one terminal device <NUM>, a set of decoders and encoders is deployed; and for the first compressed data from another terminal device <NUM>, another set of decoders and encoders is deployed. However, the deployment of the anomaly detection models is not limited to one-to-one relationships. For example, the inference server <NUM> runs fewer anomaly detection models concurrently, but with prioritization.

In an embodiment, the inference server <NUM> can request the warning system <NUM> to send a warning message for events such as falls, sleep apnea, or car accidents. For example, sending a push notification, making a phone call to the relevant authorities, or sounding a warning. In addition, the inference server <NUM> can also transmit the determination result to the warning system <NUM> periodically or based on a trigger event. The warning system <NUM> can report the overall health status.

Claim 1:
An event detection system (<NUM>), wherein the event detection system (<NUM>) comprises:
a terminal device (<NUM>) encoding a sensing result of a physiological state or a motion state into first compressed data (Z2) via an encoder (ECR); and
an inference server (<NUM>):
receiving the first compressed data (Z2) via a network;
decoding the first compressed data (Z2) into reconstructed data (X2') via a decoder (DCR) in an anomaly detection model (ADM); wherein the anomaly detection model (ADM) comprises the decoder (DCR) and the encoder (ECR);
encoding the reconstructed data (X2') into second compressed data (Z2') via the encoder (ECR); and
determining an event of the physiological state or the motion state by an error between the first compressed data (Z2) and the second compressed data (Z2').