Patent Description:
The invention exploits a technology based on radar in the mm wave band with a frequency range of <NUM> - <NUM>. phased array-technology.

The invention is based on the creation of a 3D cloud-data output from a radar scan of the environment where one or more patients are situated.

<CIT> describes a method and apparatus for tracking respiration and heart rate using an FMCW radar arranged at a distance from persons within a space. The radar may operate on a frequency of <NUM> or <NUM>. A fast Fourier transform is applied on the received data and a tracking method comprising selecting a target range bin is applied to detect the location of persons. The chest movements due to respiration and heartbeat of the detected persons are measured by phase changes and distance fluctuations. A breathing pattern or a heart rate pattern may be determined through a frequency spectrum over time in a range bin. An abnormal breathing pattern is determined by expanding the signal strength.

<CIT> describes a device and method for contactless respiration monitoring of a patient. It is based on the Doppler-Radar principle and operates at frequencies below <NUM>. The received signal is phase shifted and vectors are determined at different point in time. These vectors are defined by the time derivatives of the received signal and the phase shifted signal. The vectors provide an indicator of the change of respiration of the patient.

<CIT> describes a method for measuring respiration rate and heartbeat using an ultra-wideband impulse radar signal. The received signal is run through band-pass filters corresponding to heartbeat and respiration frequency bands. CZT (Chirp Z-transform) is applied to the band-pass filtered signals to convert the signals into the frequency axis band.

<CIT> describes a radar-based non-contact vital sign measurement system using the echo from a radar signal. A Fourier transform is done on the received signal, but how the vital signs are detected from this is poorly described.

<CIT> describes a non-contact type living body monitoring and care system using ultra-wide band in a frequency band of <NUM>-<NUM>. The received signal is run through a noise filter after which the distance of movement of the target is measured. From this respiratory state is analysed using 2D of 3D shaping.

<CIT> describes a monitoring system using WLAN receivers operating as Doppler radar to detect heartbeat or respiration motion of a chest of a patient. An analyser is coupled to one of the transmitter and receiver to determine heart attack or stroke attack. <CIT> describes a method for estimating a disease state for a patient using a non-invasive medical device that communicates with a network. <CIT> describes a wearable electronic device, which includes a hybrid wireless communication module to acquire location data from both indoor and outdoor sources, as well as to selectively transmit an LPWAN signal to provide location information based on the acquired data.

<CIT> describes a device for remote monitoring of breathing movements of a patient using a microwave radar. The radar is working on a frequency of <NUM>. A processor mixes the received signal with the transmitted signal and with a signal in quadrature with the transmitted signal to determine a breathing frequency of the patient. In some embodiments, two signals at different frequencies are emitted toward the patient and processed in order to compensate for breathing patterns that are not uniquely resolvable at a single frequency.

<CIT> describes a system using a radio frequency interferometer antenna array where the received signals are run through a clutter cancellation module to filter out static non-human related echo signals. From the filtered signals are extracted a quantified representation of position postures, movements, motions and breathing of at least one human located within the specified area. In one embodiment the human position and posture is found by dividing the surveillance space into voxels (small cubes) in cross range, down range and height. Then is the reflected electromagnetic signal from a specific voxel estimated by a back projection algorithm, and the human position is estimated by averaging the coordinates of the human reflecting voxels for each baseline, triangulating all baselines' position to generate the human position in the environment. The human posture is estimated by mapping the human related high-power voxels into the form-factor vector, and finally tracking the human movements in the surveillance space.

<CIT> describes a system for sensing human breathing at a distance. It is using a radar antenna. A clutter suppression is performed on the received data, where after one or more targets are detected in specific range bins. A breathing signal, including breathing rate, for the detected one or more targets is extracted. It is verified that each of the one or more targets are in associated range bins based on the extracted parameters. To estimate the target's range bin and distinguish between multiple targets, a combination of two signal processing techniques may be used (Singular Value Decomposition (SVD) and skewness).

<CIT> describes a system and method for breathing monitoring using a Doppler radar-based sensor system. The frequency of the radar is in the range between <NUM> and <NUM>, but preferably a frequency of about <NUM> is used. The received signal is autocorrelated and run through a Fourier transform. The value of at least two peaks of the calculated Fourier transform are determined, and from this a breathing characteristic of the person is determined.

Some of the above systems and devices are at least partially depending on the patient being connected to or wearing a device, which inevitably will limit the freedom of movement of the patient and/or the time over which the patient can be monitored.

The radar that is proposed used in the system of the invention has a built-in method for estimation of heart rate (HR) and respiration rate (RR) on an integrated circuit built by Texas Instruments. However, it has been found that this circuit stops providing estimates on both HR and RR when the distance from the radar to the subject exceeds <NUM> meter. Consequently, the built-in abilities are too poor to give the radar any usefulness in monitoring patients unless they are very limited in their freedom of movement.

<CIT>, achieves good results on RR and HR detection and estimation for short distances (up to <NUM>). However, the reference does not mention detection and estimation on longer distances than this. The reference also mentions several challenges concerning multi-person respiration detection with <NUM>.

Prior art methods used to detect and estimate respiration are in general based on traditional signal processing. These methods have certain limitations:.

<CIT> describes a system for detecting respiration of a patient using a MIMO radar. A clustering technique is used to determine which data points belong to the same person. Subsequently the amplitude of the points is determined by weighting the points that lies around the peak amplitude.

<CIT> describes a patient monitoring device. <CIT> describes a method for presence detection. <CIT> describes a system for detecting respiration and heartbeat of a patient by using a mm radar.

In the article '<NPL>', is described a system for detecting respiration and heartbeat of a patient by using a radar operating at a frequency of <NUM>. The extraction of heart rate or respiration rate is done by various types of machine learning. The articles '<NPL>' and '<NPL>' disclose people tracking.

All of the above prior art references will process all of the data points that are determined to belong to the same person in the same way. This requires a certain processing capacity. Inevitably, the number of data points that can be processed is limited by the available processing capacity. This may influence on the accuracy of the respiration or heartbeat monitoring or the possibility to follow the person when this is moving. It may also not be possible to monitor several persons at once, and if possible, the accuracy of the measurements will be lower.

An objective of the invention is to optimize a dataset from a phased-array radar to detect movement and orientation of subjects within a coverage area of the radar. This is achieved by the features of subsequent claim <NUM>.

In a preferred embodiment, the method of the invention achieves the above objectives by:.

A key item in the application of the method of the invention is a radar that is capable of detecting the presence and movement of subjects within a coverage space of the radar.

The following description will explain the technical details of the method of the invention.

<FIG> shows a block diagram illustrating a flow chart of the respiration detection optimization algorithm of the present invention.

The flow chart illustrates a method according to a preferred embodiment of the invention to determine a subject's respiratory rate with point clouds produced from radar sensors. The preferred method comprises nine steps: (<NUM>) Initialization and data acquisition from the radar configuration, (<NUM>) gathering of point cloud data with Doppler information, (<NUM>) noise filtering, (<NUM>) object identification and tracking of objects, (<NUM>) an aggregation function to group and form the grouped velocity data into a single summary signal, (<NUM>) dynamic focus of the radar based on object identification, (<NUM>) extraction of respiration signal by use of (<NUM>. A) traditional signal processing methods, (<NUM>. B) machine learning, and subsequently combining the two methods in step (<NUM>) with (<NUM>) ensemble learning, in order to obtain the (<NUM>) final optimized respiration signal. The different steps in the flow chart will be described in detail below.

The preferred radar to be used with the method of the present invention is a mm waveband radar, which operates at around <NUM> frequency. Waves with a frequency around this value will not penetrate walls, floors and ceilings between rooms to any substantial degree, and hence not be susceptible to detect persons in neighbouring rooms. However, the waves will penetrate clothes and blankets and thinner plates, such as glass and drywall board inside the room. This means that chest movements due to respiration and heartbeat of subjects, such as persons present within the coverage area of the radar, such as a room. , will be detected.

The radar can conveniently be mounted in the ceiling of a room, such as a patient room, livestock room. This will in many scenarios provide good coverage of the room. The radar may also be mounted otherwise, such as on a wall, on or in the floor, on a separate mount. It may also be mounted in other areas where it is desired to detect respiration or heartbeat induced movements, such as in onshore, offshore and under water installations, in an automobile, bus, boat, submarine, aeroplane, train, tram or other type of vehicle.

The monitoring device is preferably capable of wireless communication, both with various sensors and with a computer system in a cloud or on the premises. Adequate encryption of data ensures safety from hacking. The wireless communication can conveniently be by the Bluetooth protocol. This has a relatively short range and hence limited interference with other wireless communications.

The radar employs the several antennas arranged in a phased-array manner, which enables the beam of the antennas to be controlled in different directions without moving the radar.

<FIG> is an illustration of the principle of phased-array antennas. A row of antenna elements A is powered by a transmitter TX. The feed current for each antenna passes through a phase shifter (ϕ) controlled by a computer (C). The lines show the wave fronts of the radio waves emitted by each element. The individual wave fronts are spherical, but they superpose in front of the antenna to create a planar wave, i.e., a beam of radio waves travelling in a specific direction. The phase shifters delay the radio waves progressively going up the row of antennas, so each antenna emits its wave front a little later than the one below it. This causes the resulting planar wave to be directed at an angle θ to the antenna's axis. By changing the phase shifts Φ the computer can instantly change the angle θ of the beam. <FIG> illustrates a one-dimensional array of antennas, but most phased arrays have twodimensional arrays of antennas instead of the linear array shown here, and the beam can be steered in two dimensions. The present invention will utilize a twodimensional array.

The phased-array technology makes it possible for the radar to output 3D point cloud-data with Doppler information about motions in different points within the 3D space covered. This is information that has not been possible to obtain by radars with a single antenna. The radar is illustrated as item <NUM> in <FIG>.

The range resolution is heavily depending on the bandwidth. It has been found that a frequency of about <NUM> supports a wider bandwidth range than radars that operate in smaller bandwidth areas, such as <NUM> (UWB). The frequency can be up to about <NUM> but should preferably be within the range of <NUM>-<NUM>. This enables the radar according to the invention to detect objects and motions up to <NUM> times more accurately than UWB radars. The radar according to the invention facilitates accurate sensing with real-time decision making and complex signal processing, as will be explained below. Moreover, mm wave-based radars offer unique advantages because the sensors can detect movements through a variety of materials such as glass and drywall, which likely will be encountered in the intended application environments.

The phased-array radar is capable of creating a rich point-cloud data. One of the objects of the invention is to enable identification and separation of human motions from animals and mechanical objects and therefore reducing false detections to a great extent. Gathering point-cloud data with x, y and z co-ordinates with radial velocity provides a fine-grained range resolution. This level of resolution enables the radar to reliably detect and measure respiration movements from separate humans even though they are close to each other, like a doctor and a patient.

<FIG> illustrates a simple example of a 3D point cloud based on raw data. Motion amplitude is denoted by grey level, i.e., the darker the point is, the higher the velocity is. The raw input signal consists of a sequence of 3D point cloud frames received at a rate of <NUM> frames/s from the radar.

Position and doppler-information, which includes velocity and direction of motion, of reflected points are obtained continuously by the sensors of the radar and is segmented into frames. Each frame is associated with a timestamp and contains a collection of point data. Item <NUM> in <FIG> illustrates the 3D point data cloud created from the received radar signals.

Assumptions about the required minimum number of frames for obtaining the objectives of the present invention could be made. According to the Nyquist-Shanning sampling theorem, a periodic signal must be sampled at more than twice the highest frequency component of the signal, which can be mathematically written as fs > <NUM>fmax, where fs represent the number of frames per second and fmax represent the highest frequency that is desirable to detect. If, as an example, it is desired to estimate a respiration rate of <NUM> rpm, the theoretical minimum number of frames per second would be <NUM>. If it is desired to estimate a heart rate of <NUM> bpm, the theoretical minimum number of frames per second would be <NUM>.

This is the theoretical minimum number of frames needed, and any increase in the frame frequency would improve the resolution and hence the results.

A first step of the preferred method of the invention is noise filtering of the raw data to remove irrelevant information from the signal. This step is illustrated as item <NUM> in <FIG>. As further explained below, this is an optional step.

Various types of noise reduction techniques can be used to filter various types of noises from the frames. The goal of applying these filters is to improve the predictive quality of the repository rate model. The filtering process may include identifying and removing points with velocities that significantly exceeds the normal chest expansion speed during inspiration and expiration. This can be done by measuring and calculating the mean chest expansion velocity and the standard deviation. Points with velocities that lies within, e.g., three standard deviations σ of the mean value µ may be treated as noise and removed. That is all points with a velocity V > µV + <NUM> * σV, where <MAT> <MAT> where σV is the standard deviation of the velocity, vi is velocity of point i, µV is the mean of the velocities and N is the number of points.

In situations where static objects are making movements that falls within a frequency band that may interfere with the movements that are desirable to detect (e.g., respiration and heartbeat), such as a spinning fan, a mechanical pump, ventilator machines, patient monitors etc., the above-described way of filtering could remove such interfering objects from the point cloud if their point-velocities significantly exceeds that of, e.g., a normal chest expansion. If, however, the velocities are in the same frequency band and with similar velocities as the objects that are desirable to detect, the points need to be filtered in other ways. Such other filtering techniques could include putting constraints on the size of the point cloud, requiring that the object fits into similar measures as a body to be recognized as a living creature, e.g., a human being, requiring that the object has moved some distance during a defined time-interval, etc..

In situations where another living creature is in detection range of the radar, and the frequency and velocity overlap with the values from a human, constraints about the size and shape of the object could be applied to remove these points from the point cloud.

Other useful statistical noise reduction techniques may also be used, such as interquartile range filtering, kernel estimation, and smoothing. All of these techniques are well known per se.

The noise filtering step may improve the efficiency and quality of the subsequent steps, but is considered a non-critical step, as acceptable results for some applications could be achieved by skipping this step, given that the input point cloud data does not contain large amounts of noise. This is dependent on the environment that the radar will operate in. Depending on the data quality, the object identification step described below may sufficiently exclude erroneous data points.

The points in the 3D point cloud may originate from different objects and noise sources within the scene. In order to estimate the respiration rate of a single or even multiple humans in the scene simultaneously and with sufficient accuracy, the method must be able to separate and assign the points into different groups. The groups may represent noise or objects of interest. The identified objects must then be tracked across consecutive frames in order to isolate and extract their continuously varying movement information.

According to the invention object identification techniques (shown as item <NUM> in <FIG>) are applied in order to segment 3D point clouds into different objects, such as persons. Given a set of points, the objective of the object identification process is to group points by spatial proximity into homogeneous regions. These isolated regions, by their characteristics, represent different objects in the scene. Grouped points that are identified to derive from chest expansions may be isolated and used as input to the repository rate model. This approach allows for separation and monitoring multiple targets, such as patients within the scene.

In a preferred embodiment the method of the invention implements the use of DBSCAN as object identification algorithm for this step. This alternative is illustrated as item 4A in <FIG>. DBSCAN is a widely used object identification algorithm that is known per se, and it has several advantages which makes it a good choice for this grouping problem:.

DBSCAN is a density-based grouping algorithm where it is only needed to specify the minimum number of points N in each group and the maximum distance ε between neighbouring samples. This algorithm is, e.g., described in <NPL>.

The number of groups can be arbitrary large and do not need to be predefined. This allows for simultaneous tracking of one or several persons in the room without prior knowledge of how many persons are in the room.

The algorithm can identify less obvious noise based on the positions of the points.

DBSCAN do not require the group to have a specific geometry. This also makes It more robust to different lying and sitting positions of the person associated with the group.

The algorithm of DBSCAN can be summarized as follow:.

<FIG> illustrates the DBSCAN algorithm with two groups and one noise point. Around each point a circle with radius ε is drawn around each of the points. For the points 10a, 10b and 10c, all of the points fall within the radius ε of each other, while none of the other points wall within this radius. Consequently, these points clearly belong to the same group.

Points 11a, 11b, 11c also fall within each other's radius ε and clearly belong to the same group. Point 11d falls outside of the radius ε of points 11a and 11b, but within the radius ε of point 11c. Consequently, point 11d is also defined as a part of the same group.

Point <NUM> does not fall within the radius ε of any of the other points, and hence point <NUM> does not belong to any of the groups and is defined as noise.

Other grouping algorithms may alternatively be used to group the 3D point-cloud data, as illustrated by item 4B in <FIG>. Examples of other known per se algorithms are:.

Important characteristics of theses grouping approaches include the ability to detect the number of groups automatically and find outlier points, i.e., points with an insufficient number of neighbours in proximity. These algorithms are likely to produce comparable results as the DBSCAN algorithm, albeit with variations in their characteristics regarding complexity, speed and memory usage.

After the at least one object has been identified according to the above step, the objects must be tracked in order to follow the movement of the person(s).

Grouped points are represented with a minimum bounding box, which is the smallest box that contains the points identified as belonging to one group in the 3D space. Instead of a box, other shapes can be chosen, such as a sphere, a polyhedron or an irregularly shaped 3D boundary. A box, however, is a simple geometric shape that can be represented by few parameters, which requires limited computational power. <FIG> shows a 3D point cloud where some points have been found to belong to the same object and have been assigned a minimum bounding box that just encloses all the point of the object.

Objects are then tracked across frames by testing for overlapping geometries. In other words, grouped points found in two consecutive frames along the time axis can be determined to originate from the same object if the minimum bounding box from the grouped points overlap across the two frames. The method for object tracking is summarized as follows:.

The method assumes that the update frequency of the radar is sufficiently high so that the geometry of the grouped points that originate from the same object will always overlap across consecutive frames even if the object moves randomly at normal walking speed. This means that the frame rate has to be sufficient high that a person is not capable of moving a longer distance than the width of the minimum bounding box between two consecutive frames. For tracking human beings, a frame rate of <NUM> fps is a typical choice.

This method makes it possible to dynamically isolate and monitor objects in the room and detect if objects moves, enters or leaves the room. The geometrical properties of the minimum bounding boxes can also be leveraged in combination of the tracking procedure to keep objects from merging at close distances.

Next step of the method of the invention is to apply an aggregate function. Aggregation functions are well known per se (see for example https://www. investopedia. com/terms/a/aggregate-function. asp ), but according to the invention it is used to group and form the grouped velocity data into a single summary signal that captures all the relevant movements of the group.

The large movements (chest expansion) that occurs during respiration will increase the size of the group, and thus also the sum of magnitude points will increase. Many of these points will not necessarily have a high magnitude, and thus a summation of the points is a good aggregation function for respiration rate estimation. Other examples of aggregation functions include average, median, maximum, minimum and range. Average and median use the average of all items and the median item respectively to obtain the single summary signal, maximum and minimum uses the item with maximum and minimum value respectively while range uses the difference between the maximum and the minimum value.

<FIG> illustrates the aggregate function. At the top row of the figure are bounding boxes of several consecutive frames shown. Both the shape and size of the boxes may vary slightly from frame to frame. In this step, however, the important parameter is the combined speed, both in absolute value and direction, the important factor. Through the aggregate function, which is illustrated by the cog wheels in the second row, a graph of the combined velocity of the bounding box in each frame is plotted. In this example, the graph is a sinus curve, which would be typical when respiration is tracked. From the graph both the frequency and amplitude of the respiration can be deducted.

The next step of the method of the invention is optional but may greatly enhance the signal quality. This step is a dynamic focus adjustment, illustrated by item <NUM> in <FIG>. In this step the focus parameters of the radar sensor are focused dynamically in order to maximize signal quality of the subsequent point-cloud dataframe. When a set of points, such as objects <NUM> and <NUM> in <FIG>, are identified to belong to human beings, the focus area of the radar is centred to the centroid, i.e., the arithmetic mean position of all the points, of the bounding box.

If the system detects that one of the groups has a respiration or heartbeat outside of the normal range, the radar can focus mainly on that group to increase the resolution and improve data quality. Hence, a beginning critical condition, such as a heart attack, can be detected at an early stage.

In order to detect new persons arriving onto the space, such as a patient room, the radar will at regular intervals scan the whole space in search for new groups. When a new group is detected, the system will include that group in the focused areas.

If a person leaves the space, the group will disappear, and the system can exclude the area that that group occupied from the focused areas.

The system may also be set up to determine a likely orientation of the subject. When breathing the characteristics of the points in the group will be different if the subject, such as a person, is facing the radar or having its side towards the radar. The characteristics will again be different if the person is lying down. If the person has the back towards the radar, there will be a greater spread of points moving than if the person is facing the radar with the chest. The same techniques as explained below may be used in determining the orientation of the subject.

It is possible to equip persons with a radar reflector that when detected tells the system that this person is a caretaker. The system can then exclude the group that that person represents from the focused areas or reduce the resolution of 3D data points for that group. Alternatively, patients can be equipped with a radar reflector to tell the system to increase the resolution for the group representing that person. Visitors may also be equipped with such radar reflectors.

The focus area, or a part of the focus area, will shift in space as a person moves. This movement can be detected in various ways, such as by monitoring velocity vectors of the data points of the group or letting the focus area cover a limited portion outside the group.

By dynamically adjusting the focus area per frame to the bounding box centroids it is possible to track objects within the scene and increase signal quality. Multiple targets may also be tracked by rapidly shifting focus between them. An alternative approach to track multiple targets is expanding the focus area to cover several groups. This will however be at a possible cost of decreased spatial resolution and decreased signal-to-noise ratio. This way of dynamically focusing the radar could be achieved either through software or hardware.

The radar that is contemplated to be used in in connection with the method of the present invention is preferably of a type that can detect both regular Doppler points and micro-Doppler points. Regular Doppler points are frequency shifts of large motion movements of a target relative to the detecting radar. In addition to the large motions, the target undergoes micro-motion dynamics such as rotations and vibrations. These micro-motion dynamics induce Doppler modulations on the returned Doppler signal and is referred to as the micro-Doppler effect. A further explanation of this phenomenon can be found at https://www. researchgate. net/publication/3003947_Micro-Doppler_Effect in_Radar_Phenomenon_Model and Simulation Study. The points generated by the micro-Doppler effect are especially suited for detecting body micromotions such as respiration and heartbeat.

The radar contemplated to be used has the ability to adjust the sensitivity towards both regular Doppler points and micro-Doppler points while the radar is operational. Adjustment of, e.g., Doppler sensitivity will not affect the micro-Doppler points in itself, but the maximum number of possible Doppler and micro-Doppler points is limited by available processing power.

Focusing through software could be achieved by using a combination of both micro-Doppler and regular Doppler points. Regular Doppler points are used to track a subject's movements by creating a bounding box which defines the location of the subject. When the radar detects a stop in movement of the bounding box, the coordinates of the last known location of the bounding box is saved. The sensitivities of the regular and micro-Doppler points are then adjusted to be more sensitive towards micro-doppler points. This enables processing of a higher number of the points that are used for detecting respiration and heart rate and reduces the number of large motion points, which could introduce noise in the signals used by the estimation methods.

After adjusting the sensitivity, points outside the last known location of the bounding box are filtered out, leaving only points belonging to the subject. This step further removes micro-Doppler noise from the surrounding area, resulting in a cleaner signal that would increase the performance of the estimation methods. The remaining points after filtering are then used for estimating respiration and heart rate. Consequently, a higher resolution is achieved of the respiration and heart-beat monitoring and at the same time noise is reduced.

Even though the focus of the system is on the data points belonging to the group of micro-Doppler points, the system of the invention will still monitor enough regular Doppler points to be able to determine if the subject has moved as a whole, i.e., the Doppler shifts of several data points is larger than the range expected by the micro-Doppler shifts. When it is determined that the subject is moving, the system may adjust the sensitivity to be more sensitive towards the larger Doppler shifts in order to track the movement of the subject.

The above process could also be applied to multiple subjects allowing for focusing on multiple subjects at the same time. This would of course mean that as there is a maximum number of points that can be processed by the system per second, the resolution for each subject will be reduced. However, depending on capacity of the processor, the resolution may still be sufficient to monitor two or more subjects at the same time.

Focusing through software could be achieved by, but is not limited to, adjusting the time delays of the elements of either the transmitter (TX) or the receiver (RX). <FIG> shows dynamic focusing with time-delays applied at the TX. The applied time-delays causes the elements to transmit waves at different times, sending the combined wave front in the direction of the desired focal point. <FIG> shows an illustration of how dynamic focusing through software could be performed on the point cloud. From figure 8a, through figure 8b, to figure 8c, the bounding box has moved towards the radar, hence the focus of the radar shifts to focus at a closed distance.

Another way of achieving a dynamic adjustment of the focus is through hardware. The radar could be mounted onto some controllable hardware, such as but not limited to a robotic arm, gyroscope, servomotor, hydraulic actuators, vacuum actuators and/or other mechanical actuators. By controlling the angle of the actuator, the radar beam could be focused in the direction of the point cloud.

From this point onwards, the objects of the invention can be realized in two different ways, illustrated by the two branches 7A and 7B, respectively, in <FIG>, or they can be realized by performing the steps of both branches 7A and 7B on the dataset, after which the results are combined in a step <NUM>. This will be explained below.

The branch 7A is a frequency extraction, which is based on traditional methods. Traditional signal processing methods can be used to detect the respiration with satisfactory results. The algorithm preferably used in the present invention relies on a technique called multitaper to find dominant frequencies by estimating the spectral density of the signal. Multitaper is one of multiple methods to estimate the spectral density of a random signal from a sequence of time samples of the given random signal. Other examples of techniques for spectral density estimation are least-squares spectral analysis, non-uniform discrete Fourier transform, singular spectrum analysis, short time Fourier transform, and critical filter.

The estimated spectral density of a signal describes the distribution of power into the frequency components and is used to detect periodicities in the signal. Periodicities are found by seeking intensity peaks at different generated frequency components. This includes the peaks from the periodic movement of respiration and heartbeats. The respiration rate and heart rate can be estimated by seeking the dominant frequency components in the velocity signal within non-overlapping frequency ranges. Spectral density estimation techniques may also be performed on time samples from the velocity signal from multiple groups independently and allow tracking of the respiration rate and heart rate from multiple humans in the scene simultaneously. The frequency range of respiration rate and heart rate will be different, and there are likely to be two large periodic components in the signal: a strong lower frequency component due to respiration (<NUM> to <NUM> bpm) and a strong higher frequency component due to heart beats (<NUM> to <NUM> bpm).

The detrending and filtering step 7A2 is performed on the group signal in order to maximize the quality of the spectral density estimate. Trends are in this case not considered to be a property of the interesting signal and can be removed by subtracting the signal with its mean value.

Detrending reduces the signal component at low frequencies. A low-pass and high-pass Butterworth filter are then applied in order to smooth out noise from the signal, remove unwanted frequency components and separate the respiration signal and the heartbeat signal before estimating the spectral densities. <FIG> shows an exemplary input respiration signal before filtering. <FIG> shows an example of a Butterworth filter adapted to filter respiration signals. As a normal breathing pattern has a frequency of about <NUM>,<NUM> - <NUM>,<NUM>, the filter has a high gain around this frequency.

<FIG> shows the signal after being filtered with the filter of <FIG>. As can be seen here, the breathing pattern has become clearly visible.

The next step of branch 7A is to perform multitaper, as illustrated by step 7A2. A background on how this method in general works can be found in:
<NPL>.

One of the simplest techniques to estimate spectral density is the periodogram. The periodogram is the frequency spectrum of a set of time signals and can be defined as the Fourier transform of the autocorrelation. If it is assumed that a sample of size N of the random signal xk for k = <NUM>,<NUM>, <NUM>,. is given, an approach to obtain a nonparametric estimate of the autocovariance sequence sk is the method of moments, where the estimate can be based on the sample mean and sample autocovariance <MAT> where µ̂ denotes the sample mean, and <MAT>.

Taking the Fourier transform of the estimates above gives us the periodogram: <MAT> where Ŝ(f) is an estimate of the true power spectral density S(f) of a random signal xk, Δ denotes the sampling interval, f denotes the frequency and the Fourier transform of xk is defined as <MAT>.

The multitaper method is based on the periodogram but improves further on it by using multiple windows to form independent estimates of the spectral density. If we let <MAT> be a set of tapers, the multitaper (mt) spectral estimate for L windows is given by <MAT> where <MAT>.

The approach of combining the results from multiple windows reduces the variance of the final spectral density estimate.

An example of a multitaper spectral density estimate derived from a test subject with a respiration rate of <NUM> breaths per minute is shown in <FIG>.

Next in the method of the invention, the second branch 7B of <FIG> will be described. In this branch, the system uses machine learning to optimize the respiration and/or heartbeat signals. The application of machine learning to solve computational challenges has emerged rapidly over the last years due to the combination of improvements in algorithms and accessibility of computing power. In contrast to programming an explicit solution, machine learning trains a model to solve the problem by feeding an algorithm with historical data. This allows the model to learn complex patterns for modelling the data, which often would be infeasible to program explicitly. For the challenge of modelling respiration based on 3D point cloud data, is needed to collect labelled data. As illustrated in <FIG>, a patient <NUM> is coupled to a patient monitor <NUM>. The radar <NUM>, which is coupled to a processor <NUM> transmits and receive radar data <NUM>. The processor <NUM> stores processed data in a training database <NUM>. Data from the patient monitor <NUM> is also transferred to the database <NUM>. According to the invention this collection of large amounts of labelled data from the patient monitor <NUM> is synchronized against the radar data. The data is recorded under a range of demanding conditions (such as long distances, introduction of noise).

The synchronized dataset is then used to train the model to extract respiration directly from the 3D point cloud input data. The application of machine learning for this purpose improves the quality and robustness of the detections, especially under challenging conditions. The main contribution of using a machine learning method for this purpose is that it can learn to model the signal under a wide range of otherwise challenging conditions, provided it is trained with data that represents similar conditions that it is trained to model.

A regression analysis, as illustrated by item 7B1 in <FIG>, is a set of statistical processes of estimating relationships between an outcome variable and one or more features. The desired output of the model is detection of respiration; thus, regression methods are a good fit.

There are a wide range of well-known regression methods that can use the movement information of groups to predict respiration. An overview of these can be found in the following article: https://www. listendata. com/<NUM>/<NUM>/regressionanalysis.

Next gradient boosting, as illustrated by item 7B1A is used to produce a prediction model. Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. It builds the model in a stage-wise fashion, and it generalizes them by allowing optimization of an arbitrary differentiable loss function.

Gradient Boosting is well-described in<NPL>. <FIG> illustrates of how gradient boosting works with decision trees.

In the first iteration shown in <FIG>, a single decision tree is built to fit the training data. In this iteration data points that readily can be identified as good data (in the example having an x value below <NUM>) is classified into one group, denoted by "yes" and the rest of the datapoints that cannot readily be classified, are sorted into another group, denoted by "no".

Next, in a <NUM>nd iteration a new decision tree is built, focusing on the part of the training data that the previous model made a bad prediction on, i.e., in the "no" group. In the example the focus is here on datapoints that readily can be identified as having a y value larger than <NUM>.

The two models of the previous iterations are then combined into a single ensemble model, and the second iteration step is executed again.

In the <NUM>rd iteration, datapoints classified as bad data (labelled with "no") in the second iteration are sorted again. In the example with the criterion x smaller than <NUM>. This creates a third decision tree.

The process is repeated, where each new decision tree will attempt to correct the errors made by the combined ensemble model of previous decision trees.

The process is repeated until it starts overfitting, or the sum of the error residuals become constant.

Several Gradient Boosting algorithms are publicly available as open source implementations:.

Item 7A1B in <FIG> illustrates these alternatives in the method of the invention.

In addition to gradient boosting, several alternative methods to solve the regression problem exists. These include Random Forest Regression, Artificial Neural Network regression, K-nearest neighbour-regression, linear regression, logistic regression, ridge regression, lasso regression, polynomial regression, stepwise regression, ElasticNet regression, principal component regression, binomial regression, binary regression, Poisson regression, support vector machines (SVM), naive Bayes and expectation maximization (EM) algorithm.

Random Forest Regression is an ensemble method that operate by constructing a multitude of decision trees at training time. However, Random Forest Regression averages the predictions from all the individual regression trees at prediction time. This is illustrated by item 7B in <FIG>.

Artificial Neural Networks (ANN) are composed of individual "neurons", which has one or more input(s) and produce a single output. The neurons are organized in several layers, and the outputs from the final layer accomplish the regression task. Such ANN could be composed of linear layers, convolutional layers (CNN) and/or recurrent layers (RNN). These networks can be used for regression if the final output node of the network is configured to produce a single continuous value.

A final step of the method of the invention in reaching an optimized signal by the steps of branch 7B is to use an algorithm called CatBoost, as illustrated by item 7B2 in <FIG>.

CatBoost is a state-of-the-art machine learning algorithm based on Gradient Boosting. CatBoost introduces two critical algorithmic advances compared to existing solutions: the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. CatBoost is also significantly faster than existing methods, making it highly suitable for latency-critical tasks. This is especially true when running on a Graphical Processing Unit (GPU), which is supported out of the box.

In the next step, illustrated by <NUM> in <FIG>, is an ensemble learning technique used. In machine learning, ensemble learning describes the use of multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. This is generally described in:
<NPL>.

In a further embodiment, input from further radars can be combined with the data from the first radar. More specifically, similar processes as illustrated by the branches 7A and 7B, and explained above, are performed on the data from the further radars. The processed data from these radars are then used as input in the ensemble step in addition to the data from the first radar and hence subjected to the same ensemble learning method as described herein. This may further improve the reliability of the results. This could be particularly useful in scenarios where multiple radars are installed with an overlapping coverage area. The output from multiple radars would provide a larger confidence in the final predictions. <FIG> illustrates that the outputs of branches 7A and 7B, according to the invention, could be combined and used as input to a second machine learning model, chosen between the methods described in 7B, in order to reduce the overall error of the modelled respiration signal. This technique will learn to weight the two input signals differently to produce the final output signal.

In the following will be described experimental results using the method of the invention. Here will be described dataflow during the experiments, how the different experiments are set up and lastly, will be presented the results from the improved algorithm of the invention.

<FIG> shows the experimental setup and the dataflow during the experiments. The setup is similar to the setup of <FIG>.

In <FIG>, a subject <NUM> was connected to a patient monitor <NUM> called Recobro Vigile™ through a <NUM>-lead electrode setup. This patient monitor is a state-of-the art medical device that can be used to measure vital signs such as heart rate, respiration rate. The output of this patient monitor is used as reference when comparing the proposed method of the invention to existing methods. The radar <NUM> records 3D pulse-Doppler datapoints that are processed on a sensor CPU <NUM> by the proposed algorithm of the invention, as described above. Both the output from the CPU <NUM> and the patient monitor <NUM> are stored in a database <NUM>. Here the output from the method of the invention is compared to the vital signs from the patient monitor <NUM>.

Any type of high-quality equipment that can detect respiration or heartbeat rate can be used instead of the specific patient monitor mentioned above.

In a first experiment, a test subject was placed sitting in a chair, with the chest facing the radar <NUM>. The same experiment was performed at all the distances illustrated in <FIG>, namely <NUM>,<NUM>; <NUM>,<NUM>; <NUM>,<NUM>; <NUM>,<NUM>; and <NUM>,<NUM> distance from the radar <NUM> to the chest of the person. The results of this experiment are labelled <NUM>-<NUM> in the table of <FIG>.

In a second experiment, illustrated in <FIG>, one test subject and an additional person were sitting next to another with a separation of <NUM>. The distance to the radar was <NUM>,<NUM>, with the radar mounted in the ceiling vertically above the two persons. The results of this experiment are labelled as <NUM> in the table of <FIG>.

In a third experiment, illustrated in <FIG>, a test subject was lying on a bed with the radar vertically above at a distance of <NUM>,<NUM> from the chest of the subject. Another subject was walking around in the room next to the test subject. The results of this experiment are labelled as <NUM> in the table of <FIG>.

In a fourth experiment, illustrated in <FIG>, a test subject was lying in a bed with the radar <NUM>,<NUM> vertically above the chest. Another person was lying in another bed at the same elevation level <NUM> from the test subject. The results of this experiment are labelled as <NUM> in the table of <FIG>.

For all experiments were used a <NUM> mm-wave radar.

The table of <FIG> shows in the first column the labels of the experiments above. The second column indicates the distances from the radar to the chest of the test subject. The third gives the median difference in respirations per minute between the respiration measurements done according to the present invention and the accurate measurements done by the patient monitor <NUM>. The fourth column gives the interquartile range (lOR) at the 75th percentile between the measurements done according to the present invention and the accurate measurements done by the patient monitor <NUM>, while the final column gives the same at the 95th percentile.

As is evident from the results shown in <FIG>, the method of the invention performs very well. The difference between the actual respiration rate and the respiration rate determined by the system of the present invention is low, and it is fairly consistent. The distance between the radar and the test subject does not seem to have any significance. The ability to model respiration at longer distances is shown in <FIG>, achieving good results on distances up to <NUM> meters. The minor differences are attributed to variations in the behaviour of the test subjects in each experiment. Consequently, the method of the invention enables respiration detection and estimation for different scenarios, including single- and multi-person scenarios, under various conditions. The method is robust against noise and gives the ability to identify and track objects confidently at various distances. <FIG> shows the difference in respiration rate versus distance as a graph.

<FIG> shows the ability of the system of the invention to model respiration under noisy conditions. As can be seen from this diagram, there are no substantial differences in the results achieved in experiments with added "noise" as described in experiment setups <NUM> and <NUM> above, compared to the results of experiment <NUM>, which is without noise added. The minor differences are attributed to variations in the behaviour of the test subjects in each experiment.

<FIG> shows a 3D point cloud with bounding boxes arranged around the points that have been identified to belong to specific subjects according to experiment setup no. The ability of the method to perform multi-person detection and respiration rate estimation is shown here. The method of the invention can both capture multiple persons, as illustrated by the bounding boxes, and the method is also able to estimate respiration rates, as shown by the values assigned to each of the boxes (at the top of the figure).

Claim 1:
A method of optimizing a dataset from a phased-array radar to detect movement of subjects (<NUM>) within a coverage area of the radar, wherein said radar creates a 3D point-cloud dataset (<NUM>), said dataset (<NUM>) being fed into an object identification algorithm (<NUM>) to create at least one group of datapoints (10a, b, c; 11a, b, c) characterized by their proximity and their pattern of movement, defining that said at least one group (10a, b, c; 11a, b, c) belongs to a single subject (<NUM>) and tracking movement of datapoints of said group (10a, b, c; 11a, b, c); wherein after said point grouping, defining a minimum bounding shape, such as a box, around said at least one group (10a, b, c; 11a, b, c) and tracking said minimum bounding shape across frames of 3D point clouds taken at different consecutive times, wherein said tracking involves the following steps:
i. calculate the minimum bounding shape for each group of points in a frame,
ii. determine if any of the minimum bounding shapes found in a current frame overlap with any of the minimum bounding shapes from a previous frame,
iii. keeping track of overlapping objects and temporal movement variations, wherein said
tracking of said overlapping objects comprises:
a. updating a geometry of objects that overlaps between said current and said previous frames with a new minimum bounding shape and track a change in velocity of said minimum bounding shape,
b. add any group that does not overlap with any minimum bounding shapes from said previous frame as a new object, and
c. remove any object from said previous frame that does not overlap with any group in said current frame.