Patent Description:
Technologies which operates by weight or pressure sensing, acoustic radar (occupancy sensor), radio frequency (RF) radars, 2D- and 3D-imaging, and thermal imaging, have been used to detect vehicle occupants seat occupancy monitoring for seat belt remainders, and infant detection, for forgotten/left infants in vehicles.

<CIT> (IEE International Electronics & Engineering S. ) discloses radar for sensing of vehicle occupancy, which uses fixed filters for a radar signal.

<CIT>) discloses using fixed high pass and low pass filters to isolate a cardiac signal to detect heartbeat characteristics.

<CIT>) discloses a heart rate estimating system which uses a fixed band pass filter centered about a determined peak heart rate frequency. The band pass filter allow for accuracy in extracting the heart rate in view of fluctuating heart beats.

The present invention is directed to systems and methods for detecting vital signs of motor vehicle occupants, such as the occupants in the vehicle cabin, in dynamic environments. The detecting is performed, for example, by using radio frequency (RF) radar, to detect vital signs, based on the radar-based detection and monitoring of, for example, one or more of: respiratory or breathing rates (RR), heart rate (HR), hart rate variability (HRV) and speech state recognition. The detection of vital signs is also used to determine the presence of an occupant in the vehicle and/or the number of occupants in the vehicle, regardless of the occupant's location in the vehicle. Additionally, the present invention is that the system detects vehicle occupants out of position (OOP). Should such an OOP detection be made, this may indicate a vehicle accident, or an incapacitated occupant.

The present invention discloses methods and systems for detecting vital signs of occupants in vehicles, for example, the vehicle cabin. A signal unit transmits a radar signal to the occupant and receiving the radar signal reflected from the occupant. The reflected radar signal is analyzed with respect to vibration data of the vehicle, to produce a modified signal. The modified signal is analyzed to determine the vital signs of the occupant.

The present invention is directed to detecting vital signs, for example, breathing or respiratory rate, heart rate, and heart rate variability, of vehicle cabin occupants from the seats of the vehicle cabin.

The present invention is directed to signal units with multiple paths for the received radar signals, and each of the paths has its own analog amplification and filtering (levels/edges) which are adjusted based on the driving conditions (e.g., vibrations associated with the movement of the vehicle).

The present invention is such that the system monitors the vital signs of the driver of the vehicle during driving.

The present invention detects occupant vital signs and based on the vital signs detects driver conditions such as driver drowsiness, falling asleep and the like.

The present invention is such that the sensor units thereof may be located on the vehicle dashboard or integrated into it. The sensor units define a system, which may be independent of the vehicle's systems or can be integrated into the vehicle's systems.

Embodiments of the invention are directed to a method for determining the vital signs of an occupant in a vehicle. The method comprises: transmitting at least one analog radar signal to the occupant and receiving the at least one radar signal reflected from the occupant; obtaining vibration data including vehicle movement data and vehicle acceleration data from at least one sensor in the vehicle; filtering the received reflected at least one analog radar signal by adjusting at least one low-pass filter and at least one high-pass filter so that the cut-off frequency of the high-pass filter is the expected basic frequency of at least a heart rate of the occupant and the cut-off frequency of the low-pass filter is equal to at least sixteen multiples of the heart rate frequency, to produce a modified analog signal; converting the modified analog signal to at least one digital signal; and analyzing the at least one digital signal to determine the vital signs of the occupant.

Optionally, the method is such that the vital signs include one or more of breathing rate, heart rate and heart rate variability.

Optionally, the method is such that the radar signal is from Doppler radar.

Optionally, the method is such that the vital sign to be measured is breathing rate of the occupant and the radar signal reflected from the occupant results in a signal based on breathing rate harmonics.

Optionally, the method is such that the vital sign to be monitored includes heart rate.

Optionally, the method is such that heart rate is determined by processes including: obtaining the modified signal; dividing the modified signal into segments, each segment corresponding to a frequency, analyzing a plurality of peaks of the segment for harmonics, including, for each peak; applying weight factors to each of the harmonics; accumulate the energy from the harmonics as multiplied by the weight factors; and, determining the peak with the highest accumulated energy.

Optionally, the method is such that the peak determined to have the highest accumulated energy corresponds to the heart rate.

Optionally, the method is such that the determining heart rate variability includes the processes of: obtaining the modified signal; determine the artifacts in the modified signal; analyzing the modified signal for consecutive peaks between the artifacts; and, determining a portion of the modified signal with at least a predetermined number of consecutive peaks; and, calculating the heart rate variability parameters from the modified signal with at least a predetermined number of consecutive peaks.

Optionally, the method is such that it additionally comprises: dividing the reflected at least one analog radar signal into a first pathway for respiration rate frequencies and a second pathway for the heart rate frequencies, prior to the analyzing of the reflected at least one analog radar signal.

Embodiments of the invention are directed to a method of decreasing the impact of movement by a subject on the heart rate measurements for the subject, by filtering the heart rate fundamental frequency and determining the signal by analyzing the signal harmonics.

Embodiments of the invention are directed to a method of decreasing the impact of movement by a subject on the heart rate measurements by focusing proximately positioned radar at the aorta area.

Embodiments of the invention are directed to a method of decreasing the impact of movement by a subject on the breathing rate measurements by focusing proximately positioned radar at the diaphragm area.

Optionally, the method of decreasing the impact of movement by a subject on the breathing rate measurements is such that the aorta area is between the LI and L5 vertebrae.

Optionally, the method of decreasing the impact of movement by a subject on the breathing rate measurements is such the vibration data is obtained from an inertial measurement unit.

Embodiments of the invention are directed to a system for determining the vital signs of a subject. The system comprises: a radar transceiver for transmitting at least one analog radar signal to the occupant and receiving the at least one analog radar signal reflected from the occupant; an analog signal filter comprising at least one low-pass filter and at least one high-pass filter; a vibration detection unit for detecting vibrations associated with the vehicle and providing vibration data representative of the vibrations, including movement data and acceleration data of the vehicle; a signal converter for converting analog signals to digital signals; and, a processor in electronic communication with the radar transceiver, the analog signal filter, the vibration detector, and the signal converter, the processor programmed to: a) adjust the analog signal filter so that the cut-off frequency of the high-pass filter is the expected basic frequency of at least a heart rate of the occupant and the cut-off frequency of the low-pass filter is equal to at least sixteen multiples of the heart rate frequency, to produce a modified analog signal to produce a modified analog signal, prior to the modified analog signal being converted to at least one digital signal by the signal converter, and b) analyze the at least one digital signal to determine the vital signs of the occupant.

Optionally, the system is such that the processor programmed to analyze the at least one digital signal to determine the vital signs of the occupant, determines the vital signs including one or more of breathing rate, heart rate, and heart rate variability.

Optionally, the system is such that the vibration detection unit includes and inertial measurement unit (IMU).

Optionally, the system is such that it additionally comprises: a filtration and amplification circuit including the analog signal filter, the filtration and amplification circuit in electronic communication with the radar transceiver and the signal converter, including two passband pathways for separating respiratory rate frequencies and heart rate frequencies of the reflected at least one analog radar signal.

Optionally, the system is such that the radar transceiver, the signal converter, the processor and the vibration detection unit define a single sensor unit.

Embodiments of the invention are directed to determining a vehicle occupant based on vital signs. The method comprises: transmitting at least one analog radar signal to vehicle cabin and receiving the reflected at least one analog radar signal; obtaining vibration data including vehicle movement data and vehicle acceleration data from at least one sensor; filtering the received reflected at least one analog radar signal by adjusting at least one low-pass filter and at least one high-pass filter so that the cut-off frequency of the high-pass filter is the expected basic frequency of a heart rate of the occupant and the cut-off frequency of the low-pass filter is equal to at least sixteen multiples of the heart rate frequency, to produce a modified analog signal; converting the modified analog signal to at least one digital signal; analyzing the at least one digital to determine the presence of vital signs of an occupant in the vehicle cabin; and, should the vital signs be present, an occupant has been detected in the vehicle cabin.

Embodiments of the invention are directed to a method for determining the minimum gain level in a filtering and amplification circuit. The method comprises: generating a harmonic waveform; transmitting the waveform and receiving the reflected waveform; and, modifying the gain level of an amplifier in the filtering and amplification circuit to detect the reflected waveform.

Optionally, the method is such that it is performed in a vehicle cabin.

Optionally, the method is such that the vehicle cabin is empty of occupants.

Embodiments of the invention are directed to a method for determining the number of occupants in a vehicle. The method comprises: transmitting at least one analog radar signal into the vehicle cabin, and receiving the at least one analog radar signal reflected from one or more occupants; obtaining vibration data including vehicle movement data and vehicle acceleration data from at least one sensor; filtering the received reflected at least one analog radar signal by adjusting at least one low-pass filter and at least one high-pass filter so that the cut-off frequency of the high-pass filter is the expected basic frequency of at least a heart rate of the occupant and the cut-off frequency of the low-pass filter is equal to at least sixteen multiples of the heart rate frequency, to produce a modified analog signal; converting the modified analog signal to at least one digital signal; analyzing the at least one digital signal to determine vital signs associated with one or more occupants in the vehicle cabin; and, based on the number of vital signs detected, determining the number of occupants in the vehicle cabin.

Some embodiments of the present invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention.

Attention is now directed to the drawings, where like reference numerals or characters indicate corresponding or like components. In the drawings:.

Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and/or methods set forth in the following description and/or illustrated in the drawings.

" Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more non-transitory computer readable (storage) medium(s) having computer readable program code embodied thereon.

Throughout this document, numerous textual and graphical references are made to trademarks. These trademarks are the property of their respective owners, and are referenced only for explanation purposes herein.

The invention includes a system comprising sensor units, which include RF radar transceivers, including one or more antennas, which are connected with Transmit (TX) and Receiver (RX) blocks. TX power, operation frequency, waveform and RX Gain are configured by control signals, which come from Digital to Analog Converters (DACs) and Power supply voltages. The RX block (blocks) includes down-converters to intermediate frequency (IF) outputs. The RF radar transceiver is coupled to an amplifying and filtering analog circuit or block. The gain and the IF pass band of the analog block are configured by corresponding control signals, which originate from dedicated DACs. Outputs (output) of the analog block are connected to inputs of Analog to Digital Converter (ADC), having a digital interface with the central processing unit (CPU).

The system may have an Inertial Measurement Unit (IMU), for acceleration and/or angle measurements which will affect amplification levels and filtering edges. The IMU is used to monitor the vibration level of the dynamic environment, for example, the vehicle vibrations resulting from travel over roads, such as highways, off-road trails, surface streets, various road pavings (even and uneven surfaces), turns, internal and external noises.

The CPU is also connected with DACs, as well as other sensors (for example, mechanical vibration, temperature, doors and motor state of the vehicle). Functions of the CPU, include, for example, the calibration of RF transceiver and the analog blocks, and storage calibrated parameters, the detection of desired signals from the objects that relate to respiration and heartbeat, by optimal digital filtering of main harmonics of corresponding signals, using weight coefficients for each harmonic, which takes into account different interferences, such as movement of the objects, object speaking, and the like, detection of the signal parameters from objects related to the occupation status, using adaptive signal thresholds, which takes into account calibration factors and relationships between signals, which are received from different antennas.

The RF radar is, for example, Doppler radar, and operates in one or more modulations including, for example, continuous waveform (CW), FM/PM/AM/Pulse modulations.

During operation of the system, the RF radar transceiver generates radio waves, which propagate from the radar transmission antenna to the objects. The signals are reflected from the object and collected by receiver antenna(s) of the RF radar modules.

The receiving signal after a digitization is received by the CPU, which applies algorithms to process these received and now digital signals. The processing also accounts for vehicle parameters, such as those for vibrations, when determining breathing rates, heart rates, heart rate variabilities and driver activity, or vehicle cabin occupants.

The present invention is directed to a contactless detection and monitoring system of vital signs of vehicle occupants, which uses RF (Radio Frequency) radar to detect human or animal body vibrations/oscillations, relative to that vibration/oscillations calculations made by the system for heart rate and respiratory rate. <FIG> shows an exemplary environment for the invention. Within a vehicle <NUM> are apparatus <NUM>, known hereinafter as sensor units 101a-101i (<FIG>), which include, for example, radar transceivers <NUM> (<FIG>). These sensor units 100a-100i are linked to a network(s) <NUM>, for example, via a cellular tower <NUM>, WiFi® or the like, so as to be linked to a home server (HS) <NUM>, or main server, which together with the sensor units 101a-101i forms a system. Via the network(s) <NUM>, the home server <NUM> is linked to a multitude of other servers, devices, and the like, such as servers associated with first responders <NUM>, e.g., police, fire, ambulances, government <NUM> and governmental agencies authorities, and the like, statistical organizations <NUM>, and storage media <NUM>, such as cloud storage.

The network <NUM> of <FIG> is, for example, a communications network, such as Bluetooth®, Zigbee, Zwave, LORA, V2X, and a Local Area Network (LAN), or a Wide Area Network (WAN), including public networks such as the Internet. The network <NUM>, although shown as a single network, may be a combination of networks and/or multiple networks including, for example, in addition to the Internet, one or more cellular networks, wide area networks (WAN), and the like. "Linked" as used herein includes both wired or wireless links, either direct or indirect, and placing the computers, including, servers, components and the like, in electronic and/or data communications with each other.

Turning to <FIG>, the vehicle <NUM>, for example, the cabin 101x of the vehicle <NUM> includes sensor units 101a-101i therein. The sensor units 101a-101i are mounted, for example, on the vehicle dashboard 101a, on the ceiling 101b, 101f, <NUM>, within seats 101c1-101c3, 101e1-101e3, rear view mirror 101d, behind the seats <NUM>, the trunk 101i or baggage compartment, and the like. The sensor units 101a-101i are spaced in the vehicle cabin 101x to provide coverage of the entire vehicle cabin 101x. Each of the sensor units 101a-101i is typically used for providing specific applications for each of the various occupants. For example, seat mounted sensor units 101c1 to101c3 and 101e1 to 101e3 are used in detecting vital signs of the occupant of the respective seat 101y, 101z (as well as detecting seat status, e.g., occupied/unoccupied, by detecting vital signs of an occupant), as well classifying the occupant, such as man, woman, child, pet, and state, e.g. fatigue, stress, drunkenness, drowsiness, of each detected occupant in the seats 101y, 101z, as well as driver speech state recognition, and a seat belt reminder (SBR) for the occupant (once the seat is determined to be occupied). The vital signs detected include, for example, breathing or respiratory rate (RR), heart rate (HR), and heart rate variability (HRV), as well as driver speech state recognition.

The radar from the sensor units 101a-101i is, for example, Doppler radar and operates in one or more modulations including, for example, continuous waveform (CW), FM/PM/AM/Pulse modulations. Each sensor unit 101a-101i generates and receives signals, which, for example, monitor and detect harmonic signals generated by humans, pets, and other live beings, who are typical vehicle cabin occupants. The data associated with the radar of each sensor unit 101a-101i is typically processed in the sensor units 101a-101i (by one or more processors including a central processing unit (CPU) <NUM> (<FIG>)) with the processed data, for example, transmitted via a link to the cellular tower <NUM>, so as to be transmitted to the home server <NUM> over the network <NUM>. Alternately, some or all of the data associated with the radar of each sensor unit 101a-101i, may be transmitted to the home server <NUM>, via the link to the cellular tower <NUM>, so as to be processed by processors of the home server <NUM>. For example, the harmonic signals transmitted by the sensor units 101a-101i, monitor the pulsing of the heart, aorta or associated veins and other vessels of the heart. Each harmonic signal (e.g., of the heart/heart beat) generates multiple harmonics, as illustrated in <FIG>.

As shown in <FIG>, the signal coming from the heart is harmonic, as shown for example, as eight harmonics, first through eighth (vertical lines), of <FIG>. The radar from each sensor unit 101a-101i, for example, monitors a different section of the body, in order to monitor the heart rate and breathing rate of one or more of the vehicle occupants. The sensor units 101a-101i are able to separate vibrations of the vehicle <NUM> from the vibrations of the heart, the aorta and associated vessels, by filtering the heart rate based on the heart's frequency of beating (e.g., <NUM> to <NUM> Hertz (Hz), corresponding to <NUM> to <NUM> beats per minute (bpm).

The sensor units 101a-101i are also programmed to filter the breathing signals (RR) (typically lower frequencies from those of the heart rate (HR) signals), from the heart rate (HR) signals and vehicle vibrations, for analysis. A typical human adult breathing rate is approximately <NUM> to <NUM>, corresponding to <NUM> to <NUM> breaths per minute, and for children, approximately, <NUM> to <NUM>, corresponding to up to <NUM> breaths per minute.

For example, the heart rate signals are monitored from the Aorta, which is proximate to vertebrae Lumbar-<NUM> (L3) and Lumabr-<NUM> (L4), as shown in <FIG>. The aorta is analyzed from the L3, L4 position used, for example, as it undergoes only small movements when a person being monitored is seated in the seats 101y, 101z of the vehicle <NUM>. The breathing rate (respiratory rate) (RR) may be monitored from the diaphragm. The sensor units 101a-101i, in particular chair mounts units 101c1-101c3, 101e1-110e3, are located, for example, proximate to the L3, L4 region (for example, so the radar antenna(s) position is located near field for radar beams).

For example, the sensor units 101c1-101c3, 101e1-101e3, mounted or otherwise embedded in the seats 101y, 101z, are typically used in vital signs monitoring.

The algorithms may be performed in the sensor unit 101a-101i, for example, the CPU <NUM>, therein, or the home server <NUM>, or partially in the sensor unit(s) 101a-101i and the home server <NUM>. The algorithms are performed for numerous operations. Example operations include: detecting multiple occupants in the vehicle, by, for example, breath separation, evaluating harmonic features of the detected signals to calculate vital signs, including heart rate (HR) and breathing rate (RR) calculating the heart rate (HR) and Breathing Rate (RR), and from the heart rate, and its corresponding signal, determination of consecutive peaks in the HR signal in the time domain for determining HRV. From the RR and HR signals, the presence or absence of vehicle occupants can be determined, the type of occupant, man, woman, child, animal, as well as the vital signs of human and animal occupants. For example, determining the number of occupants, e.g., human occupants, in a vehicle is useful in the administration of High Occupancy Vehicle (HOV) roadways. Other operations detect occupancy/non-occupancy states of each seat or place for a passenger in the vehicle, and vital signs of each of the detected occupants. The vital sign detection includes, for example, determining heart rate (HR) and breathing rate (RR) of each detected occupant, and heart rate variability (HRV).

<FIG> shows sensor unit 101a, representative of the sensor units 101a-101i, as detailed above, presented as an operational unit. Each of the sensor units 101a-101i is positioned in the vehicle as shown in <FIG>, and all data is extracted inside each embedded sensor unit 101a-101i. Additional process activity, additional to that of the sensor units 101a-101i, may be performed on the network <NUM>, e.g., by the home server (HS) <NUM> (as detailed below).

The sensor unit 101a includes a power supply <NUM>, which is, for example, supplied from a vehicle accumulator. The power supply <NUM> may also be a battery or supplied directly from the vehicle, e.g., the vehicle battery. The power supply <NUM> may be controlled by the CPU <NUM>. Within the sensor unit 101a, the power supply <NUM> connects, either directly or indirectly, to all of the system elements, including: an RF radar generator/transmitter/receiver <NUM>, hereinafter "RF transceiver <NUM>" or "RF Radar Transceiver <NUM>", filtering and amplification circuit <NUM>, Analog to Digital convertor (ADC) <NUM>, Central Processing Unit (CPU) <NUM> (also known as a Signalprocessing unit, these terms used interchangeably herein), Digital to Analog (DAC) convertor <NUM>, output/input interface <NUM> (which communicates with a user interface <NUM>), and an inertial measurement unit (IMU) <NUM>.

The power supply <NUM>, RF transceiver <NUM>, and DAC <NUM>, form an RF modules array. The filtering and amplification circuit <NUM> forms an IF signal processing unit, with the Analog to Digital convertor (ADC) <NUM>. The CPU <NUM> includes one or more processors, including hardware processors, such as processors commercially available from Intel, AMD and the like.

The power supply <NUM> provides power for the RF transceiver <NUM>, DAC <NUM>, filtering and amplification circuit <NUM>, ADC <NUM>, and CPU <NUM>. The sensor unit 101a, is such that elements <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>, are typically in the sensor unit 101a as embedded elements in a single housing H.

The user interface <NUM>, may be either a wired or wireless interface. The interface <NUM> may also be integrated as part of the vehicle, as well as a smartphone, tablet, computer or any other embedded interface. This user interface <NUM>, or alternately, the output/input interface <NUM> links to the network(s) <NUM>, so as to be in electronic and data communication with the home server <NUM>, which runs the various algorithms, and sends the data outputted by these algorithms to the various entities, represented by servers <NUM>, <NUM> and <NUM>.

The RF radar transceiver <NUM> is, for example, Doppler radar and operates in one or more modulations including, for example, continuous waveform (CW), FM/PM/AM/Pulse modulations. The RF Radar Transceiver <NUM> includes one or more antennas (including radar antennas), which transmit RF high frequency signals and receive the reflected signals from the object (e.g., vehicle occupant(s)), and converters for converting the reflected (an received) high frequency RF signals to Intermediate frequency (IF) signals.

This RF radar (from the RF transceiver <NUM>) is such that if the output is a single output, for example, an analog signal, or a dual output of two signals, also analog signals, the first is "Q" refers to quadrature data and second "I" refers to in-phase data.

The RF radar transceiver <NUM> operates, for example, in one or several frequency bands. Preferably, the RF radar operates in X (<NUM> to <NUM>), Ku (<NUM> to <NUM>),K (<NUM> to <NUM>), K(ISM) (<NUM> to <NUM>) and W (<NUM> to <NUM>).

The filtering and amplification circuit <NUM> performs operations, including, for example, filtering IF (intermediate frequency) signals transmitted from the RF radar <NUM>. In this circuit <NUM>, unwanted frequencies are filtered out, letting the desired frequencies pass through to the ADC <NUM>. The filtering and amplification circuit <NUM> includes, for example, filters and amplifiers.

The filters, for example, are for various frequencies, and may be hardware, software or combinations thereof. The filters clean the IF signals of noise and prevent aliasing before data acquisition.

The amplifiers amplify the IF signals before the IF signals enter the ADC <NUM>. The amplifiers, for example, are for various frequencies, and may be hardware, software or combinations thereof.

The filtering and amplification circuit <NUM> operates in two variations of output from the RF radar <NUM>, for example, as: <NUM>) a single analog output, formed of a filtering and amplification circuit for amplifying a single analog data transfer from the RF radar <NUM>; and <NUM>) "I" (in phase) and "Q" (quadrature) outputs from RF radar <NUM>, two filtering and amplification circuits for amplifying the RF radar <NUM> "I" and "Q" outputs.

The ADC <NUM>, converts the signal-received from the filtering and amplification circuit <NUM> from an analog to a digital signal. The ADC <NUM> may be a separate module or embedded into the CPU <NUM>. When the ADC <NUM> is separate from the CPU <NUM>, it transmits digital signal (raw data) to the CPU <NUM>. The ADC <NUM> converts the filtered analog signals received from the filtering and amplification circuit <NUM>, to digital signals data to allow digital processing by the processing algorithms of CPU <NUM>, including those to determine breathing rate (RR), heart rate (HR) and heart rate variability (HRV). The CPU <NUM> receives raw data from the ADC <NUM> when the ADC <NUM> is separate from the CPU <NUM>. Alternately, the CPU <NUM>, for example, receives an analog data state from the filtering and amplification circuit <NUM>. In this case, the CPU <NUM> converts the analog signal to a digital signal (raw data). The aforementioned analog signal(s), (e.g., IF signals) from the filtering and amplification circuit <NUM>, and the corresponding signals reproduced by the CPU <NUM> from the digital signal provided by the ADC <NUM>, include one or more peaks.

The CPU <NUM> processes the data via vital signs detection algorithms, based on analyzing various signals generated by the vehicle occupants, i.e., humans, pets, and the like, and calculates the heart rate, respiratory rate and/or movement of the person or pet.

The CPU <NUM> forwards monitored data to the output interface <NUM>.

The CPU <NUM> is, for example, based on any embedded or real time processor that is suitable for operating a vital signs monitoring algorithm. The CPU <NUM> is agnostic to computer operating systems (OS).

The algorithms may be performed in the CPU <NUM>, the home server <NUM>, or partially in the CPU <NUM> and the home server <NUM>. The algorithms can also be run on any electronic control unit (ECU) of the vehicle, which includes for example, the CPU <NUM>. The algorithms are performed for numerous operations. Example operations include: detecting multiple occupants in the vehicle, by, for example, breath separation, evaluating harmonic features of the detected signals to calculate vital signs, including heart rate (HR) and breathing rate (RR) calculating the heart rate (HR) and Breathing Rate (RR), and from the heart rate, and its corresponding signal, determination of consecutive peaks in the HR signal in the time domain for determining HRV. From the RR and HR signals, the presence or absence of vehicle occupants can be determined, the type of occupant, man, woman, child, animal, as well as the vital signs of human and animal occupants. For example, determining the number of occupants, e.g., human occupants, in a vehicle is useful in the administration of High Occupancy Vehicle (HOV) roadways. Other operations detect occupancy/non-occupancy states of each seat or place for a passenger in the vehicle, and vital signs of each of the detected occupants. The vital sign detection includes, for example, determining heart rate (HR) and breathing rate (RR) of each detected occupant, and heart rate variability (HRV). Output/input interface <NUM> is, for example, a wired or wireless interface. The interface <NUM> functions, for example, to forward vital signs monitor data to a user interface <NUM>, as well as raw data, such as the data from the filtering and amplification circuit <NUM>, ADC <NUM> and the IMU <NUM>, such that the data can be processed, e.g., to detect vial signs of occupants, in the home server <NUM> and ECU (e.g., external ECU). The output/input interface is also capable to receive information such internal configurations, triggering, user data and the like.

Optionally, the sensor unit 101a can use the digital to analog convertor (DAC) <NUM> for modulation of frequency changes by applying the analog signal from the DAC <NUM> to the TRX element of the radar transceiver <NUM> VCO (voltage control oscillator) of the RF radar transceiver <NUM>. The DAC <NUM> while shown as a separate system component, may also be integrated into the CPU <NUM>. When the CPU <NUM> includes a DAC <NUM> and/or an ADC <NUM>, the DAC <NUM> and/or ADC <NUM> is bypassed or/and removed from the system of the sensor unit 101a-101i.

The IMU <NUM> includes a magnetometer, gyrometer and accelerometer, to detect various movements and vibrations of the vehicle. The IMU <NUM> links to the CPU <NUM> to provide data concerning the movements and vibrations of the vehicle. The CPU <NUM> factors this movement data into its analysis to determine RR, HR and HRV, as detailed further below.

<FIG> is a schematic diagram showing the filtering and amplification circuit <NUM>, configured for two pathways. By having the two pathways for breathing rate 406a (a higher amplitude signal than the heart rate), and heart rate 406b, higher signal to noise ratio (SNR) of the obtained signal (by the ADC <NUM>) is achieved. A first pathway 406a for breathing rate filtering, and a second pathway 406b for heart rate/heart rate variability filtering. For example, on the first pathway 406a, filtration is from approximately <NUM> to <NUM> for initial conditions and start up, while on the second pathway 406b filtration is from approximately <NUM> to <NUM> for initial conditions or start up.

<FIG> is a diagram of the components of the filtration and amplification circuit <NUM> for the breathing rate 406a and heartrate/heart rate variability 406b pathways. Both pathways 406a. 406b receive IF signal input from the RF Radar transceiver <NUM>, and output a filtered analog signal within the frequency band set along each pathway 406a, 406b, to the analog to digital converted ADC <NUM>. The frequency bands are as narrow as possible, so as to have all noise in the band eliminated, so as to have a readable signal. Each pathway 406a, 406b includes a high pass filter (HPF) 413a-<NUM>, 413b-<NUM>, an amplifier 413a-<NUM>, 413b-<NUM>, and a low pass filter 413a-<NUM>, 413b-<NUM> (LPF). The high pass filters 413a-<NUM>, 413b-<NUM>, amplifiers 413a-<NUM>, 413b-<NUM>, and low pass filters 413a-<NUM>, 413b-<NUM>, are, for example, controlled by the CPU <NUM>, as detailed below.

Along both pathways, the high pass filter (HPF) 413a-<NUM>, 413b-<NUM> sets an edge for the signal, which is at a frequency, lower than the breathing rate (RR), in the breathing rate pathway 406a, and a frequency, lower than the heart rate (HR), in the heart rate pathway 406b. The amplifiers 413a-<NUM>, 413b-<NUM> apply gain, in accordance with the calibration method detailed below, so as to amply the signal to an amplitude suitable to separate noise from the signal. The low pass filters 413a-<NUM>, 413b-<NUM>, set an upper edge for the signal, which is at a frequency, higher than the breathing rate (RR), in the breathing rate pathway 406a, and a frequency, higher than the heart rate (HR), in the heart rate pathway 406b.

A calibration method is performed in order to determine the minimal gain level for the analog amplifiers 413a-<NUM>, 413b-<NUM>, and the optimal filtering ranges. The calibration method is typically performed when the vehicle cabin 101x is empty.

The following steps are performed for a calibration. The radar transceiver <NUM> voltage control oscillator (VCO) level, is changed, from the DAC <NUM> or the CPU <NUM>. The change is based on VCO levels referenced to the heart rate frequencies. This is done by modulating the RF signal (transmitted form the radar transceiver <NUM>) at the frequency of heart rate (may be used few frequencies: lower HR, mid HR, high HR), in order to cover the entire frequency band.

If the radar transceiver <NUM> does not have a VCO, it is possible to change the voltage supply of the radar transceiver <NUM>, in order to change the transmitted (RF) frequency. This is done by changing the power supply <NUM>, in order to change the voltage level. This change in voltage level will impact on the transmitted RF frequency form the radar transceiver <NUM>.

For methods described above, the modulated signal from the radar transceiver <NUM> is typically done when the vehicle cabin 101x is empty. The reflected signal received from the radar transceiver <NUM>, at the minimal power level, which may be received by the radar transceiver <NUM>. The calibration of the gain levels of the radar RF (LNA) and IF signals should be based on the specific lower signal level which was received while the vehicle <NUM> was empty. For example, the gain level should allow for receipt of reflected calibration signals (waveforms) at the SNR ratio of at least <NUM> dB (decibel). In the case of calibration over multiple frequencies, for example, low, mid, high, the mean gain should be selected by the CPU <NUM> (by the preprogrammed initial settings).

<FIG> is a flow diagram for processes performed by the filtration and amplification circuit <NUM> of <FIG>, to isolate a signal usable for the breathing rate (RR), and heart rate (HR).

The process begins at block <NUM>. At this start block <NUM>, the filters and amplifiers are set to initial conditions, for example, <NUM> to <NUM> for filters along the breathing rate pathway 406a, and <NUM> to <NUM> for the filters along the heart rate pathway 406b, with amplifier gain initially set by the calibration method (default condition is no gain), detailed above. At block <NUM>, an IF signal, received from the RF Radar transceiver <NUM> (received as a result of the radar) is sent, and processed along the breathing rate pathway 406a and heart rate pathway 406b.

Moving along the breathing rate pathway 406a, the process moves to block <NUM>. At block <NUM>, the adjustable high pass filter (HPF) 413a-<NUM> is adjusted to filter out unwanted signals below predetermined levels. For example, the passband (e.g., predetermined level) is initially, or otherwise starts from, and at least approximately <NUM>. At block <NUM>, the analog amplifier 413a-<NUM> is referenced to the input signal, as measured by the CPU <NUM>. This adjustment is made such that the output signal from the amplifier 413a-<NUM>, remains in the linear region. The process moves to block <NUM>, where the adjustable low pass filter (LPF) 413a-<NUM>, is adjusted for the breathing rate, to filter out unwanted high frequency signals. The cut off frequency is set by the CPU <NUM> (<FIG>), with cut off values preprogrammed into the system of the CPU <NUM>, or the CPU <NUM> is provided with a look up table (LUT).

The CPU <NUM> also analyzes the signal to noise ratio, and whether the breathing rate signal is harmonic. The breathing rate signal is considered to be harmonic when the person is breathing (not speaking) periodically. For analyzing the breathing rate, at least eight harmonics should be evaluated, so as to form the breathing rate signal. For example, for a measured breathing rate frequency of <NUM>, and the signal is harmonic, the cut off frequency of the low pass filter 413a-<NUM> is determined by a multiplier, e.g., "<NUM>" (so as to, for example, evaluate at least eight harmonics, two extra harmonics resulting in <NUM> harmonics) multiplied by the breathing rate frequency, e.g., <NUM>, for a <NUM> cut off. From block <NUM>, the output signal is sent to the ADC <NUM>.

Returning to block <NUM> and moving along the heart rate pathway 406b, the process moves to block <NUM>. At block <NUM>, the adjustable high pass filter 413b-<NUM> is adjusted to filter out unwanted signals below predetermined levels. For example, the passband (e.g., predetermined level) is at least approximately <NUM>. At block <NUM>, the analog amplifier 413b-<NUM> is referenced to the input signal, as measured by the CPU <NUM>. This adjustment is made such that the output signal from the amplifier 413b-<NUM>, remains in the linear region. The process moves to block <NUM>, where the adjustable low pass filter 413b-<NUM>, is adjusted for the HR/HRV, to filter out unwanted high frequency signals. The cut off frequency is set by the CPU <NUM>, with cut off values preprogrammed into the system of the CPU <NUM> is provided with a look up table (LUT).

The CPU <NUM> also analyzes the signal to noise ratio and the harmonics weight coefficient algorithm, detailed below. Evaluation of the HR is based on the number of harmonics to be measured. For example, for a measured HR frequency of <NUM>, and the cut off frequency is a multiplier of <NUM> (the heart beat is in four motions QRST, for each of the two atria and the two ventricles- <NUM> motions by <NUM> chambers is <NUM>, the multiplier). The HR frequency, e.g., <NUM>, is multiplied the multiplier of <NUM>, for a <NUM> cut off. From block <NUM>, the output signal is sent to the ADC <NUM>.

For both pathways 406a, 406b the ADC sends the signal to the CPU <NUM>, which adjusts the signal level, with a closed loop to the respective amplifier 413a-<NUM>, 413b-<NUM>, and adjusts the filter high pass and low pass frequency cut-offs, in accordance with the process of <FIG>. For the heart rate path 406b, the high pass filter frequency cut off of block <NUM>, is set above the heart rate frequency, after determining the heart rate. This is done movements of a seated occupancy, which the vehicle is in the range of the heart rate frequency. In this case, only the heart rate harmonics are analyzed by the CPU <NUM>, and the heart rate (HR) and heart rate variability (HRV) are extracted from this analysis.

<FIG> is a diagram of process performed by the CPU <NUM> for the signal level of the frequency band, for example, by adjusting gain in the amplifiers 413a-<NUM>, 413b-<NUM>. Initially, the process starts at block <NUM>, where gain is initially set by the calibration method above. The process moves to block <NUM>, where the signal sent to the ADC <NUM> is recorded. The process moves to block <NUM>, where it is determined whether the signal level is below the supply voltage of the amplifier 413a-<NUM>, 413b-<NUM>. If no, the provcess moves to block <NUM>, where analog gain on the signal is decreased by the respective amplifier 413a-<NUM>, 413b-<NUM>. The process then moves to block <NUM>, from where it resumes.

If yes at block <NUM>, the process moves to block <NUM>. At block <NUM>, it is determined whether the signal level is above the noise level. If no, the process moves to block <NUM>, where analog gain on the signal is increased by the respective amplifier 413a-<NUM>, 413b-<NUM>. The process then moves to block <NUM>, where it continues.

If yes at block <NUM>, there is no gain adjustment by the respective amplifier 413a-<NUM>, 413b-<NUM>. The process returns to block <NUM>, from where it resumes.

<FIG> is a flow diagram detailing an exemplary process for determining the breathing or respiratory rate (RR) of a vehicle occupant, for example, by a sensor unit, such as sensor unit 101a, representative of all sensor units 101a-101i. The process starts at block <NUM>, and moves to blocks 602a and 602b, where contemporaneous, e.g., simultaneous, processes are performed.

At block 602a, a digital signal, converted from the analog signal, which was captured by the radar transceiver <NUM> for the occupant, is received in the CPU <NUM>. The digital signal is reformed by the CPU <NUM> and typically includes I and Q portions. At block 602b, data as to vibrations associated with the vehicle is obtained from the IMU <NUM>, by the CPU <NUM>. From blocks 602a and 602b, the process moves to block <NUM>, where the vehicle vibration data is determined within a defined breathing range. This predetermined (defined) breathing range, for example, is determined by the high pass 413a-<NUM> and low pass 413a-<NUM> filters and from an initial start of <NUM> to <NUM>. The process moves to block <NUM>, where unwanted frequencies, the frequencies which are measured by the IMU <NUM> within the breathing range frequency band, inside the breathing range are removed, for example, by digital filtration.

The process moves to block <NUM>, where frequencies outside the breathing range for example, is determined by the high pass 413a-<NUM> and low pass 413a-<NUM> filters and from an initial start (of <NUM> to <NUM>) are digitally filtered. At block <NUM>, the phase for the digital signal is calculated. This phase ϕ is calculated as:<MAT>.

The process moves to block <NUM>, where the peaks of the filtered signal are detected in the time domain. At block <NUM>, the peaks at the edges of the time window in which the signal is analyzed, are removed, as they may have been affected (changed) by the filtration itself. From the existing peaks, within the time window, the mean peak to peak distance is calculated, at block <NUM>. This mean peak to peak distance is the breathing or respiratory rate (RR).

<FIG> is a flow diagram detailing an exemplary process for determining the heart rate (HR) of a vehicle occupant, for example, by a sensor unit, such as sensor unit 101a, representative of all sensor units 101a-101i. The process starts at block <NUM>, and moves to blocks 702a and 702b, where contemporaneous, e.g., simultaneous, processes are performed.

At block 702a, a digital signal, converted from the analog signal, which was captured by the radar transceiver <NUM> for the occupant, is received in the CPU <NUM>. The digital signal is reformed by the CPU <NUM> and typically includes I and Q portions. At block 702b, data as to vibrations associated with the vehicle is obtained from the IMU <NUM>, by the CPU <NUM>. From blocks 702a and 702b, the process moves to block <NUM>, where the vehicle vibration data is determined within a defined heart rate range. This predetermined heart rate range, for example, is determined by the high pass 413b-<NUM> and low pass 413b-<NUM> filters and from an initial start of <NUM> to <NUM>. The process moves to block <NUM>, where unwanted frequencies, the frequencies which are measured by the IMU <NUM> within the heart rate frequency band, inside the heart rate range are removed, for example, by digital filtration.

The process moves to block <NUM>, where frequencies outside the breathing range (the breathing range, for example, is determined by the high pass 413b-<NUM> and low pass 413b-<NUM> filters and from an initial start of <NUM> to <NUM>) are digitally filtered. At block <NUM>, the phase for the digital signal is calculated. This phase ϕ is calculated as:<MAT>.

The process moves to block <NUM>, where a phase Fast Fourier Transform (FFT) is performed on the signal, to transform the signal from the time domain to the frequency domain. The process moves to block <NUM>, where peak detection for the signal is performed in the frequency domain.

The process now moves to block <NUM>, where the most probable heart rate (HR) is determined. This most probable heart rate (HR) is determined with reference to <FIG>, for example, as follows:.

The resultant signal from block <NUM> is subjected to additional digital filtering at block <NUM>. This additional digital filtering is performed to eliminate unwanted frequencies above and below a predetermined amount from both ends of the HR frequency. For example, if the HR frequency is <NUM>, the lower end frequency would be lower than <NUM>, and the upper end frequency would be higher than <NUM>, so that the unwanted frequencies eliminated are outside ± <NUM>%. (the predetermined amount).

Next, the process moves to block <NUM>, where peak detection in the time domain is performed for the signal. This peak detection is a mathematical process for finding local maxima.

The process moves to block <NUM>, where the time difference between each of the peaks, including multiple variations thereof (e.g., peak <NUM> to peak <NUM>, peak <NUM> to peak <NUM>, peak <NUM> to peak <NUM>), in the time domain is calculated.

The process moves to block <NUM>, where outlying peaks and artifacts are also removed from the signal. Artifacts include, for example, unreasonable peak differences (e.g., peak distances much shorter or much larger than that for the calculated heart rate), abnormal peak distances when compared to previous peak distances, and the like. The process then concludes at block <NUM>, where the mean peak to peak differences are calculated, from the peaks that remain. At block <NUM>, the heart rate (HR) is calculated as follows:<MAT>.

<FIG> is a flow diagram detailing an exemplary process for determining the heart rate variability (HRV) of a vehicle occupant, for example, by a sensor unit, such as sensor unit 101a, representative of all sensor units 101a-101i. The process employs the processes of 702a, 702b, <NUM>, <NUM>, <NUM> and <NUM> for determining heart rate, and continues from block <NUM>, where the phase for the digital signal is obtained from block <NUM>.

The process moves to block <NUM>, where a phase signal denoise filtering occurs. This occurs, for example, by wavelets and or wavelet decomposition. The process moves to block <NUM>, where peak detection for the signal in this time window is performed in the time domain.

The process moves to block <NUM>, where outlying peaks and artifacts are also removed from the signal. Artifacts include, for example, unreasonable peak differences (e.g., peak distances much shorter or much larger than that for the calculated heart rate), abnormal peak distances when compared to previous peak distances.

The process moves to block <NUM>, where consecutive peaks are counted based on the artifacts, which were determined in block <NUM>. The process moves to block <NUM>, where a series of consecutive peaks, of at least a predetermined number, e.g., <NUM>, is determined.

The process moves to block <NUM>, to calculate the HRV parameters, based on the peak to peak difference between consecutive peaks (the consecutive peaks received from block <NUM>). For example, the HRV parameters include the root mean square successive differences (RMSSD) and/or standard deviation normal (end) to normal (end) (SDNN) of the consecutive peaks.

Objects can also be detected in vehicle cabins, based on the processes detailed above in <FIG>, <FIG> and <FIG>. These objects include, for example, adults, children, infants and pets, who may be left in a vehicle cabin, while the vehicle ( e.g., automobile, bus, or school bus) is idling or turned off (not in motion). The method includes combining the IMU <NUM> data in order to remove the environmental impact on the analyzed signal, which can cause false detection. The method includes, detecting vital signs of potential occupants who have remained in the vehicle. The detection process includes: transmitting a radar signal to vehicle cabin and receiving the reflected signal; analyzing the reflected signal with respect to vibration data of the vehicle to produce a modified signal: and, analyzing the modified signal to determine the presence of vital signs in the vehicle cabin. Should vital signs be present, an occupant has been detected in the vehicle cabin.

This method can be easily adapted to airplanes, ships and the like, for use with caged (or uncages) pets and other animals in cargo holds. This method can also be used for enclosed spaces.

With the data having been obtained as to vehicle the system, for example at the home server <NUM> can perform various applications of the data. For example, once it is determined that the vehicle cabin 101x includes occupants, a seat belt reminder can be transmitted to the vehicle for the passengers. Passengers can be counted for tolls, cab fares, record keeping, for example for transport companies. Transport companies <NUM>, by knowing the numbers of passengers traveling on a certain route at a certain time can allocate their vehicle fleets accordingly. The number of passengers in a vehicle can be transmitted to first responders <NUM>, such as emergency vehicles and ambulances so a dispatcher can know how many ambulances to send to an accident scene.

Additionally, the number of occupants can be used to monitor traffic and traffic jams, by finding out how many people are traveling on a certain route at any given time. This way, police <NUM> and municipal authorities <NUM>, as well as statistics companies <NUM> can know: the amount of people affected by the traffic jam; and, the size of the traffic jam. The amount of people involved in a traffic jam is provided by the system of the invention. The amount of people affected by the traffic jam can be provided by mobile/vehicle applications.

While the invention is shown in use with an automobile, it is usable in multiple vehicles, such as busses, commercial vehicles, trains, boats, airplanes, space vehicles, and the like.

The invention also monitors vital signs, such as heart rate (HR), respiration rate or breathing rate (RR), and heart rate variability (HRV). The monitored vital signs of each individual occupant can be collected and stored for further use. The vital signs recorded can be used to identify a person, via a unique personal pattern as a combination of vital signs data. By one time supplying the passenger names, the system can correlate the person with his/her unique personal pattern.

Additionally, once a person is recognized via his vital signs, the vehicle can recognize the person and settings in the vehicle, can be adapted automatically for the passenger. Some settings include, for example: seat position, seatbelt configuration, seat back position, steering wheel height, and the like. Once the occupants are detected by the system, as men, women, children, and the like. Knowing this information, as sent by the system, content, e.g., music, video, and the like, from a content provider, can be sent to the vehicle, based on its occupants.

The vital sign identification provided by the system can also be indicative of a person's state, such as fatigue, drug or alcohol inebriation, and the like.

The system also recognizes vital signs of animals in the vehicle, including those being shipped as cargo.

The system can also collect personal vital signs for each passenger over the course of a journey, so as to detect sicknesses, medical conditions, and the like.

While the invention has been shown in use inside a vehicle, the invention can also be used outside of the vehicle, in other vehicles such as wheelchairs and other chairs, beds and furniture. The vehicles which the invention may be used, also include, trucks, busses, airplanes (e.g., cockpits and passenger and crew cabins), boats, ships, space vehicles, military vehicles, helicopters, and the like.

Optionally, the data processor includes a volatile memory for storing instructions and/or data and/or a non-volatile storage, for example, non-transitory storage media such as a magnetic hard-disk and/or removable media, for storing instructions and/or data.

For example, any combination of one or more non-transitory computer readable (storage) medium(s) may be utilized in accordance with the above-listed embodiments of the present invention. The non-transitory computer readable (storage) medium may be a computer readable signal medium or a computer readable storage medium.

As will be understood with reference to the paragraphs and the referenced drawings, provided above, various embodiments of computer-implemented methods are provided herein, some of which can be performed by various embodiments of apparatuses and systems described herein and some of which can be performed according to instructions stored in non-transitory computer-readable storage media described herein. Still, some embodiments of computer-implemented methods provided herein can be performed by other apparatuses or systems and can be performed according to instructions stored in computer-readable storage media other than that described herein, as will become apparent to those having skill in the art with reference to the embodiments described herein. Any reference to systems and computer-readable storage media with respect to the following computer-implemented methods is provided for explanatory purposes, and is not intended to limit any of such systems and any of such non-transitory computer-readable storage media with regard to embodiments of computer-implemented methods described above. Likewise, any reference to the following computer-implemented methods with respect to systems and computer-readable storage media is provided for explanatory purposes, and is not intended to limit any of such computer-implemented methods disclosed herein.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention.

The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the appended claims. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

The above-described processes including portions thereof can be performed by software, hardware and combinations thereof. These processes and portions thereof can be performed by computers, computer-type devices, workstations, processors, micro-processors, other electronic searching tools and memory and other non-transitory storage-type devices associated therewith. The processes and portions thereof can also be embodied in programmable non-transitory storage media, for example, compact discs (CDs) or other discs including magnetic, optical, etc., readable by a machine or the like, or other computer usable storage media, including magnetic, optical, or semiconductor storage, or other source of electronic signals.

Claim 1:
A method for determining the vital signs of an occupant in a vehicle comprising:
transmitting a at least one analog radar signal to the occupant and receiving the at least one analog radar signal as reflected from the occupant;
obtaining vibration data including vehicle movement data and vehicle acceleration data from at least one sensor in the vehicle;
filtering the received reflected at least one analog radar signal by adjusting at least one low-pass filter and at least one high-pass filter so that the cut-off frequency of the high-pass filter is the expected basic frequency of at least a heart rate (HR) of the occupant and the cut-off frequency of the low-pass filter is equal to at least sixteen multiples of the heart rate frequency, to produce a modified analog signal;
converting the modified analog signal to at least one digital signal; and,
analyzing the at least one digital signal to determine the vital signs of the occupant.