Patent Description:
A technology has been developed that uses wirelessly transmitted signals to detect a target object (for example, see Patent Literature (PTL) <NUM>).

PTL <NUM> discloses the capability of analyzing the eigenvalues of Doppler shift components included in wirelessly received signals, using Fourier transform to know the number or positions of living bodies to be detected.

Patent Application <CIT> relates to living body detection within a space, and in particular to estimating a position of at least one living body using a sensor having multiple reception and transmitting antennas. The transmitting antennas transmit radiowaves and are reflected from a living body. The reflected waves are received by the reception antennas. From the transmitted and reflected radiowaves, complex transfer functions are calculated, and living body information is extracted from the complex transfer functions. Using the living body information, a plural of position spectral functions are calculated based on MUSIC algorithm. The multiple position spectral functions are integrated, and a position of the living body is estimated from the maximum of the integrated position spectral function.

The article by <NPL> relates to living body detection using Multiple-Input Multiple-Output channel properties together with a power angular delay profile (PADP) calculated by the MUSIC algorithm. A channel transfer function in the frequency-domain is calculated from NT transmitting signals and NR reception signals, which are transmitted and received by NT transmitting antennas and NR reception antennas, respectively. For each of all elements of the channel transfer function, a power spectrum is calculated and Fourier transformed into time. The PDAP is then calculated by integrating the time-domain power spectra over all elements and all numbers of NR and NT. Patent Application <CIT> relates to living body detection within a space, and in particular to estimating a number of living bodies present in said space. A sensor estimates the presence of a living body from radiowaves transmitted by a transmitter and from received radiowaves reflected from the living body. The sensor has a complex transfer function calculator, a MUSIC spectrum calculator, and a first and second number information calculator. M antenna elements transmit radiowaves into the space where at least one living body is present. The living body reflects the radiowaves, and N receiving antenna elements receive the reflected waves. The complex transfer function calculator calculates a complex transfer function from the M transmitted and N received radiowaves. The MUSIC spectrum analyzer calculates from biological information extracted from the complex transfer function a likelihood spectrum for the at least one living body the total number of persons present in the space. Said spectrum indicates the likelihood for the presence of the at least one living body. The second number information calculator calculates a number indicating the number of living bodies by counting the number of blocks within the likelihood spectrum and outputs the number of present living bodies.

Some of the algorithms for detecting a target object require an entry of the number of target objects to be detected to the algorithms. When the number of target objects is unknown, a problem arises that a target object cannot be detected.

The present disclosure aims to provide an estimation device a method and a computer program capable of estimating information on a living body even when the number of living bodies to be detected is unknown.

These general and specific aspects may be implemented using a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or computer-readable recording media.

The estimation device according to the present disclosure is capable of estimating information on a living body even when the number of living bodies to be detected is unknown.

In relation to the detection technologies disclosed in the Background section, the inventors have found the problems described below.

Technologies have been conventionally developed of detecting a target object using wirelessly transmitted signals (for example, see PTL <NUM> to PTL <NUM>).

For example, PTL <NUM> discloses a technology of estimating the number or positions of persons to be detected by analyzing the eigenvalues of Doppler shift components, using Fourier transform. More specifically, the processing apparatus disclosed in PTL <NUM> performs Fourier transform on reception signals, determines an autocorrelation matrix of waveforms of specified frequency components extracted, and obtains eigenvalues by eigenvalue decomposition on such autocorrelation matrix. In general, an eigenvalue and an eigenvector each indicate a propagation channel, i.e., a single path, through which radio waves propagate from a transmission antenna to a reception antenna. However, the technology disclosed in PTL <NUM> removes components that do not include living body information. As such, only a path corresponding to a signal reflected by a living body, a path corresponding to the secondary reflection of such signal, and a path corresponding to noise occur as eigenvalues and eigenvectors. Here, the values of eigenvalues corresponding to noise are smaller than the values of eigenvalues corresponding to a living body. It is thus possible to estimate the number of living bodies by counting the number of ones of these eigenvalues that are larger than a predetermined threshold.

However, the technology disclosed in PTL <NUM> has the problem as described below. That is to say, when a living body to be detected is present in a relatively distant location from a detection device, or when a relatively large number of living bodies is present, a difference in the eigenvalues between the living bodies and noise becomes small. This results in a reduced accuracy of estimating the number of persons. This is because, when the Doppler effect is extremely weak, the estimation of the number of persons is affected by: internal noise of the reception device or interference waves from an object other than target objects to be detected; and the presence of an object, other than the target objects, that generates a Doppler shift. These effects make it hard to detect feeble signals having a Doppler shift. Also, the sizes of living bodies to be measured are relatively large and living body components are distributed, spanning a plurality of eigenvalues. As such, when a relatively large number of living bodies is present, the eigenvalues of the living bodies cannot be completely separated from one another, which makes it hard to estimate the number of persons.

PTL <NUM> discloses a technology of estimating the position of a target object, using a direction estimation algorithm such as MUltiple SIgnal Classification (MUSIC). More specifically, a receiving station that has received signals from a transmitting station performs Fourier transform on the received signals, determines an autocorrelation matrix of waveforms of specified frequency components extracted, and applies a direction estimation algorithm such as MUSIC method. This achieves direction estimation with high accuracy. However, MUSIC method used in PTL <NUM> requires an entry of the number of living bodies to be detected. The detection technology of PTL <NUM> thus requires a preliminary estimation of the number of persons.

For example, PTL <NUM> discloses a technology of estimating the number of incoming waves, i.e., the number of transmission devices such as mobile phones from a correlation between eigenvectors of reception signals received by a plurality of antennas and steering vectors in a range at which radio waves can arrive.

For example, PTL <NUM> discloses a technology of estimating the number of incoming waves by: using steering vectors to calculate evaluation functions for a various number of incoming waves given for reception signals received by a plurality of antennas; and estimating that the number of incoming waves with the maximum evaluation function is a true number of incoming waves.

However, the technologies disclosed in PLT <NUM> and PTL <NUM> are intended for estimating the number of transmission devices that emit radio waves, and thus cannot estimate the number of living bodies.

In view of the above, the present inventors have conceived an estimation device and so forth capable of estimating a larger number of living bodies with higher accuracy using wireless signals, without requiring target living bodies to have a special device such as a transmission device.

Note that the present disclosure can be implemented not only as a device, but also as an integrated circuit that includes the processing units included in such device, a method that includes as its steps the processing units included in the device, a program that causes a computer to execute these steps, and information, data, or signals that represent such program. Also, such program, information, data, and signals may be distributed in a recording medium such as a CD-ROM, or via a communication medium such as the Internet.

Hereinafter, certain exemplary embodiments are described in greater detail with reference to the accompanying Drawings. Each of the exemplary embodiments described below shows a general or specific example of the present disclosure. The numerical values, shapes, materials, elements, the arrangement and connection of the elements, steps, the processing order of the steps etc. shown in the following exemplary embodiments are mere examples, and therefore do not limit the present disclosure. Therefore, among the elements in the following exemplary embodiments, those not recited in any one of the independent claims are described as optional elements. Also, in the specification and the drawings, elements having substantially the same functional configuration are assigned the same reference marks and are not described to avoid redundancy.

With reference to the drawings, the following describes a method and others of estimating the number of persons performed by sensor <NUM> according to Embodiment <NUM>. Sensor <NUM> is an example of the estimation device that is capable of estimating information on a living body even when the number of living bodies to be detected is unknown.

<FIG> is a block diagram showing the configuration of sensor <NUM> according to Embodiment <NUM>. <FIG> is a conceptual diagram showing a direction-of-arrival estimation performed by sensor <NUM> according to Embodiment <NUM>.

Sensor <NUM> shown in <FIG> includes complex transfer function calculator <NUM>, living body component extractor <NUM>, correlation matrix calculator <NUM>, spectrum calculator <NUM>, and estimator <NUM>. Sensor <NUM> is connected to transmission device <NUM> and reception device <NUM>. Note that sensor <NUM> may include either or both of transmission device <NUM> and reception device <NUM>. Also, transmission device <NUM> and reception device <NUM> may be disposed in the same cabinet.

Transmission device <NUM> includes transmitter <NUM> and transmission antenna <NUM>. Transmission device <NUM> transmits radio waves in space S. Living body <NUM> is assumed to be present in space S. The following description is given of an example case where living body <NUM> is a person (i.e., human body).

Transmission antenna <NUM> includes an array antenna including MT transmission antenna elements #<NUM> through #MT. Transmission antenna <NUM> is, for example, a four-element patch array antenna with half-wavelength spacing between elements.

Transmitter <NUM> generates high-frequency signals. The high-frequency signals generated by transmitter <NUM> can be used to estimate the presence/absence of living body <NUM>, the position of living body <NUM>, or the number of living bodies. For example, transmitter <NUM> generates <NUM> continuous waves (CW), and transmits the generated CWs from transmission antenna <NUM> as radio waves, i.e., transmission waves. Note that the signals to be transmitted are not limited to CWs, and thus may be modulated signals.

Reception device <NUM> includes reception antenna <NUM> and receiver <NUM>. Reception device <NUM> receives radio waves transmitted from transmission device <NUM> in space S. The radio waves received can include reflected waves or scattered waves, which are part of the transmission waves transmitted from transmission antenna <NUM> having been reflected or scattered by living body <NUM> as signals.

Reception antenna <NUM> includes an array antenna including MR reception antenna elements #<NUM> through #MR. Reception antenna <NUM> is, for example, a four-element patch array antenna with half-wavelength spacing between elements. Reception antenna <NUM> receives the high-frequency signals by the array antenna.

Using a downconverter, for example, receiver <NUM> converts the high-frequency signals received by reception antenna <NUM> into signal-processable low-frequency signals. When transmission device <NUM> transmits modulated signals, receiver <NUM> also demodulates the received modulated signals. Receiver <NUM> transfers, to complex transfer function calculator <NUM>, the low-frequency signals obtained by the conversion.

Note that the present embodiment uses <NUM> as an exemplary frequency range, but may use, for example, <NUM> or millimeter-wave range.

Complex transfer function calculator <NUM> calculates, from the reception signals received by the array antenna of reception antenna <NUM>, complex transfer functions that represent the propagation characteristics between transmission antenna <NUM> of transmission device <NUM> and reception antenna <NUM>. More specifically, complex transfer function calculator <NUM> calculates, from a low-frequency signal transferred from receiver <NUM>, a complex transfer function representing the propagation characteristics between each of MT transmission antenna elements included in transmission antenna <NUM> and each of MR reception antenna elements included in reception antenna <NUM>.

Note that complex transfer functions calculated by complex transfer function calculator <NUM> can include components corresponding to reflected waves or scattered waves, which are part of the transmission waves transmitted from transmission antenna <NUM> having been reflected or scattered by living body <NUM> as signals (such components are also referred to as living body components). Also note that complex transfer functions calculated by complex transfer function calculator <NUM> can also include components corresponding to reflected waves that are transferred without going via living body <NUM>, such as direct waves from transmission antenna <NUM> and reflected waves from a fixed object. The amplitude and phase of signals reflected or scattered by living body <NUM>, i.e., reflected waves and scattered waves transferred via living body <NUM>, constantly changes due to activities of living body <NUM> such as breathing and heartbeat.

The following description assumes that complex transfer functions calculated by complex transfer function calculator <NUM> include living body components corresponding to reflected waves and scattered waves that are signals reflected or scattered by living body <NUM>.

Note that <FIG> shows transmission device <NUM> and reception device <NUM> that are disposed adjacent to each other, but the disposition of transmission device <NUM> and reception device <NUM> is not limited to this. Transmission device <NUM> and reception device <NUM> thus may be disposed, for example, spaced apart from each other as shown in <FIG>. Also note that a single antenna may serve both as the transmission antenna and the reception antenna. Also, the transmission antenna and the reception antenna may be shared use by hardware of a wireless device such as a Wi-Fi® router and a slave unit.

Living body component extractor <NUM> obtains, from complex transfer function calculator <NUM>, signals received by the reception array antenna of reception antenna <NUM> (such signals are also referred to as reception signals). Living body component extractor <NUM> then extracts the living body components included in the reception signals, i.e., signal components transmitted from transmission antenna <NUM> and reflected or scattered by at least one living body <NUM>.

More specifically, living body component extractor <NUM> records the complex transfer functions calculated by complex transfer function calculator <NUM> in chronological order, which is the order of signal reception. Living body component extractor <NUM> then extracts variation components derived from living body <NUM>, from among variations in the complex transfer functions recorded in chronological order. As described above, the variation components included in the complex transfer functions derived from living body <NUM> correspond to the living body components.

Example methods of extracting living body components include: a method of transforming variations in the complex transfer functions into components in the frequency domain and then extracting frequency components corresponding to living body components; and a method of extracting living body components by calculating a difference between complex transfer functions of two different times. These methods remove the components, included in the complex transfer functions, of direct waves and reflected waves that are transferred via a fixed object. As a result, the living body components transferred via living body <NUM> remain. Using complex transfer functions equivalent to five seconds, for example, components of a frequency between <NUM> and <NUM> are extracted as frequency components corresponding to living body components. Through this, it is possible to extract respiratory components derived from living body <NUM>, which are present even when living body <NUM> stays still.

Note that extraction of <NUM> to <NUM> components has been described as an example in the present embodiment. To extract components of a slower operation or a faster operation, extraction is simply required to be performed differently to extract frequency components corresponding to the desired operation.

In the present embodiment, the number of transmission antenna elements included in the transmission array antenna is MT, and the number of reception antenna elements included in the reception array antenna is MR. Stated differently, there are a plurality of transmission antenna elements and a plurality of reception antenna elements. As such, complex transfer functions corresponding to the transmission array antenna and the reception array antenna include a plurality of living body components transferred via living body <NUM>.

A plurality of living body components transferred via living body <NUM> are represented as shown in Expression <NUM> as a matrix with M rows and N columns (also referred to as living body component channel matrix F(f)). <NUM>] <MAT>.

Note that each element Fij in the living body component complex transfer function matrix, i.e., living body component channel matrix F(f), is an element obtained by extracting a variation component from each element hij of complex transfer functions. Also, the living body component complex transfer function matrix, i.e., living body component channel matrix F(f), includes functions of frequencies or difference periods similar to frequencies. Such functions are items of information corresponding to a plurality of frequencies. Note that a difference period is a time difference between two complex transfer functions used in a method of calculating a difference between two complex transfer functions of two different times to extract living body components.

Correlation matrix calculator <NUM> sorts the elements of the living body component channel matrix with M rows and N columns calculated by living body component extractor <NUM>. Through this, correlation matrix calculator <NUM> converts the living body component channel matrix into living body component channel vector Fvec (f) with (M × N) rows and one column. A method of sorting the elements is, for example, as shown in Expression <NUM>, but any operations for sorting matrix elements may be utilized and elements may be sorted into any orders. <NUM>] <MAT>.

Subsequently, correlation matrix calculator <NUM> calculates a correlation matrix from living body component channel vector Fvec (f). More specifically, correlation matrix calculator <NUM> calculates correlation matrix R of living body component channel vector Fvec (f) including a plurality of variation components derived from living body <NUM> in accordance with Expression <NUM>. <NUM>] <MAT>.

In Expression <NUM>, E[ ] represents an averaging operator and operator H represents complex conjugate transpose. Here, to calculate a correlation matrix, correlation matrix calculator <NUM> averages living body component channel vector Fvec (f) including a plurality of frequency components in the frequency direction. This enables the sensing that simultaneously uses items of information included in the respective frequencies.

Spectrum calculator <NUM> calculates likelihood spectra indicating the likelihood of the presence of living body <NUM> in space S, and calculates an integrated spectrum, using the calculated likelihood spectra. Spectrum calculator <NUM> calculates likelihood spectra, using an estimation algorithm for estimating the presence of living bodies in the case where the number of the living bodies that are present in the space has been entered. The likelihood spectra are calculated by, for example, MUSIC method. The following describes an example case of using MUSIC method. Likelihood spectra calculated by MUSIC method are also referred to as MUSIC spectra.

In general, the calculation of likelihood spectra requires the number of incoming waves. To calculate MUSIC spectra by MUSIC method, the number of incoming waves is required. The number of incoming waves corresponds to the number of living bodies <NUM> that are present in space S in the present embodiment.

Instead of using a single specific value as the number of living bodies, spectrum calculator <NUM> sequentially uses a plurality of different values as the number of living bodies to calculate MUSIC spectra.

Stated differently, to calculate MUSIC spectra, spectrum calculator <NUM> uses, as the number of living bodies, variable L by assigning different values to variable L, starting from initial value Lstart to Lend. Subsequently, spectrum calculator <NUM> calculates an integrated MUSIC spectrum by integrating a plurality of MUSIC spectra calculated, using variable L having a plurality of different values. The following describes an operation performed by MUSIC spectrum calculator <NUM>, using expressions.

The result of eigenvalue decomposition of correlation matrix R calculated by correlation matrix calculator <NUM> is written as: <MAT> where <MAT> and <MAT>.

Here, <MAT> is an eigenvector having MR elements. <MAT> are eigenvalues corresponding to the eigenvector. <MAT> is satisfied, where L represents a loop variable used as the number of living bodies, i.e., the number of persons.

Also, a steering vector (directional vector) of the transmission array antenna is defined as: <MAT> and a steering vector (directional vector) of the reception array antenna is defined as: <MAT> Note that when the antenna elements in use have no uniform complex directionality, transmission and reception steering vectors that are created on the basis of complex directionality data obtained by actual measurement may be used. Here, k represents a wavenumber.

Further, a steering vector obtained by multiplying the foregoing steering vectors in consideration of angle information of both the transmission array antenna and the reception array antenna is defined as: <MAT> to which MUSIC method is applied by assigning different values to variable L.

Stated differently, on the basis of MUSIC method, spectrum calculator <NUM> calculates evaluation function Pmusic(θT, θR) obtained by integrating a plurality of MUSIC spectra represented as Expression <NUM> below, using the steering vector obtained by the multiplication. This evaluation function is referred to as an integrated MUSIC spectrum, or simply as an integrated spectrum. <NUM>] <MAT>.

Summation is used in Expression <NUM> as an integration operation, but product may be used instead. Stated differently, in Expression <NUM>, the summation sign below may be replaced by the product sign in Math. <MAT> <MAT>.

Note that predetermined values need to be set as minimum value Lstart and maximum value Lend of variable L. For example, one is set as minimum value Lstart. Alternatively, when the minimum number of living bodies that are present in space S to be measured is already known, such known number is set as minimum value Lstart. Also, when the maximum number of living bodies that are present in space S to be measured is already known, such known number or a value greater by one to three than the known number can be set as maximum value Lend.

Alternatively, a value that is smaller than the value obtained by multiplying the number of transmission antenna elements by the number of reception antenna elements on the order of one may be set as maximum value Lend. This is because the maximum number of target objects detectable by MUSIC method is the value that is smaller by one than the value obtained by multiplying the number of transmission antenna elements by the number of reception antenna elements. Also, maximum value Lend may be the value of the number of transmission antenna elements or the number of reception antenna elements.

Stated differently, spectrum calculator <NUM> can calculate likelihood spectra, using, for example, the following natural numbers as variable L: a plurality of natural numbers less than or equal to (the number of transmission antenna elements N × the number of reception antenna elements M - <NUM>); a plurality of natural numbers less than or equal to the number of transmission antenna elements N; or a plurality of natural numbers less than or equal to the number of reception antenna elements M. This is because: living body information is more accurately defined when an estimated number of living bodies is less than or equal to the value obtained by multiplying the number of transmission antenna elements by the number of reception antenna elements; and living body information is further more accurately defined when an estimated number of living bodies is less than or equal to the number of transmission antenna elements or the number of reception antenna elements.

Spectrum calculator <NUM> can also calculate likelihood spectra, using, as variable L, a plurality of natural numbers less than or equal to the value defined as the maximum number of living bodies that can be present in space S.

Spectrum calculator <NUM> can also calculate likelihood spectra, using, as variable L, a plurality of natural numbers in a range that includes the number of living bodies indicated by living body number information stored in a storage. Here, the storage is a storage device (not illustrated) that stores the living body number information estimated by estimator <NUM> in the past.

Note that variable L is incremented by one in the foregoing example, but variable L is not required to be incremented by equal amount; variable L may be varied in a variation pattern different from that of incrementing variable L by one. The variation pattern may be preliminary defined, or may be randomly selected in the course of the processing.

Note that MUSIC spectra may be replaced by spectra obtained by Beamformer method or Capon method. It should be noted, however, that Beamformer method or Capon method is inferior to MUSIC method in terms of accuracy, and an individual use of the method cannot achieve a highly accurate estimation. In other words, MUSIC method is superior to Beamformer method or Capon method in that an individual use of MUSIC method achieves a relatively highly accurate estimation.

Estimator <NUM> estimates, from the integrated spectrum calculated by MUSIC spectrum calculator <NUM>, living body information indicating at least the number of living bodies <NUM> that are present in space S to be measured, i.e., person information indicating at least the number of persons who are present in space S, and outputs the estimated information. Estimator <NUM> may also estimate, from the integrated spectrum, living body information further indicating the positions of the living bodies that are present in space S, i.e., person information further indicating the positions of the persons who are present in space S, and output the estimated information.

In ordinary circumstances, MUSIC spectra that are calculated on the basis of a correct number of persons having been entered (i.e., the number of persons who are actually present in space S) exhibit the same number of peaks as the number of persons having been entered. However, the present embodiment integrates a plurality of MUSIC spectra obtained on the basis of various values entered as the number of persons. As such, the resulting integrated spectrum sometimes exhibit virtual images (i.e., peaks that appear in positions in which no person is actually present).

Estimator <NUM> distinguishes peaks that are not derived from virtual images among the peaks that appear in the integrated spectrum. Estimator <NUM> then calculates the number of persons on the basis of the peaks that are not derived from virtual images among the foregoing peaks, to estimate person information indicating the number of persons who are present in space S. Estimator <NUM> may also calculate the positions of the peaks that are not derived from virtual images among the foregoing peaks, to estimate person information further indicating the positions of persons who are present in space S.

Example methods of calculating the number or positions of persons include: a method that uses a ratio approach for the peak values in a spectrum; a method of counting the number of sections (or blocks) in MUSIC spectra in which likelihoods that are greater than or equal to a predetermined threshold continuously appear, i.e., sections in which likelihoods are greater than or equal to the predetermined threshold; and a method that uses machine learning such as a convolutional neural network, using a MUSIC spectrum as images. In the present embodiment, a method that uses a ratio approach to calculate person information will be described as an example method.

<FIG> is a detailed block diagram of estimator <NUM> according to Embodiment <NUM>.

Estimator <NUM> shown in <FIG> includes peak searcher <NUM>, false-peak determiner <NUM>, peak sorter <NUM>, and tester <NUM>.

Peak searcher <NUM> searches for peaks that take the local maximum value in the integrated spectrum. A group of peaks found by the search is defined as a first peak group. To exclude small peaks derived from noise, peaks of the first peak group may be limited to the peaks each taking the maximum value in a predetermined range x.

<FIG> is a conceptual diagram showing an operation performed by peak searcher <NUM> according to Embodiment <NUM>. With reference to <FIG>, the process performed by peak searcher <NUM> is described, using one-dimensional integrated spectrum <NUM>.

<FIG> shows peaks <NUM>-A, <NUM>-B, <NUM>-C, and <NUM>-D, which are four peaks included in integrated spectrum <NUM>. Among these four peaks, peaks that take the maximum value within a distance range of <NUM> or less from the corresponding peaks (i.e., ranges <NUM>-A, <NUM>-B, <NUM>-C, and <NUM>-D) are three peaks of <NUM>-A, <NUM>-B, and <NUM>-D. Peak searcher <NUM> extracts the foregoing three peaks from integrated spectrum <NUM> and obtains the extracted peaks as the first peak group.

The first peak group corresponds to at least one local maximum value, among a plurality of local maximum values in the likelihood spectra, which is the largest in a predetermined range that includes such local maximum value.

False-peak determiner <NUM> excludes relatively gentle peaks among the peaks included in the first peak group. Virtual images in integrated spectrum <NUM> appear as relatively gentle peaks. As such, virtual image-derived peaks are excluded by excluding relatively gentle peaks.

More specifically, false-peak determiner <NUM> calculates an y% value of the value included in a predetermined distance range x from each of the peak values included in the first peak group. False-peak determiner <NUM> extracts peaks whose differential from the y% value is greater than or equal to a predetermined threshold z, and obtains the extracted peaks as a second peak group. The differential between the peak value and the y% value may be the difference between the peak value and the y% value (i.e., the peak value - the y% value) or may be the ratio between the peak value and the y% value (i.e., the y% value divided by the peak value). Also, "the value included in a predetermined distance range x" may be any values included in such range, an average of the values included in the range, the maximum value or the minimum value, and so forth.

Through this, false-peak determiner <NUM> excludes relatively gentle peaks among the peaks included in the first peak group. When the predetermined distance x is <NUM>, y is <NUM>%, and z is <NUM> dB, for example, false-peak determiner <NUM> extracts values that are larger by <NUM> dB or greater than the <NUM>% value of the value included within the <NUM> range from each of the peak values included in the first peak group.

The second peak group, which is the resultant of false-peak determiner <NUM> excluding the virtual image-derived peaks from the first peak group, corresponds to at least one third local maximum value, whose differential from the value obtained by multiplying a predetermined ratio by the value included in a predetermined range that includes such third local maximum value is greater than or equal to a threshold. Here, the predetermined ratio is a predetermined value that is greater than zero and smaller than one.

Peak sorter <NUM> sorts the values of a plurality of peaks included in the second peak group in descending order. Note that peak sorter <NUM> may add, to the second peak group, the value that is smaller by w than the smallest value among the peaks included in the second peak group, as a virtual peak. The virtual peak can be utilized as the second smallest peak after the smallest peak value in comparing each of a plurality of the peaks included in the second peak group with the second smallest peak after such peak value. For example, when w is set to <NUM> dB and the smallest peak is -<NUM> dB with respect to the maximum peak, a virtual peak to be added is -<NUM> dB with respect to the maximum peak.

Tester <NUM> calculates a differential between adjacent peak values in the second peak group sorted by peak sorter <NUM>, thereby estimating the number of persons. More specifically, tester <NUM> calculates a ratio or a difference as the difference between the i-th peak and the i+<NUM>-th peak in the second peak group sorted in descending order, and outputs, as the number of persons, "i" that gives the largest difference or ratio. Here, "i" is an integer greater than or equal to one and less than or equal to the number of elements included in the second peak group.

The following describes an example case of using a difference as a differential.

<FIG> is a conceptual diagram showing an operation performed by tester <NUM> according to Embodiment <NUM>.

<FIG> shows peaks included in the second peak group, <NUM>-A, <NUM>-B, <NUM>-C, and <NUM>, which are sorted in descending order according to the peak values. Note that peak <NUM> is a virtual peak added by peak sorter <NUM>.

Peak sorter <NUM> calculates differences between adjacent peaks in the second peak group, <NUM>-A, <NUM>-B, and <NUM>-C, to determine a combination of peaks that gives the largest difference by the calculation.

In an example shown in <FIG>, the largest difference is difference <NUM>-B, i.e., the difference between the second peak <NUM>-B and the third peak <NUM>-C. As such "i" is two, and the number of persons to be calculated is two.

As thus described, tester <NUM> obtains a first local maximum value, among at least one local maximum value obtained by peak searcher <NUM>, which has the largest differential from a second local maximum value, which is the second largest after such first local maximum value, and obtains the number indicating the place of the obtained first local maximum number in descending order of the at least one local maximum value. Subsequently, estimator <NUM> estimates and outputs the number obtained by tester <NUM> as the number of persons who are present in space S.

Note that tester <NUM> may output the at least one local maximum value per se in the foregoing manner obtained by peak searcher <NUM> as person information, or may output the person information in the foregoing manner, using, as at least one local maximum value, at least one third local maximum value obtained by false-peak determiner <NUM> excluding the virtual image-derived peaks from among at least one local maximum value obtained by peak searcher <NUM>.

Note that the foregoing description provides an example case where sensor <NUM> outputs person information indicating the number of persons, but the positions of persons may be estimated using MUSIC spectra to output person information indicating the positions of the persons.

Note that the foregoing embodiment uses, as an exemplary configuration, a multiple-input, multiple-output (MIMO) configuration having a plurality of transmission antennas and a plurality of reception antennas, but a single antenna configuration may be used for one of transmission and reception. In this case, the integrated spectrum outputted by spectrum calculator <NUM> is a one-directional spectrum, but it is still possible to estimate person information by searching for peaks as in the case where the integrated spectrum is two-dimensional.

Note that a determination may be made on the basis of the magnitude of the maximum eigenvalue, power corresponding to the variation components included in the complex transfer functions, or the degree of correlation between the presence and the absence of persons only for the detection of the absence of persons in space S, i.e., the detection of zero persons, and calculation of likelihood spectra and an integrated spectrum by spectrum calculator <NUM> may be performed only for the case where any persons are present. This saves the process required to calculate likelihood spectra and an integrated spectrum, when no person is present in space S, thereby contributing to the reduction in power consumption.

The following describes a process of estimating the number of living bodies performed by sensor <NUM> with the foregoing configuration.

<FIG> is a flowchart of a process performed by sensor <NUM> according to Embodiment <NUM>.

As shown in <FIG>, in step S10, sensor <NUM> receives signals for a predetermined period by reception device <NUM>.

In step S20, sensor <NUM> calculates complex transfer functions from the reception signals.

In step S30, sensor <NUM> records the calculated complex transfer functions in chronological order, and calculates a living body component channel matrix by extracting variation components derived from the living body from the complex transfer functions recorded in chronological order.

In step S40, sensor <NUM> calculates a correlation matrix of the extracted living body component channel matrix.

In step S50, sensor <NUM> sets initial value Lstart to variable L.

In step S60, sensor <NUM> calculates likelihood spectra by MUSIC method, on the basis of variable L set in step S50 or S75 and the correlation matrix calculated in step S40.

In step S70, sensor <NUM> determines whether variable L matches Lend. Sensor <NUM> proceeds to step S80 when determining that variable L matches Lend (Yes in step S70), and proceeds to step S75 when determining that variable L does not match Lend (No in step S70).

In step S75, sensor <NUM> adds one to variable L. Subsequently, sensor <NUM> executes step S60 again.

In step S80, sensor <NUM> integrates likelihood spectra to calculate an integrated spectrum. The likelihood spectra to be integrated are likelihood spectra calculated by sensor <NUM> in the processes in steps S50, S60, S70, and S75 by incrementing variable L by one from Lstart to Lend.

In step S90, sensor <NUM> calculates the number of persons from the integrated spectrum calculated in step S80, and estimates and outputs the calculated number of persons as person information. Example methods of performing the process in step S90 include: a method that uses a ratio approach for the peak values in the integrated spectrum; a method of counting the number of blocks that are sections in the integrated spectrum in which values greater than or equal to a predetermined value continuously appear; and a method that uses machine learning such as a convolutional neural network, using the integrated spectrum as images. In the present embodiment, a method that uses a ratio approach to calculate person information will be described as an example method.

<FIG> is a flowchart of a process of calculating person information performed by sensor <NUM> according to Embodiment <NUM>. The process shown in <FIG> is an exemplary case of performing the process in step S90 by the ratio approach.

As shown in <FIG>, in step S110, sensor <NUM> extracts, from the peaks in the integrated spectrum, peaks each taking the maximum value in a predetermined range, and obtains the extracted peaks as the first peak group.

In step S120, sensor <NUM> calculates the y% value of the value included in a predetermined distance range from each of the peaks included in the first peak group.

In step S130, sensor <NUM> extracts peaks whose differential from the y% value calculated in step S120 is greater than or equal to a predetermined threshold, from the peaks extracted in step S110, and obtains the extracted peaks as the second peak group.

In step S140, sensor <NUM> sorts the peaks included in the second peak group in descending order of peak values.

In step S150, sensor <NUM> calculates a differential between the i-th peak and the (i +<NUM>)-th peak in the second peak group, and estimates and outputs, as the number of persons, person information indicating "i" that gives the largest differential. Here, "i" is an integer greater than or equal to one and less than or equal to the number of elements included in the second peak group.

Sensor <NUM> according to the present embodiment is capable of estimating, with high accuracy, the number of living bodies <NUM> that are present in space S, using wireless signals.

Some of the existing estimation methods of deriving likelihood spectra used to estimate the number of living bodies <NUM> that are present in space S require an entry of the number of living bodies in space S.

Sensor <NUM> according to the present embodiment estimates the number of living bodies that are present in space S, using an integrated spectrum obtained by integrating likelihood spectra that are calculated by use of a plurality of values as the number of living bodies that are present in space S. With this, it is possible to estimate living body information indicating the number of living bodies that are present in space S even when the number of living bodies in space S is unknown.

In Embodiment <NUM>, a method is described of using a ratio approach to estimate living body information (i.e., person information) from an integrated spectrum. In Embodiment <NUM>, a method will be described of using a method of estimating living body information by counting the number of blocks that are sections in the integrated spectrum in which likelihoods are greater than or equal to a predetermined threshold.

The sensor according to the present embodiment has the same configuration as that of sensor <NUM> according to Embodiment <NUM>, but is different in that estimator <NUM> included in sensor <NUM> according to Embodiment <NUM> is replaced by estimator <NUM>. The configuration of the present embodiment other than estimator <NUM> is the same as that of Embodiment <NUM>, and thus will not be described here.

<FIG> is a block diagram showing the configuration of estimator <NUM> according to Embodiment <NUM>. <FIG> is a conceptual diagram showing an operation performed by block detector <NUM> according to Embodiment <NUM>. Integrated spectrum <NUM> shown in <FIG> is an example of the integrated spectrum calculated by spectrum calculator <NUM>.

As shown in <FIG>, estimator <NUM> includes threshold setter <NUM> and block detector <NUM>.

Threshold setter <NUM> sets threshold <NUM> that is smaller by v[dB] than the maximum value in integrated spectrum <NUM>. Note that the values of v and threshold <NUM> may be preliminary set fixed values, or may be the values obtained by evaluating the accuracy of estimating the number of persons by assigning different values to v and threshold <NUM> beforehand to use threshed <NUM> that achieves the highest accuracy as the optimum value. In the case where sensing is performed in a <NUM>-square room by a four-element patch array antenna with half-wavelength spacing between elements using unmodulated continuous waves of <NUM>, for example, v can be set to <NUM> dB.

Block detector <NUM> detects, as blocks, sections in which likelihoods in integrated spectrum <NUM> are greater than or equal to threshold <NUM>, and obtains the number of the detected blocks.

Estimator <NUM> estimates, as the number of persons who are present in space S, the number of blocks obtained by block detector <NUM>.

In an example shown in <FIG>, two blocks, that is, blocks <NUM>-A and <NUM>-B, are detected as sections in which values in integrated spectrum <NUM> are greater than or equal to threshold <NUM>. Block detector <NUM> calculates person information indicating that the number of persons is two.

The sensor according to Embodiment <NUM> reduces the amount of computation performed by estimator <NUM> compared to sensor <NUM> according to Embodiment <NUM>. This lowers the capability standard for a processing device that is required for real-time processing, thus enabling low-cost estimation of person-related information.

In Embodiment <NUM>, a method is described of using a ratio approach to estimate living body information (i.e., person information) from an integrated spectrum. In Embodiment <NUM>, a method will be described of using a machine learning model (e.g., a convolutional neural network) to estimate living body information from the integrated spectrum.

The sensor according to the present embodiment has the same configuration as that of sensor <NUM> according Embodiment <NUM>, but different in that estimator <NUM> included in sensor <NUM> according to Embodiment <NUM> is replaced by estimator <NUM>. The configuration of the present embodiment other than estimator <NUM> is the same as that of Embodiment <NUM>, and thus will not be described here.

<FIG> is a block diagram showing the configuration of estimator <NUM> according to Embodiment <NUM>.

As shown in <FIG>, estimator <NUM> includes training data creator <NUM>, learning unit <NUM>, network storage <NUM>, image converter <NUM>, and determiner <NUM>.

Training data creator <NUM>, learning unit <NUM>, and network storage <NUM> preliminary learn a machine learning model. Image converter <NUM> and determiner <NUM> calculate person information for test data, using the preliminary learned machine learning model.

Training data creator <NUM> obtains a plurality of images representing MUSIC spectra in the case where the number of persons is known beforehand, and stores the obtained images as training data images. Here, the training data images include images representing a plurality of MUSIC spectra of the persons assumed to be present in space S. When the upper limit of the number of persons in space S to be measured is three, for example, the training data images include a plurality of training data images (e.g., <NUM> or more) for each of zero persons, one person, two persons, and three persons.

Learning unit <NUM> learns the machine learning model, using the training data images as inputs. The machine learning model is, for example, a convolutional neural network model. The training data images used as inputs are the training data images stored by training data creator <NUM>. Note that a method such as transfer learning may be used here to improve the efficiency of neural network learning.

Network storage <NUM> stores the convolutional neural network created by the learning performed by learning unit <NUM> in, for example, a memory in a computer, a recording medium such as a CD-ROM, or a server located outside of the sensor. To store the convolutional neural network in a server located outside of the sensor, data of the convolutional neural network is sent to such server by communication over a network.

Image converter <NUM> converts the integrated spectrum calculated by spectrum calculator <NUM> into a format that is processable by the convolutional neural network to generate input data. An image in the format processable by the convolutional neural network is, for example, a heatmap image, the elements of which correspond to the values of the integrated spectrum.

Determiner <NUM> obtains person information that is outputted by inputting the input data generated by image converter <NUM> to the convolutional neural network stored in network storage <NUM>.

Estimator <NUM> estimates the person information obtained by determiner <NUM> as person information indicating the number of persons who are present in space S.

The sensor according to an aspect of the present disclosure has been described above on the basis of the embodiments, but the present disclosure is not limited to these embodiments. The present disclosure also includes a variation achieved by making various modifications to the embodiments that can be conceived by those skilled in the art without departing from the essence of the present disclosure and an embodiment achieved by combining elements included in different embodiments.

Note that the present disclosure can be implemented not only as a sensor that includes such characteristic elements, but also as an estimation method, etc. that includes as its steps the characteristic elements included in the sensor and as a computer program that causes a computer to execute these characteristic steps included in the method. Such computer program can be distributed in a non-transitory computer-readable recording medium such as a CD-ROM, or via a communication network such as the Internet.

The use of machine learning by a convolutional neural network by the sensor according to Embodiment <NUM> enables automatic adjustment of various parameters such as thresholds that need to be changed depending on the environment in which the sensor is located. The present embodiment is also expected to improve the accuracy of estimating the number of persons by updating the trained network whenever necessary.

Claim 1:
An estimation device (<NUM>), comprising:
a complex transfer function calculator (<NUM>) configured to calculate a complex transfer function representing propagation characteristics between each of N transmission antenna elements (#<NUM> to #MT) and each of M reception antenna elements (#<NUM> to #MR), using radio waves that are transmitted in a space (S) as a reception signal from each of the N transmission antenna elements and received by each of the M reception antenna elements, where N and M are natural numbers greater than or equal to two, the space being a space in which at least one living body (<NUM>) is present;
characterized by the estimation device comprising:
a spectrum calculator (<NUM>) configured to:
a) calculate a plurality of likelihood spectra L, from an initial value Lstart to an end value Lend, by use of an estimation algorithm for estimating presence of each of the at least one living body from living body information, using each of the different values of L as a total number of the at least one living body, wherein the maximum value of the end value Lend is equal to or less than a maximum number obtained by multiplying the number of N transmission antennas by the number M of reception antennas minus one, the plurality of L likelihood spectra each indicates a likelihood of the presence, and the living body information being a living body component included in the complex transfer function; and
b) calculate an integrated spectrum by integrating the plurality of likelihood spectra L calculated by using each of the different values of L as the number of living bodies to be detected; and an estimator (<NUM>) configured to estimate, from the integrated spectrum, living body information indicating at least the total number of the at least one living body that is present in the space, and outputs the living body information estimated.