# Patent

## Title
Method and apparatus for measuring and recovering sparse signals

## Abstract

The invention provides two kinds of new compressive sensing technologies. In the first technical solution, there is proposed a permutation-based multi-dimensional sensing matrix and an iterative recovery algorithm with maximum likelihood (ML) local detection, which can fully exploit the digital nature of sparse signals. In the second technical solution, there is proposed a sparse measurement matrix which contains a permutation-based multi-dimensional measurement matrix, and an iterative recovery algorithm which fully utilizes the features of measurement symbols to design simple local recovery in each iteration. The second technical solution can achieve the linear decoding complexity and lower bound of sketch length empirically at the same time.

## Background
FIELD OF THE INVENTION
The application relates to compressive sensing technology.
BACKGROUND OF THE INVENTION
Compressive sensing (CS) is a recently developed technique. Considering the fact that a large part of natural and artificial signals have the sparse or near sparse property, the compressive sensing technique can find applications in many different areas like compressive imaging, compressive sampling, signal processing, data stream computing, and combinatorial group testing, etc. The basic idea of compressive sensing is that a sparse signal x (a signal is referred to as sparse if it contains much more zero elements than non-zero elements) with length-N can be accurately recovered from a linear measurement y=Ax of length-M, wherein A is the M×N measurement matrix, M<<N.
The reconstruction can be performed through minimizing ∥x∥0 that explains the measurement vector. As this minimization problem is NP hard, sub-optimal algorithms have been investigated. Major classes of computationally feasible sparse signal recovery algorithms include convex relaxation, which approximates the l0 minimization problem by an lp minimization problem with p often chosen as 1 and solves this problem using convex optimization; matching pursuit, which iteratively refines a sparse solution by successively identifying one or more components that yield the greatest quality improvement; and Bayesian framework, which assumes a priori distribution that favors sparsity for the signal vector, and uses a maximum a posteriori estimator to incorporate the observation. Despite their relatively good performance in practice, they are most suitable for signals with continuous values. For sparse signals with digital values, e.g., when dealing with monochrome images, these algorithms are less sufficient as they cannot exploit the digital nature of the source, which, if utilized properly, can greatly enhance the recovery accuracy.
Therefore, there is a need of a new compressive sensing technique that can fully exploit the digital nature of signals.
In addition, in almost all applications, it is preferred that the measurement matrix A is sparse, i.e., it contains much more zero entries than non-zero entries in each column. The advantages of sparse measurement matrices include low computational complexity in both encoding and decoding, easy incremental updates to signals, and low storage requirement, etc. Much research has been devoted to CS with sparse measurement matrices, but most of them fail to achieve the linear decoding complexity and performance bound at the same time. Typical examples of existing algorithms include matching pursuit and convex optimization. The matching pursuit type of algorithms can asymptotically achieve the lower bound of the sketch length with a linear recovery complexity. However, numerical results have shown that the empirical sketch lengths needed in this type of algorithms are always much higher than the asymptotic bound. The convex optimization type of algorithms, on the other hand, can achieve the lower bound of the sketch length both asymptotically and empirically, which indicates an advantage in terms of measurement number in practices. For example, with the number of non-zero elements K=50 and signal length N=20000, it was shown that matching pursuit needs about 2000 measurements while convex optimization needs only about 450 measurements. One major disadvantage of the convex optimization type of algorithms is their higher recovery complexity, which grows in a polynomial order with the signal length N as O(N3).
Therefore, there is a need of a new compressive sensing technique that can achieve the linear decoding complexity and lower bound of sketch length empirically at the same time, with sparse measurement matrices.
OBJECT AND SUMMARY OF THE INVENTION
To better address above two concerns, there are provided two technical solutions, one is to provide a new compressive sensing technique that can fully exploit the digital nature of sparse signals, and the other is to provide a new compressive sensing technique that can achieve the linear decoding complexity and lower bound of sketch length empirically at the same time with sparse measurement matrices.
Based thereon, in a first aspect of the invention, there is provided a method for processing digital sparse signals. The method comprises the step of: performing a linear measurement to a length-N sparse signal vector x={xi} with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein the measurement matrix A is denoted by:




  A
  =
  
    [
    
      
        
          
            
              A
              1
            
            ⁢
            
              Π
              1
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              d
            
            ⁢
            
              Π
              d
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              D
            
            ⁢
            
              Π
              D
            
          
        
      
    
    ]
  




wherein AdΠd is a sub-matrix of the measurement matrix A, for d=1 ̃D, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is denoted by:




  
    A
    d
  
  =
  
    [
    
      
        
          
            
              a
              
                1
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                1
                ,
                L
              
              
                (
                d
                )
              
            
          
        
        
          
            
          
        
        
          0
        
      
      
        
          
            
          
        
        
          ⋱
        
        
          
            
          
        
      
      
        
          0
        
        
          
            
          
        
        
          
            
              a
              
                J
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                J
                ,
                L
              
              
                (
                d
                )
              
            
          
        
      
    
    ]
  




wherein M<<N, J*L=N and each entry xi in the sparse signal vector x is taken from a finite set Q={X0=0, X1, . . . , XQ-1} with Xq a non-zero figure for q=1 ̃Q−1, and Q the size of the set, wherein the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number.
Advantageously, the method may further comprise the steps of: using D maximum likelihood detectors to respectively perform maximum likelihood detection for D dimensions, wherein the dth maximum likelihood detector is used for performing maximum likelihood detection for the dth dimension; repeating above step for a plurality of iterations until a predetermined condition is satisfied; and estimating the source symbols in the sparse signal vector based on the output of the Dth maximum likelihood detector in the last iteration.
The measurement matrix A in the first aspect of the invention allows simple maximum likelihood detection in each dimension, which fully exploits the digital nature of sparse signals and provides a computationally feasible locally optimal detection for each dimension. The multi-dimensional structure of measurement matrix enables iterative information exchange between dimensions to get a near global-optimal estimation result.
Furthermore, the measurement matrix A in the first aspect of the invention is sparse, i.e., it contains much more zero entries than non-zero entries in each column. The sparsity of measurement matrix has several attractive properties, like low computational complexity in both encoding and recovery, easy incremental updates to signals and low storage requirements, etc. These advantages make the technical solution in the first aspect of the invention a potential and practical solution to compressive sensing with sparse digital signals.
In a second aspect of the invention, there is provided a method for processing analog sparse signals. The method comprises the step of: performing a linear measurement to a length-N K-sparse signal vector x with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein the measurement matrix A is denoted by:




  A
  =
  
    [
    
      
        
          
            
              A
              1
            
            ⁢
            
              Π
              1
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              d
            
            ⁢
            
              Π
              d
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              D
            
            ⁢
            
              Π
              D
            
          
        
      
    
    ]
  




wherein AdΠd is a sub-matrix of the measurement matrix A, for d=1 ̃D, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is denoted by:




  
    A
    d
  
  =
  
    [
    
      
        
          
            
              a
              
                1
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                1
                ,
                L
              
              
                (
                d
                )
              
            
          
        
        
          
            
          
        
        
          0
        
      
      
        
          
            
          
        
        
          ⋱
        
        
          
            
          
        
      
      
        
          0
        
        
          
            
          
        
        
          
            
              a
              
                J
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                J
                ,
                L
              
              
                (
                d
                )
              
            
          
        
      
    
    ]
  




wherein K<<N, M<<N, J*L=N and the sparse signal vector x={xi}εN, wherein the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number.
Advantageously, the method may further comprise the steps of:
i. for each element {tilde over (y)}j(d) in {tilde over (y)}, for d≦1 ̃D, j=1 ̃J, judging whether the {tilde over (y)}j(d) equals to 0, wherein {tilde over (y)} is initialized as {tilde over (y)}=y, and {tilde over (y)}j(d) is the jth element in the dth dimension of {tilde over (y)};
ii. if the {tilde over (y)}j(d) equals to 0, setting xΠd−1((j−1)L+l)=0, for l=1 ̃L; wherein Πd(i) is the index of xi in the dth permutated version and Πd−1(i) the inverse operation of Πd(i);
and the method further comprises the steps of:
u. for each source symbol xi in the sparse signal vector x, for i=1 ̃N, judging whether the following equation is satisfied,




  
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              n
            
            )
          
        
        
          (
          
            d
            n
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                n
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                n
              
              )
            
          
        
        
          (
          
            d
            n
          
          )
        
      
    
  
  ,
  
    
      ∀
      
        d
        m
      
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      d
      n
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      
        d
        m
      
      ≠
      
        d
        n
      
    
    ;
  




v. if it is satisfied for a pair of (dm, dn), setting




  
    
      x
      i
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
  
  ;




wherein ji(dm)=|(Πdm(i)−1)/L| and li(dm)=mod(Πdm(i)−1, L)+1.

and after above steps, the method further comprises:
p. updating {tilde over (y)} by subtracting the recovered sparse signals from the measurement vector y, by following formula:

{tilde over (y)}=y−A{tilde over (x)}

wherein {tilde over (x)} represents the recovered sparse signal vector, in which the un-recovered symbols are set to 0;
q. repeating above steps for a plurality of iterations until a predetermined condition is satisfied.
The special structure of the measurement matrix A in the second aspect of the invention leads to some interesting features of the measurement symbols, as will be explained later, which can be exploited to design a very simple recovery algorithm in each iteration. An iterative process is used to recover the source signal step by step, starting with symbols easier to recover, and canceling the contributions of already-recovered symbols to facilitate the recovery of other symbols. By repeating such a detection and cancellation operation, a near global-optimal solution can be obtained. The complexity needed grows only linearly with the source signal length N. The multi-dimensional structure and the random permutation matrices guarantee that each measurement statistically provides (directly or indirectly) some useful information to all source symbols, which is crucial for the good performance of the proposed technique.
Furthermore, the technical solution in the second aspect of the invention can achieve the empirical sketch length lower bound and linear complexity at the same time. The good empirical performance and low complexity makes it a good and practical alternative solution to compressive sensing with sparse measurement matrices.
In addition, according to one embodiment of the invention, there is provided an apparatus for measuring and recovering sparse signals comprising: a measuring means, for performing a linear measurement to a length-N sparse signal vector x={xi} with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein the measurement matrix A is denoted by:




  A
  =
  
    [
    
      
        
          
            
              A
              1
            
            ⁢
            
              Π
              1
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              d
            
            ⁢
            
              Π
              d
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              D
            
            ⁢
            
              Π
              D
            
          
        
      
    
    ]
  




wherein AdΠd is a sub-matrix of the measurement matrix A, for d=1 ̃D, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is denoted by:




  
    A
    d
  
  =
  
    [
    
      
        
          
            
              a
              
                1
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                1
                ,
                L
              
              
                (
                d
                )
              
            
          
        
        
          
            
          
        
        
          0
        
      
      
        
          
            
          
        
        
          ⋱
        
        
          
            
          
        
      
      
        
          0
        
        
          
            
          
        
        
          
            
              a
              
                J
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                J
                ,
                L
              
              
                (
                d
                )
              
            
          
        
      
    
    ]
  




wherein M<<N, J*L=N, the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number; and a recovering means, for recovering the length-N sparse signal vector x={xi} from the length-M measurement vector y.



BRIEF DESCRIPTION OF THE DRAWINGS
The invention is explained in further detail, and by way of example, with reference to the accompanying drawings wherein:
FIG. 1 shows a flowchart of processing digital sparse signals according to one embodiment;
FIG. 2 shows the principle of the interactive algorithm used in FIG. 1;
FIG. 3a shows the comparison of the rate-distortion performance between the embodiment of FIG. 1 and two conventional techniques with p1=0.1;
FIG. 3b shows the comparison of the rate-distortion performance between the embodiment of FIG. 1 and two conventional techniques with p1=0.05;
FIG. 4 shows a flowchart of processing analog sparse signals according to another embodiment;
FIG. 5 shows the relationship between the measurement number M and K log2(N/K) of the embodiment of FIG. 4;
FIG. 6a shows the relationship between the recovery failure probability and α of the embodiment of FIG. 4 with p1=0.1; and
FIG. 6b shows the relationship between the recovery failure probability and α of the embodiment of FIG. 4 with p1=0.05.



Throughout the above drawings, like reference numerals will be understood to refer to like, similar or corresponding features or functions.
DETAILED DESCRIPTION
Hereinafter, the technical solution of the above first aspect of the invention and the technical solution of the above second aspect of the invention will be described respectively in great detail.
FIG. 1 shows a flowchart of processing digital sparse signals according to one embodiment.
In Step S11, a linear measurement (i.e. encoding) is performed to a length-N digital sparse signal vector x with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein M<<N.
Consider digital sparse signal vector x={xi} with independent and identically distributed (i.i.d.) entries. The value of each entry xi in the sparse signal vector x is taken from a finite set Q={X0=0, X1, . . . , XQ-1} with Xq a non-zero figure, for q=1 ̃Q−1, and Q the size of the set. Suppose that each entry xi has a probability p0 of being “0” and a probability pq (q=1 ̃Q−1) of being Xq. Since x is sparse,




  
    p
    0
  
  >>
  
    
      ∑
      
        q
        =
        1
      
      
        Q
        -
        1
      
    
    ⁢
    
      
    
    ⁢
    
      
        p
        q
      
      .
    
  




Based on the theory of compressive sensing, x can be recovered from the measurement vector y as below,

y=Ax+n  (1)

wherein, n is a length-M noise vector with zero mean and E(|n|2)≦Mσ2 
In this embodiment, the measurement matrix A is designed as follows:




  
    
      
        A
        =
        
          [
          
            
              
                
                  
                    A
                    1
                  
                  ⁢
                  
                    Π
                    1
                  
                
              
            
            
              
                ⋮
              
            
            
              
                
                  
                    A
                    d
                  
                  ⁢
                  
                    Π
                    d
                  
                
              
            
            
              
                ⋮
              
            
            
              
                
                  
                    A
                    D
                  
                  ⁢
                  
                    Π
                    D
                  
                
              
            
          
          ]
        
      
    
    
      
        (
        2
        )
      
    
  




wherein AdΠd (d=1 ̃D) is a sub-matrix of the measurement matrix A, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is designed as follows:




  
    
      
        
          A
          d
        
        =
        
          [
          
            
              
                
                  
                    a
                    
                      1
                      ,
                      1
                    
                    
                      (
                      d
                      )
                    
                  
                  ⁢
                  ...
                  ⁢
                  
                    
                  
                  ⁢
                  
                    a
                    
                      1
                      ,
                      L
                    
                    
                      (
                      d
                      )
                    
                  
                
              
              
                
                  
                
              
              
                0
              
            
            
              
                
                  
                
              
              
                ⋱
              
              
                
                  
                
              
            
            
              
                0
              
              
                
                  
                
              
              
                
                  
                    a
                    
                      J
                      ,
                      1
                    
                    
                      (
                      d
                      )
                    
                  
                  ⁢
                  ...
                  ⁢
                  
                    
                  
                  ⁢
                  
                    a
                    
                      J
                      ,
                      L
                    
                    
                      (
                      d
                      )
                    
                  
                
              
            
          
          ]
        
      
    
    
      
        (
        3
        )
      
    
  




wherein J*L=N, the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number. The permutation matrices {Πd} are independently and randomly generated for D dimensions. The non-zero entries in {Ad} are Gaussian random variables.
The encoding process can be implemented as follows. The source symbols in the digital sparse signal vector x are independently permutated D times. Every permutated version is divided into J=N/L groups, each containing L symbols. The symbols in the jth group of the dth permutated version are weighted by the corresponding non-zero entries in the jth row of Ad, and then linearly superimposed to generate the jth measurement symbol in this dimension. The number of total measurements M is determined by the group length L, the dimension number D, and the signal length N as M=ND/L.
After encoding, a quantization process is applied to digitalize the measurement symbols. The quantization error is represented by the noise vector n in formula (1). Let S denote the number of quantization levels, and Pquan(s) the probability that a measurement symbol is quantized to level-S. The number of bits needed to represent one quantized measurement is then




  
    
      
        b
        =
        
          
            ∑
            
              s
              =
              1
            
            S
          
          ⁢
          
            
          
          ⁢
          
            
              
                p
                quan
              
              ⁡
              
                (
                s
                )
              
            
            ·
            
              
                log
                2
              
              ⁡
              
                (
                
                  1
                  ⁢
                  
                    /
                  
                  ⁢
                  
                    
                      p
                      quan
                    
                    ⁡
                    
                      (
                      s
                      )
                    
                  
                
                )
              
            
          
        
      
    
    
      
        (
        4
        )
      
    
  




Thus the total number of bits needed to represent x is B=bM=bND/L, and the average number of bits needed per source symbol is




  
    
      
        η
        =
        
          
            B
            N
          
          =
          
            bD
            ⁢
            
              /
            
            ⁢
            L
          
        
      
    
    
      
        (
        5
        )
      
    
  




The average bit number per source symbol can be adjusted via the selection of D, L and S.
The structure of {Ad} in formula (3) enables us to use maximum likelihood detection (ML) for each group of L symbols in each dimension. By choosing small values for L, the computational complexity of ML detection can be controlled. ML detection can fully exploit the digital nature of the source signal, and provide a locally optimal solution for each dimension. The multi-dimensional structure allows the iterative information exchange between dimensions to achieve a near global-optimal estimation. The independent random permutation matrices in different dimensions guarantee that statistically each measurement can provide (directly or indirectly) some useful information to all symbols in the digital sparse signal vector x, as the symbols contributing to different measurements in one dimension might be group together and contribute to the same measurement in other dimensions. This means that the information provided by one measurement to its associated L symbols might help the detection of other symbols if these L symbols are group with other symbols in other dimensions. Such a property is fully exploited in the iterative recovery algorithm to greatly enhance the noise-resistance ability of the proposed technique.
Based thereon, in Step S 12, D maximum likelihood detectors are used to respectively perform maximum likelihood detection for D dimensions. That is to say, each maximum likelihood detector is responsible for the detection of one dimension, as shown in FIG. 2. FIG. 2 shows the principle of the interactive algorithm, where “DET-d” is the local detector of the dth dimension, “T” denotes a delay of one iteration and “/” denotes the division operation.
Within each local detector, the ML detection is performed to each group of L symbols based on their noisy measurement symbol and a priori information. The output of each detector is a posteriori information of the source symbols in the digital sparse signal vector x, which is used to refine the local detection in other dimensions in the next iteration.
The variables involved in FIG. 2 are defined as follows.
{tilde over (p)}(d)(xi=Xq): The a priori probability of xi being Xq (q=0 ̃Q−1) in the dth dimension.
p(d)(xi=Xq): The a posteriori probability of xi being Xq (q=0 ̃Q−1) in the dth dimension.
e(d)(xi=Xq): The extrinsic probability of xi being Xq (q=0 ̃Q−1) in the dth dimension.
The a priori probabilities are initialized as {tilde over (p)}(1)(xi=Xq)=pq for i=1 ̃N and q=0 ̃Q−1 in the first iteration, and the extrinsic probabilities wherein initialized as e(d)(xi=Xq)=1 for ∀ d in the first iteration.
Consider a particular measurement-j in the dth dimension, denoted by yj(d). Let {ij,l(d),l=1 ̃L} be the indices of the L source symbols that contribute to yj(d). The values of {ij,l(d), ∀j,l} are determined by the permutation matrix Πd. The a posteriori probabilities of these L symbols are calculated using the ML detection as




  
    
      
        
          
            
              p
              
                (
                d
                )
              
            
            ⁡
            
              (
              
                
                  x
                  
                    i
                    
                      j
                      ,
                      l
                    
                    
                      (
                      d
                      )
                    
                  
                
                =
                
                  X
                  q
                
              
              )
            
          
          =
          
            
              1
              
                
                  2
                  ⁢
                  
                    πσ
                    2
                  
                
              
            
            ⁢
            
              Σ
              
                
                  
                    
                      c
                      ∈
                      
                        Q
                        L
                      
                    
                  
                
                
                  
                    
                      
                        c
                        l
                      
                      =
                      
                        X
                        q
                      
                    
                  
                
              
            
            ⁢
            exp
            ⁢
            
              {
              
                -
                
                  
                    |
                    
                      
                        y
                        j
                        
                          (
                          d
                          )
                        
                      
                      -
                      
                        
                          ∑
                          
                            
                              l
                              ′
                            
                            =
                            1
                          
                          L
                        
                        ⁢
                        
                          
                        
                        ⁢
                        
                          
                            a
                            
                              j
                              ,
                              
                                l
                                ′
                              
                            
                            
                              (
                              d
                              )
                            
                          
                          ⁢
                          
                            c
                            
                              l
                              ′
                            
                          
                        
                      
                    
                    ⁢
                    
                      |
                      2
                    
                    ⁢
                    
                      ·
                      
                        
                          ∏
                          
                            
                              l
                              ′
                            
                            =
                            1
                          
                          L
                        
                        ⁢
                        
                          
                        
                        ⁢
                        
                          
                            
                              p
                              ~
                            
                            
                              (
                              d
                              )
                            
                          
                          ⁡
                          
                            (
                            
                              
                                x
                                
                                  i
                                  
                                    j
                                    ,
                                    
                                      l
                                      ′
                                    
                                  
                                  
                                    (
                                    d
                                    )
                                  
                                
                              
                              =
                              
                                c
                                
                                  l
                                  ′
                                
                              
                            
                            )
                          
                        
                      
                    
                  
                  
                    2
                    ⁢
                    
                      σ
                      2
                    
                  
                
              
              }
            
          
        
        ,
        
          
            for
            ⁢
            
              
            
            ⁢
            l
          
          =
          
            1
            ∼
            L
          
        
      
    
    
      
        (
        6
        )
      
    
  




The summation in formula (6) is over all possible vectors cεQL with the lth element fixed to Xq. The extrinsic probabilities are calculated by extracting the a priori probabilities from the a posteriori probabilities,

e(d)(xi=Xq)=p(d)(xi=Xq)/{tilde over (p)}(d)(xi=Xq)  (7)

The a posteriori probabilities generated in the dth dimension are used to update the a priori probabilities in the (mod(d, D)+1)th dimension as

{tilde over (p)}(mod(d,D)+1)(xi=Xq)=p(d)(xi=Xq)/e(mod(d,D)+1)(xi=Xq)  (8)

Note that e(mod(d,D)+1)(xi=Xq) is generated in the (mod(d, D)+1)th dimension in the previous iteration, so it should be prevented from circulating back to the (mod(d, D)+1)th dimension again, which is the basic rule of iterative detection. This is realized by the division operation in formula (8).
Then in the Step S13, repeating above procedure for a plurality of iterations until a predetermined condition is satisfied.
The predetermined condition may for example comprise any of the followings:

    
    
        a fixed number of iterations has been reached;
        the difference between the a posteriori probabilities generated in the Dth maximum likelihood detector in the two successive iterations is below a pre-defined threshold.
    
    


Last, in Step S14, the source symbols {xi} in the digital sparse signal vector x is estimated based on the output of the Dth maximum likelihood detector in the last iteration.
For example, hard decisions are made for source symbols {xi} as




  
    
      
        
          
            
              x
              ^
            
            i
          
          =
          
            X
            
              
                q
                ^
              
              i
            
          
        
        ⁢
        
          

        
        ⁢
        with
      
    
    
      
        (
        9
        )
      
    
  
  
    
      
        
          
            q
            ^
          
          i
        
        =
        
          
            
              arg
              ⁢
              
                
              
              ⁢
              max
            
            q
          
          ⁢
          
            (
            
              {
              
                
                  
                    
                      p
                      
                        (
                        D
                        )
                      
                    
                    ⁡
                    
                      (
                      
                        
                          x
                          i
                        
                        =
                        
                          X
                          q
                        
                      
                      )
                    
                  
                  |
                  q
                
                =
                
                  0
                  ∼
                  
                    Q
                    -
                    1
                  
                
              
              }
            
            )
          
        
      
    
    
      
        (
        10
        )
      
    
  




The complexity of the above proposed iterative algorithm in this embodiment is O(NDQL/L), which is much lower than that of the optimal solution O(QN) as L is much smaller than N and does not grow with N. Notice that a smaller L will lead to a larger measurement number M for a fixed D. Then by adjusting the values of D and L, different tradeoffs between the computational complexity and measurement number can be achieved.
In the following text, the performance of above proposed technical solution is illustrated using numerical results. Considering binary source signals in the digital sparse signal vector X with xi ε{0, 1}. The entries of X are i.i.d. variables with a probability p0(p1) of being “0” (“1”) and p0>>p1. The measurement matrix A is generated according to formula (2) and (3) with the non-zero entries i.i.d. Gaussian random variables with zero-mean and unit variance. The measurement matrix A is normalized such that each row has a unit norm. The quantization level is fixed to S=5 with the following quantization rule, where Quan(x) represents the value of X after quantization.




  
    
      
        
          Quan
          ⁡
          
            (
            x
            )
          
        
        =
        
          {
          
            
              
                
                  
                    -
                    0.6
                  
                  ,
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    ⁢
                    x
                  
                  <
                  
                    -
                    0.4
                  
                
              
            
            
              
                
                  
                    -
                    0.2
                  
                  ,
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    -
                    0.4
                  
                  ≤
                  x
                  <
                  0
                
              
            
            
              
                
                  0
                  ,
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    ⁢
                    x
                  
                  =
                  0
                
              
            
            
              
                
                  0.2
                  ,
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    ⁢
                    0
                  
                  <
                  x
                  ≤
                  0.4
                
              
            
            
              
                
                  0.6
                  ,
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    ⁢
                    0.4
                  
                  <
                  x
                
              
            
          
        
      
    
    
      
        (
        11
        )
      
    
  




Note that with a digital sparse signal vector X, the linear measurement symbols generated using the measurement matrix A defined in formula (2) and (3) (before quantization) have a large probability of being “0”. If a zero measurement symbol appears, we can conclude with a high accuracy that all L source symbols associated with it are zero, which will be very helpful for the detection of other source symbols in the iterative recovery algorithm. Thus, a specific quantization level for “0” in formula (20) is used to distinguish it from other values. We use Monte Carlo simulation to obtain the probabilities {pquan(s), s=1 ̃5} and calculate the average number of bits needed per source symbol η according to formula (5).
FIGS. 3a and 3b show the comparison of the rate-distortion performance between the above proposed technical solution and two conventional techniques based on random Gaussian sensing matrix, Lasso algorithm and Bayesian framework, respectively. The minimum number of bits needed per source symbol for lossless recovery is also shown for reference.
Here, set p1=0.1 and 0.05, and fix the iteration number at 5. The probabilities {pquan(s)} are listed in Table 1.







TABLE 1







The probabilities of {pquan(s)} after quantization.













pquan(1)
pquan(2)
pquan(3)
pquan(4)
pquan(5)



(x < −0.4)
(−0.4 < x < 0)
(x = 0)
(0 < x < 0.4)
(0.4 < x)


















p1 = 0.1
proposed
L = 10
0.0999
0.2321
0.3341
0.2325
0.1014




L = 12
0.0967
0.2665
0.2656
0.2685
0.1027














conventional
0.1048
0.3952
0
0.3966
0.1034














p1 = 0.05
proposed
L = 10
0.0527
0.1528
0.5873
0.153
0.0542




L = 12
0.0503
0.1845
0.5253
0.187
0.0529














conventional
0.0364
0.4642
0
0.4629
0.0365










From the table 1, the number of bits needed per source symbol η can be calculated. Adjust D from 1 to 4 and L from 10 to 12 to obtain different values of η from 0.14 to 1. The distortion, denoted by ξ, is measured as follows, which equals the average number of incorrect entries in {circumflex over (x)}={{circumflex over (x)}i} normalized by N.




  
    
      
        ξ
        =
        
          
            1
            N
          
          ⁢
          
            E
            ⁡
            
              (
              
                ||
                
                  
                    x
                    ^
                  
                  -
                  x
                
                ⁢
                
                  ||
                  2
                  2
                
              
              )
            
          
        
      
    
    
      
        (
        12
        )
      
    
  




For reference, it is shown in FIG. 3 the minimum number of bits needed per source symbol for lossless compression via entropy coding. The rate-distortion performances of two conventional approaches are also included, which are based on random Gaussian sensing matrices, and employ convex relaxation (please refer to [R. Gribonval and M. Nielsen, “Highly sparse representations from dictionaries are unique and independent of the sparseness measure,” Aalborg Univ., Aalborg, Denmark, Tech. Rep., October 2003.], and [J. A. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory, vol. 52, no. 3, pp. 1030-1051, March 2006.] for details), and Bayesian framework (please refer to [S. Ji, Y. Xue and L. Carin, “Bayesian compressive sensing,” IEEE Trans. Signal Processing, vol. 56, no. 6, pp. 2346-2356, June 2008.], and [M. E. Tipping, “Sparse Bayesian learning and the relevance vector machine,” Journal of Machine Learning Research, vol. 1, pp. 211-244, September 2001.] for details), respectively, for signal recovery. For convex relaxation, we use the Lasso algorithm (please refer to [M. J. Wainwright, “Sharp thresholds for high-dimensional and noisy sparsity recovery using 11-constrained quadratic programming (Lasso),” IEEE Trans. on Inform. Theory, vol. 55, pp. 2183-2202, May 2009.], and [R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical Society, Series B, pp. 267-288, 1996.] for details). For the Bayesian framework based algorithm, the technique introduced in the paper {S. Ji, Y. Xue and L. Carin, “Bayesian compressive sensing,” IEEE Trans. Signal Processing, vol. 56, no. 6, pp. 2346-2356, June 2008.’ } is used. Both conventional approaches use a similar quantization rule as in formula (11) except removing the quantization level dedicated to “0”, as in this case each measurement is very unlikely to be “0”. The probabilities {pquan(s), s=1 ̃5} for the conventional approaches are also listed in Table 1. From FIG. 3, it is observed that the proposed technique can achieve much better rate-distortion performance than the conventional approaches based on random sensing matrix and convex relaxation/Bayesian framework.
FIG. 4 shows a flowchart of processing analog sparse signals according to another embodiment.
In Step S41, a linear measurement (i.e. encoding) is performed to a length-N K-sparse signal vector x with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein K<<N, M<<N, and the K-sparse signal vector x={xi}εN, the measurement vector yεM.
According to the theory of compressive sensing, x can be recovered from the measurement vector y as below,

y=Ax  (13)

In this embodiment, the measurement matrix A is designed as follows:




  
    
      
        A
        =
        
          [
          
            
              
                
                  
                    A
                    1
                  
                  ⁢
                  
                    Π
                    1
                  
                
              
            
            
              
                ⋮
              
            
            
              
                
                  
                    A
                    d
                  
                  ⁢
                  
                    Π
                    d
                  
                
              
            
            
              
                ⋮
              
            
            
              
                
                  
                    A
                    D
                  
                  ⁢
                  
                    Π
                    D
                  
                
              
            
          
          ]
        
      
    
    
      
        (
        14
        )
      
    
  




wherein AdΠd(d=1 ̃D) is a sub-matrix of the measurement matrix A, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which designed as follows:




  
    
      
        
          A
          d
        
        =
        
          [
          
            
              
                
                  
                    a
                    
                      1
                      ,
                      1
                    
                    
                      (
                      d
                      )
                    
                  
                  ⁢
                  ...
                  ⁢
                  
                    
                  
                  ⁢
                  
                    a
                    
                      1
                      ,
                      L
                    
                    
                      (
                      d
                      )
                    
                  
                
              
              
                
                  
                
              
              
                0
              
            
            
              
                
                  
                
              
              
                ⋱
              
              
                
                  
                
              
            
            
              
                0
              
              
                
                  
                
              
              
                
                  
                    a
                    
                      J
                      ,
                      1
                    
                    
                      (
                      d
                      )
                    
                  
                  ⁢
                  ...
                  ⁢
                  
                    
                  
                  ⁢
                  
                    a
                    
                      J
                      ,
                      L
                    
                    
                      (
                      d
                      )
                    
                  
                
              
            
          
          ]
        
      
    
    
      
        (
        15
        )
      
    
  




wherein J*L=N, the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number. Clearly, the measurement matrix A generated in this way is sparse with only D non-zero entries per column.
The encoding operation can be implemented as follows. The source symbols in the digital sparse signal vector x are independently permutated D times. Every permutated version is divided into J=N/L groups, each containing L symbols. The symbols in the jth group of the dth permutated version are weighted by the corresponding non-zero entries in the jth row of Ad, and then linearly superimposed to generate the jth measurement in this dimension. Denote by yj(d) (j=1 ̃J) the jth measurement symbol in the dth dimension, which have




  
    
      
        
          y
          j
          
            (
            d
            )
          
        
        =
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
          
          ⁢
          
            
              a
              
                j
                ,
                l
              
              
                (
                d
                )
              
            
            ⁢
            
              
                x
                ^
              
              
                
                  
                    (
                    
                      j
                      -
                      1
                    
                    )
                  
                  ⁢
                  L
                
                +
                1
              
              
                (
                d
                )
              
            
          
        
      
    
    
      
        (
        16
        )
      
    
  




where {circumflex over (x)}(d)={{circumflex over (x)}i(d), i=1 ̃N} is the dth permutated version of x. Let Πd(i) be the index of xi in the dth permutated version and Πd−1(i) the inverse operation, which have

{circumflex over (x)}Πd(i)(d)=xi and {circumflex over (x)}i(d)=xΠd−1(i)  (17)

Every source symbol xi in the sparse signal vector X is associated with D measurements, each in one dimension. The total measurements number (i.e. sketch length) M is determined by the group length L, the dimension number D, and the source signal length N as M=ND/L.
From formula (16), it can be seen that the measurement matrix A has only D non-zero elements in each column. Also note that only non-zero elements of x lead to addition and multiplication operations in encoding. Therefore, the encoding complexity of the proposed technique is about DK multiplications and DK additions.
The block diagonal structure of {Ad} in formula (15) and the random permutation matrices in D dimensions lead to the following features of the measurement symbols, which are very useful in signal recovery. For the convenience of discussion, it is defined the degree of a measurement symbol as the number of non-zero source symbols associated with it. From (16) it can be seen that each measurement symbol yj(d) is associated with only L source symbols {{circumflex over (x)}(j−1)L+l(d), l=1 ̃L}, therefore it is easy to verify that the following two issues have relatively large probabilities to happen.
Issue-1: A measurement symbol has a degree of 0.
Issue-2: For a non-zero source symbol, at least two of its associated measurements have a degree of 1 (assume D≧2).
The probabilities of the above two issues can be calculated as follows. Let p1=K/N, and p0=1−p1. p1 and p0 represent the probabilities of a source symbol being non-zero and zero, respectively. Then the probabilities of issue-1 and issue-2 can be approximately calculated as




  
    
      
        
          
            
              P
              
                issue
                ⁢
                
                  -
                
                ⁢
                1
              
            
            ≈
            
              p
              0
              L
            
          
          =
          
            
              (
              
                
                  N
                  -
                  K
                
                N
              
              )
            
            L
          
        
        ⁢
        
          

        
        ⁢
        and
      
    
    
      
        (
        18
        )
      
    
  
  
    
      
        
          
            P
            
              issue
              ⁢
              
                -
              
              ⁢
              2
            
          
          ≈
          
            
              ∑
              
                m
                =
                2
              
              D
            
            ⁢
            
              
            
            ⁢
            
              
                (
                
                  
                    
                      D
                    
                  
                  
                    
                      m
                    
                  
                
                )
              
              ⁢
              
                
                  (
                  
                    p
                    0
                    
                      (
                      
                        L
                        -
                        1
                      
                      )
                    
                  
                  )
                
                m
              
              ⁢
              
                
                  (
                  
                    1
                    -
                    
                      p
                      0
                      
                        (
                        
                          L
                          -
                          1
                        
                        )
                      
                    
                  
                  )
                
                
                  (
                  
                    D
                    -
                    m
                  
                  )
                
              
            
          
        
        =
        
          
            ∑
            
              m
              =
              2
            
            D
          
          ⁢
          
            
          
          ⁢
          
            
              (
              
                
                  
                    D
                  
                
                
                  
                    m
                  
                
              
              )
            
            ⁢
            
              
                (
                
                  
                    N
                    -
                    K
                  
                  N
                
                )
              
              
                m
                ⁡
                
                  (
                  
                    L
                    -
                    1
                  
                  )
                
              
            
            ⁢
            
              
                (
                
                  1
                  -
                  
                    
                      (
                      
                        
                          N
                          -
                          K
                        
                        N
                      
                      )
                    
                    
                      (
                      
                        L
                        -
                        1
                      
                      )
                    
                  
                
                )
              
              
                (
                
                  D
                  -
                  m
                
                )
              
            
          
        
      
    
    
      
        (
        19
        )
      
    
  




It is easy to see that when K<<N, the two probabilities can be made relatively large by selecting proper values for L and D. For example, when K=100, N=1000, L=10, and D=4, Pissue-1=34.87% and Pissue-2=50.29%.
In the case of issue-1, the measurement symbol is zero. Also notice that if a measurement symbol is associated with one or more non-zero source symbols, these non-zero symbols are very unlikely to complete cancel each other (see formula (16)), so the measurement symbol has a probability close to 1 of being non-zero. Therefore if a zero measurement symbol appears, it can be deduced that all source symbols associated with it are zeros.
In the case of issue-2, the two 1-degree measurements of the non-zero source symbol are related as follows. Suppose that a non-zero source symbol xi has two 1-degree measurements in dimensions d1 and d2. Denote ji(dm)=|Πdm(i)−1)/L| and li(dm)=mod(Πdm(i)−1,L)+1 for m=1 and 2. The values of these two 1-degree measurements have the following relationship




  
    
      
        
          x
          i
        
        =
        
          
            
              y
              
                f
                i
                
                  (
                  
                    d
                    1
                  
                  )
                
              
              
                (
                
                  d
                  1
                
                )
              
            
            
              a
              
                
                  j
                  i
                  
                    (
                    
                      d
                      1
                    
                    )
                  
                
                ,
                
                  l
                  i
                  
                    (
                    
                      d
                      1
                    
                    )
                  
                
              
              
                (
                
                  d
                  1
                
                )
              
            
          
          =
          
            
              y
              
                f
                i
                
                  (
                  
                    d
                    2
                  
                  )
                
              
              
                (
                
                  d
                  2
                
                )
              
            
            
              a
              
                
                  j
                  i
                  
                    (
                    
                      d
                      2
                    
                    )
                  
                
                ,
                
                  l
                  i
                  
                    (
                    
                      d
                      2
                    
                    )
                  
                
              
              
                (
                
                  d
                  2
                
                )
              
            
          
        
      
    
    
      
        (
        20
        )
      
    
  




Clearly the value of xi can be calculated from formula (20).
Based on the above discussion of issues 1 and 2, then in Step S42, for each element {tilde over (y)}j(d) in {tilde over (y)} (d=1 ̃D, j=1 ̃J), it is judged whether the {tilde over (y)}j(d) equals to 0; and if the {tilde over (y)}j(d) equals to 0, setting xΠd−1((j−1)L+l)=0, for l=1 ̃L, wherein {tilde over (y)} is initialized as {tilde over (y)}=y; and
for each source symbol xi in the sparse signal vector x, for i=1 ̃N, it is judged whether the following equation is satisfied,




  
    
      
        
          
            
              
                y
                ~
              
              
                j
                i
                
                  (
                  
                    d
                    m
                  
                  )
                
              
              
                (
                
                  d
                  m
                
                )
              
            
            
              a
              
                
                  j
                  i
                  
                    (
                    
                      d
                      m
                    
                    )
                  
                
                ,
                
                  l
                  i
                  
                    (
                    
                      d
                      m
                    
                    )
                  
                
              
              
                (
                
                  d
                  m
                
                )
              
            
          
          =
          
            
              
                y
                ~
              
              
                j
                i
                
                  (
                  
                    d
                    n
                  
                  )
                
              
              
                (
                
                  d
                  n
                
                )
              
            
            
              a
              
                
                  j
                  i
                  
                    (
                    
                      d
                      n
                    
                    )
                  
                
                ,
                
                  l
                  i
                  
                    (
                    
                      d
                      n
                    
                    )
                  
                
              
              
                (
                
                  d
                  n
                
                )
              
            
          
        
        ,
        
          
            ∀
            
              d
              m
            
          
          =
          
            1
            ∼
            D
          
        
        ,
        
          
            d
            n
          
          =
          
            1
            ∼
            D
          
        
        ,
        
          
            d
            m
          
          ≠
          
            d
            n
          
        
      
    
    
      
        (
        21
        )
      
    
  




and if it is satisfied for a pair of (dm, dn), setting




  
    x
    i
  
  =
  
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
    .
  




Then in Step S43, {tilde over (y)} is updated by subtracting the recovered sparse signals from the measurement vector y, by following formula:

{tilde over (y)}=y−A{tilde over (x)}  (22)

wherein {tilde over (x)} represents the recovered sparse signal vector, in which the un-recovered symbols are set to 0.
Such an interference cancellation operation will help the recovery of other symbols in two ways. Firstly, if a measurement symbol has all but one associated source symbols recovered, then the only un-recovered source symbol can be directly estimated from the measurement (after interference cancellation). Secondly, if a non-zero source symbol is subtracted from y, the degrees of its associated measurements are decreased by 1. This might introduce new cases of issues 1 and 2 in {tilde over (y)}, so the above two rules can be applied again to re-generate more source symbols from {tilde over (y)}. This motivates the use of an iterative process along with interference cancellation to recursively recover x.
Then, in Step S44, above Steps S42 and S43 are repeated for a plurality of iterations until a predetermined condition is satisfied.
The predetermined condition may for example comprise any of the followings:

    
    
        a fixed number of iterations has been reached;
        successful recovery of all source symbols in the sparse signal vector;
        {tilde over (x)}new={tilde over (x)}old, wherein {tilde over (x)}old and {tilde over (x)}new represent {tilde over (x)} before and after one iteration.
    
    


The complexity of the above proposed iterative algorithm in this embodiment is O(N).
In the following text, the performance of above proposed technical solution is illustrated using numerical results. Consider sparse source signals x with Gaussian random variables as non-zero elements. The measurement matrix A is generated according to formula (14) and (15), with Gaussian random variables as non-zero entries. In the simulation, set p1=K/N=0.1 and 0.05, and the dimension number D=4. The group length L is adjusted to get different sketch lengths M. The iterative recovery process is terminated when any one of the following three conditions is met: a) T=20 iterations are researched; b) all source symbols have been successfully recovered; c) {tilde over (x)}new={tilde over (x)}old with {tilde over (x)}old and {tilde over (x)}new the version of {tilde over (x)} before and after one iteration. The average iteration number is Tave=6 ̃10, depending on M/N.
FIG. 5 shows the relationship between the measurement number M and K log2(N/K). The purpose of this figure is to show that the proposed technique can achieve the lower bound of the sketch length O(K log 2(N/K)). It is shown in FIG. 5 that for the above proposed technical solution, the required number of measurements M grows linearly with the signal length N. Here, set N from 1000 to 10000, and M is selected to guarantee a recovery failure probability no larger than 0.01. Note that a recovery failure occurs when ∥{tilde over (x)}final−x∥22/∥x∥22>10−2, where {tilde over (x)}final denotes {tilde over (x)} after the final iteration. FIG. 5 clearly shows that the proposed technical solution can empirically achieve the sketch length lower bound of α·K log2(N/K) with a very small value of α around 0.6 ̃0.8.
FIGS. 6a and 6b show the relationships between the recovery failure probability and α (i.e., the sketch length) for the above proposed technical solution. Here, fix N=4000 and change α from 0.5 ̃1.2. For comparison, it is also shown in FIG. 6 the recovery failure probability for existing recovery algorithms based on l1 magic and Bayesian framework. As it is can be seen, to achieve the same recovery accuracy, the above proposed technical solution requires much less measurements than the l1 magic and Bayesian framework. Such a good sketch length is achieved with a linear complexity. The good empirical performance and linear recovery complexity make the proposed technique a potential and attractive solution to compressive sensing with sparse matrices.
The technical solution of the invention has been described above from the perspective of methods, and hereinafter the technical solution of the invention will be further described from the perspective of apparatus.
According to one embodiment of the invention, there is further provided an apparatus for measuring and recovering sparse signals. The apparatus comprises: a measuring means, for performing a linear measurement to a length-N sparse signal vector x={xi} with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein the measurement matrix A is denoted by:




  A
  =
  
    [
    
      
        
          
            
              A
              1
            
            ⁢
            
              Π
              1
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              d
            
            ⁢
            
              Π
              d
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              D
            
            ⁢
            
              Π
              D
            
          
        
      
    
    ]
  




wherein AdΠd is a sub-matrix of the measurement matrix A, for d=1 ̃D, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is denoted by:




  
    A
    d
  
  =
  
    [
    
      
        
          
            
              a
              
                1
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                1
                ,
                L
              
              
                (
                d
                )
              
            
          
        
        
          
            
          
        
        
          0
        
      
      
        
          
            
          
        
        
          ⋱
        
        
          
            
          
        
      
      
        
          0
        
        
          
            
          
        
        
          
            
              a
              
                J
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                J
                ,
                L
              
              
                (
                d
                )
              
            
          
        
      
    
    ]
  




wherein M<<N, J*L=N, the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number; and
a recovering means, for recovering the length-N sparse signal vector x={xi} from the length-M measurement vector y.
Advantageously, when the sparse signals are digital sparse signals and each entry xi in the sparse signal vector x is taken from a finite set Q={X0=0, X1, . . . , XQ-1} with Xq a non-zero figure, for q=1 ̃Q−1, and Q the size of the set, the recovering means may further comprise: a detecting means including D maximum likelihood detectors, for respectively performing maximum likelihood detection for D dimensions, wherein the dth maximum likelihood detector is used for performing maximum likelihood detection for the dth dimension; wherein the detecting means repeats above detection for a plurality of iterations until a predetermined condition is satisfied; and an estimating means, for estimating the source symbols in the sparse signal vector based on the output of the Dth maximum likelihood detector in the last iteration.
To be specific, for the dth dimension, the dth maximum likelihood detector in the detecting means is used for performing a maximum likelihood detection, based on the measurement symbols of the dth dimension and a prior information in the dth dimension, to generate a posterior information of the source symbols in the sparse signal vector, wherein the a posterior information generated in the dth dimension is used for updating a prior information in the (mod(d, D)+1)th dimension.
Preferably, the a posterior information p(d)(xi=Xq) in the dth dimension is used for updating the a prior information {tilde over (p)}(mod(d,D)+1)(xi=Xq) in the (mod(d, D)+1)th dimension, by following formula:

{tilde over (p)}(mod(d,D)+1)(xi=Xq)=p(d)(xi=Xq)/e(mod(d,D)+1)(xi=Xq)

wherein the e(mod(d,D)+1)(xi=Xq) is generated in the (mod(d, D)+1)th dimension in the previous iteration, which represents the extrinsic probability of xi being Xq in the (mod(d, D)+1)th dimension, for q=0 ̃Q−1, the e(mod(d,D)+1)(xi=Xq) is initialized as 1 in the first iteration, and e(mod(d,D)+1)(xi=Xq)=p(mod(d,D)+1)(xi=Xq)/{tilde over (p)}(mod(d,D)+1)(xi=Xq).

The predetermined condition may for example comprise any of the followings:

    
    
        a fixed number of iterations has been reached;
        the difference between the a posteriori probabilities generated in the Dth maximum likelihood detector in the two successive iterations is below a pre-defined threshold.
    
    


Advantageously, when the sparse signals are analog sparse signals, the recovering means may further comprise:
a judging means, for, for each element {tilde over (y)}j(d) in {tilde over (y)}, for d=1 ̃D, j=1 ̃J, judging whether the {tilde over (y)}j(d) equals to 0, wherein {tilde over (y)} is initialized as {tilde over (y)}=y;
and if the {tilde over (y)}j(d) equals to 0, setting xΠd−1((j−1)L+l)=0, for l=1 ̃L;
wherein Πd(i) is the index of xi in the dth permutated version and Πd−1(i) the inverse operation of Πd(i);
and
for each source symbol xi in the sparse signal vector x, for i=1 ̃N, judging whether the following equation is satisfied,




  
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              n
            
            )
          
        
        
          (
          
            d
            n
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                n
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                n
              
              )
            
          
        
        
          (
          
            d
            n
          
          )
        
      
    
  
  ,
  
    
      ∀
      
        d
        m
      
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      d
      n
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      
        d
        m
      
      ≠
      
        d
        n
      
    
    ;
  




and if it is satisfied for a pair of (dm, dn), setting




  
    
      x
      i
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
  
  ;




wherein ji(dm)=|(Πdm(i)−1)/L| and li(dm)=mod(Πdm(i)−1,L)+1;
and
an updating means, for updating {tilde over (y)} by subtracting the recovered sparse signals from the measurement vector y, by following formula:

{tilde over (y)}=y−A{tilde over (x)}

wherein {tilde over (x)} represents the recovered sparse signal vector, in which the un-recovered symbols are set to 0;
wherein the judging means repeats above judgments for a plurality of iterations until a predetermined condition is satisfied.
The predetermined condition may for example comprise any of the followings:

    
    
        a fixed number of iterations has been reached;
        successful recovery of all source symbols in the sparse signal vector;
        {tilde over (x)}new={tilde over (x)}old, wherein {tilde over (x)}old and {tilde over (x)}new represent {tilde over (x)} before and after one iteration.
    
    


It should be noted that the above described embodiments are given for describing rather than limiting the invention, and it is to be understood that modifications and variations may be resorted to without departing from the spirit and scope of the invention as those skilled in the art readily understand. Such modifications and variations are considered to be within the scope of the invention and the appended claims. The protection scope of the invention is defined by the accompanying claims. In addition, any of the reference numerals in the claims should not be interpreted as a limitation to the claims. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The indefinite article “a” or “an” preceding an element or step does not exclude the presence of a plurality of such elements or steps.

## Claims

1. A method for measuring and recovering sparse signals comprising:
a. performing a linear measurement to a length-N sparse signal vector x={xi} with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein the measurement matrix A is denoted by:





  A
  =
  
    [
    
      
        
          
            
              A
              1
            
            ⁢
            
              Π
              1
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              d
            
            ⁢
            
              Π
              d
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              D
            
            ⁢
            
              Π
              D
            
          
        
      
    
    ]
  



wherein AdΠd is a sub-matrix of the measurement matrix A, for d=1 ̃D, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is denoted by:





  
    A
    d
  
  =
  
    [
    
      
        
          
            
              a
              
                1
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                1
                ,
                L
              
              
                (
                d
                )
              
            
          
        
        
          
            
          
        
        
          0
        
      
      
        
          
            
          
        
        
          ⋱
        
        
          
            
          
        
      
      
        
          0
        
        
          
            
          
        
        
          
            
              a
              
                J
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                J
                ,
                L
              
              
                (
                d
                )
              
            
          
        
      
    
    ]
  



wherein M<<N, J*L=N, the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number; and
b. recovering the length-N sparse signal vector x={xi} from the length-M measurement vector y.

2. A method according to claim 1, wherein when the sparse signals are digital sparse signals and each entry xi in the sparse signal vector x is taken from a finite set Q={X0=0, X1, . . . , XQ-1} with Xq a non-zero figure, for q=1<Q−1, and Q the size of the set, the step b comprises:
b1. using D maximum likelihood detectors to respectively perform maximum likelihood detection for D dimensions, wherein the dth maximum likelihood detector is used for performing maximum likelihood detection for the dth dimension;
b2. repeating step b1 for a plurality of iterations until a predetermined condition is satisfied;
b3. estimating the source symbols in the sparse signal vector based on the output of the Dth maximum likelihood detector in the last iteration.

3. A method according to claim 2, wherein the step b1 comprises:
for the dth dimension, performing a maximum likelihood detection, based on the measurement symbols of the dth dimension and a prior information in the dth dimension, to generate a posterior information of the source symbols in the sparse signal vector, wherein the a posterior information generated in the dth dimension is used for updating a prior information in the (mod(d, D)+1)th dimension.

4. A method according to claim 3, wherein the a posterior information p(d) (xi=Xq) in the dth dimension is used for updating the a prior information {tilde over (p)}(mod(d,D)+1)(xi=Xq) in the (mod(d, D)+1)th dimension, by following formula:


{tilde over (p)}(mod(d,D)+1)(xi=Xq)=p(d)(xi=Xq)/e(mod(d,D)+1)(xi=Xq)


wherein the e(mod(d,D)+1)(xi=Xq) is generated in the (mod(d, D)+1)th dimension in the previous iteration, which represents the extrinsic probability of xi being Xq in the (mod(d, D)+1)th dimension, for q=0 ̃Q−1, the e(mod(d,D)+1)(xi=Xq) is initialized as 1 in the first iteration, and updated in the following iterations as e(mod(d,D)+1)(xi=Xq)=p(mod(d,D)+1)(xi=Xq)/{tilde over (p)}(mod(d,D)+1)(xi=Xq).

5. A method according to claim 2, wherein the predetermined condition comprises any of the followings:
a fixed number of iterations has been reached;
the difference between the a posteriori probabilities generated in the Dth maximum likelihood detector in the two successive iterations is below a pre-defined threshold.

6. A method according to claim 1, wherein when the sparse signals are analog sparse signals, the method b comprises:
i. for each element {tilde over (y)}j(d) in {tilde over (y)}, for d=1 ̃D,j=1 ̃J, judging whether the {tilde over (y)}j(d) equals to 0, wherein {tilde over (y)} is initialized as {tilde over (y)}=y, and {tilde over (y)}j(d) is the jth element in the dth dimension of {tilde over (y)};
ii. if the {tilde over (y)}j(d) equals to 0, setting xΠd−1((j−1)L+l)=0, for l=1 ̃L;
wherein Πd(i) is the index of xi in the dth permutated version and Πd−1(i) the inverse operation of Πd(i);
and the method further comprise:
u. for each source symbol xi in the sparse signal vector xi for i=1 ̃N, judging whether the following equation is satisfied,





  
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              n
            
            )
          
        
        
          (
          
            d
            n
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                n
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                n
              
              )
            
          
        
        
          (
          
            d
            n
          
          )
        
      
    
  
  ,
  
    
      ∀
      
        d
        m
      
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      d
      n
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      
        d
        m
      
      ≠
      
        d
        n
      
    
    ;
  



v. if it is satisfied for a pair of (dm, dn), setting





  
    
      x
      i
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
  
  ;



wherein ji(dm)=|(Πdm(i)−1)/L| and li(dm)=mod(Πdm(i)−1,L)+1;
and after above steps, the method further comprises:
p. updating {tilde over (y)} by subtracting the recovered sparse signals from the measurement vector y, by following formula:


{tilde over (y)}=y−A{tilde over (x)}


wherein {tilde over (x)} represents the recovered sparse signal vector, in which the un-recovered symbols are set to 0;
q. repeating above steps for a plurality of iterations until a predetermined condition is satisfied.

7. A method according to claim 6, wherein the predetermined condition comprises any of the followings:
a fixed number of iterations has been reached;
successful recovery of all source symbols in the sparse signal vector;
{tilde over (x)}new={tilde over (x)}old wherein {tilde over (x)}old and {tilde over (x)}new represent {tilde over (x)} before and after one iteration.

8. An apparatus for measuring and recovering sparse signals comprising:
a measuring means, for performing a linear measurement to a length-N sparse signal vector x={xi} with a M×N measurement matrix A to obtain a length-M measurement vector y, wherein the measurement matrix A is denoted by:





  A
  =
  
    [
    
      
        
          
            
              A
              1
            
            ⁢
            
              Π
              1
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              d
            
            ⁢
            
              Π
              d
            
          
        
      
      
        
          ⋮
        
      
      
        
          
            
              A
              D
            
            ⁢
            
              Π
              D
            
          
        
      
    
    ]
  



wherein AdΠd is a sub-matrix of the measurement matrix A, for d=1 ̃D, Πd is a N×N random permutation matrix, and Ad is an J×N matrix which is denoted by:





  
    A
    d
  
  =
  
    [
    
      
        
          
            
              a
              
                1
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                1
                ,
                L
              
              
                (
                d
                )
              
            
          
        
        
          
            
          
        
        
          0
        
      
      
        
          
            
          
        
        
          ⋱
        
        
          
            
          
        
      
      
        
          0
        
        
          
            
          
        
        
          
            
              a
              
                J
                ,
                1
              
              
                (
                d
                )
              
            
            ⁢
            ...
            ⁢
            
              
            
            ⁢
            
              a
              
                J
                ,
                L
              
              
                (
                d
                )
              
            
          
        
      
    
    ]
  



wherein M<<N, J*L=N, the measurement symbols generated from the same permutation matrix are referred to as one dimension, and the D is the total dimension number; and
a recovering means, for recovering the length-N sparse signal vector x={xi} from the length-M measurement vector y.

9. An apparatus according to claim 8, wherein when the sparse signals are digital sparse signals and each entry xi in the sparse signal vector x is taken from a finite set Q={X0=0, X1, . . . , XQ-1} with Xq a non-zero figure, for q=1 ̃Q−1, and Q the size of the set, the recovering means comprises:
a detecting means including D maximum likelihood detectors, for respectively performing maximum likelihood detection for D dimensions, wherein the dth maximum likelihood detector is used for performing maximum likelihood detection for the dth dimension;
wherein the detecting means repeats above detection for a plurality of iterations until a predetermined condition is satisfied;
an estimating means, for estimating the source symbols in the sparse signal vector based on the output of the Dth maximum likelihood detector in the last iteration.

10. An apparatus according to claim 9, wherein for the dth dimension, the dth maximum likelihood detector in the detecting means is used for performing a maximum likelihood detection, based on the measurement symbols of the dth dimension and a prior information in the dth dimension, to generate a posterior information of the source symbols in the sparse signal vector, wherein the a posterior information generated in the dth dimension is used for updating a prior information in the (mod(d, D)+1)th dimension.

11. An apparatus according to claim 10, wherein the a posterior information p(d)(xi=Xq) in the dth dimension is used for updating the a prior information {tilde over (p)}(mod(d,D)+1)(xi=Xq) in the (mod(d, D)+1)th dimension, by following formula:


{tilde over (p)}(mod(d,D)+1)(xi=Xq)=p(d)(xi=Xq)/e(mod(d,D)+1)(xi=Xq)

wherein the e(mod(d,D)+1)(xi=Xq) is generated in the (mod(d, D)+1)th dimension in the previous iteration, which represents the extrinsic probability of xi being Xq in the (mod(d, D)+1)th dimension, for q=0 ̃Q−1, the e(mod(d,D)+1)(xi=Xq) is initialized as 1 in the first iteration, and updated in the following iterations as e(mod(d,D)+1)(xi=Xq)=p(mod(d,D)+1)(xi=Xq)/{tilde over (p)}(mod(d,D)+1)(xi=Xp).

12. An apparatus according to claim 9, wherein the predetermined condition comprises any of the followings:
a fixed number of iterations has been reached;
the difference between the a posteriori probabilities generated in the two successive iterations is below a pre-defined threshold.

13. An apparatus according to claim 8, wherein when the sparse signals are analog sparse signals, the recovering means comprises:
a judging means, for, for each element {tilde over (y)}j(d) in {tilde over (y)}, for d=1 ̃D, j=1 ̃J, judging whether the {tilde over (y)}j(d) equals to 0, wherein {tilde over (y)} is initialized as {tilde over (y)}=y, and {tilde over (y)}j(d) is the jth element in the dth dimension of {tilde over (y)};
and if the {tilde over (y)}j(d) equals to 0, setting xΠd−1((j−1)L+l)=0, for l=1 ̃L;
wherein Πd(i) is the index of xi in the dth permutated version and Πd−1 (i) the inverse operation of Πd (i);
and
for each source symbol xi in the sparse signal vector x, for i=1 ̃N, judging whether the following equation is satisfied,





  
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              n
            
            )
          
        
        
          (
          
            d
            n
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                n
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                n
              
              )
            
          
        
        
          (
          
            d
            n
          
          )
        
      
    
  
  ,
  
    
      ∀
      
        d
        m
      
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      d
      n
    
    =
    
      1
      ∼
      D
    
  
  ,
  
    
      
        d
        m
      
      ≠
      
        d
        n
      
    
    ;
  



and if it is satisfied for a pair of (dm, dn), setting





  
    
      x
      i
    
    =
    
      
        
          y
          ~
        
        
          j
          i
          
            (
            
              d
              m
            
            )
          
        
        
          (
          
            d
            m
          
          )
        
      
      
        a
        
          
            j
            i
            
              (
              
                d
                m
              
              )
            
          
          ,
          
            l
            i
            
              (
              
                d
                m
              
              )
            
          
        
        
          (
          
            d
            m
          
          )
        
      
    
  
  ;



wherein ji(dm)=|(Πdm(i)−1)/L| and li(dm)=mod(Πdm(i)−1,L)+1;
an updating means, for updating {tilde over (y)} by subtracting the recovered sparse signals from the measurement vector y, by following formula:


{tilde over (y)}=y−A{tilde over (x)}


wherein {tilde over (x)} represents the recovered sparse signal vector, in which the un-recovered symbols are set to 0;
wherein the judging means repeats above judgments for a plurality of iterations until a predetermined condition is satisfied.

14. An apparatus according to claim 13, wherein the predetermined condition comprises any of the followings:
a fixed number of iterations has been reached;
successful recovery of all source symbols in the sparse signal vector;
{tilde over (x)}new={tilde over (x)}old, wherein {tilde over (x)}old and {tilde over (x)}new represent {tilde over (x)} before and after one iteration.