Abstract:
Previously known analog transistor circuits that compute the “outer product” of two probability mass functions are extended to compute also divisions. Such circuits can be used in hardware implementations of certain algorithms including “generalized belief propagation”, which have applications in many inference problems including the decoding of error correcting codes.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     The present invention relates to a circuit and a method for signal processing. In particular, the invention relates to the computation of probability mass functions defined on finite sets. Such functions are of the form p: S→R + , where S={s 1 , . . . , s n } is a finite set, where R +  is the set of nonnegative real numbers, and where the function p satisfies the condition 
 
Σ k=1 . . . n  p(s k )=1.  (1) 
 
         [0002]     Such a function can be represented by a list (or vector) of function values (p(s 1 ), . . . , p(s n )). For sums as in (1), the simplified notation 
 
Σ s  p(s)=Σ k=1 . . . n  p(s k )  (2) 
 
 will also be used. 
 
         [0003]     In previous work (U.S. Pat. No. 6,282,559 B1; H.-A. Loeliger, F. Lustenberger, M. Helfenstein, and F. Tarkoy, “Probability propagation and decoding in analog VLSI”, Proc. 1998 IEEE Int. Symp. Inform. Th., Cambridge, Mass., USA, Aug. 16-21, 1998, p. 146; H.-A. Loeliger, F. Lustenberger, M. Helfenstein, F. Tarkoey, “Probability Propagation and Decoding in Analog VLSI,”, IEEE Transactions on Information Theory, vol. 47, no. 2, pp. 837-843, February 2001; F. Lustenberger, “On the Design of Analog VLSI Iterative Decoders”, PhD Thesis no. 13879, ETH Zurich, November 2000) analog transistor circuits were presented to compute a probability mass function p Z  (defined on some finite set S Z ={z 1 , . . . , z K }) from two probability mass functions p X  (defined on S X ={x 1 , . . . , x M }) and y (defined on S Y ={y 1 , . . . , y N }) according to the formula  
                 P   Z     ⁡     (     z   k     )       =     γ   ⁢       ∑     i   =     1   ⁢           ⁢   …   ⁢           ⁢   M         ⁢       ∑     j   =     1   ⁢           ⁢   …   ⁢           ⁢   N         ⁢         p   X     ⁡     (     x   i     )       ⁢       p   Y     ⁡     (     y   j     )       ⁢     f   ⁡     (       x   i     ,     y   j     ,     z   k       )                       (   3   )             
 
 or, equivalently, 
 
p Z (z)=γΣ x Σ y p X (x)p Y (y)f(x,y,z), (4) 
 
 where f is some (arbitrary) {0,1}-valued function (i.e. a function that returns either 0 or 1) and where y is a suitable scale factor such that Σ z  p Z (z)=1. Computations of the form (3) or (4) are the heart of the generic sum-product probability propagation algorithm, which has many applications including, in particular, the decoding of error correcting codes (see references cited above as well as H.-A. Loeliger, “An introduction to factor graphs,”, IEEE Signal Proc. Mag., January 2004, pp. 28-41). 
 
         [0004]     The core of the circuits proposed in U.S. Pat. No. 6,282,559 is shown in  FIG. 1 . The input to this circuit are the two current vectors I X (p X (x 1 ), . . . , p X (x M )) and I Y (p Y (y 1 ), . . . , p Y (y N )) with arbitrary sum currents I X  and I Y , respectively; the output of this circuit are the M-N products p X (x i )p Y (y j ), i=1 . . . N, j=1 . . . N, which are represented by currents: the term p X (x i )p y (y j ) is represented by the current  
         [0000]     I S p X (x i )p y (y j ),  
         [0000]     with sum current I S =I Y . It is then easy to compute (3) by summing currents. Note that all probabilities are represented as currents and are processed in parallel. The voltages in the circuit represent logarithms of probabilities.  
         [0005]     Recent research on improved probability propagation has produced algorithms that require the computation of expressions of the form  
                   p   Z     ⁡     (   z   )       =     γ   ⁢       ∑   x     ⁢       ∑   y     ⁢       ∑   w     ⁢       f   ⁡     (     x   ,   y   ,   z   ,   w     )       ⁢       p   X     ⁡     (   x   )       ⁢         p   Y     ⁡     (   y   )       /       p   W     ⁡     (   w   )                     ,           (   5   )             
 
 where everything is as in (4) except for the division by p W (w), where p W  is also a probability mass function. 
 
         [0006]     Examples of such algorithms include “generalized belief propagation” (J. S. Yedidia, W. T. Freeman, and Y. Weiss, “Generalized Belief Propagation”, Advances in Neural Information Processing Systems (NIPS), vol. 13, pp. 689-695, December 2000; R. J. McEliece and M. Yildirim, “Belief propagation on partially ordered sets”, in Mathematical Systems Theory in Biology, Communication, Computation, and Finance, J. Rosenthal and D. S. Gilliam, eds., IMA Volumes in Math. and Appl., vol. 134, Springer Verlag, 2003, pp. 275-299) and “structured-summary propagation” (J. Dauwels, H.-A. Loeliger, P. Merkli, and M. Ostojic, “On structured-summary propagation, LFSR synchronization, and low-complexity trellis decoding”, Proc. 41st Allerton Conf. on Communication, Control, and Computing. Monticello, Ill., Oct. 1-3, 2003, pp. 459-467). Such algorithms cannot be implemented by the circuit of  FIG. 1 .  
       BRIEF SUMMARY OF THE INVENTION  
       [0007]     Hence, it is a general object of the invention to provide a circuit and method able to calculate terms as shown in (5).  
         [0008]     Now, in order to implement these and still further objects of the invention, which will become more readily apparent as the description proceeds, in a first aspect the invention relates to a circuit for signal processing that comprises at least one circuit section, each circuit section comprising 
        Q first inputs a 1  . . . a Q ,     R second inputs b 1  . . . b R ,     a third input c,     RXQ outputs d 11  . . . d QR ,     RXQ first transistors T 11  . . . T QR , a gate of each first transistor T ij  being connected to the first input a i , a source of each first transistor T ij  being connected to the second input b j , and a drain of each first transistor T ij  being connected to the output d ij ,     Q second transistors TX 1  . . . TX Q , a gate and a drain of each second transistor TX i  being connected to the first input a i  and a source of each second transistor TX i  being connected to the third input c,     R third transistors TY 1  . . . TY R , a gate and a drain of each third transistor TY 1  being connected to a reference voltage and a source of each third transistor TY j  being connected to the second input b j , and a fourth transistor TW, a gate and a drain of the fourth transistor TW being connected to the reference voltage and a source of the fourth transistor TW being connected to the third input c.        
 
         [0016]     In a further aspect, the invention relates to a method for the parallel processing of terms 
 
p X (x m )p Y (y n )/p W (w k ) 
 
 where, p X (x m ), p y (y n ) and p W (w k ) are non-negative real-valued functions, x m  stands for an element {x 1  . . . x M } of a first finite set having M elements, y n  stands for an element {y 1  . . . y N } of a second finite set having N elements and w k  stands for an element {w 1  . . . w L } of a third finite set having L elements, wherein a plurality of the terms with differing i, j and k are calculated by providing a circuit comprising L circuit sections, wherein each circuit section comprises 
        Q≦M first inputs a 1  . . . a Q ,     R≦N second inputs b 1  . . . b R ,     a third input c,     RXQ outputs d 11  . . . d QR ,     RXQ first transistors T 11  . . . T QR , a gate of each first transistor T ij  being connected to the first input a i , a source of each first transistor T ij  being connected to the second input b j , and a drain of each first transistor T ij  being connected to the output d ij ,     Q second transistors TX 1  . . . TX Q , a gate and a drain of each second transistor TX i  being connected to the first input a i  and a source of each second transistor TX i  being connected to the third input c,     R third transistors TY 1  . . . TY R , a gate and a drain of each third transistor TY j  being connected to a reference voltage and a source of each third transistor TY j  being connected to the second input b j , and a fourth transistor TW, a gate and a drain of the fourth transistor TW being connected to the reference voltage and a source of the fourth transistor TW being connected to the third input c, said method further comprising the steps of     feeding a current proportional to p X (x m ) to each of said first inputs a i ,     feeding a current proportional to p Y (y n ) to each of said second inputs b j ,     feeding a current proportional to p W (w k ) to each of said third inputs c,     thereby generating a plurality of currents proportional to a plurality of said terms at said outputs.        
 
         [0028]     In yet a further aspect, the invention relates to a method for calculating a probability mass function p z (z) on a finite set S z  from 
 
p Z (z)=γΣ x Σ y  Σ w  f(x,y,z,w) p X (x)p Y (y)/p W (w), 
 
 wherein p X (x), P Y (y) and p W (w) are probability mass functions defined on finite sets S X , S Y  and S W , and f(x,y,z,w) is a {0, 1}-valued function, and where γ is a scaling factor, said method comprising the steps of the method of the second aspect as well as the step of adding at least some of the currents at the outputs d 11  . . . d QR . 
 
         [0029]     As is shown below, the desired terms can be calculated efficiently with one or more of the described circuit sections.  
         [0030]     The term “transistor” in the present text and claims is to be understood to designate any type of transistor, such as a FET transistor or a bipolar transistor, as well as a combination of individual transistors having equivalent properties, such as a Darlington transistor or a cascode.  
         [0031]     The term “gate” in the present text and claims refers to the control input of a transistor. Since the transistors used in the present invention can be FET as well as bipolar transistors, the term “gate” is also to be understood as designating the base if bipolar transistors are used. Similarly, the terms “drain” and “source” are to be understood as designating the collector and emitter, respectively, if bipolar transistors are used. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0032]     The invention will be better understood and objects other than those set forth above will become apparent when consideration is given to the following detailed description thereof. Such description makes reference to the annexed drawings, wherein:  
         [0033]      FIG. 1  shows a prior art multiplier,  
         [0034]      FIG. 2  shows a circuit for 8 input values and 18 output values,  
         [0035]      FIG. 3  shows a circuit for 10 input values and 8 output values calculating part of the corresponding product ratio terms,  
         [0036]      FIG. 4  shows one circuit section of a generalized version of the circuit of  FIG. 2 ,  
         [0037]      FIG. 5  is a component of an application of the invention, and  
         [0038]      FIG. 6  shows an application of the invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0039]     The invention provides a circuit to produce output currents 
 
I S p Z (x)p Y (y)/p W (w)  (6) 
        (for some reference current I S ) for all x, y, and w in parallel.  FIG. 2  shows an example of such a circuit where the sets S X  and S Y  both have M=N=3 elements and the set S W ={w 1 , . . . , w L } (the domain of p W ) has L=2 elements. To compute (5), the required output currents can be summed. As the original circuit of  FIG. 1 , copies of the new circuit of  FIG. 2  can easily be connected (and combined with circuits as in  FIG. 1 ) to large networks.        
 
         [0041]     If some term p X (x)p Y (y)/p W (w) is not used in this sum, the corresponding current must flow nonetheless; this may be achieved by connecting the corresponding output to some suitable reference voltage. However, if, for some fixed x, no such term is used, then the corresponding row of transistors may be omitted. Similarly, if, for some fixed y, no such term is used, then the corresponding column of transistors may be omitted. This is illustrated in  FIG. 3  where M=N=4, but only the terms 
        p X (x 1 )p Y (y 1 )/p W (w 1 ), p X (x 1 )p Y (y 2 )/p W (w 1 ), p X (x 2 )p Y (y 1 )/p W (w 1 ), p X (x 2 )p Y (y 2 )/p W (w 1 ), p X (x 3 )p Y (y 3 )/p W (w 2 ), p X (x 3 )p Y (y 4 )/p W (w 2 ), p X (x 4 )p Y (y 3 )/p W (w 2 ), p X (x 4 )p Y (y 4 )/p W (w 2 ), are used.        
 
         [0043]     The new circuit (exemplified by  FIGS. 2 and 3 ) works as follows. First, we note that it consists of L circuit sections  1 , where L is the cardinality of SW. In most applications, we have L&gt;2. The general form of one such circuit section is shown in  FIG. 4 . The circuit section of  FIG. 4  has 
        2≦Q≦M first inputs a 1  . . . a Q  —in the example of  FIG. 4  they carry the currents I x P x (x 1 ) . . . I x P x (x Q ); in general, the first inputs a 1  . . . a Q  carry the currents belonging to a subset of set S X , wherein each first input carries the current belonging to a different member of set S X ;     2&lt;R&lt;N second inputs b 1  . . . bR—in the example of  FIG. 4  they carry the currents I y P y (y 1 ) . . . I y P y (y R ); in general, the second inputs b 1 , . . . b R  carry the currents belonging to a subset of set S Y , wherein each first input carries the current belonging to a different member of set S Y ;     a third input c—in the example of  FIG. 3  it carries the current I W P W (w 1 ); in general, the third input c of the n-th circuit section  1  carries the current I W P W (w n );     RxQ outputs d 11  . . . d QR  carrying currents I 1,1  . . . I Q,R , which correspond to the terms (6) calculated for the applied inputs,     RxQ first transistors T 11  . . . T QR , the gate of each first transistor T ij  being connected to the first input a i , the source to the second input b j , and the drain to the output d ij ,     Q second transistors TX 1  . . . T XQ , the gate and the drain of each second transistor T Xi  being connected to the first input a i  and the source to the third input c,     R third transistors TY 1  . . . TY R , the gate and the drain of each third transistor TY j  being connected to a reference voltage V ref  and the source to the second input b j , and     a fourth transistor TW, the gate and the drain of which is connected to the reference voltage V ref  and the source to the third input c.        
 
         [0052]     All L circuit sections are of the same design but may have different R and Q.  
         [0053]     We assume that all the transistors function as voltage controlled current sources with an exponential relation between the current and the control voltage.  
         [0054]     This assumption holds both for bipolar transistors and for MOS-FET transistors in weak inversion. In the following we use the notation for MOS-FET transistors: 
 
I drain   =I   0  exp((κ· V   gate   −V   source )/ U   T ), (7) 
 
 where I drain  is the drain current, V gate  is the gate potential, V source  is the source potential, U T  is the thermal voltage, I 0  is some technology dependent current, and K is some technology dependent dimensionless constant. The currents and voltages in  FIG. 3  then satisfy both  
                         I     i   ,   j       /     (       I   Y     ⁢       p   Y     ⁡     (     y   j     )         )       =       ⁢       {       I   0     ⁢     exp   ⁡     (       (       κ   ·     V     X   ,   i         -     V     Y   ,   j         )     /     U   T       )         }     /                     ⁢     {         I   0     ⁢     exp   ⁡     (       (       κ   ·     V   ref       -     V     Y   ,   j         )     /     U   T       )         +                       ⁢       ∑     k   =     1   ⁢           ⁢   …   ⁢           ⁢   Q         ⁢       I   0     ⁢     exp   ⁡     (       (       κ   ·     V     X   ,   k         -     V     y   ,   j         )     /     U   T       )           }               =       ⁢       exp   ⁡     (     κ   ·       V     X   ,   i       /     U   T         )       /     {       exp   ⁡     (     κ   ·       V   ref     /     U   T         )       +                         ⁢       ∑     k   =     1   ⁢           ⁢   …   ⁢           ⁢   Q         ⁢     exp   ⁡     (     κ   ·       V     X   ,   k       /     U   T         )         }           ⁢     
     ⁢   and           (   8   )                         I   X     ⁢         p   X     ⁡     (     x   i     )       /     (       I   W     ⁢       p   W     ⁡     (     w   1     )         )         =       ⁢       {       I   0     ⁢     exp   ⁡     (       (       κ   ·     V     X   ,   i         -     V   W       )     /     U   T       )         }     /                     ⁢     {         I   0     ⁢     exp   ⁡     (       (       κ   ·     V   ref       -     V   W       )     /     U   T       )         +                       ⁢       ∑     k   =     1   ⁢           ⁢   …   ⁢           ⁢   Q         ⁢       I   0     ⁢     exp   ⁡     (       (       κ   ·     V     X   ,   k         -     V   W       )     /     U   T       )           }               =       ⁢       exp   ⁡     (     κ   ·       V     X   ,   i       /     U   T         )       /     {       exp   ⁡     (     κ   ·       V   ref     /     U   T         )       +                         ⁢       ∑     k   =     1   ⁢           ⁢   …   ⁢           ⁢   Q         ⁢     exp   ⁡     (     κ   ·       V     X   ,   k       /     U   T         )         }                 (   9   )             
 
         [0055]     The right-hand sides of (8) and (9) are identical, which implies 
 
 I   i,j /( I   Y   p   Y ( y   j ))= I   X   p   X ( x   i )/( I   W   p   W ( w   i ))  (10) 
 
or 
 
 I   i,j   =I   X   ·I   Y   /I   W   ·p   X (x i )·p Y (y j )/p w (w 1 ).  (11) 
 
 Note that (11) is equivalent to (6) with I S =I X ·I Y /I W . 
 
         [0056]     There is a small catch: the above analysis holds only if the condition 
 
I W p W (w 1 )≧Σ k=1 . . . Q  I X p X (x k )  (12) 
 
 is satisfied. In other words, the current fed to the third input c exceeds the sum of the currents fed to the first inputs a i . It should therefore be pointed out that, in algorithms as in J. Dauwels, H.-A. Loeliger, P. Merkli, and M. Ostojic cited above, the probability distribution P W  in (5) is not an independent input, but is derived from p X  and p Y  applied to the same circuit section  1 , as is shown in  FIG. 5 . In such applications, the condition (12) may be satisfied automatically. For example, let M=N=4 and L=2 and assume that p W  is defined by 
 
 p   W ( w   1 )=(1/2)·( p   X (x 1 )+ p   X ( x   2 )+ p   Y ( y   1 )+ p   Y ( y   2 )) 
 
and 
 
 p   W ( w   2 )=(1/2)·( p   X ( x   3 )+ p   X ( x   4 )+ p   Y ( y   3 )+ p   Y ( y   4 )). 
 
 (In other words, p W  is an average of two marginal distributions derived from p X  and from p Y , respectively; or, in yet other words, each value p W (w k ) is proportional to a sum of part of the values p X (x m ) and part of the values p Y (y n ), namely of those values that are fed to the same circuit section  1  as the given P W (w k ).) 
 
         [0057]     This may be realized as shown in  FIG. 6  with input sum currents I X =I Y . The sections  1  labeled “mult/div” represent a section  1  as shown in  FIG. 4  (one half of  FIG. 3 ) and the blocks labeled “copy” produce a copy of the current passed through it. The copied currents are added in an adder  2  by applying them in parallel to the input c. An adder is attributed to each circuit section  1 . The outputs c ij  of the circuit are proportional to 
        p X (x 1 )p Y (y 1 )/p W (w 1 ), p X (x 1 )p Y (y 2 )/p W (w 1 ), p X (x 2 )p Y (y 1 )/p W (w 1 ), p X (x 2 )p Y (y 2 )/p W (w 1 ), p X (x 3 )p Y (y 3 )/p W (w 2 ), p X (x 3 )p Y (y 4 )/p W (w 2 ), p X (x 4 )p Y (y 3 )/p W (w 2 ), p X (x 4 )p Y (y 4 )/p W (w 2 ), 
 
 represented as currents with some common sum current Is. 
       
 
         [0059]     In the examples of  FIGS. 3 and 6 , the numbers M and N divisible by L (which is equal to 2 in both embodiments) and we have Q=M/L and R=N/L for each circuit section. This is typical for most probability computations.  
         [0060]     While there are shown and described presently preferred embodiments of the invention, it is to be distinctly understood that the invention is not limited thereto but may be otherwise variously embodied and practised within the scope of the following claims.