Abstract:
Method and apparatus for performing close-loop programming of resistive memory devices in crossbar array based hardware circuits and systems. Invention provides iterative training of memristor crossbar arrays for neural networks by applying voltages corresponding to selected training patterns. Error is detected and measured as a function of the actual response to the training patterns versus the expected response to the training pattern.

Description:
PRIORITY CLAIM UNDER 35 U.S.C. §119(e) 
       [0001]    This patent application claims the priority benefit of the filing date of provisional application Ser. No. 61/917,439, having been filed in the United States Patent and Trademark Office on Dec. 18, 2013 and now incorporated by reference herein. 
     
    
     STATEMENT OF GOVERNMENT INTEREST 
       [0002]    The invention described herein may be manufactured and used by or for the Government for governmental purposes without the payment of any royalty thereon. 
     
    
     BACKGROUND OF THE INVENTION 
       [0003]    As the need for high performance computing continuously increases, the traditional Von Neumann computer architecture becomes less efficient. In recent years, neuromorphic hardware systems built upon the conventional CPU, GPU, or FPGA [4] gained a great deal of attention from people in industry, government and academia. Such systems can potentially provide the capabilities of biological perception and cognitive information processing within a compact and energy-efficient platform [1], [2]. 
         [0004]    As a highly generalized and simplified abstract of a biological system, an artificial neural network usually uses a weight matrix to represent a set of synapses. Accordingly, the net inputs of a group or groups of neurons can be transformed into matrix-vector multiplication(s). Similar to biological systems, neural network algorithms are inherently adaptive to the environment and resilient to random noise, however, hardware realizations of neural networks require a large volume of memory and are associated with high hardware cost if built with digital circuits [2]. Algorithm enhancement can alleviate the situation but cannot fundamentally resolve it. Thus, more efficient hardware-level solutions are necessary. 
         [0005]    The existence of the memristor was predicted in circuit theory about forty years ago [5]. In 2008, the physical realization of a memristor was firstly demonstrated by HP Labs through a TiO 2  thin-film structure [6]. Afterwards, many memristive materials and devices have been rediscovered. Intrinsically, a memristor behaves similarly to a synapse: it can “remember” the total electric charge/flux ever to flow through it [8], [9]. Moreover, memristor-based memories can achieve a very high integration density of 100 Gbits/cm 2 , a few times higher than flash memory technologies [7]. These unique properties make it a promising device for massively-parallel, large-scale neuromorphic systems [10], [11]. 
         [0006]    For the purpose of succinct description, the present invention uses the terminology “memristor” to represent the category of “resistive memory device”. For the remainder of the patent description, references to “memristor” shall be regarded as referring to any “resistive memory device”. 
         [0007]    Based on circuit theory, an ideal memristor with memristance M builds the relationship between the magnetic flux φ and electric charge q that passes through the device, that is, dφ=M·dq. Since the magnetic flux and the electric charge are time dependent parameters, the instantaneous memristance varies with time and reflects the historical profile of the excitations through the device. 
         [0008]    When developing actual memristive devices, many materials have demonstrated memristive behavior in theory and/or experimentation via different mechanisms. In general, a certain energy (or threshold voltage) is required to enable a state change in a memristor. When the electrical excitation through a memristor is greater than the threshold voltage, e.g., |v in |&gt;|v th |, the memristance value changes. Otherwise, the memristor behaves like a resistor. 
       BACKGROUND ON OTHER RELATED PATENT PUBLICATIONS 
       [0009]    To the best of our knowledge, a few existing U.S. patent publications [2][3][4][5][6][7][8][9] cover methods involving the crossbar array of programmable resistors, Ref [9] covers the method of resistance switching for the general device. Ref. [2] covers a re-configurable resistor crossbar architecture and methods of performing Boolean logic functions on it. Ref [3][4][5][6][7][8] cover methods of performing signal processing function using the re-configurable resistor crossbar architecture. However, none of the existing related patent publications cover a specific method or apparatus on programming the reconfigurable resistors in the crossbar array architecture. The present invention covers new method and apparatus of programming the resistive memory devices in the two-crossbar-array architecture to the desired states, which do not exist in other related patent publications or elsewhere in the prior art as can best be determined. 
       OBJECTS AND SUMMARY OF THE INVENTION 
       [0010]    The present invention relates generally to methods and apparatuses for training arrays of resistive memory devices such as but not limited to those utilized in neural networks and neuromorphic computing applications. 
         [0011]    It is an object of the present invention to provide training for arrays of resistive memory devices that detects the difference between expected and actual array response to training signals. 
         [0012]    It is another object of the present invention to provide iteratively selected and applied training patterns to arrays of resistive memory devices. 
         [0013]    In an embodiment of the present invention, a method for programming neural networks having row and column arrays of resistive memory devices where the method comprises initializing the memristors to a predetermined resistance value; determining whether a predetermined number of sequences have been trained; selecting a training pattern; inputting the training pattern to the arrays and outputting a resulting signal; detecting error by computing the difference between the input training pattern and the output signal; determining whether the detected error is a vector of all logic zero values and if so, applying a method to update the overall training progress, and returning to the step of determining whether the training is completed; otherwise, applying the method to reset the overall training progress, generating programming signals to adjust the resistance states of the memristors in the crossbars, and returning to the step of determining whether the training is completed. 
         [0014]    In another embodiment of the present invention, an apparatus for programming neural networks having row and column arrays of resistive memory devices comprises a status tracker (ST) for determining whether the neural network training process is completed; a set of training patterns residing in a group of registers for selecting a training pattern and inputting the training pattern into the neural network; an error detector for computing the difference between the training pattern input into the neural network and an output signal therefrom; an arbiter for determining whether the detected error is a vector of all logic zero values and if so, where the status tracker updates the overall training progress, continues to select the next training pattern, and checks the training progress to determine whether the training is completed; otherwise a programming signal generator generates programming signals (based on the error detector output) to adjust the resistance states of the memristors in the crossbars, and the status tracker resets the overall training progress and checks the training progress to determine whether the training is completed. 
         [0015]    Briefly stated, the present invention provides a method and apparatus for performing close-loop programming of resistive memory devices in crossbar array based hardware circuits and systems. The invention provides iterative training of memristor crossbar arrays for neural networks by applying voltages corresponding to selected training patterns. Error is detected and measured as a function of the actual response to the training patterns versus the expected response to the training pattern. The resistance states of the memristors in the crossbars are adjusted based on the error detection output. 
         [0016]    In previous works, simple learning behaviors of memristor based synaptic circuits have been analyzed [24]-[28]. However, in studying the input-output feature of memristor crossbar that employs a memristor at each intersection of horizontal and vertical metal wires, we found this typical array structure can naturally provide the capability of weight matrix storage and matrix-vector multiplication (see  FIG. 1 ). Moreover, it offers a huge number of connections. Therefore, the present invention exploits the application of the memristor crossbars in neuromorphic hardware design and use the Brain-State-in-a-Box (BSB) model [12], [13], an auto-associative neural network (see  FIG. 2 ), to illustrate the potential of memristor crossbars in complex and large scale pattern associations and classifications. 
         [0017]    In the present invention, the training method that iteratively adjusts the memristor crossbars to the required status is disclosed. Many physical constraints in circuit implementations have been considered, including limited data access, limited accuracy of signal detection, non-ideal memristor characteristics [6], process variations and defects. The present invention generates the programming signals for iterative training using the sign of input signals and the magnitude differences between the output signals and the expected outputs. By avoiding directly reading the memristance values of the crossbars, the present invention significantly reduces the design complexity and avoids analog-to-digital converter (ADC) circuits. 
       Circuit Realization of Matrix-Vector Multiplication Approximation 
       [0018]    The present invention builds upon the apparatus of performing matrix-vector multiplication approximation using two crossbar arrays of resistive memory devices and applying to realizing the auto-associative neural network recall function, which are covered by a previous US patent application [1]. The present invention covers the method and apparatus of programming the resistive memory devices in the two crossbar arrays to the desired states, which are not covered by [1]. 
         [0019]    To realize the matrix-vector multiplication approximation function y=A·x at the circuit level, the elements of the input vector x are converted to the range of input voltage levels VI. The corresponding functions for the multiplication approximation can be expressed as: 
         [0000]    
       
         
           
             
               
                 
                   
                     VO 
                     = 
                     
                       
                         
                           g 
                           s 
                         
                         
                           g 
                           max 
                         
                       
                        
                       
                         ( 
                         
                           
                             VO 
                             + 
                           
                           - 
                           
                             VO 
                             - 
                           
                         
                         ) 
                       
                     
                   
                    
                   
                     
 
                   
                    
                   where 
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       VO 
                       + 
                     
                     = 
                     
                       
                         
                           
                             
                               A 
                               ^ 
                             
                             + 
                           
                           · 
                           VI 
                         
                          
                         
                             
                         
                          
                         and 
                          
                         
                             
                         
                          
                         
                           VO 
                           - 
                         
                       
                       = 
                       
                         
                           
                             A 
                             ^ 
                           
                           - 
                         
                         · 
                         VI 
                       
                     
                   
                    
                   
                     
 
                   
                    
                   where 
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
             
               
                 
                   VI 
                   = 
                   
                     
                       
                         v 
                         bn 
                       
                       
                          
                         
                           x 
                           max 
                         
                          
                       
                     
                     · 
                     x 
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where |x max | is the maximum possible magnitude of any element of input vector x, and v bn  is the input voltage boundary, that is, −v bn ≦vi j ≦v bn  for any vi j ∈VI. In implementation, v bn  must be smaller than v th  so that the memristance values will not change during the multiplication operation. 
         [0020]    As shown in  FIG. 1 , the memristor crossbar arrays  20 ,  30  are used to realize the matrix-vector multiplication approximation operation. To obtain both positive and negative elements in the matrix, two memristor crossbar arrays M 1    20  and M 2    30  are required in the design to represent the positive and negative matrices Â +  and Â − , respectively. The memristor crossbar arrays have the same dimensions as the transposed matrix A. The input signal VI along with VO +  and VO − , the corresponding voltage outputs of two memristor crossbar arrays, are fed into a number of analog subtraction amplifier circuits. 
         [0021]    Resulting from the scaled mapping method, the required VO should be g s /g max  times the generated VO +  or VO − . In the present invention, we set R 1 =R 2 =1/g s  and R 3 =R 4 _ 32   1 /g max . The resulting output of the subtraction amplifier is: 
         [0000]    
       
         
           
             
               
                 
                   
                     vo 
                     i 
                   
                   = 
                   
                     
                       
                         
                           g 
                           s 
                         
                         
                           g 
                           max 
                         
                       
                       · 
                       
                         vo 
                         i 
                         + 
                       
                     
                     - 
                     
                       
                         
                           g 
                           s 
                         
                         
                           g 
                           max 
                         
                       
                       · 
                       
                         vo 
                         i 
                         - 
                       
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0000]    which indicates that the scaled effect (caused by mapping from A to Â +  and Â − ) has been canceled out. The M-by-N dimensional matrix requires M summing amplifiers to realize the subtraction operation in Eq. (4). Also, for subtraction amplifiers  40 , their power supplies should be adjusted to make their maximum/minimum output voltages to reflect the same scaling factor when converting the input vector x to voltage VI. Finally the resulting vector y can be obtained from VO with inversed scaling factor of x, as shown in Eq. (5). 
         [0000]    
       
         
           
             
               
                 
                   y 
                   = 
                   
                     
                       
                          
                         
                           x 
                           max 
                         
                          
                       
                       
                         v 
                         bn 
                       
                     
                     · 
                     VO 
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
       Circuit Realization of Auto-Associative Neural Network Recall Function 
       [0022]    Referring to  FIG. 2 . The Brain-State-in-a-Box (BSB) model is a typical auto-associative neural network. The mathematical model of the BSB recall function can be represented as: 
         [0000]        x ( t+ 1)= S (α ·A·x ( t )+λ· x ( t ))   (6)
 
         [0000]    where x is an N dimensional real vector, and A is an N-by-N connection matrix. A·x(t) is a matrix-vector multiplication operation, which is the main function of the recall function. α is a scalar constant feedback factor. λ is an inhibition decay constant. S(y) is the “squash” function defined as: 
         [0000]    
       
         
           
             
               
                 
                   
                     S 
                      
                     
                       ( 
                       y 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             1 
                             , 
                           
                         
                         
                           
                             
                               if 
                                
                               
                                   
                               
                                
                               y 
                             
                             ≥ 
                             1 
                           
                         
                       
                       
                         
                           
                             y 
                             , 
                           
                         
                         
                           
                             
                               if 
                                
                               
                                   
                               
                               - 
                               1 
                             
                             &lt; 
                             y 
                             &lt; 
                             1 
                           
                         
                       
                       
                         
                           
                             
                               - 
                               1 
                             
                             , 
                           
                         
                         
                           
                             
                               if 
                                
                               
                                   
                               
                                
                               y 
                             
                             ≤ 
                             
                               - 
                               1 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0023]    For a given input pattern x(0), the recall function computes Eq. (16) iteratively until convergence, that is, when all entries of x(t+1) are either “1” or “−1”. 
         [0024]    Using the same method for general matrix-vector multiplication approximation described previously, Eq. (16) converts to: 
         [0000]        x ( t+ 1)= S ( A   +   ·x ( t )− A   −   ·x ( t )+ x ( t ))   (8)
 
         [0000]    Here, for the default case we set α=λ=1. The two connection matrices A +  and A −  can be mapped to two N-by-N memristor crossbar arrays M 3    20  and M 4    30  in a scaled version Â +  and Â − , respectively. 
         [0025]    To realize the BSB recall function at the circuit level, the normalized input vector x(t) is converted to a set of input voltage signals V(t). The corresponding function for the voltage feed-back system can be expressed as: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           V 
                            
                           
                             ( 
                             
                               t 
                               + 
                               1 
                             
                             ) 
                           
                         
                         = 
                           
                          
                         
                           
                             S 
                             ′ 
                           
                            
                           
                             ( 
                             
                               
                                 
                                   
                                     A 
                                     ^ 
                                   
                                   + 
                                 
                                 · 
                                 
                                   V 
                                    
                                   
                                     ( 
                                     t 
                                     ) 
                                   
                                 
                               
                               - 
                               
                                 
                                   
                                     A 
                                     ^ 
                                   
                                   - 
                                 
                                 · 
                                 
                                   V 
                                    
                                   
                                     ( 
                                     t 
                                     ) 
                                   
                                 
                               
                               + 
                               
                                 V 
                                  
                                 
                                   ( 
                                   t 
                                   ) 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             S 
                             ′ 
                           
                            
                           
                             ( 
                             
                               
                                 
                                   V 
                                   
                                     A 
                                     + 
                                   
                                 
                                  
                                 
                                   ( 
                                   t 
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   V 
                                   
                                     A 
                                     - 
                                   
                                 
                                  
                                 
                                   ( 
                                   t 
                                   ) 
                                 
                               
                               + 
                               
                                 V 
                                  
                                 
                                   ( 
                                   t 
                                   ) 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
           
         
       
     
         [0000]    v bn  represents the input voltage boundary, that is, −v bn ≦v i (t)≦v bn  for any v i (t)∈V(t). The new saturation boundary function S′( ) needs to be modified accordingly as: 
         [0000]    
       
         
           
             
               
                 S 
                 ′ 
               
                
               
                 ( 
                 v 
                 ) 
               
             
             = 
             
               { 
               
                 
                   
                     
                       
                         v 
                         bn 
                       
                       , 
                     
                   
                   
                     
                       
                         if 
                          
                         
                             
                         
                          
                         v 
                       
                       ≥ 
                       
                         v 
                         bn 
                       
                     
                   
                 
                 
                   
                     
                       v 
                       , 
                     
                   
                   
                     
                       
                         if 
                          
                         
                             
                         
                         - 
                         
                           v 
                           bn 
                         
                       
                       &lt; 
                       v 
                       &lt; 
                       
                         v 
                         bn 
                       
                     
                   
                 
                 
                   
                     
                       
                         - 
                         
                           v 
                           bn 
                         
                       
                       , 
                     
                   
                   
                     
                       
                         if 
                          
                         
                             
                         
                          
                         v 
                       
                       ≤ 
                       
                         - 
                         
                           v 
                           bn 
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0000]    In implementation, v bn  can be adjusted based on requirements for convergence speed and accuracy. Meanwhile, v bn  must be smaller than v th  so that the memristance values will not change during the recall process. 
         [0026]    Still referring to  FIG. 2  illustrates the BSB recall circuit built based on Eq. (9). The design is an analog system consisting of three major components. The selector (switch)  60  selects V(0) as input voltage at the start of the recall computation, then selects V(t+1) afterward. We assume that “t” is discretized time, so we have t=0, 1, 2, . . . . After the output voltages are all converged, we reset t=0 so that the circuit takes the new input V(0) to be computed (recalled). Below is the detailed description. 
         [0027]    Memristor crossbar arrays: As the key component of the overall design, the memristor crossbar arrays  20 ,  30 , are used to approximate the matrix-vector multiplication functions in the BSB recall operation. To obtain both positive and negative weights in the original BSB algorithm in Eq. (6), two N-by-N memristor crossbar arrays M 3    20  and M 4    30  are required in the design to represent the connection matrices Â +  and Â − , respectively. The memristor crossbar arrays have the same dimensions as the BSB weight matrix A transposed. 
         [0028]    Summation-subtraction amplifiers: The input signal v i (t) along with v Â     +,i   (t) and v Â     . . . , j   (t), the corresponding voltage outputs of two memristor crossbar arrays, are fed into a summation-subtraction amplifier  40 . 
         [0029]    Resulting from the decayed mapping method, the required v A     +,i   (t) and v A     . . . , j   (t) should be g s /g max  times of the generated v Â     +,i   (t) and v Â     +,i   (t), respectively. In the present invention R 1 =R 4 =R 6 =1/g, and R 2 =R 3 =R 5 =R 7 =1/g max . The resulting output of the summation-subtraction amplifier  40  is: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           
                             v 
                             i 
                           
                            
                           
                             ( 
                             
                               t 
                               + 
                               1 
                             
                             ) 
                           
                         
                         = 
                           
                          
                         
                           
                             
                               
                                 g 
                                 s 
                               
                               
                                 g 
                                 max 
                               
                             
                             · 
                             
                               
                                 v 
                                 
                                   
                                     A 
                                     ^ 
                                   
                                   
                                     + 
                                     
                                       , 
                                       i 
                                     
                                   
                                 
                               
                                
                               
                                 ( 
                                 t 
                                 ) 
                               
                             
                           
                           - 
                           
                             
                               
                                 g 
                                 s 
                               
                               
                                 g 
                                 max 
                               
                             
                             · 
                             
                               
                                 v 
                                 
                                   
                                     A 
                                     ^ 
                                   
                                   
                                     - 
                                     
                                       , 
                                       i 
                                     
                                   
                                 
                               
                                
                               
                                 ( 
                                 t 
                                 ) 
                               
                             
                           
                           + 
                           
                             
                               v 
                               i 
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             
                               v 
                               
                                 A 
                                 
                                   + 
                                   
                                     , 
                                     i 
                                   
                                 
                               
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                           - 
                           
                             
                               v 
                               
                                 A 
                                 
                                   - 
                                   
                                     , 
                                     i 
                                   
                                 
                               
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                           + 
                           
                             
                               v 
                               i 
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
         [0000]    which indicates that the decayed effect has been canceled out. The N dimensional BSB model requires N summation-subtraction amplifiers  40  to realize the addition/subtraction operation in Eq. (10). Also, for the amplifiers, we should adjust their power supply levels to make their maximum/minimum output voltages to be equal to ±v bn , respectively. In the present invention the resistances R 1  through R 7  can be adjusted to match the required α and λ in Eq. (6), if they are not the default value 1. 
         [0030]    Comparator: Once a new set of voltage signals V(t+1) is generated from the summation-subtraction amplifiers  40 , the present invention sends them back as the input of the next iteration. Meanwhile, each v i (t+1)∈V(t+1) is compared to v bn  and −v bn  so that when v i  equals to either v bn  or −v bn , we deem the output i as having “converged”. The recall operation stops when all N outputs reach convergence. In total, N comparators  50  are needed to cover all the outputs. 
         [0031]    There are three major physical constraints in the circuit implementation: (1) For any v i (0)∈V(0), the voltage amplitude of initial input signal v i (0) is limited by the input circuit (2) boundary voltage v bn must be smaller than v th  of memristors; and (3) the summation-subtraction amplifier  40  has finite resolution. 
         [0032]    In the BSB recall function, the ratio between boundaries of S(y) and the initial amplitude of x i (0), x i (0)∈x(0) determines the learning space of the recall function. If the ratio is greater than the normalized value, the recall operation will take more iterations to converge with a higher accuracy. Otherwise, the procedure converges faster by lowering stability. Thus, minimizing the ratio of |v i (0)| and v bn  can help obtain the best performance. However, the real amplifier has a finite resolution and v bn  is limited within v th  of the memristor  10 . Continuously reducing |v i (0)| eventually will lose a significant amount of information in the recall circuit. 
         [0033]    The above, and other objects, features and advantages of the present invention will become apparent from the following description read in conjunction with the accompanying drawings, in which like reference numerals designate the same elements. 
       REFERENCES 
       [0000]    
       
         [1] U.S. patent application Ser. No. 13/965,459, R. W. Linderman, et al, “Apparatus for Performing Matrix-Vector Multiplication Approximation Using Crossbar Arrays of Resistive Memory Devices,” filed Aug. 13, 2013. 
         [2] U.S. Patent, US 2005/0258872 A1, G. S. Snider, “Architecture and Methods for Computing with Reconfigurable Resistor Crossbars,” November 2005. 
         [3] U.S. Pat. No. 7,302,513, B. L. Mouttet, “Programmable Crossbar Signal Processor,” November 2007. 
         [4] U.S. Pat. No. 7,342,413, B. L. Mouttet, “Programmable Crossbar Signal Processor with Input/Output Tip Interconnection,” March 2008. 
         [5] U.S. Pat. No. 7,378,870, B. L. Mouttet, “Programmable Crossbar Signal Processor with Rectification Layer,” March 2008. 
         [6] U.S. Pat. No. 7,391,235, B. L. Mouttet, “Programmable Crossbar Signal Processor with Op-Amp Outputs,” June 2008. 
         [7] U.S. Pat. No. 7,447,828, B. L. Mouttet, “Programmable Crossbar Signal Processor Used As Morphware,” November 2008. 
         [8] U.S. Pat. No. 7,459,933, B. L. Mouttet, “Programmable Crossbar Signal Processor Used in Image Processing,” December 2008. 
         [9] U.S. Pat. No. 8,391,049 B2, S. H. Jo, “Resistor Structure for a Non-Volatile Memory Device and Method,” March 2013. 
         [10] P. Camilleri, M. Giulioni, V. Dante, D. Badoni, G. Indiveri, B. Michaelis, J. Braun, and P. del Giudice, “A neuromorphic avlsi network chip with configurable plastic synapses,” in  International Conference on Hybrid Intelligent Systems,  2007, pp. 296-301. 
       
     
         [0044]    [11] P. Partzsch and R. Schuffny, “Analyzing the scaling of connectivity in neuromorphic hardware and in models of neural networks,”  IEEE Transactions on Neural Networks,  vol. 22, no. 6, pp. 919-935, 2011.
   [12] M. Wang, B. Yan, J. Hu, and P. Li, “Simulation of large neuronal networks with biophysically accurate models on graphics processors,” in  Intl. Joint Conf. on Neural Networks  ( IJCNN ), 2011, pp. 3184-3193.   [13] H. Shayani, P. Bentley, and A. Tyrrell, “Hardware implementation of a bioplausible neuron model for evolution and growth of spiking neural networks on FPGA,” in  NASA/ESA Conference on Adaptive Hardware and Systems,  2008, pp. 236-243.   [14] L. Chua, “Memristor—the missing circuit element,”  IEEE Transaction on Circuit Theory,  vol. 18, 1971, pp. 507-519.   [15] D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missing memristor found,”  Nature,  vol. 453, pp. 80-83, 2008.   [16] Y. Ho, G. M. Huang, and P. Li, “Nonvolatile memristor memory: device characteristics and design implications,” in  International Conference on Computer - Aided Design  ( ICCAD ), 2009, pp. 485-490.   [17] M. Di Ventra, Y. V. Pershin and L. O. Chua, “Circuit elements with memory: memristors, memcapacitors, and meminductors,”  Proceedings of the IEEE,  vol. 97, no. 10, pp. 1717-1724, 2009.   [18] L. Chita, “Resistance switching memories are memristors,”  Applied Physics A: Materials Science &amp; Processing,  vol. 102, no. 4, pp. 765-783, 2011.   [19] Q. Xia, W. Robinett, M. W. Cumbie, N. Banerjee, T. J. Cardinali, J. J. Yang, W. Wu, X. Li, W. M. Tong, D. B. Strukov, G. S. Snider, G. Medeiros-Ribeiro, and R. S. Williams, “Memristor-CMOS hybrid integrated circuits for reconfigurable logic,”  Nano letters,  vol. 9, no, 10, pp. 3640-3645, 2009.   [20] S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviya, P. Mazumder, and W. Lu, “Nanoscale memristor device as synapse in neuromorphic systems,”  Nano letters,  vol. 10, no. 4, pp. 1297-1301, 2010.   [21] J. Anderson, J. Silverstein, S. Ritz, and R. Jones, “Distinctive features, categorical perception, and probability learning: some applications of a neural model.”  Psychological Review,  vol. 84, no. 5, pp. 413, 1977.   [22] E. M. H. Hassoun, “Associative neural memories: theory and implementation,” in  Oxford University Press,  1993, Chapter 4.   [23] A. Schultz, “Collective recall via the brain-state-in-a-box network,”  IEEE Transactions on Neural Networks,  vol. 4, no. 4, pp. 580-587, 1993.   [24] Q. Wu, M. Bishop, R. Pino, R. Linderman, and Q. Qiu, “A multi-answer character recognition method and its implementation on a high-performance computing cluster,” in 3 rd International Conference on Future Computational Technologies and Applications,  2011, pp. 7-13.   [25] M. Hu, H. Li, Y. Chen, X. Wang, and R. Pino, “Geometry variations analysis of TiO 2  thin-film and spintronic memristors,” in  Asia and South Pacific Design Automation Conference  ( ASPDAC ), 2011, pp. 25-30.   [26] M. Hu, H. Li, Q. Wu and G. S. Rose, “Hardware realization of BSB recall function using memristor crossbar arrays,” in  Design Automation Conference  ( DAC ), 2012, pp. 498-503.   [27] W. E. Lillo, D. C. Miller, S. Hui and S. H. Zak, “Synthesis of brain-state-in-a-box (BSB) based associative memories,”  IEEE Transactions on Neural Networks,  vol. 5, no. 5, pp. 730-737, 1994.   [28] R. Perfetti, “A synthesis procedure for brain-state-in-a-box neural networks,”  IEEE Transactions on Neural networks,  vol. 6, no. 5, pp. 1071-1080, 1995.   [29] Y. Park, “Optimal and robust design of brain-state-in-a-box neural associative memories,”  IEEE Transactions on Neural Networks , vol. 23, no 2, pp. 210-218, 2010.   [30] Y. Pershin and M. Di Ventra, “Practical approach to programmable analog circuits with memristors,”  IEEE Transactions on Circuits and Systems I: Regular Papers,  vol. 57, no. 8, pp. 1857-1864, 2010.   [31] U. Rarnacher and C. V. D. Malsburg,  On the Construction of Artificial Brains.  Springer, 2010.   [32] T. Hasegawa, T. Ohno, K. Terabe, T. Tsuruoka, T. Nakayama, J. K. Girazewski, and M. Aono, “Learning abilities achieved by a single solid-state atomic switch,”  Advanced Materials,  vol. 22, no, 16, pp. 1831-1834, 2010.   [33] K. Cantley, A. Subramaniam, H. Stiegler, R. Chapman, and E. Vogel, “Hebbian learning in spiking neural networks with nano-crystalline silicon TFTs and memristive synapses,”  IEEE Transactions on Nanotechnology,  vol. 10, no 5, pp. 1066-1073, 2011.   [34] G. Howard, E. Gale, L. Bull, B. d. L. Costello, and A. Adamatzky, “Towards evolving spiking networks with memristive synapses,”  IEEE Symposium on Artificial Life  ( ALIFE ), 2011, pp. 14-21.   
 
         [0068]    [35] D. Chabi, W. Zhao, D. Querlioz, and J. O. Klein, “Robust neural logic block (NLB) based on memristor crossbar array,”  IEEE/ACM International Symposium on Nanoscale Architectures  ( NANOARCH ), 2011, pp. 137-143.
   [36] H. Kim, M. P. Sah, C. Yang, T. Roska, and L. O. Chua, “Neural synaptic weighting with a pulse-based memristor circuit,”  IEEE Transactions on Circuits and Systems I.  vol. 59, no. 1, pp. 148-158, 2012.   [37] H. Choi, H. Jung, J. Lee, J. Yoon, J. Park, D.-J. Seong, W. Lee, M. Hasan, G.-Y. Jung, and H. H. Hwang, “An electrically modifiable synapse array of resistive switching memory,”  Nanotechnology,  vol. 20, no. 34, pp. 345201, 2009.   [38] A. Heittmann and T. G. Noll, “Limits of writing raultivalued resistances in passive nano-electronic crossbars used in neuromorphic circuits,”  ACM Great Lakes Symposium on VLSI      
 
         [0072]    ( GLSVLSI ), 2012, pp. 227-232.
   [39] K.-H. Kim, S. Gaba, D. Wheeler, J. M. Cruz-Albrecht, T. Hussain, N. Srinivasa, and W. Lu, “A functional hybrid memristor crossbar array/CMOS system for data storage and neuromorphic applications,”  Nano Letters,  vol. 12, no. 1, pp. 389-395, 2012.   [40] J. J. Yang, M.-X. Zhang, M. D. Pickett, F. Miao, J. P. Strachan, W.-D, Li, W. Yi, D. A. A. Ohlberg, B. J. Choi, W. Wu, J. H. Nickel, G. Medeiros-Ribeiro, and R. S. Williams, “Engineering nonlinearity into memristors for passive crossbar applications,”  Applied Physics Letters , vol. 100, no. 11, pp. 113501, 2012.   [41] G. Medeiros-Ribeiro, F. Perner, R, Carter, H. Abdalla, M. D. Pickett, and R. S. Williams, “Lognormal switching times for titanium dioxide bipolar memristors: origin and resolution,”  Nanotechnology,  vol. 22, no. 9, pp. 095702, 2011.   [33] W. Yi, F. Perner, M. S. Qureshi. H, Abdalla, M. D. Pickett, i. J. Yang, M.-X. M. Zhang, G. Medeiros-Ribeiro, and R. S. Williams, “Feedback write scheme for memristive switching devices,”  Appl. Phys. A,  vol. 102, pp. 973-982, 2011.   [34] Richard A. Epstein, “The theory of gambling and statistical logic,” Academic Press, 2012,   
 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0078]      FIG. 1  depicts a generic matrix vector multiplication circuit comprised of arrays of memristors. 
           [0079]      FIG. 2  depicts a matrix vector multiplication circuit comprised of arrays of memristors which employs an auto associative neural network Brain-State-in-a-Box. 
           [0080]      FIG. 3  depicts the process steps to be performed in implementing the present invention. 
           [0081]      FIG. 4  depicts the functional elements of the present invention. 
           [0082]      FIG. 5  depicts the detailed circuit design of the error detection block of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
     A Simplified Delta-Rule Training Method 
       [0083]      
         [0000]    
       
         
               
             
               
               
             
           
               
                   
               
               
                 Algorithm 1. Training algorithm using Delta rule. 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 Step 0. 
                 Initialize weights (zero or small random values). 
               
               
                   
                 Initialize learning rate α. 
               
               
                 Step 1. 
                 Randomly select one prototype (training) pattern γ (k)  ∈ B n , k = 1, 
               
               
                   
                 . . ., m. B n  is the n-dimension binary space (−1, 1). 
               
               
                   
                 Set target (expected) output to the external input prototype 
               
               
                   
                 pattern γ (k) :t i  = γ i . 
               
               
                 Step 2. 
                 Compute net inputs: 
               
               
                   
               
               
                   
                 
                   
                     
                       
                         
                           y 
                           
                             in 
                             i 
                           
                         
                         = 
                         
                           
                             Σ 
                             j 
                           
                            
                           
                             γ 
                             j 
                           
                            
                           
                             w 
                             ji 
                           
                         
                       
                     
                   
                 
               
               
                   
               
               
                   
                 (Each net input is a combination of weighted signals received 
               
               
                   
                 from all units.) 
               
               
                 Step 3. 
                 Each unit determines its activation (output signal): 
               
               
                   
               
               
                   
                 
                   
                     
                       
                         
                           y 
                           i 
                         
                         = 
                         
                           
                             S 
                              
                             
                               ( 
                               
                                 y 
                                 
                                   in 
                                   i 
                                 
                               
                               ) 
                             
                           
                           = 
                           
                             { 
                             
                               
                                 
                                   
                                     1 
                                     , 
                                   
                                 
                                 
                                   
                                     
                                       y 
                                       
                                         in 
                                         i 
                                       
                                     
                                     ≥ 
                                     1 
                                   
                                 
                               
                               
                                 
                                   
                                     
                                       y 
                                       
                                         in 
                                         i 
                                       
                                     
                                     , 
                                   
                                 
                                 
                                   
                                     
                                       - 
                                       1 
                                     
                                     &lt; 
                                     
                                       y 
                                       
                                         in 
                                         i 
                                       
                                     
                                     &lt; 
                                     1 
                                   
                                 
                               
                               
                                 
                                   
                                     
                                       - 
                                       1 
                                     
                                     , 
                                   
                                 
                                 
                                   
                                     
                                       y 
                                       
                                         in 
                                         i 
                                       
                                     
                                     ≤ 
                                     
                                       - 
                                       1 
                                     
                                   
                                 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                   
               
               
                 Step 4. 
                 Update weights: Δw ij  = α · (t j  − y j ) · γ i   
               
               
                 Step 5. 
                 Repeat Steps 1-4 until the condition |t(i) − y(i)| &lt; θ is satisfied in  
               
               
                   
                 m consecutive iterations. 
               
               
                   
               
             
          
         
       
     
         [0084]    A software generated weight matrix can be mapped to the memristor crossbar arrays  20 ,  30  based on the assumption that every memristor in the crossbar could be perfectly programmed to the required resistance value, however, the traditional crossbar programming method faces accuracy and efficiency limitations due to the existence of the sneak paths [29]. Although some recent works were presented to improve the write/read ability of memristor crossbars by leveraging the device nonlinearity [31], the controllability of analog state programming is still limited. In spite of preparing the memristor crossbars with open-loop writing operations, the present invention discloses a close-loop programming method which iteratively adjusts the resistance states of the memristors in the crossbars to the target values. The present invention&#39;s technique is based on a modification of the above software training algorithm. 
         [0085]    The Delta rule in Algorithm 1 is used as an example. A weight w ij  corresponds to the analog resistance state of the memristor  10  at the cross-point of the ith row and the jth column in a crossbar array. A weight updating Δw ij  involves multiplying three analog variables: α, t j -y j , and x i . Though these variables are available in training scheme design, the hardware implementation to obtain their multiplication demands very high computation resource overhead. Therefore, we simplify the weight updating function by trading off the convergence speed as: 
         [0000]      Δ w   ij =α·sign( t   j   −y   j )·sign( x   h )   (11)
 
         [0000]    Here, sign(t j -y j ) and sign(x i ) are the polarities of t j -y j  and x i , respectively. They can be either +1 or −1. sign(t j -y j )·sign(x i ) represents the direction of the weight change. At each training iteration, the rate of the weight changes remains constant which is controlled by the learning rate α. 
         [0086]    This simplification approach minimizes the circuit design complexity while ensuring the weight change in the same direction as that of the original Delta rule. As a tradeoff, a higher number of iterations may be required to reach a converged weight matrix since the weight change is not the steepest descent as that in Algorithm 1. However, the modification will not affect the training quality much since the decision criterion of the algorithm remains as the error between the actual and expected output patterns. 
       Hardware Realization of Close-Loop Memristor Programming 
       [0087]    Referring to  FIG. 3  summarizes the operational flow of the BSB training circuit of the present invention. And the corresponding circuit diagram is illustrated in  FIG. 4 . The present invention&#39;s purpose is to provide a method to train the memristor crossbars  20 ,  30  as auto-associative memories for a set of training patterns. The training scheme leverages the recall circuit to verify the training result and generate the control signals.  FIG. 3  depicts the present invention&#39;s method as it would be performed through computer software instructions, i.e., implemented in software.  FIG. 4  depicts the present invention implemented in hardware, with the functional hardware blocks clearly depicted in communication with the BSB recall circuit being trained. The scope of the present invention is intended to include both the herein disclosed software method and hard ware apparatus embodiments to train memristive crossbar arrays. 
         [0088]    Still referring to  FIG. 4 , the major components of the design for the present invention will be described. Among these components, the BSB Recall Circuit  160  is the same as described in  FIG. 2 . The R/W Control component of the BSB Recall Circuit  160  generates the signals to perform the “Read” (recall) operation and the “Write” (programming) operation to the crossbar arrays. The Error Detection component  110  performs the error detection function as described in Step 4. The Programming Signal Generation component  120  generates the programming signals based on “Vin” and “Diff” signals, as described in Step 5; The Status Tracker (ST) &amp; Arbiter component  80 ,  130  keeps track of the overall training progress to determine whether training is completed, as well as controls the selection of the next training pattern. The Register component  170  stores all the training patterns and output one of them based on the input from the ST &amp; Arbiter  80 ,  130  component. 
         [0089]    Referring now to  FIG. 3  and  FIG. 4  concurrently: 
         [0090]    Step 1: Initializing the crossbar arrays. At the beginning of a training procedure, all memristance values  10  in M 1    20  and M 2    30  are initialized  70  to an intermediate value. The initialization  70  doesn&#39;t have to be precisely accurate. Indeed, even when all of the memristors  10  are all at either LRS or HRS, the crossbar arrays  20 ,  30  can still be successfully trained but it requires more time to reach convergence. For instance, training from HRS takes about 2,500 iterations, while initializing the memristors to intermediate states within their resistance range can reduce the iteration number to about 1,000 iterations. 
         [0091]    Step 2: Selecting a prototype pattern γ (k) ∈B n (k=1, . . . , m). Here, B n  is the n-dimension binary space (−1, 1). Assume a training set includes m prototype patterns and each pattern; γ (k)  has the same probability to be chosen every time. The ST  80  is used to keep track of the overall training progress, e.g., the patterns that have been successfully trained. When the ST is not reset, those patterns that have been trained are excluded by the arbiter  130  (see  FIG. 4 ) from the selection  90 . 
         [0092]    Step 3: Sending γ (k)  to the BSB recall circuit (as shown in  FIG. 2 ). The present invention converts γ (k)  in binary space (−1, 1) to a set of input voltages within the boundary (−0.1 V, 0.1 V). These input signals are supplied  100  to the two memristor crossbars  20 ,  30  simultaneously. The resulting signals V O  can be obtained at the output of the BSB recall circuit. 
         [0093]    Step 4: Error detection. An error is defined as the difference between the prototype pattern and the recall result; that is, the difference between the input and output signals of the recall circuit. An error detection  110  circuitry for bit i (see  FIG. 5 ) generates only the direction of the weight change based on the simplified algorithm. In total, N error detection circuitries are needed for an N×N crossbar array. 
         [0094]    Considering that the range of V out (i) could be different from that of V in (i), we apply a scalar λ to the input vector and take λ·V in (i) as the target output signal. Rather than generating λ·V in (i) in every training, the present invention uses the preset threshold voltages for error detection. Since V in (i) is either 0.1 V or −0.1 V, four thresholds are needed, including 
         [0000]        V   th     —     h   + =0.1 λ+θ, V   th     —     l   + =0.1λ−θ
 
         [0000]        V   th     —     h   − =0.1 λ−θ, V   th     —     l   − =−0.1λ+θ,   (12)
 
         [0000]    Here, θ represents the tolerable difference. 
         [0095]    The error detection output Diff(i)  110  could be −1, 0, or 1. When |V out (i)−λ·V in (i)|&lt;θ, Diff(i)=0, meaning the difference between the normalized V in (i) and V out (i) are so small that we consider them logically identical. Otherwise, Diff(1)=+1 or −1, indicating the normalized |V out (i)| is greater or less than the normalized |V in (i)|, respectively. 
         [0096]    Step 5: Training memristor crossbar arrays. If Diff  130  is not a vector of logic zero values, which means some error has been detected, the crossbar arrays need to be further tuned. The programming signal generation is based on the training rule defined by (13). In order to control the training step  120  with a finer granularity, we modify only one of the two memristor crossbars each time. For example, one could train M 1    20  or M 2    30  when the iteration number is odd or even, respectively. 
         [0097]    The training of a memristor crossbar array is conducted column-by-column. The polarity and amplitude of the programming pulse for the jth column are determined by Diff(j). The design supplies the programming pulses on all the rows of a memristor crossbar. The jth column is connected to ground and all the others are disconnected. For M 1 , the programming signals is either the current selected prototype pattern γ (k)  (when Diff(j)=1) or its element-wise negated version (when Diff(j)=−1). The programming signals to M 1    20  and M 2    30  have opposite polarities. That is, the programming signals of M 2    30  uses the current prototype pattern when Diff(j)=−1 or its element-wise negated version when Diff(j)=1. 
         [0098]    Note that the mapping method uses M 1    20  and M 2    30  to represent the positive and negative terms of the BSB weight matrix, respectively. However, the proposed training scheme operated in real design circumstance cannot and does not have to guarantee an identical mapping to software generated matrix defined in (11). In fact, what matters most is the overall effect of M 1    20  and M 2 ,  30  not exact memristance values  10  in each individual crossbar array. 
         [0099]    Step 6: If training is completed? The ST updates the training progress  150  if a prototype pattern goes through Steps 2-5 and reports no error without further tuning M 1  and M 2 . Otherwise, ST resets the training progress  140  whenever an error is detected and all of the patterns in B n  are available in Step 2. When ST  80  indicates that the entire training set has been successfully “learned” the training progress terminates. 
       Design Robustness Considerations 
       [0100]    Running the BSB recall circuit  160  (see  FIG. 2 ) under the ideal condition should obtain the same results as the mathematical algorithm. Unfortunately, the noise induced by process variations and signal fluctuations can significantly affect circuit performance. These noise sources at the component level can be modeled and their impact on the physical design challenges of the present invention, analyzed. 
       1) Process Variations 
       [0101]    Memristor Crossbar Arrays: The real memristance matrix M′ of a memristor crossbar array could be quite different from the theoretical M due to process variations. The difference can be represented by a noise matrix N M , which includes two contributors: the systematic noise N M,sys  and the random noise N M,rdm . Consequently, M′ can be expressed by: 
         [0000]        M′=M∘N   M   =M ∘(1+ N   M,sys   +N   M,rdm )   (13)
 
         [0000]    where ∘ stands for Hadamard product. The impact of N M  on the connection matrix C is too complex to be expressed by a mathematical closed-form solution. But numerical analysis shows that it can be approximated by: 
         [0000]    
       
         
           
             
               
                 
                   
                     C 
                     M 
                     ′ 
                   
                   = 
                   
                     
                       C 
                       ∘ 
                       
                         N 
                         CM 
                       
                     
                     = 
                     
                       C 
                       ∘ 
                       
                         1 
                         
                           N 
                           M 
                         
                       
                       ∘ 
                       
                         1 
                         
                           N 
                           M 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   14 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where C M ′ is the connection matrix after including memristance process variations. N CM  is the corresponding noise matrix. 
         [0102]    In the following analysis, assume N M,sys  follows a normal distribution and M M,rdm  employs the latest TiO 2  device measurement result. Coefficient Corr M  represents the correlation degree between M 1  and M 2 . When Corr M =1, the two arrays have the same systematic noise. 
         [0103]    Sensing Resistance: Similarly, classify the noise induced by R S  variations into the systematic noise N R,sys  and the random noise N R,rdm , The actual sensing resistance vector R S ′ becomes: 
         [0000]        R   S   ′=R   S   ∘N   Rs   =R   S ∘(1+ N   R,rdm )   (15)
 
         [0104]    Accordingly, C R ′, the connection matrix including noise is: 
         [0000]        C   R   ′=C∘N   CR   =C∘[N   Rs   , N   Rs   , . . . , N   Rs ]  (16)
 
         [0000]    where N CR  is the noise matrix of C after including the process variation of the sensing resistors. 
         [0105]    The mean of the r s  distribution reflects its systematic process variations. It can be obtained during post-fabrication test and its impact can be cancelled out in training procedure with proper design configuration. Thus, in the following analysis, we only consider the random noise N R,rdm  in a normal distribution. 
       2) Signal Fluctuations 
       [0106]    The electrical noise from the power supplies and the neighboring wires can significantly degrade the quality of analog signals. Different from the process variations that remain unchanged after the circuit is fabricated, signal fluctuations vary during circuit operation. Without loss of generality, we assume the runtime noise of the summing amplifiers&#39;  40  output signals follow a normal distribution, same as that of the outputs of the comparators  50 . 
       3) Physical Challenges 
       [0107]    There are three major physical constraints in the circuit implementation: (a) For any V i (0)∈V(0), the voltage amplitude of initial input signal V i (0) is limited by input circuit; (b) The boundary voltage V bn  must be smaller than V th  of the memristors  10 ; and (c) the summing amplifier  40  has a finite resolution. 
         [0108]    In the BSB recall function (2), the ratio between the boundaries of S() and the initial amplitude of x i (0), x i (0)∈x(0) determines the performance and quality of the recall operation: a larger ratio results in more iteration with higher accuracy, while a small ratio makes convergence faster by lowering system stability. Accordingly, reducing the ratio of V bn  and |V i (0)| in BSB circuit helps improve recall speed. However, the amplifier has a finite resolution and V bn  is limited within V th  of the memristor. Continuously reducing |V u (0)| eventually will lead to a loss of information. So the resolution of the summing amplifier is a key parameter to determine the optimal ratio of V bn  and |V i (0)| and the overall design cost. 
       4) Impact of Sneak Paths 
       [0109]    When utilizing crossbars as memories, only one WL is raised up and one or a few BLs are accessible at a time (see  FIG. 2 ). The other WLs and BLs remain floating. Such a single-input-single-output (SISO) access inevitably results in currents through unintended paths, called the sneak paths [29], [30]. The existence of sneak paths in the passive resistive network is a well-known issue, which greatly limits the size of crossbar arrays and their utilization in memory design. During a recall process, present invention&#39;s design accesses the crossbar in multi-input-multi-output (MIMO) mode therefore the sneak path is not an issue. 
         [0110]    The training process has subtleties. In the present invention the training voltage is slightly higher than V th  but much smaller than the switching voltage used in memory operation, because only a small change of memristance is needed in each training step. Hence, the voltage drop on the other memristors is smaller than V th therefore will not result in the unexpected memristance changes. 
         [0111]    More importantly, the sneak paths have to be well controlled in memory design because the current through the target device is a critical parameter. In contrast, the major concern in neuromorphic design design is that the association between input and output signals can be properly captured (in training) and reflected (in recall) by the memristor crossbar array. The current distribution within the crossbar is not in the area of interest. 
         [0112]    Having described preferred embodiments of the invention with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims.