Patent Publication Number: US-2023153588-A1

Title: Neuromorphic device for parallel processing of spike signals

Description:
TECHNICAL FIELD 
     The present invention relates to a neuromorphic device for parallel processing of spike signals. 
     BACKGROUND 
     Recently, along with the development of a computing technology based on artificial neural networks, research and development of hardware-based neural networks have been actively conducted. 
     Neural networks, which are currently being widely studied, started from imitation (concepts for memory, learning, and inference) of an actual biological nervous system, but only a similar network structure is adopted, and there is a difference from a nervous system in various aspects, such as a signal transmission and information expression method and a learning method. 
     Meanwhile, in relation to a hardware-based spiking neural network (SNN) which operate almost identically to the real nervous system, a learning method that outperforms existing neural networks has not yet been developed, and thus, the SNN is rarely used in the real industry. However, when a synaptic weight is derived by using the existing neural network and inference is performed by using the synaptic weight through an SNN method, a high-accuracy and ultra-low-power computing system may be implemented, and thus, research thereon is being actively conducted. 
     The SNN consists of a synaptic array that stores weighted values and a neuronal circuit that is responsible for activation. In addition, the SNN transforms an input size of a network by using coding using a concept of time, and inputs of the same size are simultaneously input depending on coding methods. The synaptic array converts the simultaneous inputs into currents corresponding to the weighted values stored in each synaptic element, and currents of the synaptic elements connected to the same output neuron have to be summed up again in the synaptic array. 
     As research advances on the known hardware-based machine learning and in-memory computing, complexity of the SNN is also increasing, and a corresponding large amount of synaptic array is required to process a huge amount of information. Accordingly, complexity of a system inevitably increases and energy efficiency decreases, and thus, a solution therefor is required. 
     SUMMARY OF INVENTION 
     Technical Problem 
     The present invention is to solve the problems described above, and an object of the present invention is to provide a neuromorphic device capable of processing a plurality of input signals in parallel. 
     However, a technical object to be solved by the present embodiment is not limited to the technical object described above, and there may be other technical objects. 
     Solution to Problem 
     As a technical means for solving the above technical problems, a neuromorphic device according to an aspect of the present invention includes a synaptic array including a plurality of word lines and a plurality of bit lines and including a plurality of synaptic elements coupled to intersections of each of the plurality of word lines and each of the plurality of bit lines, a word line signal output unit that sequentially outputs a plurality of word line signals for activating the plurality of word lines, a signal pre-processing unit that preprocesses a spike signal to modulates the spike signal into an input signal including a plurality of pulses, a weight summation unit including a plurality of weight summation circuits that respectively output a plurality of output signals obtained by applying respective weighted values stored in the synaptic array to a plurality of input signals input through the signal pre-processing unit, and a data output unit that transmits the plurality of output signals output by the weight summation unit respectively to a plurality of output neurons in response to the plurality of word line signal. 
     In addition, a neuromorphic device according to another aspect of the present invention includes a synaptic array including M word lines (M is a natural number) and N bit lines (N is a natural number) and including a plurality of synaptic elements coupled to intersections of each of the plurality of word lines and each of the plurality of bit lines, a weight summation unit including K weight summation circuits that output respectively output signals obtained by applying respective weighted values stored in the synaptic array to K input signals, and a data output unit that transmits the K output signals output from the weight summation unit respectively to output neurons for each pulse in response to M word line signals for sequentially activating the plurality of word lines, wherein each of the K input signals includes N pulses. 
     Advantageous Effects 
     According to the above-described problem solving means of the present application, a system capable of parallel processing a plurality of input signals may be constructed with respect to an SNN-based neuromorphic device. That is, an inference operation for the plurality of input signals may be performed in parallel by using a learning model stored in one synaptic array, and thus, a processing speed may be greatly improved. 
     In addition, since only one synaptic array is used, energy consumption may be reduced, and a low-power environment may be implemented. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    is a block diagram illustrating a configuration of a neuromorphic device according to an embodiment of the present invention. 
         FIG.  2    is a conceptual diagram illustrating a configuration of an SNN provided by a neuromorphic device, according to an embodiment of the present invention. 
         FIG.  3    is a circuit diagram illustrating a detailed configuration of a neuromorphic device according to an embodiment of the present invention. 
         FIG.  4    is a diagram illustrating a configuration of a signal pre-processing unit and a word line signal output unit according to an embodiment of the present invention. 
         FIG.  5    is a diagram illustrating an operation of a neuromorphic device according to an embodiment of the present invention. 
     
    
    
     BEST MODE FOR INVENTION 
     Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings such that those skilled in the art may easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly illustrate the present invention in the drawings, parts irrelevant to the descriptions are omitted, and similar reference numerals are attached to similar parts throughout the specification. 
     Throughout the specification, when a portion is “connected” to another portion, this includes not only a case of being “directly connected” but also a case of being “electrically connected” with another component therebetween. 
     Throughout the specification, when a member is said to be located “on” another member, this includes not only a case in which a member is in contact with another member but also a case in which another member is present between the two members. 
     A neuromorphic device of the present invention is manufactured to imitate a human brain in hardware by using a semiconductor process, and includes a synaptic element corresponding to a synapse of the brain, a neuron circuit corresponding to a neuron, and various peripheral circuits. 
     Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. 
       FIG.  1    is a block diagram illustrating a configuration of a neuromorphic device according to an embodiment of the present invention,  FIG.  2    is a conceptual diagram illustrating a configuration of an SNN provided by a neuromorphic device, according to an embodiment of the present invention, and  FIG.  3    is a circuit diagram illustrating a detailed configuration of a neuromorphic device according to an embodiment of the present invention. 
     As illustrated, a neuromorphic device  100  includes a synaptic array  110 , a word line signal output unit  120 , a signal pre-processing unit  130 , a weight summation unit  140 , and a data output unit  150 . 
     As illustrated in  FIG.  2   , a spiking neural network using a plurality of front-end neurons, a plurality of back-end neurons, and a plurality of synaptic devices connecting the plurality of front-end neurons to the plurality of back-end neurons is implemented through the neuromorphic device  100 . The neuromorphic device  100  operates to output several output spike signals together by applying weighted values stored in the synaptic array  110  in parallel to several input spike signals transmitted from the front-end neurons. Through this, inference operations for a plurality of input signals may be performed in parallel by using a learning model stored in one synaptic array  110 . 
     The synaptic array  110  includes a plurality of word lines and a plurality of bit lines and includes a plurality of synaptic elements coupled to intersections of the respective word lines and the respective bit lines. The synaptic array  110  is implemented to perform the same function as the brain synapse and is generally implemented based on a non-volatile memory device. The synaptic array  110  corresponds to a plurality of synaptic cells, and each synaptic cell stores a predetermined weighted value. For example, the synaptic array  110  may include synaptic cells corresponding to a multiplication of the number of front-end neuronal circuits and the number of back-end neuronal circuits. As illustrated, the synaptic array may include N bit lines (N is a natural number) and M word lines (M is a natural number), the N bit lines correspond to the number of front-end neuronal circuits, and the M word lines may correspond to the number of back-end neuronal circuits. 
     An operation of storing weighted values for the synaptic array  110  or a process of reading the stored weighted values is performed in the same principle as a program operation or a read operation performed by a general non-volatile memory device. Here, a weighted value means a weighted value that is multiplied by an input signal in a perceptron structure representing an artificial neural network model and is additionally defined as a concept including a bias which is a special weighted value having an input of 1. 
     The neuronal circuits may be classified into a front-end neuronal circuit or a pre-neuronal circuit coupled to a front end of the synaptic array  110 , and a back-end neuronal circuit or a post neuronal circuit coupled to a back end of the synaptic array  110 . A general neuronal circuit includes a signal integrator that integrates a signal transmitted through a previous synapse or so on, and a comparator that compares an integrated signal with a threshold. In addition, when a comparison result of the comparator is greater than or equal to a threshold value, the general neuronal circuit outputs a spike signal according to an activation operation. In addition, a counter for calculating the number of spike signals may be connected to each neuronal circuit. Meanwhile, in relation to a configuration of the signal integrator, an embodiment of integrating a signal by using a capacitor is known in general. 
     Next, the word line signal output unit  120  sequentially outputs a plurality of word line signals for activating the respective word lines of the synaptic array  110 . The word line signals are also provided to the data output unit  150  at the same timing. In this case, when the synaptic array  110  includes M word lines, the synaptic array  110  sequentially output a first word line signal to an M-th word line signal. In addition, the word line signal output unit  120  enables the first word line signal to the M-th word line signal to be sequentially outputted while pulses of an input signal output from the signal pre-processing unit  130  are maintained. Synchronization of the word line signal output unit  120  and the signal pre-processing unit  130  is maintained. 
     The signal pre-processing unit  130  pre-processes a spike signal received from a previous layer to modulate the spike signal into an input signal including a plurality of pulses. In particular, the signal pre-processing unit  130  causes the input signal to include N pulses which is the number of neurons or the number of bit lines in a previous stage and causes the time for each pulse to maintain a high level to be equal to a cycle in which a plurality of word line signals are sequentially output once. In addition, each pulse is maintained at the same level for the same time, but a rising time of each pulse is adjusted differently. For example, a function of the signal pre-processing unit  130  may be performed by using a sample and hold circuit. 
       FIG.  4    is a diagram illustrating a configuration of a signal pre-processing unit and a word line signal output unit according to an embodiment of the present invention. 
     A spike signal output from a previous layer is not constant in arrival time and is also not constant in time for which the signal is maintained, and thus, the signal pre-processing unit  130  modulates the spike signal into pulses of a constant size. At least, such that weighted values of respective synaptic elements included in the synaptic array  130  may be applied, the signal pre-processing unit maintains respective pulses in a high level state while the first to M-th word line signals are sequentially output once. 
     In addition, an input signal includes N pulses, and the signal pre-processing unit  130  adjusts rising times of the N pulses X 1 [ 1 ] and X 1 [ 2 ] to be different. For example, as illustrated, an interval between the rising times of respective pulses may be set to be an interval between two word line signals, which may be changed according to a designer&#39;s choice. 
     Referring back to  FIG.  1   , the weight summation unit  140  outputs an output signal obtained by applying respective weighted values stored in the synaptic array  110  to a plurality of input signals input through the signal preprocessor  130 . The weight summation unit  140  includes as many weight summation circuits  141  to  143  as the number of input signals, and through this, the weight summation unit  140  may process a plurality of input signals in parallel. 
     As illustrated in  FIG.  3   , the weight summation unit  140  has one side connected to bit lines BL 1  to BLN of the synaptic array  130  and has the other side connected to the respective weight summation circuits  141  to  143  and includes a plurality of current copy circuits  145  to  147  that copy and output currents of corresponding bit lines in response to pulses of input signals. In this case, the current copy circuits  145  to  147  are arranged as many as the number of bit lines. 
     First, the weight summation circuits  141  to  143  are arranged as many as K, which is the number of input signals, and each of the weight summation circuits  141  to  143  includes N data lines DL arranged in parallel with each other and an output terminal ODL coupled thereto. For example, a K-th input signal includes N pulses, and the weight summation circuit to which pulses of the K-th input signal are applied includes N data lines DL k  to which outputs of the current copy circuits  145  to  147  are applied and includes an output terminal ODL k  coupled thereto. 
     The data output unit  150  transmits a plurality of output signals output by the weight summation unit  140  to a plurality of output neurons in response to each word line signal. The data output unit  150  includes a plurality of switching elements which are coupled to respective output terminals ODLs of the respective weight summation circuits and to which a plurality of word line signals are respectively applied. That is, the switching elements are each coupled to each intersection of each output terminal and each word line signal supply wire. In this case, the respective switching elements transmit output of the respective weight summation circuits to the output neuron in response to word line signals WL 1  to WL M . 
     A detailed operation of the neuromorphic device of the present invention will be described by way of an example. 
       FIG.  5    is a diagram illustrating an operation of a neuromorphic device according to an embodiment of the present invention. 
     According to the illustrated SNN, it is possible to consider a synaptic array that includes a total of three front-end neurons and a total of four back-end neurons and includes 12 synaptic elements connecting the neurons to each other. According to this configuration, a synaptic array including three bit lines and four word lines is prepared. The number of input signals input to the above-described SNN is K, which may be adjusted according to a user&#39;s selection, and the present invention processes the input signals in parallel. 
     A first input signal X 1  includes pulse signals X 1 [ 1 ], X 1 [ 2 ], and X 1 [ 3 ] as many as the number of front-end neurons by the signal pre-processing unit  130 . In addition, the signal pre-processing unit  130  modulates a pulse signal to maintain each pulse signal in a high level state while the first to M-th word line signals WL 1  to WL 4  are activated. In addition, the respective pulse signals X 1 [ 1 ], X 1 [ 2 ], and X 1 [ 3 ] included in the first input signal X 1  maintains an interval between two word line signals. 
     While the first pulse signal X 1 [ 1 ] is maintained, the first word line signal WL 1  is activated, and thus, the first switching element of the current copy circuit  147  is turned on in response to the first pulse signal X 1 [ 1 ], and a weighted value of a synaptic element connected to the first word line is transmitted to the data output unit  150  via a first bit line BL 1  and a data transmission line DL 1 . In this case, the weighted value transmitted from the data transmission line DL 1  is transmitted to an output neuron N 1 [ 1 ] via a switching element (turned on in response to the first word line signal WL 1 ) coupled to the output terminal ODL 1  connected to the data transmission line DL 1 . 
     Thereafter, the second word line signal WL 2  is activated, and thus, the first switching element of the current copy circuit  147  is turned on in response to the first pulse signal X 1 [ 1 ], and a weighted value of a synaptic element connected to the second word line is transmitted to the data output unit  150  via the first bit line BL 1  and the data transmission line DL 1 . In this case, the weighted value transmitted from the data transmission line DL 1  is transmitted to an output neuron N 1 [ 2 ] via a switching element (turned on in response to the second word line signal WL 2 ) coupled to the output terminal ODL 1  connected to the data transmission line DL 1 . 
     Thereafter, the third word line signal WL 3  is activated, and thus, the first switching element of the current copy circuit  147  and the first switching element of the current copy circuit  146  are turned on in response to the first pulse signal X 1 [ 1 ] and the second pulse signal X 1 [ 2 ]. Accordingly, a weighted value of a synaptic element connected to the third word line is transmitted to the data output unit  150  via the first bit line BL 1 , the data transmission line DL 1 , and the second bit line BL 2 . In this way, currents transmitted through the plurality of data transmission lines may be summed and transmitted to the output neurons. In this case, the weighted value transmitted from the data transmission line DL 1  is transmitted to an output neuron N 1 [ 3 ] via a switching element (turned on in response to the third word line signal WL 3 ) coupled to the output terminal ODL 1  connected to the data transmission line DL 1 . 
     This process is repeatedly performed for a total of three pulses included in the first input signal, and while the first to fourth word line signals are sequentially output a total of two times, the processing for the first input signal is completed. 
     In addition, in the same manner as above, processing for the second input signal X 2  is performed in parallel. That is, while the first pulse signal X 2 [ 1 ] of the second input signal X 2  is maintained, the first word line signal WL 1  is activated, and thus, the second switching element of the current copy circuit  147  is turned on in response to the first pulse signal X 2 [ 1 ], and weighted values of each synaptic element transmitted through the first bit line BL 1  are transmitted to the data output unit  150  via the second data transmission line DL 2 . In this case, the weighted value transmitted from the data transmission line DL 2  is transmitted to the output neuron N 2 [ 1 ] via a switching element (turned on in response to the first word line signal WL 1 ) coupled to the output terminal ODL 2  connected to the second data transmission line DL 2 . 
     An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer. Computer-readable media may be any available media that may be accessed by a computer and include both volatile and nonvolatile media and removable and non-removable media. In addition, the computer-readable media may include all computer storage media. The computer storage media includes both volatile and nonvolatile media and removable and non-removable media implemented by any method or technology of storing information, such as a computer readable instruction, a data structure, a program module, and other data. 
     Although the method and system according to the present invention are described with reference to specific embodiments, some or all of their components or operations may be implemented by using a computer system having a general-purpose hardware architecture. 
     The above descriptions on the present invention are for illustration, and those skilled in the art to which the present invention pertains may understand that the descriptions may be easily modified into other specific forms without changing the technical idea or essential features of the present invention. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form. 
     The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included in the scope of the present invention. 
     REFERENCE SIGNS LIST 
     
         
         
           
               100 : neuromorphic device 
               110 : synaptic array 
               120 : word line signal output unit 
               130 : signal pre-processing unit 
               140 : weight summation unit 
               150 : data output unit