Neural array having multiple layers stacked therein for deep belief network and method for operating neural array

A neural array may include an array unit, a first processing unit, and a second processing unit. The array unit may include synaptic devices. The first processing unit may input a row input signal to the array unit, and receive a row output signal from the array unit. The second processing unit may input a column input signal to the array unit, and receive a column output signal from the array unit. The array unit may have a first array value and a second array value. When the first processing unit or the second processing unit receives an output signal based on the first array value from the array unit which has selected the first array value and then the array unit selects the second array value, it may input a signal generated based on the output signal to the array unit which has selected the second array value.

TECHNICAL FIELD

The present disclosure relates to a neural array and a method for operating the neural array.

BACKGROUND

Recently, research on a system including a neural device has continued. The system including a neural device can implement a computer different from a conventional Von Neumann type computer and may have design flexibility and energy and space efficiency.

The system including neural devices can process and learn data in a manner analogous to that of a biological brain. Typically, a neural device is connected with other neural devices through synapses of the neural device and also receives data from the other neural devices through the synapses. Also, the neural device may store the received data. In order to configure a desired system using neural devices, a number of neural devices may be needed.

Further, the neural devices may form multiple layers where a neural device of one layer may be connected to a neural device of another layer. As such, neural devices of multiple layers may be connected to one another to form a neural network. For example, a deep neural network formed using multiple layers is known to be more efficient than other neural networks. Regarding the deep neural network, algorithms such as Restricted Boltzmann Machine (RBM) and/or Error Back Propagation (BP) are known as methods for training synaptic devices constituting the deep neural network.

In a neural network including multiple layers, multiple neural devices may form a neural array. The neural array may be interconnected with another neural array to form a neural network.

SUMMARY

In an exemplary embodiment, a neural array for forming a deep neural network is generally described. The exemplary neural array may include an array unit, a first processing unit, and a second processing unit. The array unit may include one or more synaptic devices arranged in one or more rows and columns. The first processing unit may be configured to input a row input signal to the array unit and receive a row output signal from the array unit. The second processing unit may be configured to input a column input signal to the array unit and receive a column output signal from the array unit. The array unit may have multiple selectable array values. For example, the array unit may include a first array value and a second array value.

In an exemplary embodiment, the array unit may select the first array value. The array unit may generate an output signal based on an input signal and the first array value. The first processing unit or the second processing unit may receive the output signal from the array unit and generate a new input signal based on the received output signal. Then, the array unit may select the second array value. When the array unit selects the second array value, the first processing unit or the second processing unit may input the newly generated input signal to the array unit that selects the second array value.

In an additional example, each of the one or more synaptic devices of the array unit may be configured to form a NAND flash structure. The one or more synaptic devices may include a Floating Gate Metal Oxide Silicon Field Effect Transistor (FG MOSFET).

In an additional example, the array unit may be configured to operate in a time-division multiplexing mode. For example, the array unit may be configured to use the first array value in a first time section and the second array value in a second time section after the first time section.

In an additional example, the neural array may be configured to implement a RBM. Since the neural array implements a RBM, multiple neural array values can be updated.

In an additional example, the neural array may update multiple neural array values by implementing a BP process.

In an example, a method performed in a neural array configured to form a deep neural network and implement a RBM is generally described. Such neural array may have multiple selectable array values, and be configured to cause an output signal based on an array value to be an input signal for another array value. The exemplary method may include updating a first array value and updating a second array value based on the updated first array value.

The updating of the first array value may include: selecting the first array value; generating a reconstructed first input signal based on a first output signal and the first array value; generating a reconstructed first output signal based on the reconstructed first input signal and the first array value; and updating the first array value based on a first input signal, the first output signal, the reconstructed first input signal, and the reconstructed first output signal.

The updating of the second array value may include: generating a second input signal based on the first input signal and the updated first array value; and selecting the second array value.

In an example, a method to perform a BP process in a neural array forming a deep neural network is generally described. Such a neural array may have multiple selectable array values, and be configured to cause an output signal based on an array value to be an input signal for another array value. The exemplary method may include: receiving an input signal; performing forward propagation to generate an output signal based on the received input signal and multiple array values; obtaining an error value based on the output signal and a target value for the output signal; performing back propagation based on the error value and the multiple array values; and updating the multiple array values based on results of the forward propagation and the back propagation.

The above-described summary is provided for illustration purposes only and does not intend to limit in any ways. In addition to the exemplary embodiments, examples, and features described above, additional embodiments, examples, and features will become apparent by referring to the drawings and the following detailed description.

DETAILED DESCRIPTION

In the following detailed description, reference is made to the accompanying drawings, which constitutes a part of the present disclosure. In the drawings, similar symbols generally identify similar components unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be used and other changes may be made, without departing from the spirit and scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein and illustrated in the drawings, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are implicitly contemplated herein.

The present disclosure generally relates to, inter alia, a neural array including multiple synaptic devices and a method to operate the neural array.

Briefly stated, a neural array is generally described. A neural array according to some embodiments of the present disclosure may include an array unit, a first processing unit, and a second processing unit. The array unit may include one or more synaptic devices arranged in one or more rows and columns. Further, the array unit may have multiple selectable array values. In order for the array unit to have multiple selectable array values, each of the one or more synaptic devices of the array unit may be configured to form a NAND flash structure. The first processing unit may be configured to input a row input signal to the array unit and receive a row output signal from the array unit. The second processing unit may be configured to input a column input signal to the array unit and receive a column output signal from the array unit.

When the array unit selects a first array value, the first processing unit or the second processing unit may receive an output signal based on the first array value from the array unit. Then, when the array unit selects a second array value, the first processing unit or the second processing unit may use the output signal based on the first array value and input an input signal to the array unit that selects the second array value. As such, an output signal based on an array value may become an input signal for another array value, and, thus, a neural array of multiple layers without interconnection among the layers can be implemented.

Under the control of the neural array according to some embodiments of the present disclosure, a Restricted Boltzmann Machine may be implemented. The neural array may select the first array value. The neural array may generate a first output signal based on a first input signal and the first array value. The neural array may generate a reconstructed first input signal using the first output signal and the first array value and may also generate a reconstructed first output signal using the reconstructed first input signal and the first array value. The neural array may update the first array value based on the first input signal, the first output signal, the reconstructed first input signal, and the reconstructed first output signal. Then, the neural array may generate a second input signal based on the first input signal and the updated first array value and select a second array value. The neural array may update the second array value using the second input signal in the same way as the neural array updates the first array value.

The exemplary neural array may update the multiple array values by the RBM and then perform a Back Propagation Process to further update the updated multiple array values.

FIG. 1is a block diagram schematically illustrating an exemplary synaptic device100in accordance with at least some embodiments of the present disclosure. The exemplary synaptic device100may include a synaptic module110, a first input unit120, a first output unit130, a second input unit140, and a second output unit150. The synaptic device100may receive a signal from one or more other synaptic devices, and may generate a signal based on the received signal and transmit the generated signal to one or more other synaptic devices. The signals transmitted, received and generated by the synaptic device100may be various types of signals, and the transmitted, received and generated signals may be identical to or different from each other. In an example, if the synaptic device100uses a pulse signal, the synaptic device100may increase or decrease internal potential and generate a new output signal based on the internal potential. When the synaptic device100uses a pulse signal, information can be represented as a pulse width of the pulse signal. In another example, if the synaptic device100uses an analog signal, information may be represented using the intensity of a voltage or current of the analog signal. In yet another example, if the synaptic device100uses a digital signal, information may be represented as one or more discrete values.

The synaptic module110may have a coefficient value. When the synaptic device100receives an input signal, the synaptic module110may generate an output signal by applying the coefficient value to the received input signal. In some embodiments, the synaptic module110may have multiple selectable coefficient values. For example, the synaptic module110may have a first coefficient value, a second coefficient value, . . . , and a nth coefficient value. The synaptic device100may select and used one of the multiple coefficient values based on predetermined conditions. In some examples, the synaptic module110may be configured to operate in a time-division multiplexing mode. The synaptic module110may select the first coefficient value in a first time section according to the time-division multiplexing mode. The synaptic module110may select a second coefficient value in a second time section after the first time section. As such, the synaptic module110may select a nth coefficient value in a nth time section according to the time-division multiplexing mode.

In some examples, the synaptic module110may include multiple synaptic elements connected in series. In these examples, each of the synaptic elements may have a specific coefficient value. In some examples, each of the synaptic elements may be configured with a Floating Gate Metal Oxide Silicon Field Effect Transistor (FG MOSFET). Since a floating gate of the FG MOSFET is available to store information therein, a specific coefficient value of a synaptic element can be determined based on the information stored in the floating gate, thereby the synaptic module110may have multiple coefficient values.

The multiple synaptic elements connected in series in the synaptic module110may form a NAND flash structure. Since the multiple synaptic elements form a NAND flash structure, the number of synaptic elements integrated per unit area may be increased.

The first input unit120may receive a first input signal from the external or another synaptic device. The first output unit130may be configured to output a first output signal generated based on the first input signal received by the first input unit120and the coefficient value of the synaptic module110. Further, the second input unit140may receive a second input signal. In some examples, the second input signal may be generated based on the first output signal output by the first output unit130. The second output unit150may be configured to output a second output signal generated based on the coefficient value of the synaptic module110. As such, the synaptic device100may include the second input and output units140and150distinct from the first input and output units120and130. An example of the input and output of the synaptic device100will be described more clearly with reference toFIGS. 2A through 2E, and an architecture of a neural array formed using the synaptic devices100will be described more clearly with reference toFIG. 4andFIG. 5.

FIGS. 2A through 2Eare block diagrams schematically illustrating an operation of the exemplary synaptic device100illustrated inFIG. 1.FIG. 2Aillustrates that the synaptic device100receives an input signal from two input lines and outputs an output signal to two output lines. More specifically, referring toFIG. 2A, the first input unit120of the exemplary synaptic device100may receive an input signal from a row input line. The first output unit130of the synaptic device100may output an output signal generated based on the input signal received by the first input unit120to a column output line. The second input unit140may receive an input signal from a column input line. In some examples, the input signal received by the second input unit140may be associated with the output signal output by the first output unit130. The second output unit150of the synaptic device100may output an output signal generated based on the input signal received by the second input unit140to a row output line.

FIG. 2BtoFIG. 2Eillustrate an example of the input and output of the synaptic device100according to an example where the synaptic device100has a coefficient value α. As illustrated inFIG. 2B, the first input unit120may receive an input signal corresponding to a logic value “1” from the row input line. In response to the received input value “1,” the first output unit130of the synaptic device100may output an output current of α·I0to the column output line, as illustrated inFIG. 2B, in response. The first input unit120may receive an input signal corresponding to a logic value “0,” as illustrated inFIG. 2C. In response to the received input value “0,” the first output unit130of the synaptic device100may output an output current of 0, as illustrated inFIG. 2C.

Likewise, as illustrated inFIG. 2D, the second input unit140may receive an input signal corresponding to the logic value “1” from the column input line. In response to the received input value “1,” the second output unit150of the synaptic device100may output an output current of α·I0to the row output line, as illustrated inFIG. 2B. The second input unit140may receive an input signal corresponding to the logic value “0” as illustrated inFIG. 2E. In response to the received input value “0,” the second output unit150of the synaptic device100may output an output current of 0, as illustrated inFIG. 2E. AlthoughFIG. 2BtoFIG. 2Eillustrate that the synaptic device has a coefficient value of α, the synaptic device100may also have other coefficient values in addition to the coefficient value of α, and may select one of the multiple coefficient values and use the selected coefficient value. An exemplary circuit diagram for the synaptic device100having multiple coefficient values will be described in more detail with reference toFIG. 3.

FIG. 3is an exemplary circuit diagram for the exemplary synaptic device illustrated inFIG. 1. It will be understood by those skilled in the art that the circuit diagram for the synaptic device100illustrated inFIG. 3is a schematic diagram and can be modified in various ways in an actual circuit. Referring toFIG. 3, the synaptic device100includes two MOSFETs and multiple FG MOSFETs. When an input signal is received by a gate input port310of a MOSFET, an output signal is output through an output port320corresponding thereto, and when an input signal is received at a gate input port330of a MOSFET, an output signal is output through an output port340corresponding thereto. Each of the multiple FG MOSFETs may include coefficient information, and as illustrated inFIG. 3, the multiple FG MOSFETs are connected in series. Due to a structure of the FG MOSFETs connected in series, the number of synaptic devices integrated per unit area can be increased.

Bias signals bias [0], bias [1], etc. are input to a gate port of each of the multiple FG MOSFETs, and these bias signals may determine whether a FG MOSFET performs a reading operation to read information stored therein or a pass operation to operate as a closed circuit. The synaptic device100as illustrated inFIG. 3may operate in a time-division multiplexing mode. For example, in a first time section according to the time-division multiplexing mode, a FG MOSFET in a first stage may receive bias [0]=Vreadwhere the Vreadmay be a value suitable to read information stored in a floating gate of the FG MOSFET. Meanwhile, a gate of a FG MOSFET in a second stage may receive bias [1]=Vpasswhere the Vpassmay be a value suitable for the FG MOSFET to operate as a closed switch regardless of information stored in a floating gate of the FG MOSFET. As such, in the first time section, FG MOSFETs in the stages other than the FG MOSFET in the first stage may receive Vpassthrough the gates to operate as closed switches. In the first time section, the input port310may receive an input signal and the output port320may output an output signal based on the received input signal and coefficient information of the FG MOSFET in the first stage. In some examples, the input port330may receive a new input signal based on the signal output by the output port320. When the input port330receives the new input signal, the input port310may not receive an input signal, and a MOSFET corresponding to the input port310and the output port320may operate as an open switch.

As such, when the input port330receives the new input signal after the output port320outputs the output signal, the output port340may output a new output signal based on the received input signal and the coefficient information of the FG MOSFET in the first stage.

FIG. 4is a block diagram schematically illustrating an exemplary neural array400configured to constitute a deep neural network in accordance with at least some embodiments of the present disclosure. The neural array400may include an array unit410, a first processing unit420, and a second processing unit430. In some examples, the array unit410may include one or more synaptic devices arranged in one or more rows and columns. In some examples, each of the one or more synaptic devices may have multiple selectable coefficient values as described with reference toFIG. 1toFIG. 3. As described above, the synaptic devices arranged in the respective rows and columns may be configured to receive an input signal of a corresponding row and output an output signal of a corresponding column based on a selected coefficient value and the received input signal of the corresponding row. Further, the synaptic devices may be configured to receive an input signal of a corresponding column and output an output signal of a corresponding row based on a selected coefficient value and the received input signal of the corresponding column. As such, the array unit410may have array values including coefficient values included in the one or more synaptic devices arranged in one or more rows and columns. Since each synaptic device has multiple selectable coefficient values, the array unit410may also have multiple selectable array values.

As illustrated inFIG. 4, the first processing unit420may be coupled to the array unit410. The first processing unit420may input a row input signal corresponding to each row of the array unit410into the array unit410. In some examples, the first processing unit420may include one or more processing units connected corresponding to the respective rows of the array unit410, and each of the one or more processing units may input a row input signal into a corresponding row of the array unit410. The array unit410may output one or more column output signals based on the one or more row input signals received from the first processing unit420and an array value selected by the array unit410.

As illustrated inFIG. 4, the second processing unit430may also be coupled to the array unit410. The second processing unit430may receive one or more column output signals corresponding to one or more columns of the array unit410. Further, the second processing unit430may be configured to generate column input signals for the respective columns based on the received one or more column output signals. In some examples, the second processing unit430may include one or more processing units connected corresponding to the respective columns of the array unit410, and each of the one or more processing units may input a column input signal into a corresponding column of the array unit410. The array unit410may output multiple row output signals based on the one or more column input signals received from the second processing unit430and an array value selected by the array unit410.

In some examples, the array unit410may receive a row input signal from the first processing unit420. The array unit410may be configured to output a column output signal to the second processing unit430based on the row input signal received from the first processing unit420and a selected array value. Further, the array unit410may receive a column input signal from the second processing unit430. The array unit410may be configured to output a row output signal to the first processing unit420based on the column input signal received from the second processing unit430and a selected array value.

In the exemplary neural array400according to the present disclosure, a neural network in which multiple layers are stacked may be constructed. That is, the first processing unit420and the second processing unit430may correspond to layers of the neural network. The array unit410may correspond to a synapse connecting the layers. In an example, the array unit410may include a first array value and a second array value. When the array unit410selects the first array value, for example, the first processing unit420may correspond to a first layer and the second processing unit430may correspond to a second layer. When the array unit410receives an input signal from the first processing unit420(corresponding to the first layer), the array unit410may generate an output signal based on the input signal and the first array value and output the generated output signal to the second processing unit430(corresponding to the second layer).

As described above, the neural network in which multiple layers are stacked can be implemented in a neural array IC using the neural array400according to some embodiments of the present disclosure and interconnection required between neural array ICs may not be needed. It will be understood by those skilled in the art that the method of forming the neural network in which multiple layers are stacked by the neural array400according to the present disclosure is not limited to the above-described examples and can be modified without departing from the spirit or scope of the subject matter presented herein.

In some embodiments, each of the one or more synaptic devices of the array unit410may be configured to form a NAND flash structure, and, thus, the number of synaptic devices integrated per unit area can be increased. Such a synaptic device may include, for example, a FG MOSFET.

FIG. 5is a specific block diagram for the neural array400illustrated inFIG. 4and including synaptic devices. Hereinafter, examples of implementing a Restricted Boltzmann Machine (RBM) and a Back Propagation (BP) process using the exemplary neural array400according to the present disclosure will be described.

Referring toFIG. 5, i×j number of synaptic devices, such as the synaptic device100illustrated inFIG. 1, are arranged in i rows and j columns in the array unit410of the neural array400. The synaptic devices of the neural array400have coefficient values, respectively. In some examples, a synaptic device arranged in a first row and a first column may have a coefficient value α11, and a synaptic device arranged in an ith row and a jth column may have a coefficient value αij. In these examples, the array unit410may have a first array value A, and A can be represented by the following Equation 1.

The first processing unit420may include i processing units420-1,420-2, . . . ,420-iand may input i row input signals x1, x2, . . . , xiinto the array unit410. The second processing unit430may include j processing units430-1,430-2, . . . ,430-j, and as the i row input signals x1, x2, . . . , xiare input from the first processing unit420, each of the processing units430-1,430-2, . . . ,430-jmay receive Iout,1, . . . , Iout,j, and Iout,jcan be represented by the following Equation 2.

where I0represents a unit output current from a synaptic device and n represents a positive integer. The second processing unit430may calculate or determine j number of column input signals y1, y2, . . . , yjbased on Iout,j. In an example, when the second processing unit430determines a value of yj, the following Probability Equation 3 can be used.
P(yj=1)=1/(1+exp(−Σαi,jxi))  [Equation 3]

Further, the second processing unit430may input j column input signals y1, y2, . . . , yjinto the array unit410, and then, the array unit410may output an output signal Iout,ito the first processing unit420, and Iout,Ican be represented by the following Equation 4.

Herein, Y represents a matrix corresponding to j column input signals, ATrepresents a transposed matrix of an array A, and m represents a positive integer.

The first processing unit420may calculate or determine i reconstructed row input signals x′1, x′2, . . . , x′ibased on Iout,i. In an example, it will be understood by those skilled in the art that when the first processing unit420determines a value of x′i, Equation 3 can be appropriately modified for use. The second processing unit430may obtain j reconstructed column input signals y′1, y′2, . . . , y′jbased on the i reconstructed row input signals x′1, x′2, . . . , x′Iand the first array value A in a similar way as described in the above. The neural array400may update the first array value A based on the row input signals x1, x2, . . . , xi, the column input signals y1, y2, . . . , yj, the reconstructed row input signals x′1, x′2, . . . , x′I, and the reconstructed column input signals y′1, y′2, . . . , y′j. In an example, the first array value A can be updated based on the following Equation 5.
Δαi,j=<xiyj>−<x′iy′j>  [Equation 5]

In some embodiments, each of the multiple synaptic devices may have multiple selectable coefficient values and the array unit410may also have multiple selectable array values. The neural array400may operate in a time-division multiplexing mode and the array unit410may select the first array value A including αijas a component in a first time section and update the first array value A as described above, and the updated first array value is represented as A′. In the above-described example, A is described as being updated using a group of input signals x1, x2, . . . , xi, but is not limited thereto, and A may be updated multiples times using multiple groups of input signals.

In a second time section after the first time section, the second array value B including βijas a component may be selected and then may be updated using the new column input signals y1, y2, . . . , yj. An operation of updating the array value B by the neural array400is similar to the operation of updating the array value A and will be briefly described as follows. The first processing unit420may obtain i row input signals z1, z2, . . . , zibased on j column input signals y1, y2, . . . , yjand B, and the second processing unit430may obtain y′1, y′2, . . . , y′jbased on z1, z2, . . . , ziand B, and then, the first processing unit420may also obtain z′1, z′2, . . . , z′Iusing y′1, y′2, . . . , y′jand B. The second array value B may be updated using y1, y2, . . . , yj, z1, z2, . . . , zi. y′1, y′2, . . . , y′j, and z′1, z′2, . . . , z′i. As such, the second array value B can be updated multiples times using multiple groups of input signals.

FIG. 6is a flowchart illustrating an example process of implementing RBM in a neural array in accordance with at least some embodiments of the present disclosure. A process600illustrated inFIG. 6may include one or more operations, functions, or actions as shown in blocks610,620and/or630. These operations, functions, or actions can be performed by the neural array400described above with reference to, e.g.,FIG. 4. The blocks do not intend to limit the present disclosure to the described embodiments. For example, it will be appreciated by those skilled in the art that as for the present process disclosed herein, functions performed in the process and method can be implemented in a different order. Further, the briefly described operations are provided only for purposes of illustration and some of the operations can be selectively performed, can be combined with each other or can be divided up into additional operations without departing from the spirit or scope of the present disclosure. The process may start from a block610in which a nth array value is selected. In some examples, the neural array may operate in a time-division multiplexing mode, and may select a first array value in a first time section. The process600may continue from the block610to a block620in which the nth array value is updated.

In the block620, the neural array may update the selected nth array value. In some examples, the neural array may output multiple column output signals using multiple row input signals and the first array value, and then may generate multiple column input signals in response to the multiple column output signals. In these examples, the neural array may output multiple row output signals based on the first array value and the multiple column input signals, and may reconstruct the multiple row input signals based on the multiple row output signals. Further, the neural array may output multiple column output signals based on the reconstructed multiple row input signals and the first array value, and then may reconstruct the multiple column input signals. Then, the neural array may update the first array value based on the multiple row input signals, the multiple column input signals, the reconstructed multiple row input signals, and the reconstructed multiple column input signals. The process600may continue from the block620to a block630in which a value of n is increased by 1.

In some examples, when the first array value is updated, the neural array may perform a process of updating a second array value based on the first array value. For example, the neural array may obtain multiple column input values based on the updated first array value. Then, the neural array may select the second array value and update the second array value as described above using the obtained multiple column input values and the second array value. In such manner, the process600can update multiple array values, sequentially.

FIG. 7is a flowchart illustrating an exemplary process of implementing BP in a neural array in accordance with at least some embodiments of the present disclosure. A process700illustrated inFIG. 7may include one or more operations, functions, or actions as shown in blocks710,720,730,740and/or750. These operations, functions, or actions can be performed by the neural array400described above with reference to, e.g.,FIG. 4andFIG. 5. In some additional examples, the process illustrated inFIG. 7may be performed after the process600illustrated inFIG. 6. For example, the process700may be performed after multiple array values are updated by the RBM.

The blocks do not intend to limit the present disclosure to the exemplified embodiments. For example, it will be acknowledged by those skilled in the art that as for the present process disclosed herein, functions performed in the process and method can be implemented in a different order. Further, the briefly described operations are provided only for purposes of illustration and some of the operations can be selectively performed, can be combined with each other or can be divided up into additional operations without departing from the spirit or scope of the present disclosure. The process700may start from a block710in which an input signal is received. In some examples, the neural array may have multiple selectable array values as described in with reference toFIG. 4, and an output signal based on an array value may become an input signal for another array value. For example, the neural array may include a first array value A, a second array value B, and a third array value C, and may receive an input signal X. After the neural array receives the input signal, the process700may continue from the block710to a block720.

In the block720, the neural array may perform forward propagation based on the received input signal and multiple array values. In some examples, the neural array may output an output signal while selecting the first, second, and third array values A, B, and C in sequence, and a result of the forward propagation is as shown in the following Equation 6.
Y=X·A
Z=Y·B
O=Z·C[Equation 6]

As shown in Equation 6, when the neural array selects the first array value A, the neural array may determine Y based on the input signal X and the first array value A. Then, when the neural array selects the second array value B, the neural array may determine Z using the new input signal Y and the second array value B. Then, when the neural array selects the third array value C, the neural array may determine a final output signal O using the new input signal Z and the third array value C. The process700may continue from the block720to a block730.

In the block730, the neural array may obtain an error value based on the final output signal and a target value for the final output signal. In some examples, the neural array may obtain a difference (E0=target value−O) between the target value estimated based on the input value X and the final output signal O as an error value E0. The process700may continue from the block730to a block740.

In the block740, the neural array may perform back propagation based on the error value and multiple array values. In some exemplary embodiments, the neural array may perform back propagation based on the error value E0, and results of the error value E0and back propagation are as shown in the following Equation 7.
Eo=TARGET VALUE−O
Ez=Eo·CTor sign (Eo·CT)
Ey=Ez·BTor sign (EZ·BT)  [Equation 7]

where T represents a transposed matrix and sign represents a sign of a calculation result. The process700may continue from the block740to a block750.

In the block750, the neural array may update the multiple array values based on the results of the forward propagation and back propagation. In some examples, the neural array may update the first, second, and third array values based on the result X, Y, and Z of the forward propagation and the result E0, Ez, and Eyof the back propagation, and the adjusted array values can be determined based on, e.g., the following Equation 8.
ΔC=ZT·EO
ΔB=YT·Ez
ΔA=XT·Ey[Equation 8]

In the above-described examples, the neural array is described as having the three array values A, B, and C. However, it will be understood by those skilled in the art that the neural array may have a different number of array values and perform RBM and/or BP based on the array values.

Further, as described above, the neural array according to at least some embodiments of the present disclosure may have multiple selectable array values and may be configured to form a neural network in which multiple layers are stacked using the multiple array values. Furthermore, interconnection among the multiple layers may not be needed by using an output for an array value as an input for another array value. Also, a synaptic device included in the neural array is configured to form a NAND flash structure, and, thus, the number of synaptic devices integrated per unit area can be increased. Moreover, as described above, the neural array according to at least some embodiments of the present disclosure implements RBM and/or BP, and, thus, it is possible to perform training of the neural array.

The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is to be limited with the terms of the appended claims and the full scope of equivalents to which such claims are entitled. Further, it is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.

With respect to the use of substantially any plural and/or singular terms herein, those skilled in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity. Further, it will be understood by those skilled in the art that, in general, terms used herein and especially in the appended claims (e.g., the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to”, the term “having” should be interpreted as “having at least”, the term “includes” should be interpreted as “includes but is not limited to”, etc.).