Systems for introducing memristor random telegraph noise in Hopfield neural networks

Systems are provided for implementing a hardware accelerator. The hardware accelerator emulate a stochastic neural network, and includes a first memristor crossbar array, and a second memristor crossbar array. The first memristor crossbar array can be programmed to calculate node values of the neural network. The nodes values can be calculated in accordance with rules to reduce an energy function associated with the neural network. The second memristor crossbar array is coupled to the first memristor crossbar array and programmed to introduce noise signals into the neural network. The noise signals can be introduced such that the energy function associated with the neural network converges towards a global minimum and modifies the calculated node values.

DESCRIPTION OF RELATED ART

Artificial neural networks are a family of technical models based on biological nervous systems, which are used to estimate or approximate functions that depend on a large number of inputs. Neural networks may be represented as a system of interconnected “neurons” which exchange messages between each other. The connections may have numerical weights that can be tuned based on experience, making neural networks adaptive to inputs and capable of machine learning. Artificial neural networks may have a variety of applications, including function approximation, classification, data processing, robotics, and computer numerical control.

There may be various types of neural networks, including feedforward neural networks, radial basis function neural networks, recurrent neural networks, and other types. As a general description, Hopfield Neural Networks (HNNs) are a type of artificial neural network that involve computed iterative steps based on energy minimizing rules. The recurrent, or feedback characteristics of HNNs may be particularly suitable for implementing logic operations, solving optimization problems, performing analog-digital conversion, and implementing associative memories (e.g., content-addressable memories), for example. However, in some cases, models based on HNNs may experience limitations due to its computational properties. For instance, a Hopfield based system may converge at local minima of its energy function, which causes the computation to end without reaching a global minimum. This scenario has a high probability of occurring in HNN implementations, and may be problematic, as it is the global minimum of the network's energy function that typically produces optimal values, or most correct solution. Stabilization at local minima is a common drawback associated with HNNs (due to its energy minimizing properties), which affects the accuracy of its computations and serves as a deterrent from the widespread use of HNNs in various applications.

DETAILED DESCRIPTION

Various embodiments described herein are directed to hardware configured to emulate a logical neural network. Furthermore, the hardware has been adapted to include dedicated circuitry, namely a memristor crossbar array, that is designed for introducing random telegraph noise (RTN) into the emulated neural network for optimized performance.

Memristors are devices that may be used as components in a wide range of electronic circuits, such as memories, switches, radio frequency circuits, and logic circuits and systems. In a memory structure, a crossbar array of memory devices having memristors may be used. In memory devices, memristors may be used to store bits of information, 1 or 0. The resistance of a memristor may be changed by applying an electrical stimulus, such as a voltage or a current, through the memristor. Generally, at least one channel may be formed that is capable of being switched between two states—one in which the channel forms an electrically conductive path (“on”) and one in which the channel forms a less conductive path (“off”). In some other cases, conductive paths represent “off” and less conductive paths represent “on”. Furthermore, memristors may also behave as an analog component with variable conductance.

In some applications, a memory crossbar array can be used to perform vector-matrix computations. For example, an input voltage signal from each row line of the crossbar is weighted by the conductance of the resistive devices in each column line and accumulated as the current output from each column line. Ideally, if wire resistances can be ignored, the current (I) flowing out of the crossbar array will be approximately represented in the equation below:
IT=VTG(1)

where V is the input voltage and G is the conductance matrix.

The memristor crossbar array is configured to include contributions from each memristor in the crossbar array. The use of memristors at junctions or cross-points of the crossbar array enables programming the resistance (or conductance) at each such junction.

Examples disclosed herein include hardware accelerators for calculating node values for neural networks. Example hardware accelerators may include a memristor crossbar array programmed to calculate node values. Memory cells of the memristor crossbar array may be programmed according to a weight matrix. Driving input voltages mapped from an input vector through the memristor crossbar array may produce output current values which may be compared to a threshold current to generate a new input vector of new node values. In this manner, example accelerators herein provide for hardware calculations of node values for neural networks.

Referring now to the drawings,FIG.1illustrates an example hardware accelerator100according to the embodiments. Hardware accelerator100may be a hardware unit configured for calculating node values for neural networks. Hardware accelerator100may calculate new node values of a neural network by transforming an input vector in relation to a weight matrix. Hardware accelerator100may do so by calculating a vector-matrix multiplication of the input vector with the weight matrix.

In the example illustrated inFIG.1, a Hopfield Neural Network (HNN) is particularly implemented by the hardware accelerator100described herein.FIG.1shows an implementation of the example hardware accelerator100having various components, including: a memristor crossbar array structure101; multiplexer (MUX)115; comparator120; MUX decoder125; delay flip-flop (DFF) (I/O Buffer)130; and drivers135. Additionally, in the hardware accelerator ofFIG.1, the memristor crossbar array structure101includes memristor crossbar array105in combination with an additional memristor crossbar array110. Memristor crossbar array105can be configured to implement the weight matrix aspects of the HNN, as described above. Memristor crossbar array110can be configured for implementing the noise inducing aspects of the HNN, as further described in herein.

As previously described, HNNs are a type of artificial neural network that can be used as a computational mechanism for various applications. A model of an HNN is conceptually illustrated inFIG.4. Graphically, an HNN as a mathematical construct can be represented as graph400of a bidirectional (undirected) network with nodes (si)410A-410F and edges (wij)420A-4200. Edges420A-4200can be formed between each of the nodes410A-410F. A computational “problem” can be encoded (or trained) in the edge weights and a threshold function. Input node values415A-415F can be delivered to the nodes410A-410F until the computational “answer” to the problem is determined by a final state for the node values. In this manner, an HNN can be a dynamic system, and the node values can evolve based on the edge weightings to all the other node values (e.g., as a dot product operation). The dynamics follow energy minimizing rules, such that the “energy” of the system does not increase, and thus can find a minimum. The final configuration of the nodes encodes the solution. Therefore the “energy” indicates whether the network is modified when an update occurs, and the system reaches its minimum energy when it becomes stable.

The HNN can be “trained” to a set of distinct inputs. As alluded to above, HNNs can employ update rules that minimize a network's energy function (e.g., only reducing energy at each successive update). It should be also be understood that in some cases, inputs are used as initial conditions, and the HNN is allowed to evolve to a rest state (e.g., without training by changing the weights). For example, an energy function of an HNN may be represented as the equation below:
E=½Σi,jwijsisj+Σiθisi(2)

Referring now toFIG.3B, an example of a graphical representation of an energy function associated with an HNN is illustrated. It should be appreciated thatFIG.3Bdepicts the energy function corresponding to the HNN as a continuous function, whereas different implementations of an HNN can also correspond to a system with a discrete state energy function. As seen, an energy function can be represented as wave350being substantially sinusoidal in form, where an energy value for the HNN at a particular state corresponds to the vertical position (e.g., height) of the wave350along the y-axis of the graph (with respect to the x-y axis represented by dashed lines). The energy function of the HNN transitions from initial states (e.g., start of a set of inputs) having higher energy levels, illustrated as higher points310A-310E of wave350with amplitudes in the (+) y direction (with respect to the x-y axis represented by dashed lines). In between each of the higher energy states, the network propagates through subsequent states having lower energy and eventually reaches a state with minimal energy. The minimal energy states are illustrated as lower points315A-315F on wave350with an amplitude in the (−) y direction. Accordingly, node values calculated by the HNN also evolve towards energy minima, or the computational answer. Each set of inputs into the HNN can converge to its own answer, which corresponds to a respective minimum energy for that set on the energy graph, also referred to as local minima. Referring toFIG.3B, updates to the evolving node values of the HNN are represented by ball351. The position of the ball351along wave350corresponds to the system's energy, which impacts the node values calculated for that particular state. Thus, the HNN reaching a local minimum for a set of inputs can be represented on the energy graph by ball351being positioned at one of the low points315A-315F on wave350. For purposes of illustration, the local minima phenomena can generally be described in the energy graph as ball351resting in a valley. Each valley, or low point315A-315F, produces respective node values that can be considered a computational answer. As such, an HNN has the potential to converge at any of the multiple “answers” (having an energy that becomes stable at any of the points315A-315F on the graph) based on the received input.

A global minimum can be characterized as the optimally minimized energy of the HNN across multiple sets of inputs. Consequently, it is the global minimum (or global convergence) that represents the most accurate calculation by the HNN. InFIG.3B, as each of the low points315A-315F represents local minima for the energy function, the global minimum can be represented by the lowest point of wave350on the energy graph, or low point315C. Restated, although the local minima of the energy function correspond to multiple potential computational answers, node values generated by the state at global minimum can be the optimal answer. As previously described, models based on HNN may experience limitations due to its computational properties. For instance, a system can stabilize at local minima (e.g., false solution to a problem) of the energy function. Referring back to the example, ball351may settle in one of the valleys at lower points315A,315B,315D-315F in the energy graph. In order for the ball351to escape a valley, energy can be injected into the system to allow ball351to reach a higher energy level, climbing an upward slope to approach one of the higher points310A-310E. However, as described above, HNNs characteristically do not increase its energy function. Consequently, in the event that the HNN stops evolving at local minima, then the system may be stuck at that state. The energy minimizing rule of HNNs may prevent ball351from escaping the valley, in turn preventing the system from ultimately reaching the global minimum (e.g., correct solution to a problem). As a result, hardware implementations of HNNs can also suffer from the negative impacts of local minima. The embodiments address local minima constraints by adapting the hardware accelerator100inFIG.1, which emulates an HNN, to include noise inducing circuitry. In particular, the memristor crossbar array110is configured to generate random telegraph noise (RTN). Furthermore, the memristor crossbar array110introduces the RTN into the circuit in a manner that simulates injecting external energy into the energy graph of an HNN. Referring again to the example discussed in reference toFIG.3B, the memristor crossbar array110is configured to introduce RTN such that energy is injected into the system, exciting the ball enough to escape the valley, or local minima. The disclosed systems and techniques may be particularly beneficial for the various applications of HNNs, modifying the circuit design of hardware accelerator100in order to increase the overall efficiency and accuracy of these systems.FIG.3Balso depicts that the amplitude of the noise signal320is gradually decreased using a simulated annealing approach, as described herein. As an example, more noise may be needed initially for enough excitation to jump out of local minima. Eventually, once global minima is reached, the need for the hill-climbing features of added noise is no longer needed. Thus, a small amount of noise, or no noise is more desirable in the global minima state. This illustrated inFIG.3B, as the noise signal320is shown to have a gradually smaller amplitude approaching the global minima point315C.

The embodiments provide an efficient and hardware driven solution, which realizes advantages over some existing approaches to address local minima associated with HNNs. For instance, quantum annealing is a known technique to mitigate local minima. Nonetheless, quantum annealing is a temperature-dependent solution, where the environment is controlled for cooling. The lower temperatures induce particles to initially lose energy, and subsequently pull energy from the environment to assume a higher energy state (thus avoiding local minima of the energy function). In the case of hardware implemented HNNs, cryogenic temperatures are used to achieve cooling of the circuits. Thus, quantum annealing often involves high energy consumption and system inefficiency. Furthermore, there is uncertainty regarding whether advancements in quantum annealing will be successful without requiring cooling. The embodiments integrate local minima escaping mechanisms into the HNN emulating circuitry itself, therefore reducing costs and providing an alternative to the other environmental control concerns of other approaches, such as quantum annealing.

Referring back toFIG.1, hardware accelerator100can include memristor crossbar array105and memristor crossbar array110. Memristor crossbar array105can be a configuration of parallel and perpendicular lines with memory cells coupled between lines at intersections. Memristor crossbar array105may include a plurality of row lines104, a plurality of column lines106, and a plurality of memory cells106A-1061. Each of the memory cells106A-1061may be coupled between each unique combination of one row line104and one column line106. In other words, none of the memory cells106A-1061share both a row line104and a column line107.

Row lines104may be electrodes that carry current through memristor crossbar array105. In some examples, row lines104may be parallel to each other, generally with equal spacing. Row lines104may sometimes be, for example, a top electrode or a word line. Similarly, column lines106may be electrodes that run nonparallel to row lines104. Column lines106may sometimes be, for example, a bottom electrode or bit line. Row lines104and column lines106may serve as electrodes that deliver voltage and current to the memory cells106A-1061. Example materials for row lines104and column lines106may include conducting materials such as Pt, Ta, Hf, Zr, Al, Co, Ni, Fe, Nb, Mo, W, Cu, Ti, TiN, TaN, Ta2N, WN2, NbN, MoN, TiSi2, TiSi, TiSi3, TaSi2, WSi2, NbSi2, V3Si, electrically doped polycrystalline Si, electrically doped polycrystalline Ge, and combinations thereof. In the example ofFIG.1, crossbar array105may have N row lines and M column lines.

Memory cells106A-1061may be coupled between row lines104and column lines106at intersections of the row lines104and column lines106. For example, memory cells106A-1061may be positioned to calculate a new node values of an input vector of node values with respect to a weight matrix. Each of the memory cells106A-1061may have a memory device such as a resistive memory element, a capacitive memory element, or some other form of memory.

In some examples, each of the memory cells106A-1061may include a resistive memory element. A resistive memory element may have a resistance that changes with an applied voltage or current. Furthermore, in some examples, the resistive memory element may “memorize” its last resistance, either in a volatile or a non-volatile way. In this manner, each resistive memory element may be set to at least two states. In many examples, a resistive memory element may be set to multiple resistance states, which may facilitate various analog operations. The resistive memory element may accomplish these properties by having a memristor, which may be a two-terminal electrical component that provides memristive properties as described herein.

In some examples, a memristor may be nitride-based, meaning that at least a portion of the memristor is formed from a nitride-containing composition. A memristor may also be oxide-based, meaning that at least a portion of the memristor is formed from an oxide-containing material. Furthermore, a memristor may be oxy-nitride based, meaning that at least a portion of the memristor is formed from an oxide-containing material and that at least a portion of the memristor is formed from a nitride-containing material. Example materials of memristors may include tantalum oxide, hafnium oxide, titanium oxide, yttrium oxide, niobium oxide, zirconium oxide, or other like oxides, or non-transition metal oxides, such as aluminum oxide, calcium oxide, magnesium oxide, dysprosium oxide, lanthanum oxide, silicon dioxide, or other like oxides. Further examples include nitrides, such as aluminum nitride, gallium nitride, tantalum nitride, silicon nitride, and oxynitrides such as silicon oxynitride. In addition, other functioning memristors may be employed in the practice of the teachings herein.

A memristor may exhibit nonlinear or linear current-voltage behavior. Nonlinear may describe a function that grows differently than a linear function. In some implementations, a memristor may be linear or nonlinear in voltage ranges of interest. A voltage range of interest may be, for example, a range of voltages used in the operation of hardware accelerator100. In some examples, memory cells106A-1061may include other components, such as access transistors or selectors. For example, each of the memory cells106A-1061may be coupled to an access transistor between the intersections of a row line104and a column line106. Access transistors may facilitate the targeting of individual or groups of memory cells106A-1061for the purposes of reading or writing the memory cells.

Alternatively, a selector may be an electrical device that may be used in memristor devices to provide desirable electrical properties. For example, a selector may be a 2-terminal device or circuit element that admits a current that depends on the voltage applied across the terminals. A selector may be coupled to each of the memory cells106A-1061to facilitate the targeting of individual or groups of memory cells106A-1061. For example, a selector may do so by acting like an on-off switch, and it may mitigate sneak current disturbance.

The memory cells106A-1061of memristor crossbar array105may be programmed according to a weight matrix of a neural network. A weight matrix may represent a compilation of operations of a neural network. For example, a weight matrix may represent the weighted edges of HNN illustrated inFIG.4. The value stored in the memory cells106A-1061may represent the values of a weight matrix. In implementations of resistive memory, the resistance levels of each of the memory cells106A-1061may represent a value of the weight matrix. In such a manner, the weight matrix may be mapped onto crossbar array105.

Memory cells106A-1061may be programmed, for example, by having programming signals driven through them, which drives a change in the state of the memory cells106A-1061. The programming signals may define a number of values to be applied to the memory cells106A-1061. As described herein, the values of memory cells106A-1061of crossbar array105may represent a weight matrix of a neural network, such as an HNN.

In continuing reference toFIG.1, hardware accelerator100may receive an input vector of node values at the plurality of row lines104. The input vector may include node values which are to be evolved into next input values for the neural network. The input vector node values may be converted to input voltages (V′)103by a drive circuit, drivers135. A drive circuit135may deliver a set of input voltages that represents the input vector to the memristor crossbar array105. In some examples, the voltages103may be other forms of electrical stimulus such as an electrical current driven to the memory cells106A-1061. Furthermore, in some examples, the input vector may include digital values, which may be converted to analog values of the input electrical signals by a digital-to-analog converter. In other examples, the input vector may already include analog values.

Upon passing through the memristor crossbar array105, the plurality of column lines107may deliver output currents (I°)109, where the output currents109may be compared to a threshold current according to an update rule to generate a new input vector of new node values. Details of these operations is described in below.

Hardware accelerator100may also include other peripheral circuitry associated with crossbar array105. For example, an address decoder, MUX decoder125, may be used to select a row line104and activate a drive circuit135corresponding to the selected row line104. The drive circuit135, for a selected row line104, can drive a corresponding row line104with different voltages corresponding to a neural network or the process of setting resistance values within memory cells106A-1061of memristor crossbar array105. Similar drive and decode circuitry may be included for column lines107. Control circuitry may also be used to control application of voltages at the inputs and reading of voltages at the outputs of hardware accelerator100. Digital to analog circuitry and analog to digital circuitry may be used for input voltages103and the output currents109. In some examples, the peripheral circuitry above described can be fabricated using semiconductor processing techniques in the same integrated structure or semiconductor die as crossbar array.

As described herein, there are three main operations that occur during operation of the hardware accelerator100. The first operation is to program the memory cells106A-1061in the memristor crossbar array105so as to map the mathematic values in an N×M weight matrix to the array. In some examples, N and M may be the same number, and the weight matrix is symmetrical. In some examples, each of the memory cells106A-1061are programmed at a time during the programming operation. The second operation is to calculate an output current by the dot-product of input voltage and the resistance values of the memory cells of a column line107. In this operation, input voltages are applied, and output currents obtained, corresponding to the result of multiplying an N×M matrix by an N×1 vector. In some examples, the input voltages are below the programming voltages so the resistance values of the memory cells106A-1061, such as resistive memory, are not changed during the linear transformation calculation. The third operation is to compare the output currents with a threshold current. For example, comparators120may compare the output currents with the threshold current to determine a new input vector of new node values.

In an example, hardware accelerator100may calculate node values by applying a set of voltages (V′)103simultaneously along row lines104of the N×M crossbar array105and collecting the currents through column lines107and generating new node values. On each column line107, every input voltage103is weighted by the corresponding memristance (1/Gij) and the weighted summation is reflected at the output current. Using Ohm's law, the relation between the input voltages103and the output currents can be represented by a vector-matrix multiplication of the form: {VO}T=−{VI}T[G] Rs, where Gijis an N×M matrix determined by the conductance (inverse of resistance) of memristor crossbar array105, Rs is the resistance value of the sense amplifiers and T denotes the transpose of the column vectors V° and V′. The negative sign follows from use of a negative feedback operational amplifier in the sense amplifiers. From the foregoing, it follows that the hardware accelerator100can be utilized for multiplying a first vector of values {bi}Tby a matrix of values [aij] to obtain a second vector of values where i=1, N and j=1, M. The vector operation can be set forth in more detail as follows:
a11b1+a21b2+ . . . +aN1bN=c1
a1mb1+a2b2+ . . . +aNMbN=cM(3)

The vector processing or multiplication using the principles described herein generally starts by mapping a matrix of values [aij] onto crossbar array105or, stated otherwise, programming (e.g., writing) conductance values Gijinto the crossbar junctions of the memristor crossbar array105.

With reference still toFIG.1, in some examples, each of the conductance values Gijmay be set by sequentially imposing a voltage drop over each of the memory cells106A-1061. For example, the conductance value G2,3may be set by applying a voltage equal to VRow2at the second row line104of memristor crossbar array105and a voltage equal to VCol3at the third column line107of the array105. The voltage input, VRow2, may be applied to the second row line occurring at the second row line adjacent the j=1 column line. The voltage input, VCol3, will be applied to the third column line adjacent either the i=1 or i=N location. Note that when applying a voltage at a column line107, the sense circuitry for that column line may be switched out and a voltage driver switched in. The voltage difference VRow2−VCol3will generally determine the resulting conductance value G2,3based on the characteristics of the memory cell106flocated at the intersection. When following this approach, the unselected column lines107and row lines104may be addressed according to one of several schemes, including, for example, floating all unselected column lines107and row lines104or grounding all unselected column lines and row lines. Other schemes involve grounding column lines107or grounding partial column lines107. Grounding all unselected column lines and row lines is beneficial in that the scheme helps to isolate the unselected column lines and row lines to minimize the sneak path currents to the selected column line107.

In accordance examples herein, memristors used in memory cells106A-1061may have linear current-voltage relation. Linear current-voltage relations permit higher accuracy in the matrix multiplication process. However, memristor crossbar array105having linear memristors are prone to having large sneak path currents during programming of the array105, particularly when the size of memristor crossbar array105is larger than a certain size, for instance, 32×32. In such cases, the current running through a selected memristor may not be sufficient to program the memristor because most of the current runs through the sneak paths. Alternatively, the memristor may be programmed at an inaccurate value because of the sneak paths.

To alleviate the sneak path currents in such instances, and especially when larger arrays are desired, an access device, such as an access transistor or a non-linear selector, may be incorporated within or utilized together with a memristor to minimize the sneak path currents in the array. More specifically, memory cell should be broadly interpreted to include memristive devices including, for example, a resistive memory element, a memristor, a memristor and transistor, or a memristor and other components.

Following programming, operation of linear transformation accelerator100proceeds by applying the input voltages110and comparing the output currents to threshold currents. The output current delivered from column lines106may be compared, by current comparator116, with a threshold current. Current comparator116may be a circuit or device that compares two currents (i.e., output current and threshold current) and outputs a digital signal indicating which is larger. Current comparator116may have two analog input terminals and one binary digital output.

Additionally,FIG.1shows that the hardware accelerator100can include a noise inducing memristor crossbar array110implementing the local minima tolerant aspects described above. In the illustrated example ofFIG.1, the memristor crossbar array110for RTN can be additional to the N×M structure of memristor crossbar array105configured for mapping the weight matrix.FIG.2Ashows an example of a circuit configuration for the arrays comprising the memristor crossbar array structure101inFIG.1, and prominently illustrating a configuration for the noise inducing memristor crossbar array210. The memristor crossbar array210for RTN can principally include the elements of the weight matrix memristor crossbar array described in detail above. As shown inFIG.2A, the memristor crossbar array210includes a plurality of column lines207, a row line204, and memory cells211A-211C. For purposes of brevity, the elements of memristor crossbar array210are substantially similar to the memristor crossbar array discussed above in reference toFIG.1, and therefore is not discussed in detail again.

In the configuration ofFIG.2A, memristor crossbar array210can be generally described as an additional row to the memristor crossbar array205. In detail, memristor crossbar array210is configured to have the same number of column lines207as the memristor crossbar array205. Furthermore, the memristor crossbar array210can be operably coupled to memristor crossbar array205. As shown inFIG.2A, each column line207of memristor crossbar array210is coupled to a correspond column line207of the memristor crossbar array205such that each of the memory cells211A-211C is vertically aligned (in a column) with memory cells206A-2061. For example, coupling the arrays can align memory cell211A of memristor crossbar array210with memory cells206A,206D,20G of memristor crossbar array205, forming a contiguous column. The memristors of memory cells206A-206A (located at each intersection of column lines207and row lines204in array205) are configured for tuning weights of the emulated HNN. The memristors of memory cells211A-211C (located at each intersection of column lines207and row line204in array210) can be programmed for tuning noise injected in the emulated HNN. The memory cells211A-211C may also be programmed, as described above about the memory cells106A-1061.

Referring now toFIG.2B, an alternate configuration for the noise inducing memristor crossbar array260is shown. It should be appreciated that the memristor crossbar array255and the memristor crossbar array260illustrated inFIG.2Bis an example circuit configuration for the arrays that may comprise the memristor crossbar array structure101ofFIG.1. Referring back toFIG.2B, the example illustrates the memristor crossbar array260configured as an additional column coupled to the N×M structure of crossbar array255. In contrast toFIG.2A, the memristor crossbar array260includes a column line254, and multiple row lines253. Each row line254of memristor crossbar array260can be coupled to a correspond row line254of the memristor crossbar array255such that each of the memory cells261A-261C is horizontally aligned (in a row) with memory cells206A-2061. It should be appreciated that implementations of the hardware accelerator, including the noise inducing memristor crossbar array, are not limited to the examples ofFIGS.2A-2B. Accordingly, the disclosed hardware accelerator, and the components therein, can be arranged in various other circuit configurations as deemed appropriate and/or necessary. For example, the memristor crossbar array260can be configured to include multiple rows and multiple columns.

FIG.2Cillustrates another example of a weight matrix memristor crossbar array270arranged in an n×n array, coupled to the additional noise inducing crossbar array280. In an embodiment, the memristor crossbar array structure101, as seen inFIG.1, can be implemented using the circuit configuration for the memristor crossbar array270and the memristor crossbar array280shown inFIG.2C. The memory crossbar array270may be implemented as a bipolar array, comprising an array of memory cells275A-275D. In the illustrated example, an input vector298may be provided to the memristor crossbar array270. The number of memory cells may be a function of a number of values in a weight matrix. For example, if there are nine values in the weight matrix, then the weight matrix memory crossbar array270may include nine memory cells.

Each of the memory cells275A-275D may be comprised of a 2×2 memristor array290A-290D, respectively. The 2×2 memristor arrays290A-290D may be arranged in an n×n array. Thus, if there are nine memory cells275A-275D, there may be three rows of 2×2 memristor arrays290A-290D, where each row has three columns of 2×2 memristor arrays290A-290D. A plurality of rows271and a plurality of columns272may be deployed for the n×n array.

In one example, each row of 2×2 memristor arrays290A-290D may share an input276, an inverter278, and an inverted input277. In one example, each column of the memory cells275A-275D may generate an output273and274that may be fed into the noise inducing crossbar array280. As discussed in detail above, the noise inducing crossbar array280can be configured to add noise in parallel to the outputs273and274generated by the memristor crossbar array270. Additionally, the resulting output from the memristor crossbar array280, output283and284, serves as input into comparator285A. For example, a column of memory cells275A and275C of the memristor crossbar array270, including memory cell281A of the noise inducing memristor crossbar array280(vertically aligned forming a congruent column) may generate the outputs283and284that are fed to the comparator285A. The comparator285A may then generate an output286. Similarly, each column of memory cells may generate outputs that are fed to a comparator285n. The comparator285nmay then generate a respective output, and so forth.

Again, referring toFIG.2A, the memristor crossbar array210functions to introduce noise into the output of the circuitry (shown inFIG.1), thereby implementing the local minima avoidance techniques alluded to above. According to the embodiments, a conductance (GN) of the memory cells211A-211C of memristor crossbar array210can be set, or otherwise adjusted, in order to produce a noise current (IN)216. The noise current216may fluctuate (e.g., current value changing asynchronously) to replicate the bursty and random characteristics of noise. In some instances, the noise current216is a bipolar (+/−) signal. According to the embodiments, the noise current216is generated as RTN. RTN is a type of electronic noise (e.g., burst noise) that can occur in semiconductors and ultra-thin gate oxide films. It should be understood that RTN is an example of noise for purposes of discussion Thus, the disclosed memristor crossbar array210is not limited to generating RTN, being capable of producing other forms of noise as deemed appropriate. In some cases, each of the memory cells211A-211C can be configured to have a fluctuating conductance, which in turn yields fluctuating currents as an output. A voltage (VN)217can be applied to the row line204as input into memristor crossbar array210. The value for the voltage217can be selected as deemed appropriate, driving a voltage across the memory cells211A-211C that produces the noise current. According to the embodiments, the conductance for each of the memory cells211A-211C is tuned to match a desired energy function of the emulated HNN. Therefore, the noise current output from memristor crossbar array210can be summed with the output of the weight matrix memristor crossbar array205, replicating the excitation of states in the energy graph of an HNN described in reference toFIG.3B. Referring once again toFIG.3B, an example of a noise signal320that may be generated by the noise inducing crossbar array210is shown. The noise signal320can fluctuate the local energy, giving the system the necessary impulse to “hill-climb” over an energy barrier. Noise signal320can excite the system such that ball351escapes from local minima to reach a more favorable state in terms of global energy. This is implemented by the hardware, as the noise current output from the crossbar array210modifies the output current values resulting from the computations of memristor crossbar array205. In this manner, the noise current can cause the generation of new node values for the emulated HNN which may be more optimal because of the local minima tolerant aspects of noise.FIG.3Adepicts a graphical300of a RTN signal that may by generated by the memristor crossbar array210. The graph300illustrates noise as a function of current (Amperes) versus time (seconds). As seen inFIG.3A, the RTN signal can be qualitatively described as randomly fluctuating bursts.

As previously described, the column lines207of memristor crossbar array210inFIG.2Ais aligned with the column lines207of the memristor crossbar array207. The noise current generated by memristor crossbar array210is similarly output across each of the column lines207, which injects noise at every line of output from the memristor crossbar array205. As such, the configuration ofFIG.2Badds noise in parallel to the weight matrix computations of the memristor crossbar array205. Due to these parallel capabilities, the noise inducing hardware is highly compatibility with the HNN emulating hardware, maintaining advantages of the crossbar implementation (e.g., parallel computing). As an example of parallel operation, output from both the weight matrix memristor crossbar array205and the noise inducing memristor crossbar array210can be calculated in a single clock cycle. In one implementation, the memristors, of memristor crossbar array260for example, can be biased using a voltage source connected either (a) directly across the memristor or (b) connected across a series combination of a (linear or nonlinear) resistor and a plurality of memristors, such that the resistance level of the memristors is close to quantum conductance. In this implementation, the current through the memristor(s) may exhibit fluctuations, which in one example may embody random telegraph noise. The voltage across the memristor(s) may be tuned to tune the magnitude of noise and to change the behavior of the current between noisy and non-noisy. Alternately, in an additional implementation, the current through the memristor(s) can be tuned to alter the conductance of the memristor(s) in order to control and generate noise functions.

As previously discussed, adding the noise current216generated by memristor crossbar array210can produce an output-modification current for each column line207. The output-modification current can be described as the output current of the N×M portion of the memristor crossbar array205, which is a dot-product current of the input vector and the weight matrix, with nose injected therein. The output-modification current may then be used to calculate new node values. In some examples, a comparator (shown inFIG.1) may be coupled to each column lines207. Because the memristor crossbar array210has uniquely modified the output current of each column line207single comparator may be used at each column line207and a same current for comparison may be delivered to each current comparator. Based on the update rule, the hardware accelerator of the embodiments may generate a new input vector of new node values.

Now, referring back toFIG.1, the comparator210may determine a new node value for the emulated HNN. The new node values may be aggregated to generate a new input vector. For example, each output current may be compared by an update rule. A new node value corresponding to a particular output current can be set to a first value if the particular output current is greater than or equal to the threshold current, θi. The new node value can be set to a second value if the particular output current is less than the threshold current. Each output current may be represented as the sum of the products of an input vector with the weight matrix. For example, the update rule may be represented as the equation that follows:
+1 if Σjwijsj≥θi(4)si=−1 otherwise

The node values may also be programmed to attain values of +1 or 0, rather than +1 and −1 in the above equation. Any other pair of values may also be used. In some examples, the threshold currents may be delivered to the current comparators116via circuitry independent from crossbar array102. Furthermore, in some examples, column lines107may have different threshold currents associated with it. This is further described below. Alternatively, each column line106may be associated with a same threshold current.

Upon delivery of the new input vector of new node values, a controller may determine whether the new node values are final node values of the HNN. A neural network, for example, may be modeled to determine a minimum energy of a system. In such an example, a controller can determine whether the new node values, which here represents an energy of the system, are a local minimum of the system. In response to a controller determining that the new node values are not final node values, the new input vector can be converted to input voltages to be delivered to the plurality of row lines of the crossbar array105. In such a manner, the hardware accelerator100can be recurrent to calculate an iterative problem, such as determining a minimum energy of a system, implementing an HNN as hardware.

In some cases, the hardware accelerator100can be implemented as an engine in a computing device. Example computing devices that include an example accelerator may be, for example, a personal computer, a cloud server, a local area network server, a web server, a mainframe, a mobile computing device, a notebook or desktop computer, a smart TV, a point-of-sale device, a wearable device, any other suitable electronic device, or a combination of devices, such as ones connected by a cloud or internet network, that perform the functions described herein.

FIG.5depicts a block diagram of an example computer system500in which the hardware accelerator100of the embodiments described herein may be implemented. The computer system500includes a bus502or other communication mechanism for communicating information, one or more hardware processors504coupled with bus502for processing information. Hardware processor(s)504may be, for example, one or more general purpose microprocessors.

The computer system500further includes storage devices510such as a read only memory (ROM) or other static storage device coupled to bus502for storing static information and instructions for processor504. A storage device510, such as a magnetic disk, optical disk, or USB thumb drive (Flash drive), etc., is provided and coupled to bus502for storing information and instructions.

The computer system500may be coupled via bus502to a display512, such as a liquid crystal display (LCD) (or touch screen), for displaying information to a computer user. An input device514, including alphanumeric and other keys, is coupled to bus502for communicating information and command selections to processor504. Another type of user input device is cursor control516, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor504and for controlling cursor movement on display512. In some embodiments, the same direction information and command selections as cursor control may be implemented via receiving touches on a touch screen without a cursor.

FIG.6is an operational flow diagram illustrating an example of a process600for executing techniques to emulate a neural network using hardware. Additionally, process600includes introducing RTN into the emulated neural network, in accordance with the hardware and circuit configurations previously discussed herein. In some cases, process600is performed in connection with the use of the hardware accelerator. Furthermore,FIG.6shows process600as a series of executable operations performed by processor601, which can be the main processor of a computer device including the disclosed hardware accelerator (including the circuitry therein). Processor601executes the operations of process400, thereby implementing the disclosed techniques.

At operation605, a weight matrix may be converted to conductance values of a first memristor crossbar array. The first memristor crossbar array may be implemented using any of the circuit configurations described in detail in reference toFIG.2A-2C, for example. The weight matrix may represent a neural network, such as an HNN. Furthermore, the weight matrix may be used for calculating values of nodes of the neural network. As described above, values stored in memory cells of the first memristor crossbar array can represent the values of the weight matrix. In implementations or resistive memory, the resistance levels of each memory cell may represent the values of the weight matrix. In such a manner, the weight matrix may be mapped onto the memristor crossbar array.

Then, at operation610, noise values may be converted to conductance values of a second memristor crossbar array. In some cases, noise values are generated in accordance with a noise function and an energy function of a neural network (described in detail in reference toFIG.3B). That is, the noise values may represent noise, such as RTN, in manner that is introduced into the emulated neural network to avoid local minima of its a given energy function. The conversion can represent the noise values as corresponding conductance values for memristors of the second memristor crossbar array. These conductance values for memristors of the second memristor crossbar array are determined such that a desired magnitude of current is output into the circuitry to act as noise (or a noise level). In such a manner, RTN may be mapped onto the second memristor crossbar array.

In an operation615, memory cells of the first memristor crossbar array, including memristors, may be programmed according to the conductance values converted in operation610. As described previously, memory cells may be programmed, for example, by having programming signals driven through them, which drives a change in the state of the memory cells. Thus, the first crossbar array is configured to calculate nodes values in accordance with calculating a dot-product of an input vector of the neural network with the weight matrix.

Next, at operation620, the second memristor crossbar can be programmed according to the conductance values for RTN. In some cases, programming can involve biasing the memristors of the second memristor crossbar array using a voltage source, such that the resistance level of the memristors is close to conductance converted in operation610. Accordingly, the current through the memristor(s) may exhibit fluctuations, which in one example may embody random telegraph noise. Furthermore, programming can involve tuning the voltage across the memristor(s), in order to tune the magnitude of noise and to change the behavior of the current (e.g., noisy and non-noisy). In some cases, programming involves tuning the current through the memristors to alter the conductance of the memristors, thereby generating the noise functions. As a result, the second crossbar array is hardware that functions to simulate introduction of noise signals into the neural network and modifies the calculated node values of the emulated neural network generated by the first memristor crossbar array. According to the embodiments, noise inducing memristor crossbar array can simulate annealing for HNNs (e.g., escaping local minima), and thereby improving accuracy of the emulated neural network.