Arithmetic device

According to one embodiment, an arithmetic device includes one or a plurality of arithmetic units. One of the one or the arithmetic units includes a memory part including a plurality of memory regions, and an arithmetic part. At least one of the memory regions includes a line-shaped magnetic part.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2018-044541, filed on Mar. 12, 2018; the entire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an arithmetic device.

BACKGROUND

A large-scale arithmetic device is desirable.

DETAILED DESCRIPTION

According to one embodiment, an arithmetic device includes one or a plurality of arithmetic units. One of the one or the arithmetic units includes a memory part including a plurality of memory regions, and an arithmetic part. At least one of the memory regions includes a line-shaped magnetic part.

The drawings are schematic and conceptual; and the relationships between the thickness and width of portions, the proportions of sizes among portions, etc., are not necessarily the same as the actual values thereof. Further, the dimensions and proportions may be illustrated differently among drawings, even for identical portions.

In the specification and drawings, components similar to those described or illustrated in a drawing thereinabove are marked with like reference numerals, and a detailed description is omitted as appropriate.

First Embodiment

FIG. 1AandFIG. 1Bare schematic views illustrating an arithmetic device according to a first embodiment. As shown inFIG. 1A, the arithmetic device110according to the embodiment includes a memory unit10U and an arithmetic unit20U. The memory unit10U includes, for example, magnetic tracks. For example, the arithmetic unit20U performs a product-sum operation. The arithmetic unit20U may perform a determination. The arithmetic unit20U is, for example, a FPGA.

A controller51is provided in the example. The controller51controls the operations of the memory unit10U and the arithmetic unit20U.

A semiconductor memory unit18U may be further provided as in the example shown inFIG. 1A. For example, the semiconductor memory unit18U may store information relating to a part of the operations performed by the arithmetic device110. By using the semiconductor memory unit18U and the magnetic tracks included in the memory unit10U, storing that is suited to the two can be performed; and, for example, high-speed arithmetic is obtained.

The arithmetic device110includes one or multiple arithmetic units PRU. One arithmetic unit PRU includes, for example, a part of the memory unit10U and a part of the arithmetic unit20U. The memory mechanism that is included in one arithmetic unit PRU corresponds to a part of the memory unit10U. The arithmetic mechanism that is included in one arithmetic unit PRU corresponds to a part of the arithmetic unit20U.

One of the multiple arithmetic units PRU will now be described.

As shown inFIG. 1B, one of the multiple arithmetic units PRU includes a memory part10and an arithmetic part20. The memory part10includes multiple memory regions10R. At least one of the multiple memory regions10R includes a line-shaped magnetic part10L. The line-shaped magnetic part10L is, for example, a magnetic track10T. The length of the line-shaped magnetic part10L is greater than the width and the height of the line-shaped magnetic part10L. The orientations of the magnetizations in the line-shaped magnetic part10L correspond to stored information.

For example, a write operation WO and a read operation RO are performed for one memory region10R (line-shaped magnetic part10L). The write operation WO is performed by a first element described below. For example, the write operation WO is performed based on spin injection. The read operation RO is performed by a second element described below. For example, the read operation RO is performed based on a magnetoresistance effect (e.g., a tunneling magnetoresistance effect).

In the example, a magnetic noise generation operation NO is performed for the one memory region10R (the line-shaped magnetic part10L). The magnetic noise generation operation NO is performed by a third element described below. For example, the magnetic noise generation operation NO is performed based on a high frequency magnetic field generated from a spin torque oscillator.

In the arithmetic device110as shown inFIG. 1B, product-sum operations are performed in the arithmetic part20by using the multiple memory regions10R (the line-shaped magnetic parts10L). As described below, for example, it is easy to increase the scale; and higher energy conservation is easy.

An example of an overview of the configuration of the arithmetic device according to the embodiment will now be described.

FIG. 2andFIG. 3are schematic views illustrating the arithmetic device according to the first embodiment.

As shown inFIG. 2, the memory part10(the arithmetic unit PRU) shown inFIG. 1Bis multiply provided. For example, the arithmetic unit PRU corresponds to a sub-core. The arithmetic part20performs arithmetic (e.g., a product-sum operation) by using the data stored in the multiple memory parts10. The multiple memory parts10and the arithmetic part20are included in one arithmetic unit PRU. The multiple arithmetic units PRU are provided as shown inFIG. 3. Large-scale arithmetic is possible.

The arithmetic device110according to the embodiment is trainable.

High performance and high functionality are desirable for computers and electronic devices. It is desirable for the arithmetic device to be able to accommodate an enormous amount of information processing. By increasing the scale of the information processing, for example, the enormous amount of information processing of the IoT (Internet of Things), AI (Artificial Intelligence), deep learning, etc., can be accommodated.

On the other hand, the development of energy-conserving electronics also is desirable. By higher energy conservation, for example, CO2reduction which is discussed on a global scale can be accommodated. By higher energy conservation, for example, the electrical power circumstances after a large-scale disaster can be relaxed.

For such circumstances, neural networks are drawing attention as energy-conserving electronics that learn from living bodies. The relationship between neural networks and electronics has an old history. For example, the neuron model of McCulloch and Pitts presented in 1943 is known (W. S. McCulloch and W. Pitts: Bull. Math. Biophys. 5, 115 (1943)).

Subsequently, Hopfield had a major breakthrough in the field of neural networks in 1982 (J. J. Hopfield: Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982)). He showed that an interconnected network can be represented by the Hamiltonian of an Ising spin model. Thereby, it is possible to examine information processing in a neural network by using the statistical mechanics of a spin system. Further, it became possible to associate Ising spins, which can have the binary states of up or down spins, with the activity of a neuron or an information bit.

As a formal neural network, a device called the True North chip was developed jointly by IBM and Cornell University in 2014 (P. A. Merolla et al., Science 345, 668 (2014)). In this example, the device was constructed using 28-nm rule CMOS technology. As an entirety, the device operated as one million neurons. Compared to the brain of a human which is configured from 14 billion neurons, the scale of the device was small; and the power consumption was large.

A disadvantage of the concept of Hopfield's Ising machine and its hardware realized by the True North chip was that a learning function was not included. Neural network devices have been proposed to compensate for this disadvantage. For example, a restricted Boltzmann machine, a deep Boltzmann machine which is a more evolved restricted Boltzmann machine, etc., have been proposed. Further, the hardware of a neural network device generally called deep learning has been developed in recent years.

As described above, there are expectations for neural network devices having learning functions to be used as large-scale energy-conserving information processors comparable to the brain of a human. However, currently, larger scales and higher energy conservation both are exceedingly insufficient.

For example, the embodiment is applicable to a neural network device. For example, the embodiment can provide a larger scale and higher energy conservation in a neural network device. In the embodiment, for example, magnetic tracks are used in the memory part of a neural network device that has a learning function. Thereby, the neural network device (the arithmetic device) can have a larger scale. Energy conservation is possible.

One embodiment will now be described using a restricted Boltzmann machine as an example.

FIG. 4is a schematic view illustrating a model applied to an arithmetic device.

FIG. 4illustrates the configuration of a restricted Boltzmann machine. The restricted Boltzmann machine includes a visible layer including m nodes vj(j=1, 2, . . . , m), a hidden layer including n nodes hi(i=1, 2, . . . , n), and links wijconnecting the visible layer and the hidden layer. The nodes and the links correspond respectively to neurons and synapses. External data is read into the neurons vjof the visible layer. Training of the synapses wijis performed based on the external data read into the neurons vj.

The training of the restricted Boltzmann machine proceeds by repeating the update of the synapses wijby using the method shown by the arrows of Formula (1).

In Formula (1), “v(0)j” is the external data initially input to the visible layer of the machine. “v(k)j” is the data input to the visible layer after the arithmetic is repeated k times (k>=1).

The following Formula (2) is set.

In Formula (2), “p(hi=1|v(0))” is the probability of “hi=1” when “vj=v(0)j.” “p(hi=1|v(k)) is the probability of “hi=1” when “vj=v(k)j.” In Formula (2), “( )” is a sigmoid function. “ci” is the bias magnetic field of the hidden layer.

The training ends after the update is repeated for all of the external data.

In a conventionally-known restricted Boltzmann machine (a reference example), for example, an SRAM element is used as one neuron vjor hi, one synapse wij, etc. Six transistors are included in each SRAM element.

Conversely, as described above, the line-shaped magnetic part10L is used in the arithmetic device110(the restricted Boltzmann machine device) according to the embodiment. An example of the line-shaped magnetic part10L will now be described.

FIG. 5is a schematic view illustrating a part of the arithmetic device according to the first embodiment.

FIG. 5illustrates the memory region10R (the line-shaped magnetic part10L) provided in the memory part10. The line-shaped magnetic part10L includes multiple domain walls10W. For example, the domain walls10W move through the line-shaped magnetic part10L according to the current flowing through the line-shaped magnetic part10L. The region (the magnetic domain) between two domain walls10W corresponds to a region that stores memory. The orientations of the magnetizations in the line-shaped magnetic part10L correspond to the stored information.

In the embodiment, one magnetic domain (1 bit) corresponds to a neuron. One magnetic domain (1 bit) corresponds to a synapse. A size Lbit (the length or the width) of one magnetic domain (1 bit) is 50 nm or less and is small. Thereby, compared to the reference example recited above (the structure including six transistors), the chip surface area can be reduced drastically.

For example, the line-shaped magnetic part10L (e.g., the domain wall movement-type magnetic recording element) recited above is used as the mxn synapses which are plentiful compared to the (n+m) neurons. Thereby, the chip surface area can be reduced effectively.

In the embodiment, the read operation or the write operation is simple compared to the reference example recited above. Many elements (select transistors, address memory, controllers), etc., are necessary for the read operation and the write operation of the reference example recited above. In the embodiment, these circuits can be omitted. For example, it is particularly effective for the circuits to be omissible for small-scale memory of several 1000 bits. For example, small-scale memory is multiply used in an energy-conserving high-speed arithmetic device including many small-scale arithmetic units. It is particularly advantageous when the circuits can be omitted in a high-speed arithmetic device.

Generally, the update of the synapse coefficient is performed using the sigmoid function of Formula (2). In this method, for example, the probability calculation of the heat fluctuation at a temperature T is performed. Conversely, in the embodiment, the magnetic noise generation operation NO recited above can be performed. In the embodiment, for example, it is possible to use simulated annealing by using microwave irradiation from a spin torque oscillator.

Thereby, in the embodiment, it is possible to utilize a stochastic magnetization reversal phenomenon due to the application of a microwave magnetic field (magnetic field fluctuation). Thereby, the sigmoid function a of Formula (2) can be replaced with a step function σ0. Further, “p(hi|v(0))” and “p(hi|v(k))” can be replaced respectively with “h(0)i” and “h(k)i.”

Accordingly, in the embodiment, the following Formula (3) can be used instead of Formula (2).

Accordingly, Formula (1) is replaced with Formula (4).

In the hardware of one example, the arithmetic proceeds using multiple arithmetic units PRU arranged in parallel from the perspective of the energy conservation and the higher speeds described above.

One example of the multiple arithmetic units PRU is shown inFIG. 1B.

InFIG. 16, one arithmetic unit PRU corresponds to a sub-core. For example, the sub-core corresponds to the minimum unit of the arithmetic.

In the example, the sub-core SC(1, 1) includes three types of magnetic tracks10T. The first magnetic track10T corresponds to a magnetic track storing the states vj(1, −1) of the neurons of the visible layer. The second magnetic track10T corresponds to a magnetic track10T storing the states hj(1, −1) of the neurons of the hidden layer. The third magnetic track10T corresponds to magnetic tracks storing the content of the synapses wijto be trained. Four bits are allotted to the synapse wij.

An example of the operations performed by the embodiment will now be described. Here, n=m=1000 for simplification.

For example, first, the initial data v(0)j(j=1, 2, . . . , n) is written to the magnetic track10T of vjby the first element (a spin injection element) described below. h(0)1is calculated according to Formula (3); and the h1bit is updated. At this time, the reading of v(0)jis performed by the second element (e.g., a tunneling magnetoresistance element) described below. The calculation of h(0)1is performed by a product-sum operation device (the arithmetic part20). The update of the h1bit is performed by spin injection programming.

Here, w2j, . . . , w1000jtracks U=1, 2, . . . , n) are included in each of the sub-cores SC(2, 1), . . . , SC(1000, 1). The arithmetic is omissible for the magnetic tracks10T corresponding to “vj” and “hj” that are common to the SC(1, 1). By using the obtained h(0)j, v(1)jis calculated using Formula (5). Here, “bi” is the bias magnetic field of the visible layer.

v(k)jis obtained by repeating a similar calculation k times (including one time).

Here, “σ0” is a step function that is discontinuous at the origin. wijis updated according to Formula (4) by using)“h(0)i” and “h(k)i” of Formula (3).

This operation is repeated for all of the data. Thereby, the training of the synapse wij, “bj,” and “ci” ends. For example, the arithmetic unit (the arithmetic unit PRU) that includes 1000 sub-cores corresponds to the “core.” The “core” includes 106synapses. By using multiple “cores,” an even larger arithmetic device is obtained.

FIG. 6is a flowchart illustrating the operations of the arithmetic device according to the first embodiment.

FIG. 6shows an example of the arithmetic recited above. As shown inFIG. 6, the initial parameters are written (step S110). For example, the initial value of the parameter “wij” of the synapses, the initial value of the parameter “bj” (the bias magnetic field) of the visible layer, and the initial value of the parameter “ci” (the bias magnetic field) of the hidden layer are written to the line-shaped magnetic parts10L (the magnetic tracks10T).

“v(0)j” is written (step S120). First teaching data is written to the visible layer as v(0)j(j=1, 2, . . . , m).

The calculation and the writing of “h(0)i” are performed (step S130). For example, h(0)i(i=1, 2, . . . , n) is calculated from v(0)jin a state of microwave (high frequency wave) irradiation and is written to the hidden layer.

The calculation and the writing of “v(1)j” are performed (step S140). For example, v(1)jis calculated from h(0)jin the state of microwave irradiation and is written to the hidden layer.

The calculation and the writing of “h(1)i” are performed (step S150). For example, h(1)i(i=1, 2, . . . , n) is calculated from v(1)jin the state of microwave irradiation and is written to the hidden layer.

The parameters are updated (step S160). For example, the values of “wij,” “bj,” and “ci” are updated using “v(0)j,” “v(1)j,” “h(0)i,” and “h(1)i.”

“v(0)j” is written (step S170). For example, second teaching data is written to the visible layer as v(0)j(j=1, 2, . . . , m).

Then, the update of the parameters recited above (steps S120to S170) is repeated using all of the teaching data.

Second Embodiment

FIG. 7AtoFIG. 7Care schematic views illustrating a model of an arithmetic device according to a second embodiment.

FIG. 8is a flowchart illustrating the operations of the arithmetic device according to the second embodiment.

The model shown inFIG. 7AtoFIG. 7Cis one model of deep learning. In the example, the second embodiment is described using a deep Boltzmann machine (DBM) as an example. Multiple hidden layers are stacked hierarchially in the DBM.

In the training of the DBM, first, the visible layer v and the hidden layer h(1)of the first layer are focused upon; and the hidden layers of the second and higher layers are ignored. Thereby, the visible layer v and the hidden layer h(1)of the first layer can be considered to be a restricted Boltzmann machine (RBM).

As shown inFIG. 8, “w(1)ij” is trained (step S210). For example, the connections “w(1)ij” between “v” and “h(1)” are trained according to the training method of the RBM described above.

Then, “w(2)ij” is trained (step S220). For example, “w(2)ij” is trained by a method similar to a restricted Boltzmann machine by using the hidden layer “h(1)” and the hidden layer “h(2).” The initial value of “h(1)” is obtained utilizing the data input to “v” and the trained “w(1)ij.”

Then, “w(3)ij” is trained (step S230). For example, “w(3)ij” is trained by a method similar to a restricted Boltzmann machine by using the hidden layer “h(2)” and the hidden layer “h(3).” The initial value of “h(2)” is obtained utilizing the data input to “v,” the trained “w(1)ij,” and the trained “w(2)ij.”

Even in the case where the layers increase further, the connections can be trained by repeating a similar training method.

FIG. 9is a schematic view illustrating the arithmetic device according to the second embodiment.

FIG. 9shows an example of the sub-core (the arithmetic unit PRU) of the arithmetic device120according to the embodiment. As described above, the sub-core is the minimum unit of the arithmetic.

Third Embodiment

FIG. 10is a schematic view illustrating a model of an arithmetic device according to a third embodiment.

A model called an ESN (Echo State Network) is employed in the third embodiment as shown inFIG. 10. The arithmetic device according to the third embodiment is, for example, a reservoir computer (RC).

The ESN includes an inputter15a, a reservoir part15b, and an outputter15c. For example, feedforward networks are formed in the inputter15aand the outputter15c. For example, a recurrent network (RNN) is formed in the reservoir part15b. Generally, the training of the connection parameters of an RNN is exceedingly difficult. Therefore, only the connection parameters of the outputter15care trained in the RC. The number of nodes in the reservoir part15bis overwhelmingly large compared to the number of nodes in the inputter15aand in the outputter15c. Continuous variables are used as the variables. Discrete time is used as the time. In the example, the number of nodes is set to 100 for the inputter15aand for the outputter15c; and the number of nodes is set to 1000 for the reservoir part15b. The node variables and the connection parameters each are four bits.

FIG. 11toFIG. 14are schematic views illustrating the arithmetic device according to the third embodiment.

InFIG. 11, a product-sum operation is performed in the arithmetic part20. At this time, the training of “winij” is not performed. As shown inFIG. 11, the arithmetic result of the inputter15ais provided to the reservoir part15b.

InFIG. 12, a product-sum operation is performed in the arithmetic part20. At this time, the training of “wij” is not performed. As shown inFIG. 12, the arithmetic result of the reservoir part15bis provided to the outputter15c.

As shown inFIG. 13, a product-sum operation is performed in the arithmetic part20of the first outputter15d. The arithmetic result of the first outputter15dis provided to the second outputter15e.

As shown inFIG. 14, a product-sum operation is performed in the arithmetic part20of the second outputter15e. The arithmetic result of the second outputter15eis provided to the first outputter15d. For example, the update of “woutij” is performed.

FIG. 15is a flowchart illustrating the operations of the arithmetic device according to the third embodiment.

For example, “uj” is written (step S320). For example, the teaching data of the first time step (n=1) is written to the input layer (the inputter15a) as “uj.”

For example, “xi” is calculated (step S330). For example, “xi” is calculated from “uj;” and the result is supplied to the reservoir part15b. The variable x (1000 nodes) of the reservoir part15bis updated using the initial value of the reservoir part15band the data u (100 nodes) written to the inputter15ain the first time step (n=1) (referring to Formula (6)).

For example, “xj” is updated (step S340). For example, “xj” is updated from the initial value; and the result is supplied to the outputter15c. For example, among the updated values, the states of 100 nodes are output to the outputter15c(referring to Formula (7)). The connection parameters of the inputter15aand the reservoir part15bremain as the initial values without being updated.

For example, “uj” is written (step S360). The teaching data of the second time step (n=2) is written to the input layer (the inputter15a) as “uj.”

The parameter “woutij” woute is determined (step S370). The target value yt is written. For example, “woutij” is determined to minimize the difference between “y(n)” and “yt” (referring toFIG. 14). For example, “woutij” can be determined using the least-squares method, etc., because the outputter15cis a simple feedforward network.

In the example recited above, at least two of the multiple arithmetic units PRU operate in parallel. Thereby, high-speed arithmetic is possible.

In the embodiments, for example, it is favorable for the memory capacity of the memory part10included in one of the multiple arithmetic units PRU to be 104bits or less. By setting the memory capacity of one memory part10to be relatively small and by performing the arithmetic in parallel, large-scale arithmetic (e.g., training) can be performed quickly.

Examples of the first to third elements described above will now be described.

FIG. 16is a schematic cross-sectional view illustrating a part of the arithmetic device according to the embodiment.

As shown inFIG. 16, the memory part10may include a first element17A. The first element17A is, for example, a write element. The first element17A includes a first magnetic layer17aand a first nonmagnetic layer17na. The first nonmagnetic layer17nais provided between the first magnetic layer17aand a first portion p1of the line-shaped magnetic part10L.

As shown inFIG. 16, the memory part10may include a second element17B. The second element17B is, for example, a read element. The second element17B includes a second magnetic layer17band a second nonmagnetic layer17nb. The second nonmagnetic layer17nbis provided between the second magnetic layer17band a second portion p2of the line-shaped magnetic part10L.

As shown inFIG. 16, the memory part10may include a third element17C. The third element17C includes a third magnetic layer17c, a fourth magnetic layer17d, and a third nonmagnetic layer17nc. The third nonmagnetic layer17ncis provided between the third magnetic layer17cand the fourth magnetic layer17d.

For example, the write operation WO is performed by the first element17A based on spin injection. For example, the read operation RO is performed by the second element17B based on the magnetoresistance effect (e.g., the tunneling magnetoresistance effect). The magnetic noise generation operation NO is performed by the third element17C based on a high frequency magnetic field due to spin torque oscillation.

According to the embodiments, an arithmetic device can be provided in which a larger scale is possible.

Hereinabove, exemplary embodiments of the invention are described with reference to specific examples. However, the embodiments of the invention are not limited to these specific examples. For example, one skilled in the art may similarly practice the invention by appropriately selecting specific configurations of components included in arithmetic devices such as arithmetic units, memory parts, memory regions, line-shaped magnetic parts, and elements, etc., from known art. Such practice is included in the scope of the invention to the extent that similar effects thereto are obtained.

Moreover, all arithmetic devices practicable by an appropriate design modification by one skilled in the art based on the arithmetic devices described above as embodiments of the invention also are within the scope of the invention to the extent that the spirit of the invention is included.