MULTIPLICATION AND ACCUMULATION (MAC) OPERATOR AND PROCESSING-IN-MEMORY (PIM) DEVICE INCLUDING THE MAC OPERATOR

A multiplying-and-accumulating (MAC) operator for performing a MAC arithmetic operation of a weight matrix employing “M×N”-number of weight sub-matrixes as elements and a vector matrix employing “N”-number of vector sub-matrixes as elements (where “M” and “N” are natural numbers which are equal to or greater than two). The “M×N”-number of weight sub-matrixes are located at cross points of first to Mth weight matrix group rows and first to Nth weight matrix group columns, respectively. The “N”-number of vector sub-matrixes are located at cross points of first to Nth vector matrix group rows and one vector matrix group column, respectively. The MAC operator is configured to perform the MAC arithmetic operations of a matrix group column unit for the “M”-number of weight sub-matrixes arrayed in each of the first to Nth weight matrix group columns and the vector sub-matrix arrayed in each of the Nth vector matrix group rows.

BACKGROUND

1. Technical Field

Various embodiments of the present disclosure relate to processing-in-memory (PIM) systems and, more particularly, to PIM systems including a PIM device and a controller and methods of operating the PIM systems.

2. Related Art

Recently, interest in artificial intelligence (AI) has been increasing not only in the information technology industry but also in the financial and medical industries. Accordingly, in various fields, artificial intelligence, more precisely, the introduction of deep learning, is considered and prototyped. In general, techniques for effectively learning deep neural networks (DNNs) or deep networks having increased layers as compared with general neural networks to utilize the deep neural networks (DNNs) or the deep networks in pattern recognition or inference are commonly referred to as deep learning.

One cause of this widespread interest may be the improved performance of processors performing arithmetic operations. To improve the performance of artificial intelligence, it may be necessary to increase the number of layers constituting a neural network in the artificial intelligence to educate the artificial intelligence. This trend has continued in recent years, which has led to an exponential increase in the amount of computation required for the hardware that actually does the computation. Moreover, if the artificial intelligence employs a general hardware system including memory and a processor which are separated from each other, the performance of the artificial intelligence may be degraded due to limitation of the amount of data communication between the memory and the processor. In order to solve this problem, a PIM device in which a processor and memory are integrated in one semiconductor chip has been used as a neural network computing device. Because the PIM device directly performs arithmetic operations internally, data processing speed in the neural network may be improved.

SUMMARY

According to an embodiment is a multiplying-and-accumulating (MAC) operator for performing a MAC arithmetic operation of a weight matrix employing “M×N”-number of weight sub-matrixes as elements and a vector matrix employing “N”-number of vector sub-matrixes as elements (where “M” and “N” are natural numbers which are equal to or greater than two). The “M×N”-number of weight sub-matrixes are located at cross points of first to Mthweight matrix group rows and first to Nthweight matrix group columns, respectively. The “N”-number of vector sub-matrixes are located at cross points of first to Nthvector matrix group rows and one vector matrix group column, respectively. The MAC operator is configured to perform the MAC arithmetic operations of a matrix group column unit for the “M”-number of weight sub-matrixes arrayed in each of the first to Nthweight matrix group columns and the vector sub-matrix arrayed in each of the Nthvector matrix group rows. The MAC arithmetic operations of the matrix group column unit are sequentially performed “N” times from the first weight matrix group column to the Nthweight matrix group column.

According to another embodiment is a multiplying-and-accumulating (MAC) operator including a plurality of multipliers, an adder tree, and an accumulator. The plurality of multipliers is configured to perform a multiplying calculation of multiple sets of weight data and multiple sets of vector data. The adder tree includes a plurality of adders arrayed to have a tree structure and is configured to perform an adding calculation of multiple sets of multiplication result data outputted from the plurality of multipliers to generate addition result data. The accumulator is configured to perform an accumulating calculation for the addition result data outputted from the adder tree. The accumulator includes an accumulative adder and a plurality of latch circuits. The accumulative adder is configured to perform an accumulative adding calculation of the addition result data inputted to a first input terminal of the accumulative adder and feedback data inputted to a second input terminal of the accumulative adder. The plurality of latch circuits is configured to output the feedback data which are transmitted to the accumulative adder and is configured to latch accumulated addition data outputted from the accumulative adder.

According to yet another embodiment is a processing-in-memory (PIM) device including a plurality of memory banks for providing weight data, a global buffer for providing vector data, and a plurality of multiplying-and-accumulating (MAC) operators for receiving the weight data and the vector data to perform MAC arithmetic operations. Each of the plurality of MAC operators is configured to perform the MAC arithmetic operation of a weight matrix employing “M×N”-number of weight sub-matrixes as elements and a vector matrix employing “N”-number of vector sub-matrixes as elements (where, “M” and “N” are natural numbers which are equal to or greater than two). The “M×N”-number of weight sub-matrixes are located at cross points of first to Mthweight matrix group rows and first to Nthweight matrix group columns, respectively. The “N”-number of vector sub-matrixes are located at cross points of first to Nthvector matrix group rows and one vector matrix group column, respectively. Each of the “M×N”-number of weight sub-matrixes employs weight data, which are located at respective cross points of a plurality of weight matrix rows and a plurality of weight matrix columns, as elements. Each of the “N”-number of vector sub-matrixes employs vector data, which are located at respective cross points of a plurality of vector matrix rows and one vector matrix column, as elements. The weight data arrayed in the “N”-number of weight sub-matrixes located in each of the first to Mt weight matrix group rows are dispersedly stored in the plurality of memory banks in units of the weight matrix rows.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following description of embodiments, it will be understood that the terms “first” and “second” are intended to identify elements, but not used to define a particular number or sequence of elements. In addition, when an element is referred to as being located “on,” “over,” “above,” “under,” or “beneath” another element, it is intended to mean a relative positional relationship, but not used to limit certain cases in which the element directly contacts the other element, or at least one intervening element is present therebetween. Accordingly, the terms such as “on,” “over,” “above,” “under,” “beneath,” “below,” and the like that are used herein are for the purpose of describing particular embodiments only and are not intended to limit the scope of the present disclosure. Further, when an element is referred to as being “connected” or “coupled” to another element, the element may be electrically or mechanically connected or coupled to the other element directly, or may be electrically or mechanically connected or coupled to the other element indirectly with one or more additional elements therebetween.

Various embodiments are directed to PIM systems and methods of operating the PIM systems.

FIG. 1is a block diagram illustrating a PIM system according to an embodiment of the present disclosure. As illustrated inFIG. 1, the PIM system1may include a PIM device10and a PIM controller20. The PIM device10may include a data storage region11, an arithmetic circuit12, an interface (I/F)13-1, and a data (DQ) input/output (I/O) pad13-2. The data storage region11may include a first storage region and a second storage region. In an embodiment, the first storage region and the second storage region may be a first memory bank and a second memory bank, respectively. In another embodiment, the first data storage region and the second storage region may be a memory bank and buffer memory, respectively. The data storage region11may include a volatile memory element or a non-volatile memory element. For an embodiment, the data storage region11may include both a volatile memory element and a non-volatile memory element.

The arithmetic circuit12may perform an arithmetic operation on the data transferred from the data storage region11. In an embodiment, the arithmetic circuit12may include a multiplying-and-accumulating (MAC) operator. The MAC operator may perform a multiplying calculation on the data transferred from the data storage region11and perform an accumulating calculation on the multiplication result data. After MAC operations, the MAC operator may output MAC result data. The MAC result data may be stored in the data storage region11or output from the PIM device10through the data I/O pad13-2.

The interface13-1of the PIM device10may receive a command CMD and address ADDR from the PIM controller20. The interface13-1may output the command CMD to the data storage region11or the arithmetic circuit12in the PIM device10. The interface13-1may output the address ADDR to the data storage region11in the PIM device10. The data I/O pad13-2of the PIM device10may function as a data communication terminal between a device external to the PIM device10, for example the PIM controller20, and the data storage region11included in the PIM device10. The external device to the PIM device10may correspond to the PIM controller20of the PIM system1or a host located outside the PIM system1. Accordingly, data outputted from the host or the PIM controller20may be inputted into the PIM device10through the data I/O pad13-2.

The PIM controller20may control operations of the PIM device10. In an embodiment, the PIM controller20may control the PIM device10such that the PIM device10operates in a memory mode or an arithmetic mode. In the event that the PIM controller20controls the PIM device10such that the PIM device10operates in the memory mode, the PIM device10may perform a data read operation or a data write operation for the data storage region11. In the event that the PIM controller20controls the PIM device10such that the PIM device10operates in the arithmetic mode, the arithmetic circuit12of the PIM device10may receive first data and second data from the data storage region11to perform an arithmetic operation. In the event that the PIM controller20controls the PIM device10such that the PIM device10operates in the arithmetic mode, the PIM device10may also perform the data read operation and the data write operation for the data storage region11to execute the arithmetic operation. The arithmetic operation may be a deterministic arithmetic operation performed during a predetermined fixed time. The word “predetermined” as used herein with respect to a parameter, such as a predetermined fixed time or time period, means that a value for the parameter is determined prior to the parameter being used in a process or algorithm. For some embodiments, the value for the parameter is determined before the process or algorithm begins. In other embodiments, the value for the parameter is determined during the process or algorithm but before the parameter is used in the process or algorithm.

The PIM controller20may be configured to include command queue logic21, a scheduler22, a command (CMD) generator23, and an address (ADDR) generator25. The command queue logic21may receive a request REQ from an external device (e.g., a host of the PIM system1) and store the command queue corresponding to the request REQ in the command queue logic21. The command queue logic21may transmit information on a storage status of the command queue to the scheduler22whenever the command queue logic21stores the command queue. The command queue stored in the command queue logic21may be transmitted to the command generator23according to a sequence determined by the scheduler22. The command queue logic21, and also the command queue logic210ofFIGS. 2 and 20, may be implemented as hardware, software, or a combination of hardware and software. For example, the command queue logic21and/or210may be a command queue logic circuit operating in accordance with an algorithm and/or a processor executing command queue logic code.

The scheduler22may adjust a sequence of the command queue when the command queue stored in the command queue logic21is outputted from the command queue logic21. In order to adjust the output sequence of the command queue stored in the command queue logic21, the scheduler22may analyze the information on the storage status of the command queue provided by the command queue logic21and may readjust a process sequence of the command queue so that the command queue is processed according to a proper sequence.

The command generator23may receive the command queue related to the memory mode of the PIM device10and the MAC mode of the PIM device10from the command queue logic21. The command generator23may decode the command queue to generate and output the command CMD. The command CMD may include a memory command for the memory mode or an arithmetic command for the arithmetic mode. The command CMD outputted from the command generator23may be transmitted to the PIM device10.

The command generator23may be configured to generate and transmit the memory command to the PIM device10in the memory mode. The command generator23may be configured to generate and transmit a plurality of arithmetic commands to the PIM device10in the arithmetic mode. In one example, the command generator23may be configured to generate and output first to fifth arithmetic commands with predetermined time intervals in the arithmetic mode. The first arithmetic command may be a control signal for reading the first data out of the data storage region11. The second arithmetic command may be a control signal for reading the second data out of the data storage region11. The third arithmetic command may be a control signal for latching the first data in the arithmetic circuit12. The fourth arithmetic command may be a control signal for latching the second data in the arithmetic circuit12. And the fifth MAC command may be a control signal for latching arithmetic result data of the arithmetic circuit12.

The address generator25may receive address information from the command queue logic21and generate the address ADDR for accessing a region in the data storage region11. In an embodiment, the address ADDR may include a bank address, a row address, and a column address. The address ADDR outputted from the address generator25may be inputted to the data storage region11through the interface (I/F)13-1.

FIG. 2is a block diagram illustrating a PIM system1-1according to a first embodiment of the present disclosure. As illustrated inFIG. 2, the PIM system1-1may include a PIM device100and a PIM controller200. The PIM device100may include a first memory bank (BANK0)111, a second memory bank (BANK1)112, a MAC operator120, an interface (I/F)131, and a data input/output (I/O) pad132. For an embodiment, the MAC operator120represents a MAC operator circuit. The first memory bank (BANK0)111, the second memory bank (BANK1)112, and the MAC operator120included in the PIM device100may constitute one MAC unit. In another embodiment, the PIM device100may include a plurality of MAC units. The first memory bank (BANK0)111and the second memory bank (BANK1)112may represent a memory region for storing data, for example, a DRAM device. Each of the first memory bank (BANK0)111and the second memory bank (BANK1)112may be a component unit which is independently activated and may be configured to have the same data bus width as data I/O lines in the PIM device100. In an embodiment, the first and second memory banks111and112may operate through interleaving such that an active operation of the first and second memory banks111and112is performed in parallel while another memory bank is selected. Each of the first and second memory banks111and112may include at least one cell array which includes memory unit cells located at cross points of a plurality of rows and a plurality of columns.

Although not shown in the drawings, a core circuit may be disposed adjacent to the first and second memory banks111and112. The core circuit may include X-decoders XDECs and Y-decoders/IO circuits YDEC/IOs. An X-decoder XDEC may also be referred to as a word line decoder or a row decoder. The X-decoder XDEC may receive a row address ADD_R from the PIM controller200and may decode the row address ADD_R to select and enable one of the rows (i.e., word lines) coupled to the selected memory bank. Each of the Y-decoders/IO circuits YDEC/IOs may include a Y-decoder YDEC and an I/O circuit IO. The Y-decoder YDEC may also be referred to as a bit line decoder or a column decoder. The Y-decoder YDEC may receive a column address ADDR_C from the PIM controller200and may decode the column address ADDR_C to select and enable at least one of the columns (i.e., bit lines) coupled to the selected memory bank. Each of the I/O circuits may include an I/O sense amplifier for sensing and amplifying a level of a read datum outputted from the corresponding memory bank during a read operation for the first and second memory banks111and112. In addition, the I/O circuit may include a write driver for driving a write datum during a write operation for the first and second memory banks111and112.

The interface131of the PIM device100may receive a memory command M_CMD, MAC commands MAC_CMDs, a bank selection signal BS, and the row/column addresses ADDR_R/ADDR_C from the PIM controller200. The interface131may output the memory command M_CMD, together with the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C, to the first memory bank111or the second memory bank112. The interface131may output the MAC commands MAC_CMDs to the first memory bank111, the second memory bank112, and the MAC operator120. In such a case, the interface131may output the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C to both of the first memory bank111and the second memory bank112. The data I/O pad132of the PIM device100may function as a data communication terminal between a device external to the PIM device100and the MAC unit (which includes the first and second memory banks111and112and the MAC operator120) included in the PIM device100. The external device to the PIM device100may correspond to the PIM controller200of the PIM system1-1or a host located outside the PIM system1-1. Accordingly, data outputted from the host or the PIM controller200may be inputted into the PIM device100through the data I/O pad132.

The PIM controller200may control operations of the PIM device100. In an embodiment, the PIM controller200may control the PIM device100such that the PIM device100operates in a memory mode or a MAC mode. In the event that the PIM controller200controls the PIM device100such that the PIM device100operates in the memory mode, the PIM device100may perform a data read operation or a data write operation for the first memory bank111and the second memory bank112. In the event that the PIM controller200controls the PIM device100such that the PIM device100operates in the MAC mode, the PIM device100may perform a MAC arithmetic operation for the MAC operator120. In the event that the PIM controller200controls the PIM device100such that the PIM device100operates in the MAC mode, the PIM device100may also perform the data read operation and the data write operation for the first and second memory banks111and112to execute the MAC arithmetic operation.

The PIM controller200may be configured to include command queue logic210, a scheduler220, a memory command generator230, a MAC command generator240, and an address generator250. The command queue logic210may receive a request REQ from an external device (e.g., a host of the PIM system1-1) and store a command queue corresponding to the request REQ in the command queue logic210. The command queue logic210may transmit information on a storage status of the command queue to the scheduler220whenever the command queue logic210stores the command queue. The command queue stored in the command queue logic210may be transmitted to the memory command generator230or the MAC command generator240according to a sequence determined by the scheduler220. When the command queue outputted from the command queue logic210includes command information requesting an operation in the memory mode of the PIM device100, the command queue logic210may transmit the command queue to the memory command generator230. On the other hand, when the command queue outputted from the command queue logic210is command information requesting an operation in the MAC mode of the PIM device100, the command queue logic210may transmit the command queue to the MAC command generator240. Information on whether the command queue relates to the memory mode or the MAC mode may be provided by the scheduler220.

The scheduler220may adjust a timing of the command queue when the command queue stored in the command queue logic210is outputted from the command queue logic210. In order to adjust the output timing of the command queue stored in the command queue logic210, the scheduler220may analyze the information on the storage status of the command queue provided by the command queue logic210and may readjust a process sequence of the command queue such that the command queue is processed according to a proper sequence. The scheduler220may output and transmit to the command queue logic210information on whether the command queue outputted from the command queue logic210relates to the memory mode of the PIM device100or relates to the MAC mode of the PIM device100. In order to obtain the information on whether the command queue outputted from the command queue logic210relates to the memory mode or the MAC mode, the scheduler220may include a mode selector221. The mode selector221may generate a mode selection signal including information on whether the command queue stored in the command queue logic210relates to the memory mode or the MAC mode, and the scheduler220may transmit the mode selection signal to the command queue logic210.

The memory command generator230may receive the command queue related to the memory mode of the PIM device100from the command queue logic210. The memory command generator230may decode the command queue to generate and output the memory command M_CMD. The memory command M_CMD outputted from the memory command generator230may be transmitted to the PIM device100. In an embodiment, the memory command M_CMD may include a memory read command and a memory write command. When the memory read command is outputted from the memory command generator230, the PIM device100may perform the data read operation for the first memory bank111or the second memory bank112. Data which are read out of the PIM device100may be transmitted to an external device through the data I/O pad132. The read data outputted from the PIM device100may be transmitted to a host through the PIM controller200. When the memory write command is outputted from the memory command generator230, the PIM device100may perform the data write operation for the first memory bank111or the second memory bank112. In such a case, data to be written into the PIM device100may be transmitted from the host to the PIM device100through the PIM controller200. The write data inputted to the PIM device100may be transmitted to the first memory bank111or the second memory bank112through the data I/O pad132.

The MAC command generator240may receive the command queue related to the MAC mode of the PIM device100from the command queue logic210. The MAC command generator240may decode the command queue to generate and output the MAC commands MAC_CMDs. The MAC commands MAC_CMDs outputted from the MAC command generator240may be transmitted to the PIM device100. The data read operation for the first memory bank111and the second memory bank112of the PIM device100may be performed by the MAC commands MAC_CMDs outputted from the MAC command generator240, and the MAC arithmetic operation of the MAC operator120may also be performed by the MAC commands MAC_CMDs outputted from the MAC command generator240. The MAC commands MAC_CMDs and the MAC arithmetic operation of the PIM device100according to the MAC commands MAC_CMDs will be described in detail with reference toFIG. 3.

The address generator250may receive address information from the command queue logic210. The address generator250may generate the bank selection signal BS for selecting one of the first and second memory banks111and112and may transmit the bank selection signal BS to the PIM device100. In addition, the address generator250may generate the row address ADDR_R and the column address ADDR_C for accessing a region (e.g., memory cells) in the first or second memory bank111or112and may transmit the row address ADDR_R and the column address ADDR_C to the PIM device100.

FIG. 3illustrates the MAC commands MAC_CMDs outputted from the MAC command generator240included in the PIM system1-1according to the first embodiment of the present disclosure. As illustrated inFIG. 3, the MAC commands MAC_CMDs may include first to sixth MAC command signals. In an embodiment, the first MAC command signal may be a first MAC read signal MAC_RD_BK0, the second MAC command signal may be a second MAC read signal MAC_RD_BK1, the third MAC command signal may be a first MAC input latch signal MAC_L1, the fourth MAC command signal may be a second MAC input latch signal MAC_L2, the fifth MAC command signal may be a MAC output latch signal MAC_L3, and the sixth MAC command signal may be a MAC latch reset signal MAC_L_RST.

The first MAC read signal MAC_RD_BK0may control an operation for reading first data (e.g., weight data) out of the first memory bank111to transmit the first data to the MAC operator120. The second MAC read signal MAC_RD_BK1may control an operation for reading second data (e.g., vector data) out of the second memory bank112to transmit the second data to the MAC operator120. The first MAC input latch signal MAC_L1may control an input latch operation of the weight data transmitted from the first memory bank111to the MAC operator120. The second MAC input latch signal MAC_L2may control an input latch operation of the vector data transmitted from the second memory bank112to the MAC operator120. If the input latch operations of the weight data and the vector data are performed, the MAC operator120may perform the MAC arithmetic operation to generate MAC result data corresponding to the result of the MAC arithmetic operation. The MAC output latch signal MAC_L3may control an output latch operation of the MAC result data generated by the MAC operator120. And, the MAC latch reset signal MAC_L_RST may control an output operation of the MAC result data generated by the MAC operator120and a reset operation of an output latch included in the MAC operator120.

The PIM system1-1according to the present embodiment may be configured to perform a deterministic MAC arithmetic operation. The term “deterministic MAC arithmetic operation” used in the present disclosure may be defined as the MAC arithmetic operation performed in the PIM system1-1during a predetermined fixed time. Thus, the MAC commands MAC_CMDs transmitted from the PIM controller200to the PIM device100may be sequentially generated with fixed time intervals. Accordingly, the PIM controller200does not require any extra end signals of various operations executed for the MAC arithmetic operation to generate the MAC commands MAC_CMDs for controlling the MAC arithmetic operation. In an embodiment, latencies of the various operations executed by MAC commands MAC_CMDs for controlling the MAC arithmetic operation may be set to have fixed values in order to perform the deterministic MAC arithmetic operation. In such a case, the MAC commands MAC_CMDs may be sequentially outputted from the PIM controller200with fixed time intervals corresponding to the fixed latencies.

For example, the MAC command generator240is configured to output the first MAC command at a first point in time. The MAC command generator240is configured to output the second MAC command at a second point in time when a first latency elapses from the first point in time. The first latency is set as the time it takes to read the first data out of the first storage region based on the first MAC command and to output the first data to the MAC operator. The MAC command generator240is configured to output the third MAC command at a third point in time when a second latency elapses from the second point in time. The second latency is set as the time it takes to read the second data out of the second storage region based on the second MAC command and to output the second data to the MAC operator. The MAC command generator240is configured to output the fourth MAC command at a fourth point in time when a third latency elapses from the third point in time. The third latency is set as the time it takes to latch the first data in the MAC operator based on the third MAC command. The MAC command generator240is configured to output the fifth MAC command at a fifth point in time when a fourth latency elapses from the fourth point in time. The fourth latency is set as the time it takes to latch the second data in the MAC operator based on the fourth MAC command and to perform the MAC arithmetic operation of the first and second data which are latched in the MAC operator. The MAC command generator240is configured to output the sixth MAC command at a sixth point in time when a fifth latency elapses from the fifth point in time. The fifth latency is set as the time it takes to perform an output latch operation of MAC result data generated by the MAC arithmetic operation.

FIG. 4illustrates an example of the MAC operator120of the PIM device100included in the PIM system1-1according to the first embodiment of the present disclosure. Referring toFIG. 4, MAC operator120may be configured to include a data input circuit121, a MAC circuit122, and a data output circuit123. The data input circuit121may include a first input latch121-1and a second input latch121-2. The MAC circuit122may include a multiplication logic circuit122-1and an addition logic circuit122-2. The data output circuit123may include an output latch123-1, a transfer gate123-2, a delay circuit123-3, and an inverter123-4. In an embodiment, the first input latch121-1, the second input latch121-2, and the output latch123-1may be realized using flip-flops.

The data input circuit121of the MAC operator120may be synchronized with the first MAC input latch signal MAC_L1to latch first data DA1transferred from the first memory bank111to the MAC circuit122through an internal data transmission line. In addition, the data input circuit121of the MAC operator120may be synchronized with the second MAC input latch signal MAC_L2to latch second data DA2transferred from the second memory bank112to the MAC circuit122through another internal data transmission line. Because the first MAC input latch signal MAC_L1and the second MAC input latch signal MAC_L2are sequentially transmitted from the MAC command generator240of the PIM controller200to the MAC operator120of the PIM device100with a predetermined time interval, the second data DA2may be inputted to the MAC circuit122of the MAC operator120after the first data DA1is inputted to the MAC circuit122of the MAC operator120.

The MAC circuit122may perform the MAC arithmetic operation of the first data DA1and the second data DA2inputted through the data input circuit121. The multiplication logic circuit122-1of the MAC circuit122may include a plurality of multipliers122-11. Each of the multipliers122-11may perform a multiplying calculation of the first data DA1outputted from the first input latch121-1and the second data DA2outputted from the second input latch121-2and may output the result of the multiplying calculation. Bit values constituting the first data DA1may be separately inputted to the multipliers122-11. Similarly, bit values constituting the second data DA2may also be separately inputted to the multipliers122-11. For example, if the first data DA1is represented by an ‘N’-bit binary stream, the second data DA2is represented by an ‘N’-bit binary stream, and the number of the multipliers122-11is ‘M’, then ‘N/M’-bit portions of the first data DA1and ‘N/M’-bit portions of the second data DA2may be inputted to each of the multipliers122-11.

The addition logic circuit122-2of the MAC circuit122may include a plurality of adders122-21. Although not shown in the drawings, the plurality of adders122-21may be disposed to provide a tree structure including a plurality of stages. Each of the adders122-21disposed at a first stage may receive two sets of multiplication result data from two of the multipliers122-11included in the multiplication logic circuit122-1and may perform an adding calculation of the two sets of multiplication result data to output the addition result data. Each of the adders122-21disposed at a second stage may receive two sets of addition result data from two of the adders122-21disposed at the first stage and may perform an adding calculation of the two sets of addition result data to output the addition result data. The adder122-21disposed at a last stage may receive two sets of addition result data from two adders122-21disposed at the previous stage and may perform an adding calculation of the two sets of addition result data to output the addition result data. Although not shown in the drawings, the addition logic circuit122-2may further include an additional adder for performing an accumulative adding calculation of MAC result data DA_MAC outputted from the adder122-21disposed at the last stage and previous MAC result data DA_MAC stored in the output latch123-1of the data output circuit123.

The data output circuit123may output the MAC result data DA_MAC outputted from the MAC circuit122to a data transmission line. Specifically, the output latch123-1of the data output circuit123may be synchronized with the MAC output latch signal MAC_L3to latch the MAC result data DA_MAC outputted from the MAC circuit122and to output the latched data of the MAC result data DA_MAC. The MAC result data DA_MAC outputted from the output latch123-1may be fed back to the MAC circuit122for the accumulative adding calculation. In addition, the MAC result data DA_MAC may be inputted to the transfer gate123-2. The output latch123-1may be initialized if a latch reset signal LATCH_RST is inputted to the output latch123-1. In such a case, all of data latched by the output latch123-1may be removed. In an embodiment, the latch reset signal LATCH_RST may be activated by generation of the MAC latch reset signal MAC_L_RST and may be inputted to the output latch123-1.

The MAC latch reset signal MAC_L_RST outputted from the MAC command generator240may be inputted to the transfer gate123-2, the delay circuit123-3, and the inverter123-4. The inverter123-4may inversely buffer the MAC latch reset signal MAC_L_RST to output the inversely buffered signal of the MAC latch reset signal MAC_L_RST to the transfer gate123-2. The transfer gate123-2may transfer the MAC result data DA_MAC from the output latch123-1to the data transmission line in response to the MAC latch reset signal MAC_L_RST. The delay circuit123-3may delay the MAC latch reset signal MAC_L_RST by a certain time to generate and output a latch control signal PINSTB.

FIG. 5illustrates an example of the MAC arithmetic operation performed in the PIM system1-1according to the first embodiment of the present disclosure. As illustrated inFIG. 5, the MAC arithmetic operation performed by the PIM system1-1may be executed though a matrix calculation. Specifically, the PIM device100may execute a matrix multiplying calculation of an ‘M×N’ weight matrix (e.g., ‘8×8’ weight matrix) and a ‘N×1’ vector matrix (e.g., ‘8×1’ vector matrix) according to control of the PIM controller200(where, ‘M’ and ‘N’ are natural numbers). Elements W0.0, . . . , and W7.7constituting the weight matrix may correspond to the first data DA1inputted to the MAC operator120from the first memory bank111. Elements X0.0, . . . , and X7.0constituting the vector matrix may correspond to the second data DA2inputted to the MAC operator120from the second memory bank112. Each of the elements W0.0, . . . , and W7.7constituting the weight matrix may be represented by a binary stream having a plurality of bit values. In addition, each of the elements X0.0, . . . , and X7.0constituting the vector matrix may also be represented by a binary stream having a plurality of bit values. The number of bits included in each of the elements W0.0, . . . , and W7.7constituting the weight matrix may be equal to the number of bits included in each of the elements X0.0, . . . , and X7.0constituting the vector matrix.

The matrix multiplying calculation of the weight matrix and the vector matrix may be appropriate for a multilayer perceptron-type neural network structure (hereinafter, referred to as an ‘MLP-type neural network’). In general, the MLP-type neural network for executing deep learning may include an input layer, a plurality of hidden layers (e.g., at least three hidden layers), and an output layer. The matrix multiplying calculation (i.e., the MAC arithmetic operation) of the weight matrix and the vector matrix illustrated inFIG. 5may be performed in one of the hidden layers. In a first hidden layer of the plurality of hidden layers, the MAC arithmetic operation may be performed using vector data inputted to the first hidden layer. However, in each of second to last hidden layers among the plurality of hidden layers, the MAC arithmetic operation may be performed using a calculation result of the previous hidden layer as the vector data.

FIG. 6is a flowchart illustrating processes of the MAC arithmetic operation described with reference toFIG. 5, which are performed in the PIM system1-1according to the first embodiment of the present disclosure. In addition,FIGS. 7 to 13are block diagrams illustrating the processes of the MAC arithmetic operation illustrated inFIG. 5, which are performed in the PIM system1-1according to the first embodiment of the present disclosure. Referring toFIGS. 6 to 13, before the MAC arithmetic operation is performed, the first data (i.e., the weight data) may be written into the first memory bank111at a step301. Thus, the weight data may be stored in the first memory bank111of the PIM device100. In the present embodiment, it may be assumed that the weight data are the elements W0.0, . . . , and W7.7constituting the weight matrix ofFIG. 5. The integer before the decimal point is one less than a row number, and the integer after the decimal point is one less than a column number. Thus, for example, the weight W0.0represents the element of the first row and the first column of the weight matrix.

At a step302, whether an inference is requested may be determined. An inference request signal may be transmitted from an external device located outside of the PIM system1-1to the PIM controller200of the PIM system1-1. An inference request, in some instances, may be based on user input. An inference request may initiate a calculation performed by the PIM system1-1to reach a determination based on input data. In an embodiment, if no inference request signal is transmitted to the PIM controller200, the PIM system1-1may be in a standby mode until the inference request signal is transmitted to the PIM controller200. Alternatively, if no inference request signal is transmitted to the PIM controller200, the PIM system1-1may perform operations (e.g., data read/write operations) other than the MAC arithmetic operation in the memory mode until the inference request signal is transmitted to the PIM controller200. In the present embodiment, it may be assumed that the second data (i.e., the vector data) are transmitted together with the inference request signal. In addition, it may be assumed that the vector data are the elements X0.0, . . . , and X7.0constituting the vector matrix ofFIG. 5. If the inference request signal is transmitted to the PIM controller200at the step302, then the PIM controller200may write the vector data transmitted with the inference request signal into the second memory bank112at a step303. Accordingly, the vector data may be stored in the second memory bank112of the PIM device100.

At a step304, the MAC command generator240of the PIM controller200may generate and transmit the first MAC read signal MAC_RD_BK0to the PIM device100, as illustrated inFIG. 7. In such a case, the address generator250of the PIM controller200may generate and transmit the bank selection signal BS and the row/column address ADDR_R/ADDR_C to the PIM device100. The bank selection signal BS may be generated to select the first memory bank111of the first and second memory banks111and112. Thus, the first MAC read signal MAC_RD_BK0may control the data read operation for the first memory bank111of the PIM device100. The first memory bank111may output and transmit the elements W0.0, . . . , and W0.7in the first row of the weight matrix of the weight data stored in a region of the first memory bank111, which is selected by the row/column address ADDR_R/ADDR_C, to the MAC operator120in response to the first MAC read signal MAC_RD_BK0. In an embodiment, the data transmission from the first memory bank111to the MAC operator120may be executed through a global input/output (hereinafter, referred to as ‘GIO’) line which is provided as a data transmission path in the PIM device100. Alternatively, the data transmission from the first memory bank111to the MAC operator120may be executed through a first bank input/output (hereinafter, referred to as ‘BIO’) line which is provided specifically for data transmission between the first memory bank111and the MAC operator120.

At a step305, the MAC command generator240of the PIM controller200may generate and transmit the second MAC read signal MAC_RD_BK1to the PIM device100, as illustrated inFIG. 8. In such a case, the address generator250of the PIM controller200may generate and transmit the bank selection signal BS for selecting the second memory bank112and the row/column address ADDR_R/ADDR_C to the PIM device100. The second MAC read signal MAC_RD_BK1may control the data read operation for the second memory bank112of the PIM device100. The second memory bank112may output and transmit the elements X0.0, . . . , and X7.0in the first column of the vector matrix corresponding to the vector data stored in a region of the second memory bank112, which is selected by the row/column address ADDR_R/ADDR_C, to the MAC operator120in response to the second MAC read signal MAC_RD_BK1. In an embodiment, the data transmission from the second memory bank112to the MAC operator120may be executed through the GIO line in the PIM device100. Alternatively, the data transmission from the second memory bank112to the MAC operator120may be executed through a second BIO line which is provided specifically for data transmission between the second memory bank112and the MAC operator120.

At a step306, the MAC command generator240of the PIM controller200may generate and transmit the first MAC input latch signal MAC_L1to the PIM device100, as illustrated inFIG. 9. The first MAC input latch signal MAC_L1may control the input latch operation of the first data for the MAC operator120of the PIM device100. The elements W0.0, . . . , and W0.7in the first row of the weight matrix may be inputted to the MAC circuit122of the MAC operator120by the input latch operation, as illustrated inFIG. 11. The MAC circuit122may include the plurality of multipliers122-11(e.g., eight multipliers122-11), the number of which is equal to the number of columns of the weight matrix. In such a case, the elements W0.0, . . . , and W0.7in the first row of the weight matrix may be inputted to the eight multipliers122-11, respectively.

At a step307, the MAC command generator240of the PIM controller200may generate and transmit the second MAC input latch signal MAC_L2to the PIM device100, as illustrated inFIG. 10. The second MAC input latch signal MAC_L2may control the input latch operation of the second data for the MAC operator120of the PIM device100. The elements X0.0, . . . , and X7.0in the first column of the vector matrix may be inputted to the MAC circuit122of the MAC operator120by the input latch operation, as illustrated inFIG. 11. In such a case, the elements X0.0, . . . , and X7.0in the first column of the vector matrix may be inputted to the eight multipliers122-11, respectively.

At a step308, the MAC circuit122of the MAC operator120may perform the MAC arithmetic operation of an Rthrow of the weight matrix and the first column of the vector matrix, which are inputted to the MAC circuit122. An initial value of ‘R’ may be set as ‘1’. Thus, the MAC arithmetic operation of the first row of the weight matrix and the first column of the vector matrix may be performed a first time. For example, the scalar product is calculated of the Rth‘1×N’ row vector of the ‘M×N’ weight matrix and the ‘N×1’ vector matrix as an ‘R×1’ element of the ‘M×1’ MAC result matrix. For R=1, the scalar product of the first row of the weight matrix and the first column of the vector matrix shown inFIG. 5is W0.0*X0.0+W0.1*X1.0+W0.2*X2.0+W0.3*X3.0+W0.4*X4.0+W0.5*X5.0+W0.6*X6.0+W0.7*X7.0. Specifically, each of the multipliers122-11of the multiplication logic circuit122-1may perform a multiplying calculation of the inputted data, and the result data of the multiplying calculation may be inputted to the addition logic circuit122-2. The addition logic circuit122-2, as illustrated inFIG. 11, may include four adders122-21A disposed at a first stage, two adders122-21B disposed at a second stage, and an adder122-21C disposed at a third stage.

Each of the adders122-21A disposed at the first stage may receive output data of two of the multipliers122-11and may perform an adding calculation of the output data of the two multipliers122-11to output the result of the adding calculation. Each of the adders122-21B disposed at the second stage may receive output data of two of the adders122-21A disposed at the first stage and may perform an adding calculation of the output data of the two adders122-21A to output the result of the adding calculation. The adder122-21C disposed at the third stage may receive output data of two of the adders122-21B disposed at the second stage and may perform an adding calculation of the output data of the two adders122-21B to output the result of the adding calculation. The output data of the addition logic circuit122-2may correspond to result data (i.e., MAC result data) of the MAC arithmetic operation of the first row included in the weight matrix and the column included in the vector matrix. Thus, the output data of the addition logic circuit122-2may correspond to an element MAC0.0located at a first row of an ‘8×1’ MAC result matrix having eight elements of MAC0.0, . . . , and MAC7.0, as illustrated inFIG. 5. The output data MAC0.0of the addition logic circuit122-2may be inputted to the output latch123-1disposed in the data output circuit123of the MAC operator120, as described with reference toFIG. 4.

At a step309, the MAC command generator240of the PIM controller200may generate and transmit the MAC output latch signal MAC_L3to the PIM device100, as illustrated inFIG. 12. The MAC output latch signal MAC_L3may control the output latch operation of the MAC result data MAC0.0performed by the MAC operator120of the PIM device100. The MAC result data MAC0.0inputted from the MAC circuit122of the MAC operator120may be outputted from the output latch123-1in synchronization with the MAC output latch signal MAC_L3, as described with reference toFIG. 4. The MAC result data MAC0.0outputted from the output latch123-1may be inputted to the transfer gate123-2of the data output circuit123.

At a step310, the MAC command generator240of the PIM controller200may generate and transmit the MAC latch reset signal MAC_L_RST to the PIM device100, as illustrated inFIG. 13. The MAC latch reset signal MAC_L_RST may control an output operation of the MAC result data MAC0.0generated by the MAC operator120and a reset operation of the output latch included in the MAC operator120. As described with reference toFIG. 4, the transfer gate123-2receiving the MAC result data MAC0.0from the output latch123-1of the MAC operator120may be synchronized with the MAC latch reset signal MAC_L_RST to output the MAC result data MAC0.0. In an embodiment, the MAC result data MAC0.0outputted from the MAC operator120may be stored into the first memory bank111or the second memory bank112through the first BIO line or the second BIO line in the PIM device100.

At a step311, the row number ‘R’ of the weight matrix for which the MAC arithmetic operation is performed may be increased by ‘1’. Because the MAC arithmetic operation for the first row among the first to eight rows of the weight matrix has been performed during the previous steps, the row number of the weight matrix may change from ‘1’ to ‘2’ at the step311. At a step312, whether the row number changed at the step311is greater than the row number of the last row (i.e., the eighth row of the current example) of the weight matrix may be determined. Because the row number of the weight matrix is changed to ‘2’ at the step311, a process of the MAC arithmetic operation may be fed back to the step304.

If the process of the MAC arithmetic operation is fed back to the step304from the step312, then the same processes as described with reference to the steps304to310may be executed again for the increased row number of the weight matrix. That is, as the row number of the weight matrix changes from ‘1’ to ‘2’, the MAC arithmetic operation may be performed for the second row of the weight matrix instead of the first row of the weight matrix with the vector matrix. If the process of the MAC arithmetic operation is fed back to the step304at the step312, then the processes from the step304to the step311may be iteratively performed until the MAC arithmetic operation is performed for all of the rows of the weight matrix with the vector matrix. If the MAC arithmetic operation for the eighth row of the weight matrix terminates and the row number of the weight matrix changes from ‘8’ to ‘9’ at the step311, the MAC arithmetic operation may terminate because the row number of ‘9’ is greater than the last row number of ‘8’ at the step312.

FIG. 14illustrates another example of a MAC arithmetic operation performed in the PIM system1-1according to the first embodiment of the present disclosure. As illustrated inFIG. 14, the MAC arithmetic operation performed by the PIM system1-1may further include an adding calculation of the MAC result matrix and a bias matrix. Specifically, as described with reference toFIG. 5, the PIM device100may execute the matrix multiplying calculation of the ‘8×8’ weight matrix and the ‘8×1’ vector matrix according to control of the PIM controller200. As a result of the matrix multiplying calculation of the ‘8×8’ weight matrix and the ‘8×1’ vector matrix, the ‘8×1’ MAC result matrix having the eight elements MAC0.0, . . . , and MAC7.0may be generated. The ‘8×1’ MAC result matrix may be added to a ‘8×1’ bias matrix. The ‘8×1’ bias matrix may have elements B0.0, . . . , and B7.0corresponding to bias data. The bias data may be set to reduce an error of the MAC result matrix. As a result of the adding calculation of the MAC result matrix and the bias matrix, a ‘8×1’ biased result matrix having eight elements Y0.0, . . . , and Y7.0may be generated.

FIG. 15is a flowchart illustrating processes of the MAC arithmetic operation described with reference toFIG. 14in the PIM system1-1according to the first embodiment of the present disclosure. Moreover,FIG. 16illustrates an example of a configuration of a MAC operator120-1for performing the MAC arithmetic operation ofFIG. 14in the PIM system1-1according to the first embodiment of the present disclosure. InFIG. 16, the same reference numerals or the same reference symbols as used inFIG. 4denote the same elements, and the detailed descriptions of the same elements as indicated in the previous embodiment will be omitted hereinafter. Referring toFIG. 15, the first data (i.e., the weight data) may be written into the first memory bank111at a step321to perform the MAC arithmetic operation in the PIM device100. Thus, the weight data may be stored in the first memory bank111of the PIM device100. In the present embodiment, it may be assumed that the weight data are the elements W0.0, . . . , and W7.7constituting the weight matrix ofFIG. 14.

At a step322, whether an inference is requested may be determined. An inference request signal may be transmitted from an external device located outside of the PIM system1-1to the PIM controller200of the PIM system1-1. In an embodiment, if no inference request signal is transmitted to the PIM controller200, the PIM system1-1may be in a standby mode until the inference request signal is transmitted to the PIM controller200. Alternatively, if no inference request signal is transmitted to the PIM controller200, the PIM system1-1may perform operations (e.g., data read/write operations) other than the MAC arithmetic operation in the memory mode until the inference request signal is transmitted to the PIM controller200. In the present embodiment, it may be assumed that the second data (i.e., the vector data) are transmitted together with the inference request signal. In addition, it may be assumed that the vector data are the elements X0.0, . . . , and X7.0constituting the vector matrix ofFIG. 14. If the inference request signal is transmitted to the PIM controller200at the step322, the PIM controller200may write the vector data transmitted with the inference request signal into the second memory bank112at a step323. Accordingly, the vector data may be stored in the second memory bank112of the PIM device100.

At a step324, the output latch of the MAC operator may be initially set to have the bias data and the initially set bias data may be fed back to an accumulative adder of the MAC operator. This process is executed to perform the matrix adding calculation of the MAC result matrix and the bias matrix, which is described with reference toFIG. 14. In other words, the output latch123-1in the data output circuit123-A of the MAC operator (120-1) is set to have the bias data. Because the matrix multiplying calculation is executed for the first row of the weight matrix, the output latch123-1may be initially set to have the element B0.0located at a cross point of the first row and the first column of the bias matrix as the bias data. The output latch123-1may output the bias data B0.0, and the bias data B0.0outputted from the output latch123-1may be inputted to the accumulative adder122-21D of the addition logic circuit122-2, as illustrated inFIG. 16.

In an embodiment, in order to output the bias data B0.0out of the output latch123-1and to feed back the bias data B0.0to the accumulative adder122-21D, the MAC command generator240of the PIM controller200may transmit the MAC output latch signal MAC_L3to the MAC operator120-1of the PIM device100. When a subsequent MAC arithmetic operation is performed, the accumulative adder122-21D of the MAC operator120-1may add the MAC result data MAC0.0outputted from the adder122-21C disposed at the last stage to the bias data B0.0which is fed back from the output latch123-1to generate the biased result data Y0.0and may output the biased result data Y0.0to the output latch123-1. The biased result data Y0.0may be outputted from the output latch123-1in synchronization with the MAC output latch signal MAC_L3transmitted in a subsequent process.

In a step325, the MAC command generator240of the PIM controller200may generate and transmit the first MAC read signal MAC_RD_BK0to the PIM device100. In addition, the address generator250of the PIM controller200may generate and transmit the bank selection signal BS and the row/column address ADDR_R/ADDR_C to the PIM device100. The step325may be executed in the same way as described with reference toFIG. 7. In a step326, the MAC command generator240of the PIM controller200may generate and transmit the second MAC read signal MAC_RD_BK1to the PIM device100. In addition, the address generator250of the PIM controller200may generate and transmit the bank selection signal BS for selecting the second memory bank112and the row/column address ADDR_R/ADDR_C to the PIM device100. The step326may be executed in the same way as described with reference toFIG. 8.

At a step327, the MAC command generator240of the PIM controller200may generate and transmit the first MAC input latch signal MAC_L1to the PIM device100. The step327may be executed in the same way as described with reference toFIG. 9. The first MAC input latch signal MAC_L1may control the input latch operation of the first data for the MAC operator120of the PIM device100. The input latch operation of the first data may be performed in the same way as described with reference toFIG. 11. At a step328, the MAC command generator240of the PIM controller200may generate and transmit the second MAC input latch signal MAC_L2to the PIM device100. The step328may be executed in the same way as described with reference toFIG. 10. The second MAC input latch signal MAC_L2may control the input latch operation of the second data for the MAC operator120of the PIM device100. The input latch operation of the second data may be performed in the same way as described with reference toFIG. 11.

At a step329, the MAC circuit122of the MAC operator120may perform the MAC arithmetic operation of an Rthrow of the weight matrix and the first column of the vector matrix, which are inputted to the MAC circuit122. An initial value of ‘R’ may be set as ‘1’. Thus, the MAC arithmetic operation of the first row of the weight matrix and the first column of the vector matrix may be performed a first time. Specifically, each of the multipliers122-11of the multiplication logic circuit122-1may perform a multiplying calculation of the inputted data, and the result data of the multiplying calculation may be inputted to the addition logic circuit122-2. The addition logic circuit122-2may include the four adders122-21A disposed at the first stage, the two adders122-21B disposed at the second stage, the adder122-21C disposed at the third stage, and the accumulative adder122-21D, as illustrated inFIG. 16. The accumulative adder122-21D may add output data of the adder122-21C to feedback data fed back from the output latch123-1to output the result of the adding calculation. The output data of the adder122-21C may be the matrix multiplying result MAC0.0, which corresponds to the result of the matrix multiplying calculation of the first row of the weight matrix and the first column of the vector matrix. The accumulative adder122-21D may add the output data MAC0.0of the adder122-21C to the bias data B0.0fed back from the output latch123-1to output the result of the adding calculation. The output data Y0.0of the accumulative adder122-21D may be inputted to the output latch123disposed in a data output circuit123-A of the MAC operator120-1.

At a step330, the MAC command generator240of the PIM controller200may generate and transmit the MAC output latch signal MAC_L3to the PIM device100. The step330may be executed in the same way as described with reference toFIG. 12. The MAC output latch signal MAC_L3may control the output latch operation of the MAC result data MAC0.0, which is performed by the MAC operator120-1of the PIM device100. The biased result data Y0.0transmitted from the MAC circuit122of the MAC operator120to the output latch123-1may be outputted from the output latch123-1in synchronization with the MAC output latch signal MAC_L3. The biased result data Y0.0outputted from the output latch123may be inputted to the transfer gate123-2.

At a step331, the MAC command generator240of the PIM controller200may generate and transmit the MAC latch reset signal MAC_L_RST to the PIM device100. The step331may be executed in the same way as described with reference toFIG. 13. The MAC latch reset signal MAC_L_RST may control an output operation of the biased result data Y0.0generated by the MAC operator120and a reset operation of the output latch123-1included in the MAC operator120. The transfer gate123-2receiving the biased result data Y0.0from the output latch123-1of the data output circuit123-A included in the MAC operator120may be synchronized with the MAC latch reset signal MAC_L_RST to output the biased result data Y0.0. In an embodiment, the biased result data Y0.0outputted from the MAC operator120may be stored into the first memory bank111or the second memory bank112through the first BIO line or the second BIO line in the PIM device100.

At a step332, the row number ‘R’ of the weight matrix for which the MAC arithmetic operation is performed may be increased by ‘1’. Because the MAC arithmetic operation for the first row among the first to eight rows of the weight matrix has been performed during the previous steps, the row number of the weight matrix may change from ‘1’ to ‘2’ at the step332. At a step333, whether the row number changed at the step332is greater than the row number of the last row (i.e., the eighth row of the current example) of the weight matrix may be determined. Because the row number of the weight matrix is changed to ‘2’ at the step332, a process of the MAC arithmetic operation may be fed back to the step324.

If the process of the MAC arithmetic operation is fed back to the step324from the step333, then the same processes as described with reference to the steps324to331may be executed again for the increased row number of the weight matrix. That is, as the row number of the weight matrix changes from ‘1’ to ‘2’, the MAC arithmetic operation may be performed for the second row of the weight matrix instead of the first row of the weight matrix with the vector matrix and the bias data B0.0in the output latch123-1initially set at the step324may be changed into the bias data B1.0. If the process of the MAC arithmetic operation is fed back to the step324at the step333, the processes from the step324to the step332may be iteratively performed until the MAC arithmetic operation is performed for all of the rows of the weight matrix with the vector matrix. If the MAC arithmetic operation for the eighth row of the weight matrix terminates and the row number of the weight matrix changes from ‘8’ to ‘9’ at the step332, the MAC arithmetic operation may terminate because the row number of ‘9’ is greater than the last row number of ‘8’ at the step333.

FIG. 17illustrates yet another example of a MAC arithmetic operation performed in the PIM system1-1according to the first embodiment of the present disclosure. As illustrated inFIG. 17, the MAC arithmetic operation performed by the PIM system1-1may further include a process for applying the biased result matrix to an activation function. Specifically, as described with reference toFIG. 14, the PIM device100may execute the matrix multiplying calculation of the ‘8×8’ weight matrix and the ‘8×1’ vector matrix according to control of the PIM controller200to generate the MAC result matrix. In addition, the MAC result matrix may be added to the bias matrix to generate biased result matrix.

The biased result matrix may be applied to the activation function. The activation function means a function which is used to calculate a unique output value by comparing a MAC calculation value with a critical value in an MLP-type neural network. In an embodiment, the activation function may be a unipolar activation function which generates only positive output values or a bipolar activation function which generates negative output values as well as positive output values. In different embodiments, the activation function may include a sigmoid function, a hyperbolic tangent (Tanh) function, a rectified linear unit (ReLU) function, a leaky ReLU function, an identity function, and a maxout function.

FIG. 18is a flowchart illustrating processes of the MAC arithmetic operation described with reference toFIG. 17in the PIM system1-1according to the first embodiment of the present disclosure. Moreover,FIG. 19illustrates an example of a configuration of a MAC operator120-2for performing the MAC arithmetic operation ofFIG. 17in the PIM system1-1according to the first embodiment of the present disclosure. InFIG. 19, the same reference numerals or the same reference symbols as used inFIG. 4denote the same elements, and the detailed descriptions of the same elements as mentioned in the previous embodiment will be omitted hereinafter. Referring toFIG. 18, the first data (i.e., the weight data) may be written into the first memory bank111at a step341to perform the MAC arithmetic operation in the PIM device100. Thus, the weight data may be stored in the first memory bank111of the PIM device100. In the present embodiment, it may be assumed that the weight data are the elements W0.0, . . . , and W7.7constituting the weight matrix ofFIG. 17.

At a step342, whether an inference is requested may be determined. An inference request signal may be transmitted from an external device located outside of the PIM system1-1to the PIM controller200of the PIM system1-1. In an embodiment, if no inference request signal is transmitted to the PIM controller200, the PIM system1-1may be in a standby mode until the inference request signal is transmitted to the PIM controller200. Alternatively, if no inference request signal is transmitted to the PIM controller200, the PIM system1-1may perform operations (e.g., the data read/write operations) other than the MAC arithmetic operation in the memory mode until the inference request signal is transmitted to the PIM controller200. In the present embodiment, it may be assumed that the second data (i.e., the vector data) are transmitted together with the inference request signal. In addition, it may be assumed that the vector data are the elements X0.0, . . . , and X7.0constituting the vector matrix ofFIG. 17. If the inference request signal is transmitted to the PIM controller200at the step342, then the PIM controller200may write the vector data transmitted with the inference request signal into the second memory bank112at a step343. Accordingly, the vector data may be stored in the second memory bank112of the PIM device100.

At a step344, an output latch of a MAC operator may be initially set to have bias data and the initially set bias data may be fed back to an accumulative adder of the MAC operator. This process is executed to perform the matrix adding calculation of the MAC result matrix and the bias matrix, which is described with reference toFIG. 17. That is, as illustrated inFIG. 19, the output latch123-1of the MAC operator (120-2ofFIG. 19) may be initially set to have the bias data of the bias matrix. Because the matrix multiplying calculation is executed for the first row of the weight matrix, the element B0.0located at first row and the first column of the bias matrix may be initially set as the bias data in the output latch123-1. The output latch123-1may output the bias data B0.0, and the bias data B0.0outputted from the output latch123-1may be inputted to the accumulative adder122-21D of the MAC operator120-2.

In an embodiment, in order to output the bias data B0.0out of the output latch123-1and to feed back the bias data B0.0to the accumulative adder122-21D, the MAC command generator240of the PIM controller200may transmit the MAC output latch signal MAC_L3to the MAC operator120-2of the PIM device100. When a subsequent MAC arithmetic operation is performed, the accumulative adder122-21D of the MAC operator120-2may add the MAC result data MAC0.0outputted from the adder122-21C disposed at the last stage to the bias data B0.0which is fed back from the output latch123-1to generate the biased result data Y0.0and may output the biased result data Y0.0to the output latch123-1. As illustrated inFIG. 19, the biased result data Y0.0may be transmitted from the output latch123-1to an activation function logic circuit123-5disposed in a data output circuit123-B of the MAC operator120-2in synchronization with the MAC output latch signal MAC_L3transmitted in a subsequent process.

In a step345, the MAC command generator240of the PIM controller200may generate and transmit the first MAC read signal MAC_RD_BK0to the PIM device100. In addition, the address generator250of the PIM controller200may generate and transmit the bank selection signal BS and the row/column address ADDR_R/ADDR_C to the PIM device100. The step345may be executed in the same way as described with reference toFIG. 7. In a step346, the MAC command generator240of the PIM controller200may generate and transmit the second MAC read signal MAC_RD_BK1to the PIM device100. In addition, the address generator250of the PIM controller200may generate and transmit the bank selection signal BS for selecting the second memory bank112and the row/column address ADDR_R/ADDR_C to the PIM device100. The step346may be executed in the same way as described with reference toFIG. 8.

At a step347, the MAC command generator240of the PIM controller200may generate and transmit the first MAC input latch signal MAC_L1to the PIM device100. The step347may be executed in the same way as described with reference toFIG. 9. The first MAC input latch signal MAC_L1may control the input latch operation of the first data for the MAC operator120of the PIM device100. The input latch operation of the first data may be performed in the same way as described with reference toFIG. 11. At a step348, the MAC command generator240of the PIM controller200may generate and transmit the second MAC input latch signal MAC_L2to the PIM device100. The step348may be executed in the same way as described with reference toFIG. 10. The second MAC input latch signal MAC_L2may control the input latch operation of the second data for the MAC operator120of the PIM device100. The input latch operation of the second data may be performed in the same way as described with reference toFIG. 11.

At a step349, the MAC circuit122of the MAC operator120may perform the MAC arithmetic operation of an Rthrow of the weight matrix and the first column of the vector matrix, which are inputted to the MAC circuit122. An initial value of ‘R’ may be set as ‘1’. Thus, the MAC arithmetic operation of the first row of the weight matrix and the first column of the vector matrix may be performed a first time. Specifically, each of the multipliers122-11of the multiplication logic circuit122-1may perform a multiplying calculation of the inputted data, and the result data of the multiplying calculation may be inputted to the addition logic circuit122-2. The addition logic circuit122-2may include the four adders122-21A disposed at the first stage, the two adders122-21B disposed at the second stage, the adder122-21C disposed at the third stage, and the accumulative adder122-21D, as illustrated inFIG. 19. The accumulative adder122-21D may add output data of the adder122-21C to feedback data fed back from the output latch123-1to output the result of the adding calculation. The output data of the adder122-21C may be the element MAC0.0of the ‘8×1’ MAC result matrix, which corresponds to the result of the matrix multiplying calculation of the first row of the weight matrix and the first column of the vector matrix. The accumulative adder122-21D may add the output data MAC0.0of the adder122-21C to the bias data B0.0fed back from the output latch123-1to output the result of the adding calculation. The output data Y0.0of the accumulative adder122-21D may be inputted to the output latch123-1disposed in the data output circuit123-A of the MAC operator120.

At a step350, the MAC command generator240of the PIM controller200may generate and transmit the MAC output latch signal MAC_L3to the PIM device100. The step350may be executed in the same way as described with reference toFIG. 12. The MAC output latch signal MAC_L3may control the output latch operation of the output latch123-1included in the MAC operator120of the PIM device100. The biased result data Y0.0transmitted from the MAC circuit122of the MAC operator120to the output latch123-1may be outputted from the output latch123-1in synchronization with the MAC output latch signal MAC_L3. The biased result data Y0.0outputted from the output latch123-1may be inputted to the activation function logic circuit123-5. At a step351, the activation function logic circuit123-5may apply an activation function to the biased result data Y0.0to generate a final output value, and the final output value may be inputted to the transfer gate (123-2ofFIG. 4). This, for example, is the final output value for the current of R which is incremented in step354.

At a step352, the MAC command generator240of the PIM controller200may generate and transmit the MAC latch reset signal MAC_L_RST to the PIM device100. The step352may be executed in the same way as described with reference toFIG. 13. The MAC latch reset signal MAC_L_RST may control an output operation of the final output value generated by the MAC operator120and a reset operation of the output latch123-1included in the MAC operator120. The transfer gate123-2receiving the final output value from the activation function logic circuit123-5of the data output circuit123-B included in the MAC operator120may be synchronized with the MAC latch reset signal MAC_L_RST to output the final output value. In an embodiment, the final output value outputted from the MAC operator120may be stored into the first memory bank111or the second memory bank112through the first BIO line or the second BIO line in the PIM device100.

At a step353, the row number ‘R’ of the weight matrix for which the MAC arithmetic operation is performed may be increased by ‘1’. Because the MAC arithmetic operation for the first row among the first to eight rows of the weight matrix has been performed during the previous steps, the row number of the weight matrix may change from ‘1’ to ‘2’ at the step353. At a step354, whether the row number changed at the step353is greater than the row number of the last row (i.e., the eighth row) of the weight matrix may be determined. Because the row number of the weight matrix is changed to ‘2’ at the step353, a process of the MAC arithmetic operation may be fed back to the step344.

If the process of the MAC arithmetic operation is fed back to the step344from the step354, the same processes as described with reference to the steps344to354may be executed again for the increased row number of the weight matrix. That is, as the row number of the weight matrix changes from ‘1’ to ‘2’, the MAC arithmetic operation may be performed for the second row of the weight matrix instead of the first row of the weight matrix with the vector matrix, and the bias data B0.0in the output latch123-1initially set at the step344may be changed to the bias data B1.0. If the process of the MAC arithmetic operation is fed back to the step344from the step354, the processes from the step344to the step354may be iteratively performed until the MAC arithmetic operation is performed for all of the rows of the weight matrix with the vector matrix. For an embodiment, a plurality of final output values, namely, one final output value for each incremented value of R, represents an ‘N×1’ final result matrix. If the MAC arithmetic operation for the eighth row of the weight matrix terminates and the row number of the weight matrix changes from ‘8’ to ‘9’ at the step354, the MAC arithmetic operation may terminate because the row number of ‘9’ is greater than the last row number of ‘8’ at the step354.

FIG. 20is a block diagram illustrating a PIM system1-2according to a second embodiment of the present disclosure. InFIG. 20, the same reference numerals or the same reference symbols as used inFIG. 2denote the same elements. As illustrated inFIG. 20, the PIM system1-2may be configured to include a PIM device400and a PIM controller500. The PIM device400may be configured to include a memory bank (BANK)411corresponding to a storage region, a global buffer412, a MAC operator420, an interface (I/F)431, and a data input/output (I/O) pad432. For an embodiment, the MAC operator420represents a MAC operator circuit. The memory bank (BANK)411and the MAC operator420included in the PIM device400may constitute one MAC unit. In another embodiment, the PIM device400may include a plurality of MAC units. The memory bank (BANK)411may represent a memory region for storing data, for example, a DRAM device. The global buffer412may also represent a memory region for storing data, for example, a DRAM device or an SRAM device. The memory bank (BANK)411may be a component unit which is independently activated and may be configured to have the same data bus width as data I/O lines in the PIM device400. In an embodiment, the memory bank411may operate through interleaving such that an active operation of the memory bank411is performed in parallel while another memory bank is selected. The memory bank411may include at least one cell array which includes memory unit cells located at cross points of a plurality of rows and a plurality of columns.

Although not shown in the drawings, a core circuit may be disposed adjacent to the memory bank411. The core circuit may include X-decoders XDECs and Y-decoders/IO circuits YDEC/IOs. An X-decoder XDEC may also be referred to as a word line decoder or a row decoder. The X-decoder XDEC may receive a row address ADDR_R from the PIM controller500and may decode the row address ADDR_R to select and enable one of the rows (i.e., word lines) coupled to the selected memory bank. Each of the Y-decoders/IO circuits YDEC/IOs may include a Y-decoder YDEC and an I/O circuit IO. The Y-decoder YDEC may also be referred to as a bit line decoder or a column decoder. The Y-decoder YDEC may receive a column address ADD_C from the PIM controller500and may decode the column address ADD_C to select and enable at least one of the columns (i.e., bit lines) coupled to the selected memory bank. Each of the I/O circuits may include an I/O sense amplifier for sensing and amplifying a level of a read datum outputted from the corresponding memory bank during a read operation for the memory bank411. In addition, the I/O circuit may include a write driver for driving a write datum during a write operation for the memory bank411.

The MAC operator420of the PIM device400may have mostly the same configuration as the MAC operator120described with reference toFIG. 4. That is, the MAC operator420may be configured to include the data input circuit121, the MAC circuit122, and the data output circuit123, as described with reference toFIG. 4. The data input circuit121may be configured to include the first input latch121-1and the second input latch121-2. The MAC circuit122may be configured to include the multiplication logic circuit122-1and the addition logic circuit122-2. The data output circuit123may be configured to include the output latch123-1, the transfer gate123-2, the delay circuit123-3, and the inverter123-4. In an embodiment, the first input latch121-1, the second input latch121-2, and the output latch123-1may be realized using flip-flops.

The MAC operator420may be different from the MAC operator120in that a MAC input latch signal MAC_L1is simultaneously inputted to both of clock terminals of the first and second input latches121-1and121-2. As indicated in the following descriptions, the weight data and the vector data may be simultaneously transmitted to the MAC operator420of the PIM device400included in the PIM system1-2according to the present embodiment. That is, the first data DA1(i.e., the weight data) and the second data DA2(i.e., the vector data) may be simultaneously inputted to both of the first input latch121-1and the second input latch121-2constituting the data input circuit121, respectively. Accordingly, it may be unnecessary to apply an extra control signal to the clock terminals of the first and second input latches121-1and121-2, and thus the MAC input latch signal MAC_L1may be simultaneously inputted to both of the clock terminals of the first and second input latches121-1and121-2included in the MAC operator420.

In another embodiment, the MAC operator420may be realized to have the same configuration as the MAC operator120-1described with reference toFIG. 16to perform the operation illustrated inFIG. 14. Even in such a case, the MAC operator420may have the same configuration as described with reference toFIG. 16except that the MAC input latch signal MAC_L1is simultaneously inputted to both of the clock terminals of the first and second input latches121-1and121-2constituting the data input circuit121. In yet another embodiment, the MAC operator420may be realized to have the same configuration as the MAC operator120-2described with reference toFIG. 19to perform the operation illustrated inFIG. 17. Even in such a case, the MAC operator420may have the same configuration as described with reference toFIG. 19except that the MAC input latch signal MAC_L1is simultaneously inputted to both of the clock terminals of the first and second input latches121-1and121-2constituting the data input circuit121.

The interface431of the PIM device400may receive the memory command M_CMD, the MAC commands MAC_CMDs, the bank selection signal BS, and the row/column addresses ADDR_R/ADDR_C from the PIM controller500. The interface431may output the memory command M_CMD, together with the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C, to the memory bank411. The interface431may output the MAC commands MAC_CMDs to the memory bank411and the MAC operator420. In such a case, the interface431may output the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C to the memory bank411. The data I/O pad432of the PIM device400may function as a data communication terminal between a device external to the PIM device400, the global buffer412, and the MAC unit (which includes the memory bank411and the MAC operator420) included in the PIM device400. The external device to the PIM device400may correspond to the PIM controller500of the PIM system1-2or a host located outside the PIM system1-2. Accordingly, data outputted from the host or the PIM controller500may be inputted into the PIM device400through the data I/O pad432. In addition, data generated by the PIM device400may be transmitted to the external device to the PIM device400through the data I/O pad432.

The PIM controller500may control operations of the PIM device400. In an embodiment, the PIM controller500may control the PIM device400such that the PIM device400operates in the memory mode or the MAC mode. In the event that the PIM controller500controls the PIM device400such that the PIM device400operates in the memory mode, the PIM device400may perform a data read operation or a data write operation for the memory bank411. In the event that the PIM controller500controls the PIM device400such that the PIM device400operates in the MAC mode, the PIM device400may perform the MAC arithmetic operation for the MAC operator420. In the event that the PIM controller500controls the PIM device400such that the PIM device400operates in the MAC mode, the PIM device400may also perform the data read operation and the data write operation for the memory bank411and the global buffer412to execute the MAC arithmetic operation.

The PIM controller500may be configured to include the command queue logic210, the scheduler220, the memory command generator230, a MAC command generator540, and an address generator550. The scheduler220may include the mode selector221. The command queue logic210may receive the request REQ from an external device (e.g., a host of the PIM system1-2) and store a command queue corresponding the request REQ in the command queue logic210. The command queue stored in the command queue logic210may be transmitted to the memory command generator230or the MAC command generator540according to a sequence determined by the scheduler220. The scheduler220may adjust a timing of the command queue when the command queue stored in the command queue logic210is outputted from the command queue logic210. The scheduler210may include the mode selector221that generates a mode selection signal including information on whether command queue stored in the command queue logic210relates to the memory mode or the MAC mode. The memory command generator230may receive the command queue related to the memory mode of the PIM device400from the command queue logic210to generate and output the memory command M_CMD. The command queue logic210, the scheduler220, the mode selector221, and the memory command generator230may have the same function as described with reference toFIG. 2.

The MAC command generator540may receive the command queue related to the MAC mode of the PIM device400from the command queue logic210. The MAC command generator540may decode the command queue to generate and output the MAC commands MAC_CMDs. The MAC commands MAC_CMDs outputted from the MAC command generator540may be transmitted to the PIM device400. The data read operation for the memory bank411of the PIM device400may be performed by the MAC commands MAC_CMDs outputted from the MAC command generator540, and the MAC arithmetic operation of the MAC operator420may also be performed by the MAC commands MAC_CMDs outputted from the MAC command generator540. The MAC commands MAC_CMDs and the MAC arithmetic operation of the PIM device400according to the MAC commands MAC_CMDs will be described in detail with reference toFIG. 21.

The address generator550may receive address information from the command queue logic210. The address generator550may generate the bank selection signal BS for selecting a memory bank where, for example, the memory bank411represents multiple memory banks. The address generator550may transmit the bank selection signal BS to the PIM device400. In addition, the address generator550may generate the row address ADDR_R and the column address ADDR_C for accessing a region (e.g., memory cells) in the memory bank411and may transmit the row address ADDR_R and the column address ADDR_C to the PIM device400.

FIG. 21illustrates the MAC commands MAC_CMDs outputted from the MAC command generator540included in the PIM system1-2according to the second embodiment of the present disclosure. As illustrated inFIG. 21, the MAC commands MAC_CMDs may include first to fourth MAC command signals. In an embodiment, the first MAC command signal may be a MAC read signal MAC_RD_BK, the second MAC command signal may be a MAC input latch signal MAC_L1, the third MAC command signal may be a MAC output latch signal MAC_L3, and the fourth MAC command signal may be a MAC latch reset signal MAC_L_RST.

The MAC read signal MAC_RD_BK may control an operation for reading the first data (e.g., the weight data) out of the memory bank411to transmit the first data to the MAC operator420. The MAC input latch signal MAC_L1may control an input latch operation of the weight data transmitted from the first memory bank411to the MAC operator420. The MAC output latch signal MAC_L3may control an output latch operation of the MAC result data generated by the MAC operator420. And, the MAC latch reset signal MAC_L_RST may control an output operation of the MAC result data generated by the MAC operator420and a reset operation of an output latch included in the MAC operator420.

The PIM system1-2according to the present embodiment may also be configured to perform the deterministic MAC arithmetic operation. Thus, the MAC commands MAC_CMDs transmitted from the PIM controller500to the PIM device400may be sequentially generated with fixed time intervals. Accordingly, the PIM controller500does not require any extra end signals of various operations executed for the MAC arithmetic operation to generate the MAC commands MAC_CMDs for controlling the MAC arithmetic operation. In an embodiment, latencies of the various operations executed by MAC commands MAC_CMDs for controlling the MAC arithmetic operation may be set to have fixed values in order to perform the deterministic MAC arithmetic operation. In such a case, the MAC commands MAC_CMDs may be sequentially outputted from the PIM controller500with fixed time intervals corresponding to the fixed latencies.

FIG. 22is a flowchart illustrating processes of the MAC arithmetic operation described with reference toFIG. 5, which are performed in the PIM system1-2according to the second embodiment of the present disclosure. In addition,FIGS. 23 to 26are block diagrams illustrating the processes of the MAC arithmetic operation illustrated inFIG. 5, which are performed in the PIM system1-2according to the second embodiment of the present disclosure. Referring toFIGS. 22 to 26, the first data (i.e., the weight data) may be written into the memory bank411at a step361to perform the MAC arithmetic operation. Thus, the weight data may be stored in the memory bank411of the PIM device400. In the present embodiment, it may be assumed that the weight data are the elements W0.0, . . . , and W7.7constituting the weight matrix ofFIG. 5.

At a step362, whether an inference is requested may be determined. An inference request signal may be transmitted from an external device located outside of the PIM system1-2to the PIM controller500of the PIM system1-2. In an embodiment, if no inference request signal is transmitted to the PIM controller500, the PIM system1-2may be in a standby mode until the inference request signal is transmitted to the PIM controller500. Alternatively, if no inference request signal is transmitted to the PIM controller500, the PIM system1-2may perform operations (e.g., data read/write operations) other than the MAC arithmetic operation in the memory mode until the inference request signal is transmitted to the PIM controller500. In the present embodiment, it may be assumed that the second data (i.e., the vector data) are transmitted together with the inference request signal. In addition, it may be assumed that the vector data are the elements X0.0, . . . , and X7.0constituting the vector matrix ofFIG. 5. If the inference request signal is transmitted to the PIM controller500at the step362, then the PIM controller500may write the vector data transmitted with the inference request signal into the global buffer412at a step363. Accordingly, the vector data may be stored in the global buffer412of the PIM device400.

At a step364, the MAC command generator540of the PIM controller500may generate and transmit the MAC read signal MAC_RD_BK to the PIM device400, as illustrated inFIG. 23. In such a case, the address generator550of the PIM controller500may generate and transmit the row/column address ADDR_R/ADDR_C to the PIM device400. Although not shown in the drawings, if a plurality of memory banks are disposed in the PIM device400, the address generator550may transmit a bank selection signal for selecting the memory bank411among the plurality of memory banks as well as the row/column address ADDR_R/ADDR_C to the PIM device400. The MAC read signal MAC_RD_BK inputted to the PIM device400may control the data read operation for the memory bank411of the PIM device400. The memory bank411may output and transmit the elements W0.0, . . . , and W0.7in the first row of the weight matrix of the weight data stored in a region of the memory bank411, which is designated by the row/column address ADDR_R/ADDR_C, to the MAC operator420in response to the MAC read signal MAC_RD_BK. In an embodiment, the data transmission from the memory bank411to the MAC operator420may be executed through a BIO line which is provided specifically for data transmission between the memory bank411and the MAC operator420.

Meanwhile, the vector data X0.0, . . . , and X7.0stored in the global buffer412may also be transmitted to the MAC operator420in synchronization with a point in time when the weight data are transmitted from the memory bank411to the MAC operator420. In order to transmit the vector data X0.0, . . . , and X7.0from the global buffer412to the MAC operator420, a control signal for controlling the read operation for the global buffer412may be generated in synchronization with the MAC read signal MAC_RD_BK outputted from the MAC command generator540of the PIM controller500. The data transmission between the global buffer412and the MAC operator420may be executed through a GIO line. Thus, the weight data and the vector data may be independently transmitted to the MAC operator420through two separate transmission lines, respectively. In an embodiment, the weight data and the vector data may be simultaneously transmitted to the MAC operator420through the BIO line and the GIO line, respectively.

At a step365, the MAC command generator540of the PIM controller500may generate and transmit the MAC input latch signal MAC_L1to the PIM device400, as illustrated inFIG. 24. The MAC input latch signal MAC_L1may control the input latch operation of the weight data and the vector data for the MAC operator420of the PIM device400. The elements W0.0, . . . , and W0.7in the first row of the weight matrix and the elements X0.0, . . . , and X7.0in the first column of the vector matrix may be inputted to the MAC circuit122of the MAC operator420by the input latch operation. The MAC circuit122may include the plurality of multipliers (e.g., the eight multipliers122-11), the number of which is equal to the number of columns of the weight matrix and the number of rows of the vector matrix. The elements W0.0, . . . , and W0.7in the first row of the weight matrix may be inputted to the first to eighth multipliers122-11, respectively, and the elements X0.0, . . . , and X7.0in the first column of the vector matrix may also be inputted to the first to eighth multipliers122-11, respectively.

At a step366, the MAC circuit122of the MAC operator420may perform the MAC arithmetic operation of an R row of the weight matrix and the first column of the vector matrix, which are inputted to the MAC circuit122. An initial value of ‘R’ may be set as ‘1’. Thus, the MAC arithmetic operation of the first row of the weight matrix and the first column of the vector matrix may be performed a first time. Specifically, as described with reference toFIG. 4, each of the multipliers122-11of the multiplication logic circuit122-1may perform a multiplying calculation of the inputted data, and the result data of the multiplying calculation may be inputted to the addition logic circuit122-2. The addition logic circuit122-2may receive output data from the multipliers122-11and may perform the adding calculation of the output data of the multipliers122-11to output the result data of the adding calculation. The output data of the addition logic circuit122-2may correspond to result data (i.e., MAC result data) of the MAC arithmetic operation of the first row included in the weight matrix and the column included in the vector matrix. Thus, the output data of the addition logic circuit122-2may correspond to the element MAC0.0located at the first row of the ‘8×1’ MAC result matrix having the eight elements of MAC0.0, . . . , and MAC7.0illustrated inFIG. 5. The output data MAC0.0of the addition logic circuit122-2may be inputted to the output latch123-1disposed in the data output circuit123of the MAC operator420, as described with reference toFIG. 4.

At a step367, the MAC command generator540of the PIM controller500may generate and transmit the MAC output latch signal MAC_L3to the PIM device400, as illustrated inFIG. 25. The MAC output latch signal MAC_L3may control the output latch operation of the MAC result data MAC0.0performed by the MAC operator420of the PIM device400. The MAC result data MAC0.0transmitted from the MAC circuit122of the MAC operator420to the output latch123-1may be outputted from the output latch123-1by the output latch operation performed in synchronization with the MAC output latch signal MAC_L3, as described with reference toFIG. 4. The MAC result data MAC0.0outputted from the output latch123-1may be inputted to the transfer gate123-2of the data output circuit123.

At a step368, the MAC command generator540of the PIM controller500may generate and transmit the MAC latch reset signal MAC_L_RST to the PIM device400, as illustrated inFIG. 26. The MAC latch reset signal MAC_L_RST may control an output operation of the MAC result data MAC0.0generated by the MAC operator420and a reset operation of the output latch123-1included in the MAC operator420. As described with reference toFIG. 4, the transfer gate123-2receiving the MAC result data MAC0.0from the output latch123-1of the MAC operator420may be synchronized with the MAC latch reset signal MAC_L_RST to output the MAC result data MAC0.0. In an embodiment, the MAC result data MAC0.0outputted from the MAC operator420may be stored into the memory bank411through the BIO line in the PIM device400.

At a step369, the row number ‘R’ of the weight matrix for which the MAC arithmetic operation is performed may be increased by ‘1’. Because the MAC arithmetic operation for the first row among the first to eight rows of the weight matrix has been performed during the previous steps, the row number of the weight matrix may change from ‘1’ to ‘2’ at the step369. At a step370, whether the row number changed at the step369is greater than the row number of the last row (i.e., the eighth row) of the weight matrix may be determined. Because the row number of the weight matrix is changed to ‘2’ at the step370, a process of the MAC arithmetic operation may be fed back to the step364.

If the process of the MAC arithmetic operation is fed back to the step364from the step370, the same processes as described with reference to the steps364to370may be executed again for the increased row number of the weight matrix. That is, as the row number of the weight matrix changes from ‘1’ to ‘2’, the MAC arithmetic operation may be performed for the second row of the weight matrix instead of the first row of the weight matrix with the vector matrix. If the process of the MAC arithmetic operation is fed back to the step364from the step370, the processes from the step364to the step370may be iteratively performed until the MAC arithmetic operation is performed for all of the rows of the weight matrix with the vector matrix. If the MAC arithmetic operation for the eighth row of the weight matrix terminates and the row number of the weight matrix changes from ‘8’ to ‘9’ at the step369, the MAC arithmetic operation may terminate because the row number of ‘9’ is greater than the last row number of ‘8’ at the step370.

FIG. 27is a flowchart illustrating processes of the MAC arithmetic operation described with reference toFIG. 14, which are performed in the PIM system1-2according to the second embodiment of the present disclosure. In order to perform the MAC arithmetic operation according to the present embodiment, the MAC operator420of the PIM device400may have the same configuration as the MAC operator120-1illustrated inFIG. 16. Referring toFIGS. 20 and 27, the first data (i.e., the weight data) may be written into the memory bank411at a step381to perform the MAC arithmetic operation. Thus, the weight data may be stored in the memory bank411of the PIM device400. In the present embodiment, it may be assumed that the weight data are the elements W0.0, . . . , and W7.7constituting the weight matrix ofFIG. 14.

At a step382, whether an inference is requested may be determined. An inference request signal may be transmitted from an external device located outside of the PIM system1-2to the PIM controller500of the PIM system1-2. In an embodiment, if no inference request signal is transmitted to the PIM controller500, the PIM system1-2may be in a standby mode until the inference request signal is transmitted to the PIM controller500. Alternatively, if no inference request signal is transmitted to the PIM controller500, the PIM system1-2may perform operations (e.g., data read/write operations) other than the MAC arithmetic operation in the memory mode until the inference request signal is transmitted to the PIM controller500. In the present embodiment, it may be assumed that the second data (i.e., the vector data) are transmitted together with the inference request signal. In addition, it may be assumed that the vector data are the elements X0.0, . . . , and X7.0constituting the vector matrix ofFIG. 14. If the inference request signal is transmitted to the PIM controller500at the step382, then the PIM controller500may write the vector data transmitted with the inference request signal into the global buffer412at a step383. Accordingly, the vector data may be stored in the global buffer412of the PIM device400.

At a step384, an output latch of a MAC operator420may be initially set to have bias data and the initially set bias data may be fed back to an accumulative adder of the MAC operator420. This process is executed to perform the matrix adding calculation of the MAC result matrix and the bias matrix, which is described with reference toFIG. 14. That is, as illustrated inFIG. 16, the output latch123-1of the data output circuit123-A included in the MAC operator420may be initially set to have the bias data of the bias matrix. Because the matrix multiplying calculation is executed for the first row of the weight matrix, the element B0.0located at first row of the bias matrix may be initially set as the bias data in the output latch123-1. The output latch123-1may output the bias data B0.0, and the bias data B0.0outputted from the output latch123-1may be inputted to the accumulative adder122-21D of the addition logic circuit122-2included in the MAC operator420.

In an embodiment, in order to output the bias data B0.0out of the output latch123-1and to feed back the bias data B0.0to the accumulative adder122-21D, the MAC command generator540of the PIM controller500may transmit the MAC output latch signal MAC_L3to the MAC operator420of the PIM device400. When a subsequent MAC arithmetic operation is performed, the accumulative adder122-21D of the MAC operator420may add the MAC result data MAC0.0outputted from the adder122-21C disposed at the last stage to the bias data B0.0which is fed back from the output latch123-1to generate the biased result data Y0.0and may output the biased result data Y0.0to the output latch123-1. The biased result data Y0.0may be outputted from the output latch123-1in synchronization with the MAC output latch signal MAC_L3transmitted in a subsequent process.

At a step385, the MAC command generator540of the PIM controller500may generate and transmit the MAC read signal MAC_RD_BK to the PIM device400, as illustrated inFIG. 23. In such a case, the address generator550of the PIM controller500may generate and transmit the row/column address ADDR_R/ADDR_C to the PIM device400. The MAC read signal MAC_RD_BK inputted to the PIM device400may control the data read operation for the memory bank411of the PIM device400. The memory bank411may output and transmit the elements W0.0, . . . , and W0.7in the first row of the weight matrix of the weight data stored in a region of the memory bank411, which is designated by the row/column address ADDR_R/ADDR_C, to the MAC operator420in response to the MAC read signal MAC_RD_BK. In an embodiment, the data transmission from the memory bank411to the MAC operator420may be executed through a BIO line which is provided specifically for data transmission between the memory bank411and the MAC operator420.

Meanwhile, the vector data X0.0, . . . , and X7.0stored in the global buffer412may also be transmitted to the MAC operator420in synchronization with a point in time when the weight data are transmitted from the memory bank411to the MAC operator420. In order to transmit the vector data X0.0, . . . , and X7.0from the global buffer412to the MAC operator420, a control signal for controlling the read operation for the global buffer412may be generated in synchronization with the MAC read signal MAC_RD_BK outputted from the MAC command generator540of the PIM controller500. The data transmission between the global buffer412and the MAC operator420may be executed through a GIO line. Thus, the weight data and the vector data may be independently transmitted to the MAC operator420through two separate transmission lines, respectively. In an embodiment, the weight data and the vector data may be simultaneously transmitted to the MAC operator420through the BIO line and the GIO line, respectively.

At a step386, the MAC command generator540of the PIM controller500may generate and transmit the MAC input latch signal MAC_L1to the PIM device400, as illustrated inFIG. 24. The MAC input latch signal MAC_L1may control the input latch operation of the weight data and the vector data for the MAC operator420of the PIM device400. The elements W0.0, . . . , and W0.7in the first row of the weight matrix and the elements X0.0, . . . , and X7.0in the first column of the vector matrix may be inputted to the MAC circuit122of the MAC operator420by the input latch operation. The MAC circuit122may include the plurality of multipliers (e.g., the eight multipliers122-11), the number of which is equal to the number of columns of the weight matrix and the number of rows of the vector matrix. The elements W0.0, . . . , and W0.7in the first row of the weight matrix may be inputted to the first to eighth multipliers122-11, respectively, and the elements X0.0, . . . , and X7.0in the first column of the vector matrix may also be inputted to the first to eighth multipliers122-11, respectively.

At a step387, the MAC circuit122of the MAC operator420may perform the MAC arithmetic operation of an Rthrow of the weight matrix and the first column of the vector matrix, which are inputted to the MAC circuit122. An initial value of ‘R’ may be set as ‘1’. Thus, the MAC arithmetic operation of the first row of the weight matrix and the first column of the vector matrix may be performed a first time. Specifically, each of the multipliers122-11of the multiplication logic circuit122-1may perform a multiplying calculation of the inputted data, and the result data of the multiplying calculation may be inputted to the addition logic circuit122-2. The addition logic circuit122-2may receive output data of the multipliers122-11and may perform the adding calculation of the output data of the multipliers122-11to output the result data of the adding calculation to the accumulative adder122-21D. The output data of the adder122-21C included in the addition logic circuit122-2may correspond to result data (i.e., MAC result data) of the MAC arithmetic operation of the first row included in the weight matrix and the column included in the vector matrix. The accumulative adder122-21D may add the output data MAC0.0of the adder122-21C to the bias data B0.0fed back from the output latch123-1and may output the result data of the adding calculation. The output data (i.e., the biased result data Y0.0) of the accumulative adder122-21D may be inputted to the output latch123-1disposed in the data output circuit123-A of the MAC operator420.

At a step388, the MAC command generator540of the PIM controller500may generate and transmit the MAC output latch signal MAC_L3to the PIM device400, as described with reference toFIG. 25. The MAC output latch signal MAC_L3may control the output latch operation for the output latch123-1of the MAC operator420included in the PIM device400. The output latch123-1of the MAC operator420may output the biased result data Y0.0according to the output latch operation performed in synchronization with the MAC output latch signal MAC_L3. The biased result data Y0.0outputted from the output latch123-1may be inputted to the transfer gate123-2of the data output circuit123-A.

At a step389, the MAC command generator540of the PIM controller500may generate and transmit the MAC latch reset signal MAC_L_RST to the PIM device400, as illustrated inFIG. 26. The MAC latch reset signal MAC_L_RST may control an output operation of the biased result data Y0.0generated by the MAC operator420and a reset operation of the output latch123-1included in the MAC operator420. The transfer gate123-2receiving the biased result data Y0.0from the output latch123-1of the MAC operator420may be synchronized with the MAC latch reset signal MAC_L_RST to output the biased result data Y0.0. In an embodiment, the biased result data Y0.0outputted from the MAC operator120may be stored into the memory bank411through the BIO line in the PIM device400.

At a step390, the row number ‘R’ of the weight matrix for which the MAC arithmetic operation is performed may be increased by ‘1’. Because the MAC arithmetic operation for the first row among the first to eight rows of the weight matrix has been performed at the previous steps, the row number of the weight matrix may change from ‘1’ to ‘2’ at the step390. At a step391, whether the row number changed at the step390is greater than the row number of the last row (i.e., the eighth row) of the weight matrix may be determined. Because the row number of the weight matrix is changed to ‘2’ at the step390, a process of the MAC arithmetic operation may be fed back to the step384.

If the process of the MAC arithmetic operation is fed back to the step384at the step391, the same processes as described with reference to the steps384to391may be executed again for the increased row number of the weight matrix. That is, as the row number of the weight matrix changes from ‘1’ to ‘2’, the MAC arithmetic operation may be performed for the second row of the weight matrix instead of the first row of the weight matrix with the vector matrix. If the process of the MAC arithmetic operation is fed back to the step384at the step391, then the processes from the step384to the step390may be iteratively performed until the MAC arithmetic operation is performed for all of the rows of the weight matrix with the vector matrix. If the MAC arithmetic operation for the eighth row of the weight matrix terminates and the row number of the weight matrix changes from ‘8’ to ‘9’ at the step390, then the MAC arithmetic operation may terminate because the row number of ‘9’ is greater than the last row number of ‘8’ at the step391.

FIG. 28is a flowchart illustrating processes of the MAC arithmetic operation described with reference toFIG. 17, which are performed in the PIM system1-2according to the second embodiment of the present disclosure. In order to perform the MAC arithmetic operation according to the present embodiment, the MAC operator420of the PIM device400may have the same configuration as the MAC operator120-2illustrated inFIG. 19. Referring toFIGS. 19 and 28, the first data (i.e., the weight data) may be written into the memory bank411at a step601to perform the MAC arithmetic operation. Thus, the weight data may be stored in the memory bank411of the PIM device400. In the present embodiment, it may be assumed that the weight data are the elements W0.0, . . . , and W7.7constituting the weight matrix ofFIG. 17.

At a step602, whether an inference is requested may be determined. An inference request signal may be transmitted from an external device located outside of the PIM system1-2to the PIM controller500of the PIM system1-2. In an embodiment, if no inference request signal is transmitted to the PIM controller500, the PIM system1-2may be in a standby mode until the inference request signal is transmitted to the PIM controller500. Alternatively, if no inference request signal is transmitted to the PIM controller500, the PIM system1-2may perform operations (e.g., data read/write operations) other than the MAC arithmetic operation in the memory mode until the inference request signal is transmitted to the PIM controller500. In the present embodiment, it may be assumed that the second data (i.e., the vector data) are transmitted together with the inference request signal. In addition, it may be assumed that the vector data are the elements X0.0, . . . , and X7.0constituting the vector matrix ofFIG. 17. If the inference request signal is transmitted to the PIM controller500at the step602, then the PIM controller500may write the vector data transmitted with the inference request signal into the global buffer412at a step603. Accordingly, the vector data may be stored in the global buffer412of the PIM device400.

At a step604, an output latch of a MAC operator420may be initially set to have bias data and the initially set bias data may be fed back to an accumulative adder of the MAC operator420. This process is executed to perform the matrix adding calculation of the MAC result matrix and the bias matrix, which is described with reference toFIG. 17. That is, as described with reference toFIG. 19, the output latch123-1of the data output circuit123-B included in the MAC operator420may be initially set to have the bias data of the bias matrix. Because the matrix multiplying calculation is executed for the first row of the weight matrix, the element B0.0located at first row of the bias matrix may be initially set as the bias data in the output latch123-1. The output latch123-1may output the bias data B0.0, and the bias data B0.0outputted from the output latch123-1may be inputted to the accumulative adder122-21D of the addition logic circuit122-2included in the MAC operator420.

In an embodiment, in order to output the bias data B0.0out of the output latch123-1and to feed back the bias data B0.0to the accumulative adder122-21D, the MAC command generator540of the PIM controller500may transmit the MAC output latch signal MAC_L3to the MAC operator420of the PIM device400. When a subsequent MAC arithmetic operation is performed, the accumulative adder122-21D of the MAC operator420may add the MAC result data MAC0.0outputted from the adder122-21C disposed at the last stage of the addition logic circuit122-2to the bias data B0.0which is fed back from the output latch123-1to generate the biased result data Y0.0and may output the biased result data Y0.0to the output latch123-1. The biased result data Y0.0may be outputted from the output latch123-1in synchronization with the MAC output latch signal MAC_L3transmitted in a subsequent process.

At a step605, the MAC command generator540of the PIM controller500may generate and transmit the MAC read signal MAC_RD_BK to the PIM device400, as illustrated inFIG. 23. In such a case, the address generator550of the PIM controller500may generate and transmit the row/column address ADDR_R/ADDR_C to the PIM device400. The MAC read signal MAC_RD_BK inputted to the PIM device400may control the data read operation for the memory bank411of the PIM device400. The memory bank411may output and transmit the elements W0.0, . . . , and W0.7in the first row of the weight matrix of the weight data stored in a region of the memory bank411, which is designated by the row/column address ADDR_R/ADDR_C, to the MAC operator420in response to the MAC read signal MAC_RD_BK. In an embodiment, the data transmission from the memory bank411to the MAC operator420may be executed through a BIO line which is provided specifically for data transmission between the memory bank411and the MAC operator420.

Meanwhile, the vector data X0.0, . . . , and X7.0stored in the global buffer412may also be transmitted to the MAC operator420in synchronization with a point in time when the weight data are transmitted from the memory bank411to the MAC operator420. In order to transmit the vector data X0.0, . . . , and X7.0from the global buffer412to the MAC operator420, a control signal for controlling the read operation for the global buffer412may be generated in synchronization with the MAC read signal MAC_RD_BK outputted from the MAC command generator540of the PIM controller500. The data transmission between the global buffer412and the MAC operator420may be executed through a GIO line. Thus, the weight data and the vector data may be independently transmitted to the MAC operator420through two separate transmission lines, respectively. In an embodiment, the weight data and the vector data may be simultaneously transmitted to the MAC operator420through the BIO line and the GIO line, respectively.

At a step606, the MAC command generator540of the PIM controller500may generate and transmit the MAC input latch signal MAC_L1to the PIM device400, as described with reference toFIG. 24. The MAC input latch signal MAC_L1may control the input latch operation of the weight data and the vector data for the MAC operator420of the PIM device400. The elements W0.0, . . . , and W0.7in the first row of the weight matrix and the elements X0.0, . . . , and X7.0in the first column of the vector matrix may be inputted to the MAC circuit122of the MAC operator420by the input latch operation. The MAC circuit122may include the plurality of multipliers (e.g., the eight multipliers122-11), the number of which is equal to the number of columns of the weight matrix and the number of rows of the vector matrix. The elements W0.0, . . . , and W0.7in the first row of the weight matrix may be inputted to the first to eighth multipliers122-11, respectively, and the elements X0.0, . . . , and X7.0in the first column of the vector matrix may also be inputted to the first to eighth multipliers122-11, respectively.

At a step607, the MAC circuit122of the MAC operator420may perform the MAC arithmetic operation of an Rthrow of the weight matrix and the first column of the vector matrix, which are inputted to the MAC circuit122. An initial value of ‘R’ may be set as ‘1’. Thus, the MAC arithmetic operation of the first row of the weight matrix and the first column of the vector matrix may be performed a first time. Specifically, each of the multipliers122-11of the multiplication logic circuit122-1may perform a multiplying calculation of the inputted data, and the result data of the multiplying calculation may be inputted to the addition logic circuit122-2. The addition logic circuit122-2may receive output data of the multipliers122-11and may perform the adding calculation of the output data of the multipliers122-11to output the result data of the adding calculation to the accumulative adder122-21D. The output data of the adder122-21C included in the addition logic circuit122-2may correspond to result data (i.e., the MAC result data MAC0.0) of the MAC arithmetic operation of the first row included in the weight matrix and the column included in the vector matrix. The accumulative adder122-21D may add the output data MAC0.0of the adder122-21C to the bias data B0.0fed back from the output latch123-1and may output the result data of the adding calculation. The output data (i.e., the biased result data Y0.0) of the accumulative adder122-21D may be inputted to the output latch123-1disposed in the data output circuit123-A of the MAC operator420.

At a step608, the MAC command generator540of the PIM controller500may generate and transmit the MAC output latch signal MAC_L3to the PIM device400, as described with reference toFIG. 25. The MAC output latch signal MAC_L3may control the output latch operation for the output latch123-1of the MAC operator420included in the PIM device400. The output latch123-1of the MAC operator420may output the biased result data Y0.0according to the output latch operation performed in synchronization with the MAC output latch signal MAC_L3. The biased result data Y0.0outputted from the output latch123-1may be inputted to the activation function logic circuit123-5, which is illustrated inFIG. 19. At a step610, the activation function logic circuit123-5may apply an activation function to the biased result data Y0.0to generate a final output value, and the final output value may be inputted to the transfer gate (123-2ofFIG. 4).

At a step610, the MAC command generator540of the PIM controller500may generate and transmit the MAC latch reset signal MAC_L_RST to the PIM device400, as described with reference toFIG. 26. The MAC latch reset signal MAC_L_RST may control an output operation of the final output value generated by the MAC operator420and a reset operation of the output latch123-1included in the MAC operator420. The transfer gate123-2receiving the final output value from the activation function logic circuit123-5of the data output circuit123-B included in the MAC operator420may be synchronized with the MAC latch reset signal MAC_L_RST to output the final output value. In an embodiment, the final output value outputted from the MAC operator420may be stored into the memory bank411through the BIO line in the PIM device400.

At a step611, the row number ‘R’ of the weight matrix for which the MAC arithmetic operation is performed may be increased by ‘1’. Because the MAC arithmetic operation for the first row among the first to eight rows of the weight matrix has been performed at the previous steps, the row number of the weight matrix may change from ‘1’ to ‘2’ at the step611. At a step612, whether the row number changed at the step611is greater than the row number of the last row (i.e., the eighth row) of the weight matrix may be determined. Because the row number of the weight matrix is changed to ‘2’ at the step611, a process of the MAC arithmetic operation may be fed back to the step604.

If the process of the MAC arithmetic operation is fed back to the step604from the step612, the same processes as described with reference to the steps604to612may be executed again for the increased row number of the weight matrix. That is, as the row number of the weight matrix changes from ‘1’ to ‘2’, the MAC arithmetic operation may be performed for the second row of the weight matrix instead of the first row of the weight matrix with the vector matrix to generate the MAC result data (corresponding to the element MAC1.0located in the second row of the MAC result matrix) and the bias data (corresponding to the element B1.0located in the second row of the bias matrix). If the process of the MAC arithmetic operation is fed back to the step604from the step612, the processes from the step604to the step612may be iteratively performed until the MAC arithmetic operation is performed for all of the rows (i.e., first to eighth rows) of the weight matrix with the vector matrix. If the MAC arithmetic operation for the eighth row of the weight matrix terminates and the row number of the weight matrix changes from ‘8’ to ‘9’ at the step611, the MAC arithmetic operation may terminate because the row number of ‘9’ is greater than the last row number of ‘8’ at the step612.

FIG. 29is a block diagram illustrating a PIM system1-3according to a third embodiment of the present disclosure. As illustrated inFIG. 29, the PIM system1-3may have substantially the same configuration as the PIM system1-1illustrated inFIG. 2except that a PIM controller200A of the PIM system1-3further includes a mode register set (MRS)260as compared with the PIM controller200of the PIM system1-1. Thus, the same explanation as described with reference toFIG. 2will be omitted hereinafter. The mode register set260in the PIM controller200A may receive an MRS signal instructing arrangement of various signals necessary for the MAC arithmetic operation of the PIM system1-3. In an embodiment, the mode register set260may receive the MRS signal from the mode selector221included in the scheduler220. However, in another embodiment, the MRS signal may be provided by an extra logic circuit other than the mode selector221. The mode register set260receiving the MRS signal may transmit the MRS signal to the MAC command generator240. For an embodiment, the MRS260represents a MRS circuit.

In an embodiment, the MRS signal may include timing information on when the MAC commands MAC_CMDs are generated. In such a case, the deterministic operation of the PIM system1-3may be performed by the MRS signal provided by the MRS260. In another embodiment, the MRS signal may include information on the timing related to an interval between the MAC modes or information on a mode change between the MAC mode and the memory mode. In an embodiment, generation of the MRS signal in the MRS260may be executed before the vector data are stored in the second memory bank112of the PIM device100by the inference request signal transmitted from an external device to the PIM controller200A. Alternatively, the generation of the MRS signal in the MRS260may be executed after the vector data are stored in the second memory bank112of the PIM device100by the inference request signal transmitted from an external device to the PIM controller200A.

FIG. 30is a block diagram illustrating a PIM system1-4according to a fourth embodiment of the present disclosure. As illustrated inFIG. 30, the PIM system1-4may have substantially the same configuration as the PIM system1-2illustrated inFIG. 20except that a PIM controller500A of the PIM system1-4further includes the mode register set (MRS)260as compared with the PIM controller500of the PIM system1-2. Thus, the same explanation as described with reference toFIG. 20will be omitted hereinafter. The mode register set260in the PIM controller500A may receive an MRS signal instructing arrangement of various signals necessary for the MAC arithmetic operation of the PIM system1-4. In an embodiment, the mode register set260may receive the MRS signal from the mode selector221included in the scheduler220. However, in another embodiment, the MRS signal may be provided by an extra logic circuit other than the mode selector221. The mode register set260receiving the MRS signal may transmit the MRS signal to the MAC command generator540.

In an embodiment, the MRS signal may include timing information on when the MAC commands MAC_CMDs are generated. In such a case, the deterministic operation of the PIM system1-4may be performed by the MRS signal provided by the MRS260. In another embodiment, the MRS signal may include information on the timing related to an interval between the MAC modes or information on a mode change between the MAC mode and the memory mode. In an embodiment, generation of the MRS signal in the MRS260may be executed before the vector data are stored in the global buffer412of the PIM device400by the inference request signal transmitted from an external device to the PIM controller500A. Alternatively, the generation of the MRS signal in the MRS260may be executed after the vector data are stored in the global buffer412of the PIM device400by the inference request signal transmitted from an external device to the PIM controller500A.

FIG. 31is a block diagram illustrating a PIM device400A according to an embodiment of the present disclosure. Referring toFIG. 31, the PIM device400A may include “L”-number of memory banks (i.e., first to Lthmemory banks BK(0)˜BK(L−1)), a global buffer GB, “L”-number of MAC operators (i.e., first to L MAC operators MAC(0)˜MAC(L−1)), and a command/address decoder450(where, “L” is a natural number which is equal to or greater than two). In an embodiment, the first to Lthmemory banks BK(0)˜BK(L−1) may correspond to the first storage region of the data storage region11included in the PIM device10illustrated inFIG. 1, and the global buffer GB may correspond to the second storage region of the data storage region11included in the PIM device10illustrated inFIG. 1. The first to LthMAC operators MAC(0)˜MAC(L−1) may constitute the MAC operator120of the PIM device10illustrated inFIG. 1.

In an embodiment, the PIM device400of the PIM system1-2described with reference toFIG. 20may be replaced with the PIM device400A according to the present embodiment. The first to Lthmemory banks BK(0)˜BK(L−1) and the first to LthMAC operators MAC(0)˜MAC(L−1) may constitute first to LthMAC units. For example, the first memory bank BK(0) and the first MAC operator MAC(0) may constitute the first MAC unit. The MAC operator MAC constituting a certain MAC unit may receive weight data used for the MAC arithmetic operation from the memory bank BK constituting the certain MAC unit. For example, the first MAC operator MAC(0) may receive the weight data from the first memory bank BK(0).

The global buffer GB may be configured to output vector data used for the MAC arithmetic operation to the first to LthMAC operators MAC(0)˜MAC(L−1). In order that the global buffer GB outputs the vector data to the first to LthMAC operators MAC(0)˜MAC(L−1), the global buffer GB may receive the vector data through a controller according to a request outputted from a host and may store the vector data therein. In an embodiment, the vector data may be transmitted from the global buffer GB to the MAC operators MAC(0)˜MAC(L−1) through a global input/output (GIO) line. The vector data outputted from the global buffer GB may be transmitted to all of the MAC operators MAC(0)˜MAC(L−1).

The command/address decoder450may receive a command CMD and an address ADDR from an external device such as a controller. The command/address decoder450may decode the command CMD and the address ADDR to generate and output control signals for controlling operations of the memory banks BK(0)˜BK(L−1), the global buffer GB, and the MAC operators MAC(0)˜MAC(L−1) as well as an address signal ADDR_S. The control signals may include a read signal RD, a write signal WT, a MAC signal MAC, a result read signal RD_RST, an update signal UPDATE, and an accumulation latch selection signal ALS. The read signal RD may control a read operation of the memory banks BK(0)˜BK(L−1) and the global buffer GB, and the write signal WT may control a write operation of the memory banks BK(0)˜BK(L−1) and the global buffer GB. The MAC signal MAC may control the MAC arithmetic operation of the MAC operators MAC(0)˜MAC(L−1). The result read signal RD_RST may control an operation for outputting MAC result data of the MAC operators MAC(0)˜MAC(L−1). The update signal UPDATE and the accumulation latch selection signal ALS may control a latch operation of an accumulator included in each of the MAC operators MAC(0)˜MAC(L−1).

FIG. 32illustrates an example of a matrix multiplying calculation executed by the PIM device400A illustrated inFIG. 31. The matrix multiplying calculation according to the present embodiment may be different from the matrix multiplying calculation described with reference toFIG. 5in terms of the number of MAC operators executing the matrix multiplying calculation. However, fundamentally, the matrix multiplying calculation described with reference toFIG. 5may be equally applicable to the matrix multiplying calculation according to the present embodiment.

Specifically, referring toFIG. 32, the MAC operators MAC(0)˜MAC(L−1) may perform the matrix multiplying calculation of a weight matrix and a vector matrix to generate a result matrix corresponding to a result of the MAC arithmetic operation. Hereinafter, it may be assumed that the weight matrix has 32 rows and 32 columns. That is, the weight matrix may have 32 matrix rows (i.e., first to 32ndmatrix rows MR1˜MR32) and 32 matrix columns (i.e., first to 32ndmatrix columns MC1˜MC32). Accordingly, the weight matrix have “32×32”-number of elements W1.1, . . . , W1.32, . . . , W32.1, . . . , and W32.32. The elements W1.1, . . . , W1.32, . . . , W32.1, . . . , and W32.32of the weight matrix may correspond to weight data. The vector matrix may have 32 elements V1, . . . , and V32. The elements V1, . . . , and V32of the vector matrix may correspond to vector data. The result matrix may have 32 elements MAC_RST1, . . . , and MAC_RST32. The elements MAC_RST1, . . . , and MAC_RST32of the result matrix may correspond to MAC result data. Hereinafter, it may be assumed that the terms “elements of the weight matrix”, “elements of the vector matrix”, and “elements of the result matrix” have the same meanings, respectively, as the terms “weight data”, “vector data”, and “MAC result data”. In an embodiment, each of the weight data W1.1˜W32.32and the vector data V1˜V32may be a multibit binary stream, for example, a 16-bit binary stream.

FIG. 33illustrates a process for storing the weight data W1.1˜W32.32included in the weight matrix illustrated inFIG. 32into the memory banks BK(0)˜BK(7) of the PIM device400A illustrated inFIG. 31. Hereinafter, it may be assumed that the PIM device400A includes eight memory banks, for example, first to eighth memory banks BK(0)˜BK(7). Referring toFIGS. 32 and 33, the weight data W1.1˜W32.32may be stored into the first to eighth memory banks BK(0)˜BK(7) in units of matrix rows MR. In the present embodiment, the weight data arrayed in one of the first to 32ndmatrix rows MR1˜MR32may be stored into any one of rows ROW in the first to eighth memory banks BK(0)˜BK(7). However, the present embodiment may be merely an example of the present disclosure, and the present disclosure is not limited to the present embodiment. Accordingly, in some other embodiments, the weight data arrayed in two or more rows of the first to 32ndmatrix rows MR1˜MR32may be stored into any one of rows ROW in the first to eighth memory banks BK(0)˜BK(7).

Specifically, the weight data W1.1˜W1.32, . . . , and W8.1˜W8.32arrayed in respective ones of the first to eighth matrix rows MR1˜MR8may be stored into first rows ROW0of the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the weight data W1.1˜W1.32in the first matrix row MR1may be stored into the first row ROW0of the first memory bank BK(0), and the weight data W2.1˜W2.32in the second matrix row MR2may be stored into the first row ROW0of the second memory bank BK(1). In addition, the weight data W3.1˜W3.32in the third matrix row MR3may be stored into the first row ROW0of the third memory bank BK(2), and the weight data W4.1˜W4.32in the fourth matrix row MR4may be stored into the first row ROW0of the fourth memory bank BK(3). Similarly, the weight data W8.1˜W8.32in the eighth matrix row MR8may be stored into the first row ROW0of the eighth memory bank BK(7).

The weight data W9.1˜W32.32arrayed in the ninth to 32ndmatrix rows MR9˜MR32of the weight matrix may also be stored into the first to eighth memory banks BK(0)˜BK(7) in the same way as the weight data W1.1˜W8.32are stored into the first to eighth memory banks BK(0)˜BK(7). Thus, the weight data W9.1˜W9.32, . . . , and W16.1˜W16.32arrayed in the ninth to sixteenth matrix rows MR9˜MR16of the weight matrix may be stored into second rows ROW1of the first to eighth memory banks BK(0)˜BK(7), and the weight data W17.1˜W17.32, . . . , and W24.1˜W24.32arrayed in the 17thto 24thmatrix rows MR17˜MR24of the weight matrix may be stored into third rows ROW2of the first to eighth memory banks BK(0)˜BK(7). In addition, the weight data W25.1˜W25.32, . . . , and W32.1˜W32.32arrayed in the 25thto 32ndmatrix rows MR25-MR32of the weight matrix may be stored into fourth rows ROW3of the first to eighth memory banks BK(0)˜BK(7).

The MAC arithmetic operation performed by each of the first to eighth MAC operators MAC(0)˜MAC(7) of the PIM device400A may include the multiplying calculations performed by the multipliers122-11included inFIG. 4. Thus, a size of the weight data which are capable of being transmitted from the first to eighth memory banks BK(0)˜BK(7) to the first to eighth MAC operators MAC(0)˜MAC(7) may be limited by a unit MAC arithmetic amount of each of the first to eighth MAC operators MAC(0)˜MAC(7). Similarly, a size of the vector data which are capable of being transmitted from the global buffer GB to the first to eighth MAC operators MAC(0)˜MAC(7) may also be limited by the unit MAC arithmetic amount of each of the first to eighth MAC operators MAC(0)˜MAC(7). The term “unit MAC arithmetic amount” may be defined as a size (i.e., the number of bits) of the weight data (or the vector data) which are capable of being processed by the multipliers included in each of the MAC operators BK(0)˜BK(7). The unit MAC arithmetic amount may be less than the total number of bits of all of the weight data stored in any one of the rows included in each of the memory banks BK(0)˜BK(7). In such a case, the MAC arithmetic operation for the weight data stored in first, second, third, or fourth rows ROW0, ROW1, ROW2, or ROW3) of the first to eighth memory banks BK(0)˜BK(7) cannot be performed by a single MAC calculation. That is, in order to complete the MAC arithmetic operation for the weight data stored in first, second, third, or fourth rows ROW0, ROW1, ROW2, or ROW3) of the first to eighth memory banks BK(0)˜BK(7), it may be necessary to successively perform a plurality of MAC calculations. Hereinafter, it may be assumed that the unit MAC arithmetic amount is 256 bits (corresponding to 16 sets of 16-bit weight data). In such a case, each of the first to eighth MAC operators MAC(0)˜MAC(7) may include 16 multipliers, and each of the multipliers may perform a multiplying calculation of 16-bit weight data (one set of weight data) and 16-bit vector data (one set of vector data).

FIG. 34illustrates a process for dividing the weight matrix and the vector matrix illustrated inFIG. 32into a plurality of weight sub-matrixes WSMs and a plurality of vector sub-matrixes VSMs. Referring toFIG. 34, the weight matrix and the vector matrix, which are used for the MAC calculation of the PIM device400A, may be divided into the plurality of weight sub-matrixes WSMs and the plurality of vector sub-matrixes VSMs according to the number of the memory banks BKs and the unit MAC arithmetic amount. That is, the weight matrix may be configured to employ the plurality of weight sub-matrixes WSMs as its elements, and the vector matrix may be configured to employ the plurality of vector sub-matrixes VSMs as its elements. In such a case, the weight matrix may have a plurality of weight matrix group rows WMGRs and a plurality of weight matrix group columns WMGCs. Meanwhile, the vector matrix may have only a plurality of vector matrix group rows VMGRs.

The number of the plurality of weight matrix group rows WMGRs in the weight matrix may be determined by the number of memory banks BKs. Meanwhile, the number of the plurality of weight matrix group columns WMGCs in the weight matrix may be determined by the unit MAC arithmetic amount. In an embodiment, the number of the plurality of weight matrix group rows WMGRs in the weight matrix may be determined by dividing the number of the matrix rows MRs of the weight matrix into the number of the memory banks BKs. The number of the plurality of weight matrix group columns WMGCs in the weight matrix may be determined by dividing the number of the matrix columns MCs of the weight matrix into the number of sets of the weight data corresponding to the unit MAC arithmetic amount. When the number of the matrix rows MR1˜MR32of the weight matrix is 32 and the number of the memory banks BK(0)˜BK(7) is 8 as in the present embodiment, the weight matrix may have four (32/8) weight matrix group rows (i.e., first to fourth weight matrix group rows WMGR1˜WMGR4). In addition, when the number of the matrix columns MC1-MC32of the weight matrix is 32 and the number of the sets of the weight data corresponding to the unit MAC arithmetic amount is 16 as in the present embodiment, the weight matrix may have two (32/16) weight matrix group columns (i.e., first and second weight matrix group columns WMGC1and WMGC2).

Two weight sub-matrixes WSM11and WSM12belonging to the first weight matrix group row WMGR1may include the weight data W1.1˜W1.32, . . . , and W8.1˜W8.32arrayed in the first to eighth matrix rows MR1˜MR8of the weight matrix. As described with reference toFIG. 33, the weight data W1.1˜W1.32, . . . , and W8.1˜W8.32arrayed in first to eighth rows of the weight sub-matrixes WSM11and WSM12may be stored into the first rows ROW0of the first to eighth memory banks BK(0)˜BK(7). Two weight sub-matrixes WSM21and WSM22belonging to the second weight matrix group row WMGR2may include the weight data W9.1˜W9.32, . . . , and W16.1˜W16.32arrayed in the ninth to sixteenth matrix rows MR9˜MR16of the weight matrix. As described with reference toFIG. 33, the weight data W9.1˜W9.32, . . . , and W16.1˜W16.32arrayed in first to eighth rows of the weight sub-matrixes WSM21and WSM22may be stored into the second rows ROW1of the first to eighth memory banks BK(0)˜BK(7).

Two weight sub-matrixes WSM31and WSM32belonging to the third weight matrix group row WMGR3may include the weight data W17.1˜W17.32, . . . , and W24.1˜W24.32arrayed in the seventeenth to twenty fourth matrix rows MR17˜MR24of the weight matrix. The weight data W17.1˜W17.32, . . . , and W24.1˜W24.32arrayed in first to eighth rows of the weight sub-matrixes WSM31and WSM32may be stored into the third rows ROW2of the first to eighth memory banks BK(0)˜BK(7). Two weight sub-matrixes WSM41and WSM42belonging to the fourth weight matrix group row WMGR4may include the weight data W25.1˜W25.32, . . . , and W32.1˜W32.32arrayed in the 25thto 32ndmatrix rows MR25˜MR32of the weight matrix. The weight data W25.1˜W25.32, . . . , and W32.1˜W32.32arrayed in first to eighth rows of the weight sub-matrixes WSM41and WSM42may be stored into the fourth rows ROW3of the first to eighth memory banks BK(0)˜BK(7).

The four weight sub-matrixes WSM11, WSM21, WSM31, and WSM41belonging to the first weight matrix group column WMGC1may include the weight data W1.1˜W1.16, . . . , and W32.1˜W32.16arrayed in the first to sixteenth matrix columns MC1˜MC16of the weight matrix. The four weight sub-matrixes WSM12, WSM22, WSM32, and WSM42belonging to the second weight matrix group column WMGC2may include the weight data W1.17˜W1.32, . . . , and W32.17˜W32.32arrayed in the seventeenth to 32ndmatrix columns MC17˜MC32of the weight matrix.

The weight data W1.1˜W1.32, . . . , and W8.1˜W8.32arrayed in the first to eighth rows of the two weight sub-matrixes WSM11and WSM12belonging to the first weight matrix group row WMGR1may be used for first MAC arithmetic operations of the first to eighth MAC operators MAC(0)˜MAC(7), as described with reference toFIG. 33. Specifically, the weight data W1.1˜W1.32, which are arrayed in the first matrix row MR1, among the weight data W1.1˜W1.32, . . . , and W8.1˜W8.32may be used for performing the first MAC arithmetic operation of the first MAC operator MAC(0). The weight data W2.1˜W2.32, which are arrayed in the second matrix row MR2, among the weight data W1.1˜W1.32, . . . , and W8.1˜W8.32may be used for performing the first MAC arithmetic operation of the second MAC operator MAC(1). Similarly, the weight data W8.1˜W8.32, which are arrayed in the eighth matrix row MR8, among the weight data W1.1˜W1.32, . . . , and W8.1˜W8.32may be used for performing the first MAC arithmetic operation of the eighth MAC operator MAC(7).

The weight data W9.1˜W9.32, . . . , and W16.1˜W16.32included in the two weight sub-matrixes WSM21and WSM22belonging to the second weight matrix group row WMGR2may be used for second MAC arithmetic operations of the first to eighth MAC operators MAC(0)˜MAC(7), as described with reference toFIG. 33. Specifically, the weight data W9.1˜W9.32, which are arrayed in the ninth matrix row MR9, among the weight data W9.1˜W9.32, . . . , and W16.1˜W16.32may be used for performing the second MAC arithmetic operation of the first MAC operator MAC(0). The weight data W10.1˜W10.32, which are arrayed in the tenth matrix row MR10, among the weight data W9.1˜W9.32, . . . , and W16.1˜W16.32may be used for performing the second MAC arithmetic operation of the second MAC operator MAC(1). Similarly, the weight data W16.1˜W16.32, which are arrayed in the sixteenth matrix row MR16, among the weight data W9.1˜W9.32, . . . , and W16.1˜W16.32may be used for performing the second MAC arithmetic operation of the eighth MAC operator MAC(7).

Moreover, the weight data W17.1˜W17.32, . . . , and W24.1˜W24.32included in the two weight sub-matrixes WSM31and WSM32belonging to the third weight matrix group row WMGR3may be used for third MAC arithmetic operations of the first to eighth MAC operators MAC(0)˜MAC(7). Furthermore, the weight data W25.1˜W25.32, . . . , and W32.1˜W32.32included in the two weight sub-matrixes WSM41and WSM42belonging to the fourth weight matrix group row WMGR4may be used for fourth MAC arithmetic operations of the first to eighth MAC operators MAC(0)˜MAC(7).

The number of the vector matrix group rows VMGRs in the vector matrix may be determined by dividing the dividing the number of the matrix rows MRs of the vector matrix into the number of sets of the vector data corresponding to the unit MAC arithmetic amount. When the number of the matrix rows MR1˜MR32of the vector matrix is 32 and the number of the sets of the vector data corresponding to the unit MAC arithmetic amount is 16 as in the present embodiment, the vector matrix may have two vector matrix group rows (i.e., first and second vector matrix group rows VMGR1and VMGR2). That is, the vector matrix may be configured to employ a first vector sub-matrix VSM11in the first vector matrix group row VMGR1and a second vector sub-matrix VSM21in the second vector matrix group row VMGR2as its elements. The number of the vector matrix group rows VMGRs may be equal to the number of the weight matrix group columns WMGCs. The first vector sub-matrix VSM11of the vector matrix may include the vector data V1˜V16arrayed in first to sixteenth rows of the first vector sub-matrix VSM11(i.e., the first to sixteenth matrix rows MR1˜MR16of the vector matrix). The second vector sub-matrix VSM21of the vector matrix may include the vector data V17˜V32arrayed in first to sixteenth rows of the second vector sub-matrix VSM21(i.e., the seventeenth to 32ndmatrix rows MR17˜MR32of the vector matrix).

The first to eighth MAC operators MAC(0)˜MAC(7) of the PIM device400A according to the present disclosure may perform the MAC arithmetic operations by executing the matrix multiplying calculations for the weight sub-matrixes WSMs of the weight matrix and the vector sub-matrixes VSMs of the vector matrix. In an embodiment, the first to eighth MAC operators MAC(0)˜MAC(7) may perform the first MAC arithmetic operations in units of matrix group columns by executing the matrix multiplying calculations for the weight sub-matrixes WSM11, WSM21, WSM31, and WSM41in the first weight matrix group column WMGC1and the vector sub-matrix VSM11in the first vector matrix group row VMGR1. Next, the first to eighth MAC operators MAC(0)˜MAC(7) may perform the second MAC arithmetic operations in units of matrix group columns by executing the matrix multiplying calculations for the weight sub-matrixes WSM12, WSM22, WSM32, and WSM42in the second weight matrix group column WMGC2and the vector sub-matrix VSM21in the second vector matrix group row VMGR2.

Specifically, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a first MAC arithmetic operation of a first matrix group column unit for the weight sub-matrix WSM11, which is located at a cross point of the first weight matrix group row WMGR1and the first weight matrix group column WMGC1, and the vector sub-matrix VSM11located in the first vector matrix group row VMGR1. Next, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a first MAC arithmetic operation of a second matrix group column unit for the weight sub-matrix WSM21, which is located at a cross point of the second weight matrix group row WMGR2and the first weight matrix group column WMGC1, and the vector sub-matrix VSM11located in the first vector matrix group row VMGR1.

Subsequently, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a first MAC arithmetic operation of a third matrix group column unit for the weight sub-matrix WSM31, which is located at a cross point of the third weight matrix group row WMGR3and the first weight matrix group column WMGC1, and the vector sub-matrix VSM11located in the first vector matrix group row VMGR1. Next, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a first MAC arithmetic operation of a fourth matrix group column unit for the weight sub-matrix WSM41, which is located at a cross point of the fourth weight matrix group row WMGR4and the first weight matrix group column WMGC1, and the vector sub-matrix VSM11located in the first vector matrix group row VMGR1. As such, the first MAC arithmetic operations of the matrix group column unit may be completed by sequentially performing the first MAC arithmetic operations of the first to fourth matrix group column units. While the first MAC arithmetic operations of the matrix group column unit are performed, the first MAC arithmetic operations may be performed using different weight sub-matrixes and the same vector sub-matrix VSM11.

After the first MAC arithmetic operations of the matrix group column unit are completed, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a second MAC arithmetic operation of the first matrix group column unit for the weight sub-matrix WSM12, which is located at a cross point of the first weight matrix group row WMGR1and the second weight matrix group column WMGC2, and the vector sub-matrix VSM21located in the second vector matrix group row VMGR2. Next, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a second MAC arithmetic operation of the second matrix group column unit for the weight sub-matrix WSM22, which is located at a cross point of the second weight matrix group row WMGR2and the second weight matrix group column WMGC2, and the vector sub-matrix VSM21located in the second vector matrix group row VMGR2.

Subsequently, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a second MAC arithmetic operation of the third matrix group column unit for the weight sub-matrix WSM32, which is located at a cross point of the third weight matrix group row WMGR3and the second weight matrix group column WMGC2, and the vector sub-matrix VSM21located in the second vector matrix group row VMGR2. Next, the first to eighth MAC operators MAC(0)˜MAC(7) may perform a second MAC arithmetic operation of the fourth matrix group column unit for the weight sub-matrix WSM42, which is located at a cross point of the fourth weight matrix group row WMGR4and the second weight matrix group column WMGC2, and the vector sub-matrix VSM21located in the second vector matrix group row VMGR2. As such, the second MAC arithmetic operations of the matrix group column unit may be completed by sequentially performing the second MAC arithmetic operations of the first to fourth matrix group column units. While the second MAC arithmetic operations of the matrix group column unit are performed, the second MAC arithmetic operations may be performed using different weight sub-matrixes and the same vector sub-matrix VSM21. That is, while all of the MAC arithmetic operations are performed, the vector sub-matrix VSM may be changed only when the MAC arithmetic operation is shifted from the first MAC arithmetic operation to the second MAC arithmetic operation or vice versa.

FIG. 35is a block diagram illustrating an example of a configuration of the first MAC operator MAC(0) included in the PIM device400A illustrated inFIG. 31. The configuration of the first MAC operator MAC(0) may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7). Referring toFIG. 35, the first MAC operator MAC(0) may include a plurality of multipliers (e.g., first to sixteenth multipliers MUL0˜MUL15), an adder tree including a plurality of adders, an accumulator1220, and an output circuit1230.

Each of the first to sixteenth multipliers MUL0˜MUL15may receive one of weight data W1˜W16(corresponding to the weight data W1.1˜W1.16of the weight sub-matrix WSM11illustrated inFIG. 34) from the first memory bank BK(0) and may also receive one of the vector data V1˜V16included in the vector sub-matrix VSM11illustrated inFIG. 34from the global buffer GB. Each of the first to sixteenth multipliers MUL0˜MUL15may perform a multiplying calculation of the weight data W and the vector data V to generate and output multiplication result data DM. For example, the first multiplier MUL0may perform a multiplying calculation of the weight data W1and the vector data V1to generate and output first multiplication result data DM0, and the second multiplier MUL1may perform a multiplying calculation of the weight data W2and the vector data V2to generate and output second multiplication result data DM1. Similarly, the sixteenth multiplier MUL15may perform a multiplying calculation of the weight data W16and the vector data V16to generate and output sixteenth multiplication result data DM15.

The adder tree may include a plurality of adders which are arrayed to have a hierarchical structure such as a tree structure. In the present embodiment, the adder tree may include half-adders. However, the present embodiment may be merely an example of the present disclosure. Thus, in some other embodiments, the adder tree may include full-adders. In the present embodiment, eight adders may be disposed in a first stage located at a highest level of the adder tree, and four adders may be disposed in a second stage located at a second highest level of the adder tree. In addition, two adders may be disposed in a third stage located at a third highest level of the adder tree, and one adder may be disposed in a fourth stage located at a lowest level of the adder tree.

Each of the adders disposed in the first stage may perform an adding calculation of two sets of multiplication result data DMs which are outputted from two multipliers among the first to sixteenth multipliers MUL0˜MUL5, thereby generating and outputting addition result data. For example, a first adder of the eight adders in the first stage may perform an adding calculation of the first multiplication result data DM0outputted from the first multiplier MUL0and the second multiplication result data DM1outputted from the second multiplier MUL1, thereby generating and outputting addition result data. In addition, each of the adders disposed in the second stage may perform an adding calculation of two sets of addition result data which are outputted from two adders among the eight adders disposed in the first stage, thereby generating and outputting addition result data. In the same way, the adder disposed in the fourth stage may perform an adding calculation of two sets of addition result data which are outputted from the two adders disposed in the third stage, thereby generating and outputting addition result data DMA of the adder tree.

The accumulator1220may receive the addition result data DMA from the adder tree to perform an accumulative adding calculation. In order to perform the accumulative adding calculation of the accumulator1220, the accumulator1220may include an accumulative adder and a latch circuit. The accumulative adder may perform an accumulative adding calculation of the addition result data DMA outputted from the adder tree and feedback data outputted from the latch circuit. The latch circuit may latch output data of the accumulative adder. The latched data of the latch circuit may be fed back to the accumulative adder to be used as the feedback data. In addition, the latched data of the latch circuit may be transmitted to the output circuit1230. An operation of the latch circuit included in the accumulator1220may be controlled by the update signal UPDATE and the accumulation latch selection signal ALS which are outputted from the command/address decoder (450ofFIG. 31). The accumulator1220will be described in more detail with reference toFIG. 36later.

The output circuit1230may receive the output data of the latch circuit included in the accumulator1220. The output circuit1230may output the output data of the accumulator1220as MAC result data MAC_RST which is transmitted to an external device of the first MAC operator MAC(0). In an embodiment, the MAC result data MAC_RST outputted from the output circuit1230may be transmitted to the memory banks BK(0)˜BK(7) or the global buffer GB. In another embodiment, the MAC result data MAC_RST outputted from the output circuit1230may be transmitted to a host through an external device (e.g., a controller) coupled to the PIM device400A. An operation for outputting the MAC result data MAC_RST from the output circuit1230may be performed in response to the result read signal RD_RST which is outputted from the command/address decoder (450ofFIG. 31).

FIG. 36is a circuit diagram illustrating an example of a configuration of the accumulator1220included in the first MAC operator MAC(0) illustrated inFIG. 35. Referring toFIG. 36, the accumulator1220may include an accumulative adder1221, a latch circuit1222, a latch circuit selector1223, and an input selector1224. The accumulative adder1221may have a first input terminal, a second input terminal, and an output terminal. The accumulative adder1221may receive the addition result data DMA from the adder tree through the first input terminal. The accumulative adder1221may receive the feedback data from the input selector1224through the second input terminal. That is, the accumulative adder1221may receive previous accumulated addition data DMACC0, which are outputted from the latch circuit1222, through the second input terminal. The accumulative adder1221may perform an adding calculation of the addition result data DMA and the previous accumulated addition data DMACC0to generate and output the addition result as current accumulated addition data DMACC1through the output terminal. The current accumulated addition data DMACC1outputted from the accumulative adder1221may be transmitted to the latch circuit1222.

The latch circuit1222may include a plurality of latch circuits, for example, first to fourth latch circuits FF1˜FF4. The number of the latch circuits may be equal to the number of the weight matrix group rows WMGR1˜WMGR4described with reference toFIG. 34. In an embodiment, each of the first to fourth latch circuits FF1˜FF4may be realized using a flip-flop having a latch function. All of input terminals of the first to fourth latch circuits FF1˜FF4may be coupled to the output terminal of the accumulative adder1221. Thus, the current accumulated addition data DMACC outputted from the accumulative adder1221may be transmitted to all of the first to fourth latch circuits FF1˜FF4. One of the first to fourth latch circuits FF1˜FF4may latch the current accumulated addition data DMACC1, which are outputted from the accumulative adder1221, in response to a first logic level signal having a logic “high” level inputted to a clock terminal and may output the latched data of the current accumulated addition data DMACC1through an output terminal Q. Selecting one of the first to fourth latch circuits FF1˜FF4to latch and output the current accumulated addition data DMACC1may be achieved by output signals of the latch circuit selector1223.

The latch circuit selector1223may include an output selector1223A and first to fourth AND gates1223B˜1223E. The output selector1223A may have an input terminal IN, first to fourth output terminals OUT1˜OUT4, and a selection control terminal S1. In an embodiment, the output selector1223A may be realized using a 1-to-4 demultiplexer. A logic high level signal HI may be inputted to the input terminal IN of the output selector1223A. The accumulation latch selection signal ALS[1:0] corresponding to a selection control signal may be inputted to the selection control terminal S1of the output selector1223A. In such a case, the output selector1223A may output the logic high level signal HI through one of the first to fourth output terminals OUT1˜OUT4, which is selected by the accumulation latch selection signal ALS[1:0], and the output selector1223A may output a logic low level signal LO through the remaining non-selected output terminals. In an embodiment, the output selector1223A may output the logic high level signal HI through the first output terminal OUT1when the accumulation latch selection signal ALS[1:0] has a logic level combination of “00”, and the output selector1223A may output the logic high level signal HI through the second output terminal OUT2when the accumulation latch selection signal ALS[1:0] has a logic level combination of “01”. Moreover, the output selector1223A may output the logic high level signal HI through the third output terminal OUT3when the accumulation latch selection signal ALS[1:0] has a logic level combination of “10”, and the output selector1223A may output the logic high level signal HI through the fourth output terminal OUT4when the accumulation latch selection signal ALS[1:0] has a logic level combination of “11”.

The update signal UPDATE may be transmitted from the command/address decoder450to first input terminals of the first to fourth AND gates1223B˜1223E. A second input terminal of the first AND gate1223B may be coupled to the first output terminal OUT1of the output selector1223A, and an output terminal of the first AND gate1223B may be coupled to the clock terminal of the first latch circuit FF1. A second input terminal of the second AND gate1223C may be coupled to the second output terminal OUT2of the output selector1223A, and an output terminal of the second AND gate1223C may be coupled to the clock terminal of the second latch circuit FF2. A second input terminal of the third AND gate1223D may be coupled to the third output terminal OUT3of the output selector1223A, and an output terminal of the third AND gate1223D may be coupled to the clock terminal of the third latch circuit FF3. A second input terminal of the fourth AND gate1223E may be coupled to the fourth output terminal OUT4of the output selector1223A, and an output terminal of the fourth AND gate1223E may be coupled to the clock terminal of the fourth latch circuit FF4.

The first AND gate1223B may perform a logical AND operation of the update signal UPDATE and an output signal outputted through the first output terminal OUT1of the output selector1223A to generate a first clock signal. The first clock signal generated by the logical AND operation of the first AND gate1223B may be transmitted to a clock terminal of the first latch circuit FF1. The second AND gate1223C may perform a logical AND operation of the update signal UPDATE and an output signal outputted through the second output terminal OUT2of the output selector1223A to generate a second clock signal. The second clock signal generated by the logical AND operation of the second AND gate1223C may be transmitted to a clock terminal of the second latch circuit FF2. The third AND gate1223D may perform a logical AND operation of the update signal UPDATE and an output signal outputted through the third output terminal OUT3of the output selector1223A to generate a third clock signal. The third clock signal generated by the logical AND operation of the third AND gate1223D may be transmitted to a clock terminal of the third latch circuit FF3. The fourth AND gate1223E may perform a logical AND operation of the update signal UPDATE and an output signal outputted through the fourth output terminal OUT4of the output selector1223A to generate a fourth clock signal. The fourth clock signal generated by the logical AND operation of the fourth AND gate1223E may be transmitted to a clock terminal of the fourth latch circuit FF4.

The input selector1224may have first to fourth input terminals IN1˜IN4, an output terminal OUT, and a selection control terminal S2. In an embodiment, the input selector1224may be realized using a 4-to-1 multiplexer. The first input terminal IN1of the input selector1224may be coupled to an output terminal Q of the first latch circuit FF1. The second input terminal IN2of the input selector1224may be coupled to an output terminal Q of the second latch circuit FF2. The third input terminal IN3of the input selector1224may be coupled to an output terminal Q of the third latch circuit FF3. The fourth input terminal IN4of the input selector1224may be coupled to an output terminal Q of the fourth latch circuit FF4. The output terminal OUT of the input selector1224may be coupled to the second input terminal of the accumulative adder1221. In addition, the output terminal OUT of the input selector1224may also be coupled to the output circuit (1230ofFIG. 35), as described with reference toFIG. 35.

The accumulation latch selection signal ALS[1:0] corresponding to a selection control signal may be inputted to the selection control terminal S2of the input selector1224. The input selector1224may output the data inputted to one of the first to fourth input terminals IN1˜IN4, which is selected by the accumulation latch selection signal ALS[1:0], through the output terminal OUT. In an embodiment, the data inputted to the first input terminal IN1(i.e., the data outputted from the first latch circuit FF1) may be outputted through the output terminal OUT of the input selector1224when the accumulation latch selection signal ALS[1:0] has a logic level combination of “00”, and the data inputted to the second input terminal IN2(i.e., the data outputted from the second latch circuit FF2) may be outputted through the output terminal OUT of the input selector1224when the accumulation latch selection signal ALS[1:0] has a logic level combination of “01”. Moreover, the data inputted to the third input terminal IN3(i.e., the data outputted from the third latch circuit FF3) may be outputted through the output terminal OUT of the input selector1224when the accumulation latch selection signal ALS[1:0] has a logic level combination of “10”, and the data inputted to the fourth input terminal IN4(i.e., the data outputted from the fourth latch circuit FF4) may be outputted through the output terminal OUT of the input selector1224when the accumulation latch selection signal ALS[1:0] has a logic level combination of “11”.

FIG. 37illustrates a first MAC arithmetic operation of a first matrix group column unit of the PIM device400A illustrated inFIG. 31. The first MAC arithmetic operation of the first matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM11and the vector sub-matrix VSM11, as described with reference toFIG. 34. Referring toFIG. 37, when a MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the update signal UPDATE having a logic “high” level at a point in time when a certain amount of time elapses from a point in time when the MAC signal MAC having a logic “high” level is outputted. In an embodiment, the certain amount of time may be set as an interval of time from a point in time when the MAC signal MAC having a logic “high” level is outputted from the command/address decoder450until a point in time when the accumulative adding calculations are executed by the accumulators of the first to eighth MAC operators MAC(0)˜MAC(7). The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W1.1˜W1.16, . . . , and W8.1˜W8.16, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM11, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W1.1˜W1.16, which are arrayed in the first row of the weight sub-matrix WSM11, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W2.1˜W2.16, which are arrayed in the second row of the weight sub-matrix WSM11, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W8.1˜W8.16, which are arrayed in the eighth row of the weight sub-matrix WSM11, from the eighth memory bank BK(7). Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the vector data V1˜V16arrayed in the vector sub-matrix VSM11from the global buffer GB.

FIG. 38illustrates an operation performed by the first MAC operator MAC(0) during the first MAC arithmetic operation of the first matrix group column unit of the PIM device400A illustrated inFIG. 37. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 38, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W1.1˜W1.16and the vector data V1˜V16. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM11_0˜DM11_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM11_0˜DM11_15outputted from the first to sixteenth multipliers MUL0˜MUL15, thereby generating and outputting addition result data DMA11. The addition result data DMA11outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA11outputted from the adder tree and feedback data DF outputted from the input selector1224, thereby generating and outputting accumulated addition data DMACC11. Because the first latch circuit FF1of the latch circuit1222has an initialized status, the feedback data DF transmitted from the input selector1224to the accumulative adder1221may have a value of zero. Thus, the accumulated addition data DMACC11outputted from the accumulative adder1221may have the same value as the addition result data DMA11outputted from the adder tree. The accumulated addition data DMACC11outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the first output terminal OUT1in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. Thus, the logic high level signal HI may be inputted to the second input terminal of the first AND gate1223B. In such a case, the output selector1223A may output a logic low level signal LO through the second, third, and fourth output terminals OUT2, OUT3, and OUT4. Thus, the logic low level signal LO may be inputted to the second input terminals of the second to fourth AND gates1223C,1223D, and1223E. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the first AND gate1223B may output the logic high level signal HI to the clock terminal of the first latch circuit FF1while the second to fourth AND gates1223C˜1223E output the logic low level signal LO to the clock terminals of the second to fourth latch circuits FF2, FF3, and FF4. The first latch circuit FF1may latch the accumulated addition data DMACC11outputted from the accumulative adder1221in response to the logic high level signal HI outputted from the first AND gate1223B and may output the latched data of the accumulated addition data DMACC11to the first input terminal IN1of the input selector1224.

FIG. 39illustrates a first MAC arithmetic operation of a second matrix group column unit of the PIM device400A illustrated inFIG. 31. The first MAC arithmetic operation of the second matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM21and the vector sub-matrix VSM11, as described with reference toFIG. 34. Referring toFIG. 39, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the update signal UPDATE having a logic “high” level at a point in time when the certain amount of time elapses from a point in time when the MAC signal MAC having a logic “high” level is outputted. The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “01”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W9.1˜W9.16, . . . , and W16.1˜W16.16, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM21, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W9.1˜W9.16, which are arrayed in the first row of the weight sub-matrix WSM21, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W10.1˜W10.16, which are arrayed in the second row of the weight sub-matrix WSM21, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W16.1˜W16.16, which are arrayed in the eighth row of the weight sub-matrix WSM21, from the eighth memory bank BK(7). Meanwhile, the vector data V1˜V16previously transmitted to each of the first to eighth MAC operators MAC(0)˜MAC(7) are not changed during the first MAC arithmetic operation of the second matrix group column unit of the PIM device400A.

FIG. 40illustrates an operation performed by the first MAC operator MAC(0) during the first MAC arithmetic operation of the second matrix group column unit of the PIM device400A illustrated inFIG. 39. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 40, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W9.1˜W9.16and the vector data V1˜V16. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM12_0˜DM12_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM12_0˜DM12_15outputted from the first to sixteenth multipliers MUL0˜MUL15, thereby generating and outputting addition result data DMA12. The addition result data DMA12outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA12outputted from the adder tree and the feedback data DF outputted from the input selector1224, thereby generating and outputting accumulated addition data DMACC12. Because the second latch circuit FF2of the latch circuit1222has an initialized status, the feedback data DF transmitted from the input selector1224to the accumulative adder1221may have a value of zero. Thus, the accumulated addition data DMACC12outputted from the accumulative adder1221may have the same value as the addition result data DMA12outputted from the adder tree. The accumulated addition data DMACC12outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the second output terminal OUT2in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “01”. Thus, the logic high level signal HI may be inputted to the second input terminal of the second AND gate1223C. In such a case, the output selector1223A may output a logic low level signal LO through the first, third, and fourth output terminals OUT1, OUT3, and OUT4. Thus, the logic low level signal LO may be inputted to the second input terminals of the first, third, and fourth AND gates1223B,1223D, and1223E. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the second AND gate1223C may output the logic high level signal HI to the clock terminal of the second latch circuit FF2while the first, third, and fourth AND gates1223B,1223D, and1223E output the logic low level signal LO to the clock terminals of the first, third, and fourth latch circuits FF1, FF3, and FF4. The second latch circuit FF2may latch the accumulated addition data DMACC12outputted from the accumulative adder1221in response to the logic high level signal HI outputted from the second AND gate1223C and may output the latched data of the accumulated addition data DMACC12to the second input terminal IN2of the input selector1224.

FIG. 41illustrates a first MAC arithmetic operation of a third matrix group column unit of the PIM device400A illustrated inFIG. 31. The first MAC arithmetic operation of the third matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM31and the vector sub-matrix VSM11, as described with reference toFIG. 34. Referring toFIG. 41, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “10”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W17.1˜W17.16, . . . , and W24.1˜W24.16, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM31, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W17.1˜W17.16, which are arrayed in the first row of the weight sub-matrix WSM31, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W18.1˜W18.16, which are arrayed in the second row of the weight sub-matrix WSM31, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W24.1˜W24.16, which are arrayed in the eighth row of the weight sub-matrix WSM31, from the eighth memory bank BK(7). Meanwhile, the vector data V1˜V16previously transmitted to each of the first to eighth MAC operators MAC(0)˜MAC(7) are not changed during the first MAC arithmetic operation of the third matrix group column unit of the PIM device400A.

FIG. 42illustrates an operation performed by the first MAC operator MAC(0) during the first MAC arithmetic operation of the third matrix group column unit of the PIM device400A illustrated inFIG. 41. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 42, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W17.1˜W17.16and the vector data V1˜V16. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM13_0˜DM13_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM13_0˜DM13_15outputted from the first to sixteenth multipliers MUL0˜MUL15, thereby generating and outputting addition result data DMA13. The addition result data DMA13outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA13outputted from the adder tree and the feedback data DF outputted from the input selector1224, thereby generating and outputting accumulated addition data DMACC13. Because the third latch circuit FF3of the latch circuit1222has an initialized status, the feedback data DF transmitted from the input selector1224to the accumulative adder1221may have a value of zero. Thus, the accumulated addition data DMACC13outputted from the accumulative adder1221may have the same value as the addition result data DMA13outputted from the adder tree. The accumulated addition data DMACC13outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the third output terminal OUT3in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “10”. Thus, the logic high level signal HI may be inputted to the second input terminal of the third AND gate1223D. In such a case, the output selector1223A may output a logic low level signal LO through the first, second, and fourth output terminals OUT1, OUT2, and OUT4. Thus, the logic low level signal LO may be inputted to the second input terminals of the first, second, and fourth AND gates1223B,1223C, and1223E. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the third AND gate1223D may output the logic high level signal HI to the clock terminal of the third latch circuit FF3while the first, second, and fourth AND gates1223B,1223C, and1223E output the logic low level signal LO to the clock terminals of the first, second, and fourth latch circuits FF1, FF2, and FF4. The third latch circuit FF3may latch the accumulated addition data DMACC13outputted from the accumulative adder1221in synchronization with the logic high level signal HI outputted from the third AND gate1223D and may output the latched data of the accumulated addition data DMACC13to the third input terminal IN3of the input selector1224.

FIG. 43illustrates a first MAC arithmetic operation of a fourth matrix group column unit of the PIM device400A illustrated inFIG. 31. The first MAC arithmetic operation of the fourth matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM41and the vector sub-matrix VSM11, as described with reference toFIG. 34. Referring toFIG. 43, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “11”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W25.1˜W25.16, . . . , and W32.1˜W32.16, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM41, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W25.1˜W25.16, which are arrayed in the first row of the weight sub-matrix WSM41, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W26.1˜W26.16, which are arrayed in the second row of the weight sub-matrix WSM41, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W32.1˜W32.16, which are arrayed in the eighth row of the weight sub-matrix WSM41, from the eighth memory bank BK(7). Meanwhile, the vector data V1˜V16previously transmitted to each of the first to eighth MAC operators MAC(0)˜MAC(7) are not changed during the first MAC arithmetic operation of the fourth matrix group column unit of the PIM device400A.

FIG. 44illustrates an operation performed by the first MAC operator MAC(0) during the first MAC arithmetic operation of the fourth matrix group column unit of the PIM device400A illustrated inFIG. 43. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 44, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W25.1˜W25.16and the vector data V1˜V16. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM14_0˜DM14_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM14_0˜DM14_15outputted from the first to sixteenth multipliers MUL0˜MUL5, thereby generating and outputting addition result data DMA14. The addition result data DMA14outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA14outputted from the adder tree and the feedback data DF outputted from the input selector1224, thereby generating and outputting accumulated addition data DMACC14. Because the fourth latch circuit FF4of the latch circuit1222has an initialized status, the feedback data DF transmitted from the input selector1224to the accumulative adder1221may have a value of zero. Thus, the accumulated addition data DMACC14outputted from the accumulative adder1221may have the same value as the addition result data DMA14outputted from the adder tree. The accumulated addition data DMACC14outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the fourth output terminal OUT4in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “11”. Thus, the logic high level signal HI may be inputted to the second input terminal of the fourth AND gate1223E. In such a case, the output selector1223A may output a logic low level signal LO through the first, second, and third output terminals OUT1, OUT2, and OUT3. Thus, the logic low level signal LO may be inputted to the second input terminals of the first, second, and third AND gates1223B,1223C, and1223D. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the fourth AND gate1223E may output the logic high level signal HI to the clock terminal of the fourth latch circuit FF4while the first, second, and third AND gates1223B,1223C, and1223D output the logic low level signal LO to the clock terminals of the first, second, and third latch circuits FF1, FF2, and FF3. The fourth latch circuit FF4may latch the accumulated addition data DMACC14outputted from the accumulative adder1221in synchronization with the logic high level signal HI outputted from the fourth AND gate1223E and may output the latched data of the accumulated addition data DMACC14to the fourth input terminal IN4of the input selector1224.

As described with reference toFIGS. 37 to 44, the first MAC arithmetic operations of the first to fourth matrix group column units of the PIM device400A may be sequentially performed to complete the first MAC arithmetic operations of the matrix group column unit for the weight sub-matrixes WSM11, WSM21, WSM31, and WSM41in the first weight matrix group column WMGC1and the vector sub-matrixes VSM11in the first vector matrix group row VMGR1. While the first MAC arithmetic operations of the matrix group column unit are performed, only the weight data transmitted to the MAC operators may be changed without any alteration of the vector data transmitted from the global buffer to the MAC operators. As a result of the first MAC arithmetic operations of the matrix group column unit, the accumulated addition data DMACC11generated by the first MAC arithmetic operation of the first matrix group column unit may be latched in the first latch circuit FF1of the accumulator1220and the accumulated addition data DMACC12generated by the first MAC arithmetic operation of the second matrix group column unit may be latched in the second latch circuit FF2of the accumulator1220. In addition, the accumulated addition data DMACC13generated by the first MAC arithmetic operation of the third matrix group column unit may be latched in the third latch circuit FF3of the accumulator1220, and the accumulated addition data DMACC14generated by the first MAC arithmetic operation of the fourth matrix group column unit may be latched in the fourth latch circuit FF4of the accumulator1220.

FIG. 45illustrates a second MAC arithmetic operation of a first matrix group column unit of the PIM device400A illustrated inFIG. 31. The second MAC arithmetic operation of the first matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM12and the vector sub-matrix VSM21, as described with reference toFIG. 34. Referring toFIG. 45, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the update signal UPDATE having a logic “high” level at a point in time when a certain amount of time elapses from a point in time when the MAC signal MAC having a logic “high” level is outputted. In an embodiment, the certain time may be set as an interval of time from a point in time when the MAC signal MAC having a logic “high” level is outputted from the command/address decoder450until a point in time when the accumulative adding calculations are executed by the accumulators of the first to eighth MAC operators MAC(0)˜MAC(7). The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W1.17˜W1.32, . . . , and W8.17˜W8.32, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM12, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W1.17˜W1.32, which are arrayed in the first row of the weight sub-matrix WSM12, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W2.17˜W2.32, which are arrayed in the second row of the weight sub-matrix WSM12, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W8.17˜W8.32, which are arrayed in the eighth row of the weight sub-matrix WSM12, from the eighth memory bank BK(7).

As described with reference toFIG. 34, while the vector data V1˜V16of the vector sub-matrix VSM11are used for the first MAC arithmetic operations of the first to fourth matrix group column units, the vector data V17˜V32of the vector sub-matrix VSM21may be used for the second MAC arithmetic operations of the first to fourth matrix group column units. Thus, the vector data V17˜V32of the vector sub-matrix VSM21have to be transmitted to the first to eighth MAC operators MAC(0)˜MAC(7) in order to perform the second MAC arithmetic operation of the first matrix group column unit after the termination of the first MAC arithmetic operations of the first to fourth matrix group column units. Accordingly, in order to perform the second MAC arithmetic operation of the first matrix group column unit, each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the vector data V17˜V32arrayed in the vector sub-matrix VSM21from the global buffer GB.

FIG. 46illustrates an operation performed by the first MAC operator MAC(0) during the second MAC arithmetic operation of the first matrix group column unit of the PIM device400A illustrated inFIG. 45. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 46, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W1.17˜W1.32and the vector data V17˜V32. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM21_0˜DM21_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM21_0˜DM21_15outputted from the first to sixteenth multipliers MUL0˜MUL5, thereby generating and outputting addition result data DMA21. The addition result data DMA21outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The input selector1224of the accumulator1220may feedback the accumulated addition data DMACC11, which are transmitted from the first latch circuit FF1to the first input terminal IN1of the input selector1224by the first MAC arithmetic operation of the first matrix group column unit, to the accumulative adder1221in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA21outputted from the adder tree and the accumulated addition data DMACC11fed back from the input selector1224, thereby generating and outputting accumulated addition data DMACC21. Thus, the accumulated addition data DMACC21outputted from the accumulative adder1221may correspond to accumulation data that the result data of the second MAC arithmetic operation of the first matrix group column unit are added to the result data of the first MAC arithmetic operation of the first matrix group column unit. The accumulated addition data DMACC21outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the first output terminal OUT1in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. Thus, the logic high level signal HI may be inputted to the second input terminal of the first AND gate1223B. In such a case, the output selector1223A may output a logic low level signal LO through the second, third, and fourth output terminals OUT2, OUT3, and OUT4. Thus, the logic low level signal LO may be inputted to the second input terminals of the second to fourth AND gates1223C,1223D, and1223E. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the first AND gate1223B may output the logic high level signal HI to the clock terminal of the first latch circuit FF1while the second to fourth AND gates1223C˜1223E output the logic low level signal LO to the clock terminals of the second to fourth latch circuits FF2, FF3, and FF4. The first latch circuit FF1may latch the accumulated addition data DMACC21outputted from the accumulative adder1221in synchronization with the logic high level signal HI outputted from the first AND gate1223B and may output the latched data of the accumulated addition data DMACC21to the first input terminal IN1of the input selector1224.

FIG. 47illustrates a second MAC arithmetic operation of a second matrix group column unit of the PIM device400A illustrated inFIG. 31. The second MAC arithmetic operation of the second matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM22and the vector sub-matrix VSM21, as described with reference toFIG. 34. Referring toFIG. 47, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the update signal UPDATE having a logic “high” level at a point in time when the certain amount of time elapses from a point in time when the MAC signal MAC having a logic “high” level is outputted. The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “01”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W9.17˜W9.32, . . . , and W16.17˜W16.32, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM22, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W9.17˜W9.32, which are arrayed in the first row of the weight sub-matrix WSM22, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W10.17˜W10.32, which are arrayed in the second row of the weight sub-matrix WSM22, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W16.17˜W16.32, which are arrayed in the eighth row of the weight sub-matrix WSM22, from the eighth memory bank BK(7). Meanwhile, the vector data V17˜V32previously transmitted to each the first to eighth MAC operators MAC(0)˜MAC(7) are not changed during the second MAC arithmetic operation of the second matrix group column unit of the PIM device400A.

FIG. 48illustrates an operation performed by the first MAC operator MAC(0) during the second MAC arithmetic operation of the second matrix group column unit of the PIM device400A illustrated inFIG. 47. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 48, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W9.17˜W9.32and the vector data V17˜V32. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM22_0˜DM22_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM22_0˜DM22_15outputted from the first to sixteenth multipliers MUL0˜MUL15, thereby generating and outputting addition result data DMA22. The addition result data DMA22outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The input selector1224of the accumulator1220may feedback the accumulated addition data DMACC12, which are transmitted from the second latch circuit FF2to the second input terminal IN2of the input selector1224by the first MAC arithmetic operation of the second matrix group column unit, to the accumulative adder1221in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “01”. The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA22outputted from the adder tree and the accumulated addition data DMACC12fed back from the input selector1224, thereby generating and outputting accumulated addition data DMACC22. Thus, the accumulated addition data DMACC22outputted from the accumulative adder1221may correspond to accumulation data that the result data of the second MAC arithmetic operation of the second matrix group column unit are added to the result data of the first MAC arithmetic operation of the second matrix group column unit. The accumulated addition data DMACC22outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the second output terminal OUT2in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “01”. Thus, the logic high level signal HI may be inputted to the second input terminal of the second AND gate1223C. In such a case, the output selector1223A may output a logic low level signal LO through the first, third, and fourth output terminals OUT1, OUT3, and OUT4. Thus, the logic low level signal LO may be inputted to the second input terminals of the first, third, and fourth AND gates1223B,1223D, and1223E. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the second AND gate1223C may output the logic high level signal HI to the clock terminal of the second latch circuit FF2while the first, third, and fourth AND gates1223B,1223D, and1223E output the logic low level signal LO to the clock terminals of the first, third, and fourth latch circuits FF1, FF3, and FF4. The second latch circuit FF2may latch the accumulated addition data DMACC22outputted from the accumulative adder1221in synchronization with the logic high level signal HI outputted from the second AND gate1223C and may output the latched data of the accumulated addition data DMACC22to the second input terminal IN2of the input selector1224.

FIG. 49illustrates a second MAC arithmetic operation of a third matrix group column unit of the PIM device400A illustrated inFIG. 31. The second MAC arithmetic operation of the third matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM32and the vector sub-matrix VSM21, as described with reference toFIG. 34. Referring toFIG. 49, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the update signal UPDATE having a logic “high” level at a point in time when the certain amount of time elapses from a point in time when the MAC signal MAC having a logic “high” level is outputted. The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “10”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W17.17˜W17.32, . . . , and W24.17˜W24.32, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM32, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W17.17˜W17.32, which are arrayed in the first row of the weight sub-matrix WSM32, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W18.17˜W18.32, which are arrayed in the second row of the weight sub-matrix WSM32, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W24.17˜W24.32, which are arrayed in the eighth row of the weight sub-matrix WSM32, from the eighth memory bank BK(7). Meanwhile, the vector data V17˜V32previously transmitted to each the first to eighth MAC operators MAC(0)˜MAC(7) are not changed during the second MAC arithmetic operation of the third matrix group column unit of the PIM device400A.

FIG. 50illustrates an operation performed by the first MAC operator MAC(0) during the second MAC arithmetic operation of the third matrix group column unit of the PIM device400A illustrated inFIG. 49. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 50, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W17.17˜W17.32and the vector data V17˜V32. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM23_0˜DM23_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM23_0˜DM23_15outputted from the first to sixteenth multipliers MUL0˜MUL15, thereby generating and outputting addition result data DMA23. The addition result data DMA23outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The input selector1224of the accumulator1220may feedback the accumulated addition data DMACC13, which are transmitted from the third latch circuit FF3to the third input terminal IN3of the input selector1224by the first MAC arithmetic operation of the third matrix group column unit, to the accumulative adder1221in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “10”. The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA23outputted from the adder tree and the accumulated addition data DMACC13fed back from the input selector1224, thereby generating and outputting accumulated addition data DMACC23. Thus, the accumulated addition data DMACC23outputted from the accumulative adder1221may correspond to accumulation data that the result data of the second MAC arithmetic operation of the third matrix group column unit are added to the result data of the first MAC arithmetic operation of the third matrix group column unit. The accumulated addition data DMACC23outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the third output terminal OUT3in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “10”. Thus, the logic high level signal HI may be inputted to the second input terminal of the third AND gate1223D. In such a case, the output selector1223A may output a logic low level signal LO through the first, second, and fourth output terminals OUT1, OUT2, and OUT4. Thus, the logic low level signal LO may be inputted to the second input terminals of the first, second, and fourth AND gates1223B,1223C, and1223E. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the third AND gate1223D may output the logic high level signal HI to the clock terminal of the third latch circuit FF3while the first, second, and fourth AND gates1223B,1223C, and1223E output the logic low level signal LO to the clock terminals of the first, second, and fourth latch circuits FF1, FF2, and FF4. The third latch circuit FF3may latch the accumulated addition data DMACC23outputted from the accumulative adder1221in synchronization with the logic high level signal HI outputted from the third AND gate1223D and may output the latched data of the accumulated addition data DMACC23to the third input terminal IN3of the input selector1224.

FIG. 51illustrates a second MAC arithmetic operation of a fourth matrix group column unit of the PIM device400A illustrated inFIG. 31. The second MAC arithmetic operation of the fourth matrix group column unit of the PIM device400A may be performed by the matrix multiplying calculation of the weight sub-matrix WSM42and the vector sub-matrix VSM21, as described with reference toFIG. 34. Referring toFIG. 51, when the MAC command MAC_CMD is transmitted to the PIM device400A, the command/address decoder450may generate and output the MAC signal MAC having a logic “high” level, the update signal UPDATE having a logic “high” level, and the accumulation latch selection signal ALS[1:0]. The command/address decoder450may output the update signal UPDATE having a logic “high” level at a point in time when the certain amount of time elapses from a point in time when the MAC signal MAC having a logic “high” level is outputted. The command/address decoder450may output the accumulation latch selection signal ALS[1:0] having a logic level combination of “11”.

Each of the first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data Ws and the vector data Vs in response to the MAC signal MAC outputted from the command/address decoder450. The first to eighth MAC operators MAC(0)˜MAC(7) may receive the weight data W25.17˜W25.32, . . . , and W32.17˜W32.32, which are arrayed in respective ones of the first to eight rows of the weight sub-matrix WSM42, from the first to eighth memory banks BK(0)˜BK(7), respectively. For example, the first MAC operator MAC(0) may receive the weight data W25.17˜W25.32, which are arrayed in the first row of the weight sub-matrix WSM42, from the first memory bank BK(0); and the second MAC operator MAC(1) may receive the weight data W26.17˜W26.32, which are arrayed in the second row of the weight sub-matrix WSM42, from the second memory bank BK(1). Similarly, the eighth MAC operator MAC(7) may receive the weight data W32.17˜W32.32, which are arrayed in the eighth row of the weight sub-matrix WSM42, from the eighth memory bank BK(7). Meanwhile, the vector data V17˜V32previously transmitted to each the first to eighth MAC operators MAC(0)˜MAC(7) are not changed during the second MAC arithmetic operation of the fourth matrix group column unit of the PIM device400A.

FIG. 52illustrates an operation performed by the first MAC operator MAC(0) during the second MAC arithmetic operation of the fourth matrix group column unit of the PIM device400A illustrated inFIG. 51. The operation of the first MAC operator MAC(0) described hereinafter may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7) except alteration of the weight data. Referring toFIG. 52, the first to sixteenth multipliers MUL0˜MUL15of the first MAC operator MAC(0) may perform multiplying calculations of the weight data W25.17˜W25.32and the vector data V17˜V32. The first to sixteenth multipliers MUL0˜MUL15may output first to sixteenth multiplication data DM24_-DM24_15generated by the multiplying calculations, respectively. The adder tree may perform adding calculations of the first to sixteenth multiplication data DM24_0˜DM24_15outputted from the first to sixteenth multipliers MUL0˜MUL5, thereby generating and outputting addition result data DMA24. The addition result data DMA24outputted from the adder tree may be transmitted to the accumulative adder1221of the accumulator1220.

The input selector1224of the accumulator1220may feedback the accumulated addition data DMACC4, which are transmitted from the fourth latch circuit FF4to the fourth input terminal IN4of the input selector1224by the first MAC arithmetic operation of the fourth matrix group column unit, to the accumulative adder1221in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “11”. The accumulative adder1221of the accumulator1220may perform an adding calculation of the addition result data DMA24outputted from the adder tree and the accumulated addition data DMACC14fed back from the input selector1224, thereby generating and outputting accumulated addition data DMACC24. Thus, the accumulated addition data DMACC24outputted from the accumulative adder1221may correspond to accumulation data that the result data of the second MAC arithmetic operation of the fourth matrix group column unit are added to the result data of the first MAC arithmetic operation of the fourth matrix group column unit. The accumulated addition data DMACC24outputted from the accumulative adder1221may be transmitted to each of the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector1223A of the latch circuit selector1223may output a logic high level signal HI through the fourth output terminal OUT4in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “11”. Thus, the logic high level signal HI may be inputted to the second input terminal of the fourth AND gate1223E. In such a case, the output selector1223A may output a logic low level signal LO through the first, second, and third output terminals OUT1, OUT2, and OUT3. Thus, the logic low level signal LO may be inputted to the second input terminals of the first, second, and third AND gates1223B,1223C, and1223D. The update signal UPDATE having a logic “high” level may be transmitted from the command/address decoder450to the first input terminals of the first to fourth AND gates1223B˜1223E. Accordingly, the fourth AND gate1223E may output the logic high level signal HI to the clock terminal of the fourth latch circuit FF4while the first, second, and third AND gates1223B,1223C, and1223D output the logic low level signal LO to the clock terminals of the first, second, and third latch circuits FF1, FF2, and FF3. The fourth latch circuit FF4may latch the accumulated addition data DMACC24outputted from the accumulative adder1221in synchronization with the logic high level signal HI outputted from the fourth AND gate1223E and may output the latched data of the accumulated addition data DMACC24to the fourth input terminal IN4of the input selector1224.

As described with reference toFIGS. 37 to 44, the first MAC operator MAC(0) may perform the first MAC arithmetic operations of the first to fourth matrix group column units for multiplying the weight data W1.1˜W1.16arrayed in the first row of the weight sub-matrix WSM11, the weight data W9.1˜W9.16arrayed in the first row of the weight sub-matrix WSM21, the weight data W17.1˜W17.16arrayed in the first row of the weight sub-matrix WSM31, and the weight data W25.1˜W25.16arrayed in the first row of the weight sub-matrix WSM41by the vector data V1˜V16arrayed in the vector sub-matrix VSM11(refer toFIG. 34). In addition, as described with reference toFIGS. 45 to 52, the first MAC operator MAC(0) may perform the second MAC arithmetic operations of the first to fourth matrix group column units for multiplying the weight data W1.17˜W1.32arrayed in the first row of the weight sub-matrix WSM12, the weight data W9.17˜W9.32arrayed in the first row of the weight sub-matrix WSM22, the weight data W17.17˜W17.32arrayed in the first row of the weight sub-matrix WSM32, and the weight data W25.17˜W25.32arrayed in the first row of the weight sub-matrix WSM42by the vector data V17˜V32arrayed in the vector sub-matrix VSM21(refer toFIG. 34). Accordingly, the MAC arithmetic operation (i.e., the matrix multiplying calculation) for the weight matrix and the vector matrix illustrated inFIG. 32may be completed by the first MAC arithmetic operations of the matrix group column unit and the second MAC arithmetic operations of the matrix group column unit.

FIG. 53illustrates an operation for outputting MAC result data MAC_RST1from the first MAC operator MAC(0) included in the PIM device400A illustrated inFIG. 31. The MAC result data MAC_RST1corresponding to the element arrayed in the first row of the result matrix illustrated inFIG. 32may be the accumulated addition data DMACC21which are obtained by the matrix multiplying calculation of the weigh data W1.1˜W1.32arrayed in the first row of the weight matrix inFIG. 32and the vector data V1˜V32arrayed in the vector matrix inFIG. 32. The output operation of the MAC result data MAC_RST1may be performed after the accumulated addition data DMACC21are transmitted from the first latch circuit FF1to the first input terminal IN1of the input selector1224by the second MAC arithmetic operation of the first matrix group column unit described with reference toFIG. 46.

Referring toFIG. 53, in order to control the output operation of the MAC result data MAC_RST1, the command/address decoder450may output the result read signal RD_RST having a logic “high” level and the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. The input selector1224may output the accumulated addition data DMACC21, which are inputted to the first input terminal IN1, through the output terminal OUT in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. The accumulated addition data DMACC21outputted from the input selector1224may be transmitted to the output circuit1230. The output circuit1230may output the accumulated addition data DMACC21as the MAC result data MAC_RST1in response to the result read signal RD_RST having a logic “high” level which is outputted from the command/address decoder450.

The output process of the MAC result data MAC_RST1may be equally applicable to each of output processes of the remaining MAC result data MAC_RST9, MAC_RST17, and MAC_RST25. In such case, in order to output the accumulated addition data DMACC22as the MAC result data MAC_RST9corresponding to the element arrayed in the ninth row of the result matrix illustrated inFIG. 32, the command/address decoder450may output the result read signal RD_RST having a logic “high” level and the accumulation latch selection signal ALS[1:0] having a logic level combination of “01”. In addition, in order to output the accumulated addition data DMACC23as the MAC result data MAC_RST17corresponding to the element arrayed in the 17throw of the result matrix illustrated inFIG. 32, the command/address decoder450may output the result read signal RD_RST having a logic “high” level and the accumulation latch selection signal ALS[1:0] having a logic level combination of “10”. Furthermore, in order to output the accumulated addition data DMACC24as the MAC result data MAC_RST25corresponding to the element arrayed in the 25throw of the result matrix illustrated inFIG. 32, the command/address decoder450may output the result read signal RD_RST having a logic “high” level and the accumulation latch selection signal ALS[1:0] having a logic level combination of “11”.

While the MAC result data MAC_RST are read out of the first MAC operator MAC(0), both of the MAC signal MAC and the update signal UPDATE delayed by the certain time from the MAC signal MAC may maintain a logic “low(LO)” level. Thus, the first to fourth AND gates1223B,1223C,1223D, and1223E may output logic low level signals LO which are transmitted to respective ones of the clock terminals of the first to fourth latch circuits FF1˜FF4. Accordingly, while the MAC result data MAC_RST1are outputted from the first MAC operator MAC(0), no latch operation is performed by the first to fourth latch circuits FF1˜FF4.

FIG. 54is a block diagram illustrating a PIM device400B according to another embodiment of the present disclosure. Referring toFIG. 54, the PIM device400B may include “L”-number of memory banks (i.e., first to Lthmemory banks BK(0)˜BK(L−1)), a global buffer GB, “L”-number of MAC operators (i.e., first to LhMAC operators MAC(0)˜MAC(L−1)), and a command/address decoder460(where, “L” is a natural number which is equal to or greater than two). The memory banks BK(0)˜BK(L−1) included in the PIM device400B may have the same configuration and function as the memory banks BK(0)˜BK(L−1) of the PIM device400A described with reference toFIG. 31, and the global buffer GB of the PIM device400B may have the same configuration and function as the global buffer GB of the PIM device400A described with reference toFIG. 31. Each of the first to LthMAC operators MAC(0)˜MAC(L−1) included in the PIM device400B may also have the same configuration and function as each of the first to LthMAC operators MAC(0)˜MAC(L−1) of the PIM device400A described with reference toFIG. 31except the accumulator. That is, each of the first to LthMAC operators MAC(0)˜MAC(L−1) included in the PIM device400B may be different from each of the first to LthMAC operators MAC(0)˜MAC(L−1) included in the PIM device400A in terms of a configuration of only the accumulator.

The command/address decoder460may receive a command CMD and an address ADDR from an external device such as a controller. The command/address decoder460may decode the command CMD and the address ADDR to generate and output various control signals RD, WT, MAC, RD_RST, UPDATE, and ALS for controlling operations of the memory banks BK(0)˜BK(L−1), the global buffer GB, and the MAC operators MAC(0)˜MAC(L−1) as well as generating an address signal ADDR_S, like the command/address decoder450described with reference toFIG. 31. In addition, the command/address decoder460of the PIM device400B may further output a temporary copy signal TC and a temporary storage signal TS acting as the control signals. The temporary copy signal TC may control an operation of a first input selector (2224A ofFIG. 56) disposed in an accumulator (2220ofFIG. 56) included in each of the MAC operators MAC(0)˜MAC(L−1). The temporary storage signal TS may control an operation of a latch circuit (2222ofFIG. 56) included in each of the MAC operators MAC(0)˜MAC(L−1).

FIG. 55is a block diagram illustrating an example of a configuration of the first MAC operator MAC(0) included in the PIM device400B illustrated inFIG. 54. InFIG. 55, the same reference numerals and symbols as used inFIG. 35denote the same components. In the present embodiment, the configuration of the first MAC operator MAC(0) may be equally applicable to each of the second to eighth MAC operators MAC(1)˜MAC(7). Referring toFIG. 55, the first MAC operator MAC(0) may include a plurality of multipliers (e.g., first to sixteenth multipliers MUL0˜MUL15), an adder tree including a plurality of adders, an accumulator2220, and an output circuit1230. The first to sixteenth multipliers MUL0˜MUL15, the adder tree, and the output circuit1230included in the PIM device400B may have the same configuration as the first to sixteenth multipliers MUL0˜MUL5, the adder tree, and the output circuit1230included in the PIM device400A described with reference toFIG. 35. Thus, descriptions of the first to sixteenth multipliers MUL0˜MUL5, the adder tree, and the output circuit1230included in the PIM device400B will be omitted hereinafter to avoid duplicate explanation.

The accumulator2220may receive the addition result data DMA from the adder tree to perform an accumulative adding calculation. In order to perform the accumulative adding calculation of the accumulator2220, the accumulator2220may include an accumulative adder and a plurality of latch circuits. The accumulative adder may perform an accumulative adding calculation of the addition result data DMA outputted from the adder tree and feedback data outputted from one of the plurality of latch circuits. The plurality of latch circuits may latch output data of the accumulative adder. The latched data of the latch circuits may be selectively fed back to the accumulative adder to be used as the feedback data. In addition, the latched data of the latch circuits may be selectively transmitted to the output circuit1230. Operations of the latch circuits included in the accumulator2220may be controlled by the update signal UPDATE, the accumulation latch selection signal ALS, the temporary copy signal TC, and the temporary storage signal TS which are outputted from the command/address decoder (460ofFIG. 54). The accumulator2220will be described in more detail hereinafter with reference toFIG. 56.

FIG. 56is a circuit diagram illustrating an example of a configuration of the accumulator2220included in the first MAC operator MAC(0) illustrated inFIG. 55. Referring toFIG. 56, the accumulator2220may include an accumulative adder2221, a latch circuit2222, a latch circuit selector2223, the first input selector2224A, a second input selector2224B, and a temporary latch circuit FF0. The accumulative adder2221may have a first input terminal coupled to the adder tree, a second input terminal coupled to an output terminal Q of the temporary latch circuit FF0, and an output terminal coupled to a first input terminal IN1of the first input selector2224A. The accumulative adder2221may receive the addition result data DMA from the adder tree through the first input terminal and may receive feedback data DF from the temporary latch circuit FF0through the second input terminal. The accumulative adder2221may perform an adding calculation of the addition result data DMA and the feedback data DF to generate and output the addition result as accumulated addition data DMACC through the output terminal. The accumulated addition data DMACC outputted from the accumulative adder2221may be transmitted to the first input terminal IN1of the first input selector2224A.

The first input selector2224A may have the first input terminal IN1coupled to the output terminal of the accumulative adder2221, a second input terminal IN2coupled to an output terminal OUT of the second input selector2224B, a selection control terminal receiving the temporary copy signal TC[0] from the command/address decoder (460ofFIG. 54), and an output terminal coupled to an input terminal of the temporary latch circuit FF0. In an embodiment, the first input selector2224A may be realized using a 2-to-1 multiplexer. The first input selector2224A may selectively output the accumulated addition data DMACC inputted to the first input terminal IN1or the output data of the second input selector2224B inputted to the second input terminal IN2, in response to the temporary copy signal TC[0]. In an embodiment, when the temporary copy signal TC[0] has a logic “low” level, the first input selector2224A may selectively output the accumulated addition data DMACC inputted to the first input terminal IN1. To the contrary, when the temporary copy signal TC[O] has a logic “high” level, the first input selector2224A may selectively output the output data of the second input selector2224B, which are inputted to the second input terminal IN2. The output data of the first input selector2224A may be transmitted to the input terminal of the temporary latch circuit FF0.

The temporary latch circuit FF0may have an input terminal coupled to the output terminal of the first input selector2224A, a clock terminal receiving the update signal UPDATE from the command/address decoder (460ofFIG. 54), and an output terminal coupled to the second input terminal of the accumulative adder2221and all of the input terminals of first to fourth latch circuits FF1˜FF4included in the latch circuit2222. In an embodiment, the temporary latch circuit FF0may be realized using a flip-flop having a latch function. The temporary latch circuit FF0may be synchronized with a rising edge of the update signal UPDATE to latch and output the output data of the first input selector2224A through the output terminal Q.

The latch circuit2222may include the first to fourth the latch circuits FF1˜FF4. The number of the latch circuits included in the latch circuit2222may be equal to the number of the weight matrix group rows WMGR1˜WMGR4in the weight matrix described with reference toFIG. 34. Each of the first to fourth latch circuits FF1˜FF4may have an input terminal coupled to the output terminal of the temporary latch circuit FF0, a clock terminal respectively coupled to one of first to fourth AND gates2223B˜2223E included in the latch circuit selector2223, and an output terminal Q respectively coupled to one of first to fourth input terminals IN1˜IN4of the second input selector2224B. In an embodiment, each of the first to fourth latch circuits FF1˜FF4may be realized using a flip-flop having a latch function. Each of the first to fourth latch circuits FF1˜FF4may be synchronized with a rising edge of a signal inputted to the clock terminal to latch and output the output data of the temporary latch circuit FF0through the output terminal Q.

The latch circuit selector2223may include an output selector2223A and the first to fourth AND gates2223B˜2223E. The output selector2223A may have an input terminal IN receiving a logic high level signal HI, a selection control terminal S1receiving the accumulation latch selection signal ALS[1:0] from the command/address decoder (460ofFIG. 54), and first to fourth output terminals OUT1˜OUT4. In an embodiment, the output selector2223A may be realized using a 1-to-4 demultiplexer. The output selector2223A may output the logic high level signal HI through one of the first to fourth output terminals OUT1˜OUT4in response to the accumulation latch selection signal ALS[1:0]. In an embodiment, the output selector2223A may output the logic high level signal HI through the first output terminal OUT1when the accumulation latch selection signal ALS[1:0] has a logic level combination of “00”, and the output selector2223A may output the logic high level signal HI through the second output terminal OUT2when the accumulation latch selection signal ALS[1:0] has a logic level combination of “01”. Moreover, the output selector2223A may output the logic high level signal HI through the third output terminal OUT3when the accumulation latch selection signal ALS[1:0] has a logic level combination of “10”, and the output selector2223A may output the logic high level signal HI through the fourth output terminal OUT4when the accumulation latch selection signal ALS[1:0] has a logic level combination of “11”.

The temporary storage signal TS[0] may be transmitted from the command/address decoder (460ofFIG. 54) to first input terminals of the first to fourth AND gates2223B˜2223E. The first AND gate2223B may receive the output signal of the output selector2223A, which is outputted through the first output terminal OUT1of the output selector2223A, through a second input terminal of the first AND gate2223B. An output terminal of the first AND gate2223B may be coupled to the clock terminal of the first latch circuit FF1included in the latch circuit2222. The second AND gate2223C may receive the output signal of the output selector2223A, which is outputted through the second output terminal OUT2of the output selector2223A, through a second input terminal of the second AND gate2223C. An output terminal of the second AND gate2223C may be coupled to the clock terminal of the second latch circuit FF2included in the latch circuit2222. The third AND gate2223D may receive the output signal of the output selector2223A, which is outputted through the third output terminal OUT3of the output selector2223A, through a second input terminal of the third AND gate2223D. An output terminal of the third AND gate2223D may be coupled to the clock terminal of the third latch circuit FF3included in the latch circuit2222. The fourth AND gate2223E may receive the output signal of the output selector2223A, which is outputted through the fourth output terminal OUT4of the output selector2223A, through a second input terminal of the fourth AND gate2223E. An output terminal of the fourth AND gate2223E may be coupled to the clock terminal of the fourth latch circuit FF4included in the latch circuit2222.

The first AND gate2223B may perform a logical AND operation of the temporary storage signal TS[0] and the output signal outputted through the first output terminal OUT1of the output selector2223A to generate a first clock signal. The first clock signal generated by the logical AND operation of the first AND gate2223B may be transmitted to the clock terminal of the first latch circuit FF1. The second AND gate2223C may perform a logical AND operation of the temporary storage signal TS[0] and the output signal outputted through the second output terminal OUT2of the output selector2223A to generate a second clock signal. The second clock signal generated by the logical AND operation of the second AND gate2223C may be transmitted to the clock terminal of the second latch circuit FF2. The third AND gate2223D may perform a logical AND operation of the temporary storage signal TS[0] and the output signal outputted through the third output terminal OUT3of the output selector2223A to generate a third clock signal. The third clock signal generated by the logical AND operation of the third AND gate2223D may be transmitted to the clock terminal of the third latch circuit FF3. The fourth AND gate2223E may perform a logical AND operation of the temporary storage signal TS[0] and the output signal outputted through the fourth output terminal OUT4of the output selector2223A to generate a fourth clock signal. The fourth clock signal generated by the logical AND operation of the fourth AND gate2223E may be transmitted to the clock terminal of the fourth latch circuit FF4.

The second input selector2224B may have the first to fourth input terminals IN1˜IN4, the output terminal OUT, and a selection control terminal S2. In an embodiment, the second input selector2224B may be realized using a 4-to-1 multiplexer. The first input terminal IN1of the second input selector2224B may be coupled to the output terminal Q of the first latch circuit FF1. The second input terminal IN2of the second input selector2224B may be coupled to the output terminal Q of the second latch circuit FF2. The third input terminal IN3of the second input selector2224B may be coupled to the output terminal Q of the third latch circuit FF3. The fourth input terminal IN4of the second input selector2224B may be coupled to the output terminal Q of the fourth latch circuit FF4. The output terminal OUT of the second input selector2224B may be coupled to the second input terminal IN2of the first input selector2224A. In addition, the output terminal OUT of the second input selector2224B may also be coupled to the output circuit (1230ofFIG. 55), as described with reference toFIG. 55.

The accumulation latch selection signal ALS[1:0] corresponding to a selection control signal may be inputted to the selection control terminal S2of the second input selector2224B. The second input selector2224B may output the data inputted to one of the first to fourth input terminals IN1˜IN4, which is selected by the accumulation latch selection signal ALS[1:0], through the output terminal OUT. In an embodiment, the data inputted to the first input terminal IN1(i.e., the data outputted from the first latch circuit FF1) may be outputted through the output terminal OUT of the second input selector2224B when the accumulation latch selection signal ALS[1:0] has a logic level combination of “00”, and the data inputted to the second input terminal IN2(i.e., the data outputted from the second latch circuit FF2) may be outputted through the output terminal OUT of the second input selector2224B when the accumulation latch selection signal ALS[1:0] has a logic level combination of “01”. Moreover, the data inputted to the third input terminal IN3(i.e., the data outputted from the third latch circuit FF3) may be outputted through the output terminal OUT of the second input selector2224B when the accumulation latch selection signal ALS[1:0] has a logic level combination of “10”, and the data inputted to the fourth input terminal IN4(i.e., the data outputted from the fourth latch circuit FF4) may be outputted through the output terminal OUT of the second input selector2224B when the accumulation latch selection signal ALS[1:0] has a logic level combination of “11”.

FIGS. 57 and 58illustrate a first MAC arithmetic operation of a first matrix group column unit of the PIM device400B illustrated inFIG. 54. Specifically,FIG. 57illustrates an accumulative adding calculation of the first MAC arithmetic operation of the first matrix group column unit, andFIG. 58illustrates a latch operation of the first MAC arithmetic operation of the first matrix group column unit. In the present embodiment, the first MAC arithmetic operations of the matrix group column unit may be achieved by sequentially performing the first MAC arithmetic operation of the first matrix group column unit for the weight sub-matrix WSM11in the first weight matrix group column WMGC1and the vector sub-matrix VSM11in the first vector matrix group row VMGR1, the first MAC arithmetic operation of the second matrix group column unit for the weight sub-matrix WSM21in the first weight matrix group column WMGC1and the vector sub-matrix VSM11in the first vector matrix group row VMGR1, the first MAC arithmetic operation of the third matrix group column unit for the weight sub-matrix WSM31in the first weight matrix group column WMGC1and the vector sub-matrix VSM11in the first vector matrix group row VMGR1, and the first MAC arithmetic operation of the fourth matrix group column unit for the weight sub-matrix WSM41in the first weight matrix group column WMGC1and the vector sub-matrix VSM11in the first vector matrix group row VMGR1, as described with reference toFIG. 34.

First, referring toFIG. 57, the accumulative adder2221of the accumulator2220may receive addition result data DMA11from the adder tree. A process that the addition result data DMA11are outputted from the adder tree may be the same as the process described with reference toFIGS. 37 and 38. Prior to the accumulative adding calculation of the accumulative adder2221, the command/address decoder (460ofFIG. 54) may output the temporary copy signal TC[0] having a logic “high” level, the temporary storage signal TS[0] having a logic “high” level, and the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”.

The output selector2223A may output a logic high level signal HI, which is transmitted to the second input terminal of the first AND gate2223B, through the first output terminal OUT1in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. Because the temporary storage signal TS[0] having a logic “high” level is inputted to the first input terminal of the first AND gate2223B, the first AND gate2223B may output a logic high level signal HI to the clock terminal of the first latch circuit FF1. The first latch circuit FF1may output an initial value of “0” to the first input terminal IN1of the second input selector2224B. The second input selector2224B may output the data (i.e., the data having a value of “0” outputted from the first latch circuit FF1), which are inputted to the first input terminal IN1selected by the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”, through the output terminal OUT of the second input selector2224B. The data having a value of “0” outputted from the second input selector2224B may be transmitted to the second input terminal IN2of the first input selector2224A.

The first input selector2224A may output the data (having a value of “0”), which are inputted to the second input terminal IN2selected by the temporary copy signal TC[0] having a logic “high” level, to the input terminal of the temporary latch circuit FF0. The temporary latch circuit FF0may be synchronized with a rising edge of the update signal UPDATE to latch and output the data having a value of “0”. The data having a value of “0” outputted from the temporary latch circuit FF0may be transmitted to the second input terminal of the accumulative adder2221to be used as the feedback data DF. When the feedback data DF are outputted from the temporary latch circuit FF0, the command/address decoder (460ofFIG. 54) may change a logic level of the update signal UPDATE from a logic “high” level into a logic “low” level. The accumulative adder2221may perform an adding calculation of the addition result data DMA11inputted to the first input terminal and the feedback data DF having a value of “0” inputted to the second input terminal, thereby generating and outputting accumulated addition data DMACC11. As described with reference toFIG. 38, the accumulated addition data DMACC11may correspond to data which are generated by a matrix multiplying calculation of the weight data W1.1˜W1.16arrayed in the first row of the weigh sub-matrix WSM11illustrated inFIG. 34and the vector data V1˜V16in the vector sub-matrix VSM11illustrated inFIG. 34.

Next, referring toFIG. 58, when the feedback data DF are transmitted to the accumulative adder2221, the command/address decoder (460ofFIG. 54) may change a logic level of the temporary copy signal TC[O] from a logic “high” level into a logic “low” level. In addition, the command/address decoder (460ofFIG. 54) may change a logic level of the update signal UPDATE from a logic “low” level into a logic “high” level. The accumulative adder2221may output the accumulated addition data DMACC11to the first input terminal IN1of the first input selector2224A. The first input selector2224A may output the data (i.e., the accumulated addition data DMACC11), which are inputted to the first input terminal IN1selected by the temporary copy signal TC[0] having a logic “low” level, to the input terminal of the temporary latch circuit FF0. The temporary latch circuit FF0may be synchronized with a rising edge of the update signal UPDATE to latch and output the accumulated addition data DMACC11. The accumulated addition data DMACC11outputted from the temporary latch circuit FF0may be transmitted to the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector2223A of the latch circuit selector2223may output a logic high level signal HI through the first output terminal OUT1in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. Thus, the logic high level signal HI may be inputted to the second input terminal of the first AND gate2223B. In such a case, the temporary storage signal TS[0] having a logic “high” level may be transmitted from the command/address decoder460to the first input terminals of the first to fourth AND gates2223B˜2223E. Thus, while the first AND gate2223B outputs the logic high level signal HI to the clock terminal of the first latch circuit FF1, the second to fourth AND gates2223C,2223D, and2223E may output the logic low level signals LO to respective clock terminals of the second to fourth latch circuits FF2˜FF4. The first latch circuit FF1may be synchronized with the logic high level signal HI outputted from the first AND gate2223B to latch and output the accumulated addition data DMACC11, which are outputted from the output terminal Q of the temporary latch circuit FF0, to the first input terminal IN1of the second input selector2224B.

FIG. 59illustrates a status of the accumulator2220after termination of the first MAC arithmetic operations of the first to fourth matrix group column units of the PIM device400B illustrated inFIG. 54. As described with reference toFIGS. 57 and 58, when the first MAC arithmetic operation of the first matrix group column unit terminates, the accumulated addition data DMACC11may be latched in the first latch circuit FF1. After the first MAC arithmetic operation of the first matrix group column unit terminates, the PIM device400B may sequentially perform the first MAC arithmetic operations of the second to fourth matrix group column units with alteration of the weight data in the same way as the first MAC arithmetic operation of the first matrix group column unit. Accordingly, the accumulated addition data DMACC12may be latched in the second latch circuit FF2by the first MAC arithmetic operation of the second matrix group column unit, and the accumulated addition data DMACC13may be latched in the third latch circuit FF3by the first MAC arithmetic operation of the third matrix group column unit. In addition, the accumulated addition data DMACC14may be latched in the fourth latch circuit FF4by the first MAC arithmetic operation of the fourth matrix group column unit.

As described with reference toFIGS. 39 to 44, the accumulated addition data DMACC12may correspond to data which are generated by a matrix multiplying calculation of the weight data W9.1˜W9.16arrayed in the first row of the weigh sub-matrix WSM21illustrated inFIG. 34and the vector data V1˜V16in the vector sub-matrix VSM11illustrated inFIG. 34, and the accumulated addition data DMACC13may correspond to data which are generated by a matrix multiplying calculation of the weight data W17.1˜W17.16arrayed in the first row of the weigh sub-matrix WSM31illustrated inFIG. 34and the vector data V1˜V16in the vector sub-matrix VSM11illustrated inFIG. 34. Moreover, the accumulated addition data DMACC14may correspond to data which are generated by a matrix multiplying calculation of the weight data W25.1˜W25.16arrayed in the first row of the weigh sub-matrix WSM41illustrated inFIG. 34and the vector data V1-V16in the vector sub-matrix VSM11illustrated inFIG. 34.

When the logic high level signal HI is inputted to the clock terminal of the first latch circuit FF1while the accumulated addition data DMACC11˜DMACC14are latched in the respective first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC11may be transmitted from the first latch circuit FF1to the first input terminal IN1of the second input selector2224B. In addition, when the logic high level signal HI is inputted to the clock terminal of the second latch circuit FF2while the accumulated addition data DMACC11˜DMACC14are latched in the respective first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC12may be transmitted from the second latch circuit FF2to the second input terminal IN2of the second input selector2224B. Moreover, when the logic high level signal HI is inputted to the clock terminal of the third latch circuit FF3while the accumulated addition data DMACC11˜DMACC4are latched in the respective first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC13may be transmitted from the third latch circuit FF3to the third input terminal IN3of the second input selector2224B. Furthermore, when the logic high level signal HI is inputted to the clock terminal of the fourth latch circuit FF4while the accumulated addition data DMACC11˜DMACC14are latched in the respective first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC14may be transmitted from the fourth latch circuit FF4to the fourth input terminal IN4of the second input selector2224B.

FIGS. 60 and 61illustrate a second MAC arithmetic operation of the first matrix group column unit of the PIM device400B illustrated inFIG. 54. Specifically,FIG. 60illustrates an accumulative adding calculation of the second MAC arithmetic operation of the first matrix group column unit, andFIG. 61illustrates a latch operation of the second MAC arithmetic operation of the first matrix group column unit. In the present embodiment, the second MAC arithmetic operations of the matrix group column unit in the PIM device400B may be achieved by sequentially performing the second MAC arithmetic operation of the first matrix group column unit for the weight sub-matrix WSM12in the second weight matrix group column WMGC2and the vector sub-matrix VSM21in the second vector matrix group row VMGR2, the second MAC arithmetic operation of the second matrix group column unit for the weight sub-matrix WSM22in the second weight matrix group column WMGC2and the vector sub-matrix VSM21in the second vector matrix group row VMGR2, the second MAC arithmetic operation of the third matrix group column unit for the weight sub-matrix WSM32in the second weight matrix group column WMGC2and the vector sub-matrix VSM21in the second vector matrix group row VMGR2, and the second MAC arithmetic operation of the fourth matrix group column unit for the weight sub-matrix WSM42in the second weight matrix group column WMGC2and the vector sub-matrix VSM21in the second vector matrix group row VMGR2, as described with reference toFIG. 34.

First, referring toFIG. 60, the accumulative adder2221of the accumulator2220may receive addition result data DMA21from the adder tree. A process that the addition result data DMA21are outputted from the adder tree may be the same as the process described with reference toFIGS. 45 and 46. Prior to the accumulative adding calculation of the accumulative adder2221, the command/address decoder (460ofFIG. 54) may output the temporary copy signal TC[0] having a logic “high” level, the temporary storage signal TS[0] having a logic “high” level, and the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”.

The output selector2223A may output a logic high level signal HI, which is transmitted to the second input terminal of the first AND gate2223B, through the first output terminal OUT1in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. Because the temporary storage signal TS[0] having a logic “high” level is inputted to the first input terminal of the first AND gate2223B, the first AND gate2223B may output a logic high level signal HI to the clock terminal of the first latch circuit FF1. The first latch circuit FF1may output the latched data (i.e., the accumulated addition data DMACC11which are generated by the first MAC arithmetic operation of the first matrix group column unit) to the first input terminal IN1of the second input selector2224B. The second input selector2224B may output the data (i.e., the accumulated addition data DMACC11outputted from the first latch circuit FF1), which are inputted to the first input terminal IN1selected by the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”, through the output terminal OUT of the second input selector2224B. The accumulated addition data DMACC11outputted from the second input selector2224B may be transmitted to the second input terminal IN2of the first input selector2224A.

The first input selector2224A may output the data (i.e., the accumulated addition data DMACC11), which are inputted to the second input terminal IN2selected by the temporary copy signal TC[0] having a logic “high” level, to the input terminal of the temporary latch circuit FF0. The temporary latch circuit FF0may be synchronized with a rising edge of the update signal UPDATE to latch and output the accumulated addition data DMACC11. The accumulated addition data DMACC11outputted from the temporary latch circuit FF0may be transmitted to the second input terminal of the accumulative adder2221to be used as the feedback data DF. When the accumulated addition data DMACC11are outputted from the temporary latch circuit FF0, the command/address decoder (460ofFIG. 54) may change a logic level of the update signal UPDATE from a logic “high” level into a logic “low” level. The accumulative adder2221may perform an adding calculation of the addition result data DMA21inputted to the first input terminal and the accumulated addition data DMACC11(corresponding to the feedback data DF) inputted to the second input terminal, thereby generating and outputting accumulated addition data DMACC21. As described with reference toFIG. 38, the accumulated addition data DMACC21may correspond to data which are generated by a matrix multiplying calculation of the weight data W1.17˜W1.32arrayed in the first row of the weigh sub-matrix WSM12illustrated inFIG. 34and the vector data V17˜V32in the vector sub-matrix VSM21illustrated inFIG. 34.

Next, referring toFIG. 61, when the feedback data DF are transmitted to the accumulative adder2221, the command/address decoder (460ofFIG. 54) may change a logic level of the temporary copy signal TC[0] from a logic “high” level into a logic “low” level. In addition, the command/address decoder (460ofFIG. 54) may change a logic level of the update signal UPDATE from a logic “low” level into a logic “high” level. The accumulative adder2221may output the accumulated addition data DMACC21to the first input terminal IN1of the first input selector2224A. The first input selector2224A may output the data (i.e., the accumulated addition data DMACC21), which are inputted to the first input terminal IN1selected by the temporary copy signal TC[0] having a logic “low” level, to the input terminal of the temporary latch circuit FF0. The temporary latch circuit FF0may be synchronized with a rising edge of the update signal UPDATE to latch and output the accumulated addition data DMACC21. The accumulated addition data DMACC21outputted from the temporary latch circuit FF0may be transmitted to the input terminals of the first to fourth latch circuits FF1˜FF4.

The output selector2223A of the latch circuit selector2223may output a logic high level signal HI through the first output terminal OUT1in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. Thus, the logic high level signal HI may be inputted to the second input terminal of the first AND gate2223B. In such a case, the temporary storage signal TS[0] having a logic “high” level may be transmitted from the command/address decoder460to the first input terminals of the first to fourth AND gates2223B˜2223E. Thus, while the first AND gate2223B outputs the logic high level signal HI to the clock terminal of the first latch circuit FF1, the second to fourth AND gates2223C,2223D, and2223E may output the logic low level signals LO to respective ones of the clock terminals of the second to fourth latch circuits FF2˜FF4. The first latch circuit FF1may be synchronized with the logic high level signal HI outputted from the first AND gate2223B to latch and output the accumulated addition data DMACC21, which are outputted from the output terminal Q of the temporary latch circuit FF0, to the first input terminal IN1of the second input selector2224B.

FIG. 62illustrates a status of the accumulator2220after termination of the second MAC arithmetic operations of the first to fourth matrix group column units of the PIM device400B illustrated inFIG. 54. As described with reference toFIGS. 60 and 61, when the second MAC arithmetic operation of the first matrix group column unit terminates, the accumulated addition data DMACC21may be latched in the first latch circuit FF1. After the second MAC arithmetic operation of the first matrix group column unit terminates, the PIM device400B may sequentially perform the second MAC arithmetic operations of the second to fourth matrix group column units with alteration of the weight data in the same way as the second MAC arithmetic operation of the first matrix group column unit. Accordingly, the accumulated addition data DMACC22may be latched in the second latch circuit FF2by the second MAC arithmetic operation of the second matrix group column unit, and the accumulated addition data DMACC23may be latched in the third latch circuit FF3by the second MAC arithmetic operation of the third matrix group column unit. In addition, the accumulated addition data DMACC24may be latched in the fourth latch circuit FF4by the second MAC arithmetic operation of the fourth matrix group column unit.

As described with reference toFIGS. 47 to 52, the accumulated addition data DMACC22may correspond to data which are generated by a matrix multiplying calculation of the weight data W9.17˜W9.32arrayed in the first row of the weigh sub-matrix WSM22illustrated inFIG. 34and the vector data V17˜V32in the vector sub-matrix VSM21illustrated inFIG. 34, and the accumulated addition data DMACC23may correspond to data which are generated by a matrix multiplying calculation of the weight data W17.17˜W17.32arrayed in the first row of the weigh sub-matrix WSM32illustrated inFIG. 34and the vector data V17˜V32in the vector sub-matrix VSM21illustrated inFIG. 34. Moreover, the accumulated addition data DMACC24may correspond to data which are generated by a matrix multiplying calculation of the weight data W25.17˜W25.32arrayed in the first row of the weigh sub-matrix WSM42illustrated inFIG. 34and the vector data V17˜V32in the vector sub-matrix VSM21illustrated inFIG. 34.

When the logic high level signal HI is inputted to the clock terminal of the first latch circuit FF1while the accumulated addition data DMACC21˜DMACC24are latched in respective one of the first to fourth latch circuits FF1˜FF4by the second MAC arithmetic operations of the first to fourth matrix group column units, the accumulated addition data DMACC21may be transmitted from the first latch circuit FF1to the first input terminal IN1of the second input selector2224B. In addition, when the logic high level signal HI is inputted to the clock terminal of the second latch circuit FF2while the accumulated addition data DMACC21˜DMACC24are latched in respective one of the first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC22may be transmitted from the second latch circuit FF2to the second input terminal IN2of the second input selector2224B. Moreover, when the logic high level signal HI is inputted to the clock terminal of the third latch circuit FF3while the accumulated addition data DMACC21˜DMACC24are latched in respective one of the first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC23may be transmitted from the third latch circuit FF3to the third input terminal IN3of the second input selector2224B. Furthermore, when the logic high level signal HI is inputted to the clock terminal of the fourth latch circuit FF4while the accumulated addition data DMACC21˜DMACC24are latched in respective one of the first to fourth latch circuits FF1˜FF4, the accumulated addition data DMACC24may be transmitted from the fourth latch circuit FF4to the fourth input terminal IN4of the second input selector2224B.

FIG. 63illustrates an operation for outputting the MAC result data MAC_RST1from the first MAC operator MAC(0) included in the PIM device400B illustrated inFIG. 54. Referring toFIG. 63, in order to control the output operation of the MAC result data MAC_RST1, the command/address decoder460may output the result read signal RD_RST having a logic “high” level and the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. In addition, the command/address decoder460may output the temporary storage signal TS[0] having a logic “high” level. In such a case, a logic high level signal HI may be inputted to the clock terminal of the first latch circuit FF1by the accumulation latch selection signal ALS[1:0] having a logic level combination of “00” and the temporary storage signal TS[0] having a logic “high” level. The first latch circuit FF1may be synchronized with the logic high level signal HI to output the accumulated addition data DMACC21latched in the first latch circuit FF1. The accumulated addition data DMACC21may be transmitted to the first input terminal IN1of the second input selector2224B. The second input selector2224B may output the accumulated addition data DMACC21, which are inputted to the first input terminal IN1, through the output terminal OUT in response to the accumulation latch selection signal ALS[1:0] having a logic level combination of “00”. The accumulated addition data DMACC21outputted from the second input selector2224B may be transmitted to the output circuit1230. The output circuit1230may output the accumulated addition data DMACC21as the MAC result data MAC_RST1in response to the result read signal RD_RST having a logic “high” level which is outputted from the command/address decoder460.

Although not shown in the drawings, when the accumulation latch selection signal ALS[1:0] having a logic level combination of “01” is transmitted from the command/address decoder460to the output selector2223A and the second input selector2224B, the accumulated addition data DMACC22latched in the second latch circuit FF2may be transmitted to the second input terminal IN2of the second input selector2224B. The accumulated addition data DMACC22may be outputted through the output terminal OUT of the second input selector2224B and may be transmitted to the output circuit1230. The output circuit1230may output the accumulated addition data DMACC22as MAC result data MAC_RST9in response to the result read signal RD_RST having a logic “high” level. In addition, when the accumulation latch selection signal ALS[1:0] having a logic level combination of “10” is transmitted from the command/address decoder460to the output selector2223A and the second input selector2224B, the accumulated addition data DMACC23latched in the third latch circuit FF3may be transmitted to the third input terminal IN3of the second input selector2224B. The accumulated addition data DMACC23may be outputted through the output terminal OUT of the second input selector2224B and may be transmitted to the output circuit1230. The output circuit1230may output the accumulated addition data DMACC23as MAC result data MAC_RST17in response to the result read signal RD_RST having a logic “high” level. Furthermore, when the accumulation latch selection signal ALS[1:0] having a logic level combination of “11” is transmitted from the command/address decoder460to the output selector2223A and the second input selector2224B, the accumulated addition data DMACC24latched in the fourth latch circuit FF4may be transmitted to the fourth input terminal IN4of the second input selector2224B. The accumulated addition data DMACC24may be outputted through the output terminal OUT of the second input selector2224B and may be transmitted to the output circuit1230. The output circuit1230may output the accumulated addition data DMACC24as MAC result data MAC_RST25in response to the result read signal RD_RST having a logic “high” level.

FIG. 64is a block diagram illustrating a PIM device400C according to yet another embodiment of the present disclosure. Referring toFIG. 64, the PIM device400C may include “L”-number of memory banks (i.e., first to Lthmemory banks BK(0)˜BK(L−1)), a global buffer GB, “L”-number of MAC operators (i.e., first to LthMAC operators MAC(0)˜MAC(L−1)), and a command/address decoder470(where, “L” is a natural number which is equal to or greater than two). The memory banks BK(0)˜BK(L−1) included in the PIM device400C may have the same configuration and function as the memory banks BK(0)˜BK(L−1) of the PIM device400A described with reference toFIG. 31, and the global buffer GB of the PIM device400C may have the same configuration and function as the global buffer GB of the PIM device400A described with reference toFIG. 31. Each of the first to LthMAC operators MAC(0)˜MAC(L−1) included in the PIM device400C may also have the same configuration and function as each of the first to LthMAC operators MAC(0)˜MAC(L−1) of the PIM device400A described with reference toFIG. 31except the accumulator. That is, each of the first to LthMAC operators MAC(0)˜MAC(L−1) included in the PIM device400C may be different from each of the first to LthMAC operators MAC(0)˜MAC(L−1) included in the PIM device400A in terms of a configuration of only the accumulator.

The command/address decoder470may receive a command CMD and an address ADDR from an external device such as a controller. The command/address decoder470may decode the command CMD and the address ADDR to generate and output various control signals RD, WT, MAC, RD_RST, UPDATE, ALS, TC, TS, and T_RD_RST for controlling operations of the memory banks BK(0)˜BK(L−1), the global buffer GB, and the MAC operators MAC(0)˜MAC(L−1) as well as generating an address signal ADDR_S. The read signal RD, the write signal WT, the MAC signal MAC, the result read signal RD_RST, the update signal UPDATE, and the accumulation latch selection signal ALS illustrated inFIG. 64may be the same signals as the read signal RD, the write signal WT, the MAC signal MAC, the result read signal RD_RST, the update signal UPDATE, and the accumulation latch selection signal ALS described with reference toFIG. 31. The temporary copy signal TC and the temporary storage signal TS illustrated inFIG. 64may be the same signals as the temporary copy signal TC and the temporary storage signal TS described with reference toFIG. 54. The temporary result read signal T_RD_RST among the various control signals RD, WT, MAC, RD_RST, UPDATE, ALS, TC, TS, and T_RD_RST may control an operation of an accumulator included in each of the MAC operators MAC(0)˜MAC(L−1) outputting interim result data during the MAC arithmetic operation.

FIG. 65is a block diagram illustrating an example of a configuration of an accumulator3220and an output circuit3230included in each of the MAC operators MAC(0)˜MAC(L−1) of the PIM device400C illustrated inFIG. 64. The accumulator3220may have the same configuration as the accumulator2220described with reference toFIG. 56except for an output line of the temporary latch circuit FF0. Thus, inFIG. 65, the same reference numerals and symbols as used inFIG. 56denote the same components. Accordingly, descriptions of the accumulator3220will be omitted or briefly mentioned hereinafter to avoid duplicate explanation. Referring toFIG. 65, each of the MAC operators MAC(0)˜MAC(L−1) included in the PIM device400C may be different from the MAC operator illustrated inFIG. 63in that the output circuit3230includes a first output circuit3231and a second output circuit3232. The first output circuit3231of the output circuit3230may be the same as the output circuit1230described with reference toFIG. 63. The second output circuit3232of the output circuit3230may have an input terminal coupled to the output terminal of the temporary latch circuit FF0included in the accumulator3220and an output terminal coupled to the output terminal of the first output circuit3231. The second output circuit3232may output the accumulated addition data DMACC, which are outputted from the temporary latch circuit FF0, as MAC result data MAC_RST in response to the temporary result read signal T_RD_RST having a logic “high” level. According to the MAC operator illustrated inFIG. 65, the accumulated addition data DMACC generated prior to termination of the MAC arithmetic operation (i.e., during the MAC arithmetic operation) may be outputted from the MAC operator.

A limited number of possible embodiments for the present teachings have been presented above for illustrative purposes. Those of ordinary skill in the art will appreciate that various modifications, additions, and substitutions are possible. While this patent document contains many specifics, these should not be construed as limitations on the scope of the present teachings or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.