PIM device, computing system including the PIM device, and operating method of the PIM device

A processing in memory (PIM) device includes a memory configured to receive data through a first path from a host processor provided outside the PIM device, and an information gatherer configured to receive the data through a second path connected to the first path when the data is transferred to the memory via the first path, and to generate information by processing the data received through the second path.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit under 35 USC § 119(a) of Korean Patent Application No. 10-2019-0178165, filed on Dec. 30, 2019, in the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes.

BACKGROUND

The present disclosure relates to process in memory (PIM) devices, computing systems including the PIM devices, and methods of operating the PIM devices.

2. Description of Related Art

Functions of semiconductor memory devices have been separated from the functions of a processor that performs computational operations. Accordingly, for applications such as neural networks, big data, and the Internet of things (IoT), which require operations on large amounts of data, data bottlenecks frequently occur as a large amount of data is transmitted and received between a semiconductor memory device and a processor.

In order to solve such a problem, a research on a process in memory (PIM) as a semiconductor memory device in which memory functions are combined with the functions of a processor performing various computational operations is being conducted.

SUMMARY

In one general aspect, there is provided a processing in memory (PIM) device including a memory configured to receive data through a first path from a host processor provided outside the PIM device, and an information gatherer configured to receive the data through a second path connected to the first path when the data is transferred to the memory via the first path, and to generate information by processing the data received through the second path.

The memory may be configured to return the data and the information gatherer is further configured to return the information according to a request of the host processor.

According to a request of the host processor, the memory may return the data to the host processor through the first path, and the information gatherer may return the information to the host processor through the second path.

According to a request of the host processor, the memory may return the data to the host processor via a third path, and the information gatherer may return the information to the host processor through a fourth path that connects the third path to the information gatherer.

The PIM device may include a memory buffer to temporarily store the data transferred from the host processor, wherein the memory buffer may transfer the data to the memory through the first path and may transfer the data to the information gatherer through the second path.

The information gatherer may include a processor configured to process the data to generate the information, and at least one register configured to store settings regarding the information, the settings being received from the host processor.

The information gatherer may include at least one of an enabling register to store information whether or not to gather the information from the data, a range register to store a range for gathering the information, a mask register to store a type of information, or an information register file to store the information generated according to the settings.

The PIM device may be any one of a dynamic random access memory (DRAM), a high band memory (HBM), or a load reduced dual in-line memory module (LRDIMM).

In another general aspect, there is provided a computing system including a host processor, and a processing in memory (PIM) device comprising a memory to receive data from the host processor and an information gatherer configured to receive the data and to generate information by processing the data, wherein the PIM device is configured to transfer the data to the memory via a first path and to transfer the information to the information gatherer via a second path connected to the first path, the host processor requests any one or any combination of the data and the information from the PIM device, and the PIM device is further configured to return the data from the memory in response to a request for the data, and to return the information from the information gatherer in response to a request for the information.

In another general aspect, there is provided an operating method of a processing in memory (PIM) device, the operating method including transferring data received from a host processor provided outside the PIM device to a memory through a first path, transferring the data to an information gatherer via a second path connected to the first path, in response to the data being transferred to the memory via the first path, and generating information by processing the data by the information gatherer.

The operating method may include returning the data from the memory or returning the information from the information gatherer according to a request of the host processor.

The returning of the data from the memory may include returning the data to the host processor through the first path, and the returning of the information from the information gatherer may include returning the information to the host processor through the second path.

The returning of the data from the memory may include returning the data to the host processor through a third path, and the returning of the information from the information gatherer may include returning the information to the host processor through a fourth path that connects the third path to the information gatherer.

The operating method may include temporarily storing the data transferred from the host processor to a memory buffer, and transferring the data stored at the memory buffer to the memory through the first path and to the information gatherer through the second path.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particular examples only, and is not to be used to limit the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any one and any combination of any two or more of the associated listed items. As used herein, the terms “include,” “comprise,” and “have” specify the presence of stated features, numbers, operations, elements, components, and/or combinations thereof, but do not preclude the presence or addition of one or more other features, numbers, operations, elements, components, and/or combinations thereof.

Also, in the description of example embodiments, detailed description of structures or functions that are thereby known after an understanding of the disclosure of the present application will be omitted when it is deemed that such description will cause ambiguous interpretation of the example embodiments.

Hereinafter, examples will be described in detail with reference to the accompanying drawings, and like reference numerals in the drawings refer to like elements throughout.

FIG. 1is a diagram of illustrating an example of a configuration of a computing system1000including a PIM device100.

Referring toFIG. 1, the computing system1000may include a host processor200and the PIM device100. In the computing system1000inFIG. 1, although only components related to the embodiments are shown, but it should be understood that other general purpose components, such as, for example, an interfacing module for connecting with other electronic devices, an input/output module for receiving user input and outputting information may be used without departing from the spirit and scope of the illustrative examples described.

The computing system1000may be various devices and/or systems such as, for example, a smart phone, a mobile phone, a wearable device, (such as, a ring, a watch, a pair of glasses, glasses-type device, a bracelet, an ankle bracket, a belt, a necklace, an earring, a headband, a helmet, a device embedded in the cloths, or an eye glass display (EGD)), a computing device, for example, a server, a laptop, a notebook, a subnotebook, a netbook, an ultra-mobile PC (UMPC), a tablet personal computer (tablet), a phablet, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA), an ultra mobile personal computer (UMPC), a portable lab-top PC, electronic product, for example, a robot, a digital camera, a digital video camera, a portable game console, an MP3 player, a portable/personal multimedia player (PMP), a handheld e-book, a global positioning system (GPS) navigation, a personal navigation device, portable navigation device (PND), a handheld game console, an e-book, a television (TV), a high definition television (HDTV), a smart TV, a smart appliance, a smart home device, or a security device for gate control, a smart speaker, a robot, various Internet of Things (IoT) devices, or a kiosk and may be performed by an application, middleware, or an operating system installed on a user device, or a program of a server interoperating with the corresponding application.

The host processor200is, for example, a hardware apparatus configured to execute instructions or programs, or to control an overall operation of the computing system1000. The host processor200may include one processor core (single core) or a plurality of processor cores (multi-core). The host processor200may be implemented as a central processing unit (CPU), a graphic processing unit (GPU), an application processor (AP), a reconfigurable processor, a multicore processor, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), or any other type of multi- or single-processor configuration.

The host processor200may process or execute programs and/or data stored in the PIM device100. In an example, the host processor200may control the function of a neural network device by executing programs stored in the PIM device100.

The host processor200may include a memory controller for controlling the PIM device100. The memory controller controls the operation of a memory device by applying a command CMD and an address ADDR to control the memory device.

When writing data, the host processor200may transfer data to be written and an address corresponding to a memory space in which each data is to be stored to the PIM device100. The PIM device100may write data in a memory space corresponding to a received address. When reading data, the host processor200may transmit an address corresponding to a memory space in which data to be read is written to the PIM device100and may receive the data stored in the corresponding address from the PIM device100.

The computing system1000may further include an input/output device (I/O device) (not shown), and in an example, the host processor200may include control the I/O device and the PIM device100according to a memory-mapped I/O (MMIO) method.

The host processor200may transfer various data through a path connecting the host processor200and the PIM device100. For example, the host processor200may transfer various data and information to the PIM device100through a data bus for transmitting data and a control bus for transmitting a command CMD and an address ADDR.

The PIM device100may store programs, data, or instructions. In another example, the PIM device100may store input values required for a computation process of a neural network device and intermediate and final results generated as a result of the computation.

The PIM device100may include a memory110that stores data received from the host processor200provided outside the PIM device100, and an information gatherer120that acquires data transferring to the memory110on a data bus and processes the acquired data.

The PIM device100may include other general components in addition to the components shown inFIG. 1. For example, the PIM device100may further include an internal processor packaged together in a chip package of the memory110to process data therein and a memory buffer to temporarily store the transferred data.

In an example, the PIM device100may correspond to random access memory (RAM), such as, for example, dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory, or a device, such as, for example, High Bandwidth Memory (HBM), Load Reduced Dual In-line Memory Module (LRDIMM).

The memory110may receive and store data from the host processor200provided outside the PIM device100. Types of data to be received and stored by the memory110may vary. For example, the memory110may receive and store data such as, for example, voice recognition data, image data, and biometric information.

In an example, the memory110may receive data from the host processor200through a path L1. The path L1is a data moving path, for example, may be a data bus. The memory110stores data as-received through the path L1. In other words, when receiving data, the memory110stores the data without performing an additional operation or processing.

The memory110may return data at a request of the host processor200. In this case, the memory110may return data through a path L3.

In an example, the path L1through which the memory110receives data and the path L3for returning data may be the same path. In an example, the path L1for receiving data for the memory110and the path L3for returning the data may be designed differently in consideration of factors such as, electronic components through which the data passes for data processing and other paths that cross each other.

In an example, the memory110may be configured as a memory array including a plurality of memories110, and the memory array may form a memory bank. The host processor200may independently access each of the memory banks.

The information gatherer120may include a processor that performs data processing and computation and a memory device that stores information generated from data. Since the processor of the information gatherer120is designed according to a PIM architecture implemented along with the memory110on a single chip, it is possible to have a fast memory access with a low-latency. The memory device of the information gatherer120may be implemented along with various types of the memory110, and the memory device may be, for example, SRAM or a register.

When data is transferred from the host processor200to the memory110of the PIM device100, the information gatherer120may receive the data through a path L2that is connected to the path L1.

The host processor200may access the information gatherer120by using an address assigned to the information gatherer120according to a MMIO method and may write data or read information in the information gatherer120.

The processor of the information gatherer120may process the received data to generate information and store the generated information in a register. The information generated from the data is less than the data itself, and thus, may be temporarily stored in the register of the information gatherer120.

Data received by the information gatherer120and information generated from the data may vary. In an example, the data received by the information gatherer120may include data such as, for example, voice recognition data, image data, and biometric information.

In an example, the information gatherer120may generate statistical information from data. The statistical information generated by the information gatherer120may be used to perform neural network related functions in the host processor200, which will be described in detail with reference toFIG. 2.

It is inefficient for the host processor200to read all of the data stored in the memory110in the PIM device100to generate information. Accordingly, when the host processor200transmits data to the memory110to write the data in the memory110, the information gatherer120may also receive the data and may generate information by processing the data.

Thereafter, the information gatherer120may provide information according to a request of the host processor200. The host processor200may read data written in an address of the memory110by transferring the address of the memory110to the PIM device100and may read information written at the address of the information gatherer120by transferring the address of the information gatherer120.

As a result, the PIM device100may generate information through a single data processing process by using the information gatherer120. Execution time and energy needed for processing data and generating information may be reduced by providing information stored in the information gatherer120without additional operation in response to repeated information read request from the host processor200.

In an example, the information gatherer120may return the generated information to the host processor200through a path L4in response to a request of the host processor200. In an example, the path L4may be connected to the path L3through which the memory110returns data.

In an example, the path L2through which the information gatherer120receives data and the path L4for returning the information may be the same path. In another example, the path L2through which the information gatherer120receives data and the path L4for returning the information may be designed differently in consideration of electronic components for data processing and crossing other paths.

In an example, the memory110may include a plurality of memory banks, and the plurality of information gatherers120may each be independently connected to the memory bank. Accordingly, each of the information gatherers120may generate information by processing data received from the connected memory bank. As a result, when each of the plurality of information gatherers120is independently connected to the memory bank, information may be generated in units of memory banks, and thus, resolution may be increased when compared to generating information in units of the entire memory110.

The PIM device100may further include a memory buffer. To address differences in transfer rates that may occur between the respective components in the PIM device100and the host processor200, a memory buffer may temporarily store data and information transferred between the host processor200and the PIM device100.

In detail, the memory buffer may temporarily store data transferred from the host processor200to the memory110, data transferred from the host processor200to the information gatherer120, data transferred from the memory110to the host processor200, and Information transmitted from the information gatherer120to the host processor200.

In an example, the memory buffer may be located on the paths L1, L2, L3, and L4or may be connected to the paths L1, L2, L3, and L4. The memory buffer may transfer data to the memory110via the path L1, may transfer data to the information gatherer120via the path L2, may transfer data from the memory110to the host processor200via the path L3, and may transfer information from the information gatherer120to the host processor200via the path L4.

FIG. 2is a diagram illustrating an example of a configuration of the information gatherer120in the PIM device100.

Referring toFIG. 2, a processor122and a plurality of registers may be included in the information gatherer120.

The information gatherer120may increase a data processing speed of the processor122connected to the registers by writing information to a register which is a high speed storage device. The register may store intermediate results during data processing and final results of the processor122.

The host processor200may control the operation of the information gatherer120and settings regarding information through registers. Depending on the operation of the information gatherer120and the settings regarding the information, the number and type of registers may be varied. In an example, the host processor200may receive settings from a user and transfer the settings to each register. Each register may store the transmitted settings. Each register has a unique address, and the host processor200may transmit the settings of the information gatherer120and an address of the register in which the settings are to be stored to the PIM device100.

In an example, the information gatherer120may include at least any one of an enabling register123, a mask register124, a range register125, and an information register file126.

In an example, the enabling register123may store settings regarding whether information gathering is performed from data or not. The host processor200may control the enablement of the information gatherer120through the enabling register123.

The mask register124may store settings regarding the type of information to be generated from the data. The host processor200may control the type of information to be generated from the information gatherer120through the mask register124. For example, when the information gatherer120generates statistical information from data, the mask register124may set the type of statistical information to be generated, such as minimum and maximum values of the data, a sum of the data, a sum of the squares of the data, the number of zeros in the data. The minimum and maximum values of the data may be used for quantization, the sum of the data and the sum of the squares of the data may be used for normalization, and the number of zeros in the data may be used for load balancing of the processor.

The range register125may store settings regarding a range of data for generating information. The host processor200may control a range of addresses of data for generating information through the range register125. For example, when the information gatherer120generates statistical information from the image data, the range register125may set a pixel area of an image data to generate the statistical information.

The information register file126is a gathering of a plurality of registers and may store information generated from data. The information register file126may return the stored information at the request of the host processor200.

FIG. 3is a diagram illustrating an example of a method of operating the PIM device100. The operations inFIG. 3may be performed in the sequence and manner as shown, although the order of some operations may be changed or some of the operations omitted without departing from the spirit and scope of the illustrative examples described. Many of the operations shown inFIG. 3may be performed in parallel or concurrently. The blocks of the image processing method ofFIG. 3, and combinations of the blocks, are performed by an image processing apparatus. In an example, the image processing apparatus is implemented by special purpose hardware-based computer, and devices such as a processor, that perform the specified functions, or combinations of special purpose hardware and computer instructions included in the image processing apparatus. In addition to the description ofFIG. 3below, the descriptions ofFIGS. 1-2is also applicable toFIG. 3and are incorporated herein by reference. Thus, the above description may not be repeated here.

Referring toFIG. 3, the PIM device100may transfer data received from the host processor200provided outside the PIM device100to the memory110through a first path (S1100). The first path is a path L1connecting the host processor200to the memory110.

The PIM device100may receive data and an address through the path L1connecting the host processor200to the memory110and write data to the memory110designated according to the address. The PIM device100stores data in the memory110as it is received without processing the data. Afterwards, the PIM device100may return at least some of the data received from the memory110at the request of the host processor200.

When data is transferred to the memory110through the first path, the PIM device100may transfer data to the information gatherer120through a second path (S1200). The second path is a path L2connecting the path L1and the information gatherer120.

The PIM device100may generate and store information (S1300). The host processor200may receive input from a user such as, for example, whether to generate information from data, a range, type of information to be generated. The host processor200may transfer inputs received from the user to the PIM device100. The PIM device100may store inputs received from a user in a register of the information gatherer120and generate information from data according to the inputs received from the user.

The PIM device100may return information from the information collecting unit120according to a request of the host processor200(S1400). As a result, the PIM device100may generate information in advance by a single data processing through the information gatherer120, and provide the information for a plurality of information read requests of host processor200without an additional operation. This operation increases the efficiency and speed of procession because the need of processing the data at a host processor after reading the data from the memory device through the host processor is eliminated to generate information from data written in a memory device.

In the computing system1000including the PIM device100, information may be generated from data without additional access to the memory110, thereby reducing execution time and energy.

Also, the information gatherer120may be applied to a memory device without modifying an interface of the memory device, and thus, is advantageous for introduction to the memory device.

FIG. 4is a diagram illustrating an example of a configuration of a computing system1100including a DRAM300.

Referring toFIG. 4, the PIM device100is the DRAM300. The DRAM300may include various types, such as SDRAM, RDRAM, DDR SDRAM, LRDIMM, and HBM, andFIG. 4describes the application of an information gatherer320to a write and read path of a general DRAM.

Descriptions given with reference toFIGS. 1 to 3may be applied to the computing system1100including the DRAM300ofFIG. 4. In addition to the description ofFIG. 4below, the descriptions ofFIGS. 1-3is also applicable toFIG. 4and are incorporated herein by reference. Thus, the above description may not be repeated here. A data-in buffer340and the data-out buffer350are one type of memory buffer. The data-in buffer340may temporarily store data transferred from the host processor200to a bank memory array310. The data out buffer350may temporarily store data transferred from the bank memory array310to the host processor200.

The DRAM300may receive data from the host processor200through a path L1connected to the data-in buffer340and write the data to the bank memory array310. The DRAM300may return data from the bank memory array310to the host processor200through a path L3connected to the data-out buffer350. At this time, an I/O gate330is a kind of buffer and temporarily stores data to be read and written.

The DRAM300may provide data to the information gatherer320through a path L2connected to the data-in buffer340. The DRAM300may return information generated from the data from the information gatherer320to the host processor200through a path L4connected to the data-out buffer350.

FIG. 5is a diagram illustrating an example of a configuration of a computing system1200including an LRDIMM400.

Referring toFIG. 5, the PIM device100is the LRDIMM400. The LRDIMM400includes a memory buffer450. The LRDIMM400may distribute a load by sequentially arranging the load in the memory buffer450without depending on rank.

The LRDIMM400is a type of DRAM.FIG. 5shows an information gatherer420that is applied to the memory buffer450of the LRDIMM400. Descriptions given with reference toFIGS. 1 to 3may be applied to the computing system1200including the LRDIMM400ofFIG. 5. In addition to the description ofFIG. 5below, the descriptions ofFIGS. 1-3is also applicable toFIG. 5and are incorporated herein by reference. Thus, the above description may not be repeated here.

The LRDIMM400may write data to memory chips411,412,413, and414from the host processor200through a path L1or read data from the memory chips411,412,413, and414. The LRDIMM400may write data in the information gatherer420through a path L2, or read information generated by processing data from the information gatherer420.

The memory buffer450may temporarily store data transferred from the host processor200to the memory chips411,412,413, and414. When the data is transferred to the memory chips411,412,413, and414through the memory buffer450, the information gatherer420may receive data from the memory buffer450. Also, the information gatherer420may return information generated by processing the data to the host processor200through the memory buffer450.

FIG. 6is a diagram illustrating an example of a configuration of a computing system1300including an HBM500.

Referring toFIG. 6, the PIM device100is an HBM500. The HBM500is a type of DRAM. InFIG. 6, an information gatherer520is applied to a buffer-die530of the HBM500. Descriptions given with reference toFIGS. 1 to 3may be applied to the computing system1300including the HBM500. In addition to the description ofFIG. 6below, the descriptions ofFIGS. 1-3is also applicable toFIG. 6and are incorporated herein by reference. Thus, the above description may not be repeated here.

The HBM500stacks a plurality of core-dies540including a memory510, and may perform communication between the memory510and a host processor200through a through silicon via (TSV) of the core-die540. The HBM500may include a buffer-die530that performs as an interface between the core-dies540and the host processor200.

The HBM500may receive data from the host processor200through a path L1provided inside the buffer-die530and transfer the data to the core-dies540. The HBM500may receive data from the host processor200through a deserializer DES and alignment ALIGN that convert data in a physical layer PHY into a parallel form.

The HBM500may return data from the core-dies540to the host processor200via a path L3provided inside the buffer-die530. The HBM500may transfer data from the core-dies540to the host processor200through a serializer SER that converts a first-in first-out (FIFO) buffer and data of the physical layer PHY into a serial form.

The information gatherer520of the HBM500may be disposed in the buffer-die530. The HBM500may transfer data received from the host processor200to the information gatherer520through a path L2that is provided inside the buffer-die530and is connected to the path L1. The path L2may be connected to one point of the path L1located after the physical layer PHY based on a data flow direction.

The HBM500may transfer information generated from data in the information gatherer520to the host processor200through a path L4that is provided inside the buffer-die530and is connected to the path L3. The path L4may be connected to one point of the path L3located before the physical layer PHY based on the data flow direction.

Instructions or software to control a processor to implement the hardware components and perform the methods as described above are written as computer programs, code segments, instructions or any combination thereof, for individually or collectively instructing or configuring the processor or computer to operate as a machine or special-purpose computer to perform the operations performed by the hardware components and the methods as described above. In one example, the instructions or software include machine code that is directly executed by the processor or computer, such as machine code produced by a compiler. In another example, the instructions or software include higher-level code that is executed by the processor or computer using an interpreter.

Instructions or software to control. The above-described method of operating the PIM device100may be recorded in a non-transitory computer-readable recording medium are written as computer programs, code segments, instructions or any combination thereof, for individually or collectively instructing or configuring the PIM device100to operate as a machine or special-purpose computer to perform the operations performed by the hardware components and the methods as described above. Programmers of ordinary skill in the art can readily write the instructions or software based on the block diagrams and the flow charts illustrated in the drawings and the corresponding descriptions in the specification, which disclose algorithms for performing the operations performed by the hardware components and the methods as described above. Examples of non-transitory computer-readable recording media include read-only memory (ROM), random-access programmable read only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, blue-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), flash memory, card type memory such as multimedia card, secure digital (SD) card, or extreme digital (XD) card, magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state disks, and any other device that is configured to store the instructions or software and any associated data, data files, and data structures in a non-transitory manner and providing the instructions or software and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the instructions.

According to the present disclosure, when writing data received from an external device to a memory device, a PIM device generates information by processing the data and stores the information, and afterwards, the PIM device may return the generated information to the external device without additional access to the memory device. As a result, an execution time and energy required for data processing and information generation may be reduced.