Patent ID: 12189551

DETAILED DESCRIPTION

Specific structural or functional descriptions of examples of embodiments in accordance with concepts which are disclosed in this specification are illustrated only to describe the examples of embodiments in accordance with the concepts and the examples of embodiments in accordance with the concepts may be carried out by various forms but the descriptions are not limited to the examples of embodiments described in this specification.

FIG.1is a diagram illustrating a computing system10000according to an embodiment of the present disclosure.

Referring toFIG.1, the computing system10000may include a memory system1000and a host device2000.

The memory system1000may be configured to store data in response to control of the host device2000. Examples of the memory system1000may include a cellular phone, a smartphone, an MP3 player, a laptop computer, a desktop computer, a game player, a display, a tablet PC, and an in-vehicle infotainment system.

The memory system1000may be manufactured as one of various types of storage devices according to a host interface corresponding to a communication method with the host device2000. For example, the memory system1000may be configured as any one of various types of storage devices such as a solid state drive (SSD), a multimedia card in the form of an MMC, an eMMC, an RS-MMC and a micro-MMC, a secure digital card in the form of an SD, a mini-SD and a micro-SD, a universal serial bus (USB) storage device, a universal flash storage (UFS) device, a Personal Computer Memory Card International Association (PCMCIA) card type storage device, a peripheral component interconnection (PCI) card type storage device, a PCI express (PCI-E) card type storage device, a compact flash (CF) card, a smart media card, and a memory stick.

The memory system1000may be manufactured as any of various types of packages. For example, the memory system1000may be manufactured in any of various package types, such as a package-on-package (POP), a system-in-package (SIP), a system-on-chip (SOC), a multi-chip package (MCP), a chip-on-board (COB), a wafer-level fabricated package (WFP), and a wafer-level stack package (WSP).

The memory system1000may include a deep learning-based algorithm for training and inference with high performance so as to effectively use or provide a large amount of data. However, in a deep learning-based system, since an embedding operation is performed mainly by the host device2000, bandwidth constraints may be caused. In addition, since a large amount of service data is required, a memory capacity may become scarce.

The computing system10000according to an embodiment of the present disclosure may operate in response to control of the host device2000. More specifically, the host device2000may include a host processor and a host memory. The host processor may include a general-purpose processor such as a CPU, an Application Processor (AP), or a Digital Signal Processor (DSP); a graphics dedicated processor, such as a Graphics Processing Unit (GPU) or a Vision Processing Unit (VPU); or an artificial intelligence dedicated processor such as a Neural-network Processing Unit (NPU). In addition, the host memory may include an operating system or an application program for providing a recommended system service.

More specifically, the host device2000may broadcast a query to the memory system1000. In addition, the host device2000may control the memory system1000to acquire data corresponding to the query from the memory system1000by using the query. In addition, the host device2000may control the memory system1000to perform a pooling operation to generate embedding data by using the acquired data, and to provide the embedding data to the host device2000.

According to an embodiment, the computing system10000may solve the lack of the memory capacity by a pooled memory architecture which may enable additional memory capacity, and may alleviate bandwidth constraints by configuring to enable near data processing (NDP).

FIG.2is a diagram illustrating internal and external bandwidths of the memory system1000according to an embodiment of the present disclosure.

Referring toFIG.2, a computing system10000which includes the memory system1000and the host device2000is briefly described. According to an embodiment of the present disclosure, the computing system10000may determine a resource to perform an operation on the basis of a first bandwidth10and a second bandwidth20. The first bandwidth10may refer to the amount of data transmission between a near data processor (NDP) and a plurality of memory devices in the memory system1000. The first bandwidth10may refer to the amount of data transmitted per unit time from the plurality of memory devices to the NDP. In the same manner, the second bandwidth20may refer to the amount of data transmission per unit time from the memory system1000to the host device2000. The first bandwidth10and the second bandwidth20may be constant values which are determined depending on hardware. The host device2000may determine resources to perform a first operation and a second operation on raw data on the basis of the first bandwidth10and the second bandwidth20.

FIG.3is a diagram illustrating communication packets and a pre-processing operation according to an embodiment of the present disclosure.

Referring toFIG.3, communication packets between the host device2000and the memory system1000are shown. More specifically, a first communication packet11may be a message which is transferred from the host device2000to the memory system1000. In addition, the first communication packet11may include ‘Task ID’, ‘Opcode’, ‘Source Address’, ‘Source Size’ and ‘Destination Address’.

The first communication packet11may include 91 bits, in total, and four bits may be allocated to ‘Task ID’. The ‘Task ID’ may indicate an operational state of the host device2000. For example, the ‘Task ID’ may indicate whether an operation of the host device2000is running or terminated. The host device2000may rearrange an operation of the memory system1000.

In addition, three bits may be allocated to the ‘Opcode’, and data for differentiating a plurality of embedding operations may be included in the ‘Opcode’. More specifically, the host device2000may include the ‘Opcode’ to differentiate initialization, an inference operation, and a learning operation from each other. According to an embodiment, the host device2000may identify a type of an operation to be performed with reference to the ‘Opcode’.

In addition, 32 bits may be allocated to the ‘Source Address’. The memory system1000may include data about a source address of a query or a gradient. More specifically, the host device2000may include data about an address of a query or a gradient that the memory system1000needs to fetch from the host memory by using the ‘Source Address’. The gradient may refer to data for updating the embedding table, for example, embedding data which includes a weight.

In addition, 20 bits may be allocated to the ‘Source Size’, and the ‘Source Size’ may include data about a size of a query or a gradient. In addition, the ‘Destination Address’ may include an address of a host memory which receives a result of performing an internal operation by the memory system1000.

The host device2000may communicate with the memory system1000using the first communication packet11. In addition, the memory system1000may transmit a second communication packet12, which is a response message, when receiving the first communication packet11from the host device2000. The second communication packet12may include ‘Task ID’ and ‘Opcode’.

FIG.4is a diagram illustrating near data processing (NDP) according to an embodiment of the present disclosure.

Referring toFIG.4, a first memory system1000amay include a memory device100and a near data processor (NDP)200. The first memory system1000amay be a part of the memory system1000as shown inFIG.1orFIG.2.

The memory device100may store data or utilize stored data. More specifically, the memory device100may operate in response to control of a memory controller210. In addition, the memory device100may include a plurality of memory dies, each of which may include a memory cell array including a plurality of memory cells that store data.

The memory cell array may include a plurality of memory blocks. Each of the memory blocks may include a plurality of memory cells. Each memory block may include a plurality of pages. A page may be a unit for storing data in the memory device100, or a unit for reading data stored in the memory device100

Examples of the memory device100may include Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), Low Power Double Data Rate4 (LPDDR4) SDRAM, Graphics Double Data Rate (GDDR) SDRAM, Low Power DDR (LPDDR), Rambus Dynamic Random Access Memory (RDRAM), NAND flash memory, vertical NAND flash memory, NOR flash memory, resistive random access memory (RRAM), phase-change random access memory (PRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), and spin transfer torque random access memory (STT-RAM).

The memory device100may receive a command and an address from the memory controller210. The memory device100may be configured to access an area selected in response to the received address in the memory cell array. When the memory device100accesses the selected area, it may mean that the memory device100may perform an operation corresponding to the received command on the selected area. For example, the memory device100may perform a write operation (program operation), a read operation and an erase operation. A program operation may refer to an operation by which the memory device100writes data into the area selected by the address. A read operation may refer to an operation by which the memory device100reads data from the area selected by the address. An erase operation may refer to an operation in which the memory device100erases the data stored in the area selected by the address.

The NDP200may control the general operations of the first memory system1000aand perform an operation in response to a request of the host device2000. Examples of the operation may include a pooling operation, a vector multiplication, a vector sum, and a matrix multiplication.

More specifically, the NDP200may include the memory controller210, an NDP core220, and an interface230.

The memory controller210may control the memory device100to perform memory operations including a read operation, an erase operation, and a program operation. More specifically, the memory controller210may control the memory device100to perform a program operation, a read operation, or an erase operation in response to a request from the host device2000. During a program operation, the memory controller210may provide a program command, a physical block address, and data to the memory device100. During a read operation, the memory controller210may provide a read command and a physical block address to the memory device100. During an erase operation, the memory controller210may provide an erase command and a physical block address to the memory device100.

The NDP core220may perform an operation on data received from the memory device100in response to a request of the host device2000. The NDP core220may include a general-purpose processor such as a CPU, an AP, or a Digital Signal Processor (DSP); a graphics dedicated processor, such as a GPU or a VPU (Vision Processing Unit); or an artificial intelligence dedicated processor such as an NPU.

The interface230may assist the NDP200in performing communication with the host device2000. The NDP200may perform communication with the host device2000using the interface230.

The interface230may include a mailbox which serves as a communicator when data is moved between the NDP200and the host device2000. In addition, the interface230may include a generic interrupt controller (GIC) which handles an interrupt between the NDP200and the host device2000. In addition, the interface230may include a direct memory access (DMA) which allows data to be transferred to a memory of the host device2000or the memory device100without intervention of the processor including the NDP core220. The interface230may be connected to the host device2000by PCIe and include a protocol to perform communication using the PCIe.

According to an embodiment, the NDP core220may perform an entirety of, or a portion of operations of data required by the computing system10000. For example, the NDP core220may generate intermediate data by performing an operation on the raw data stored in the memory device100. In addition, the NDP core220may generate the final data by performing an operation on the intermediate data. An operation to be performed by the NDP core220may be determined in response to control of the host device2000.

FIG.5is a diagram illustrating an interface operation according to an embodiment of the present disclosure.

Referring toFIG.5, the NDP200may perform an interface operation with the host device2000.

First, the NDP200may receive a request to perform an operation on predetermined data (e.g., N query data) from the host device2000. More specifically, the memory controller210may receive the predetermined data (e.g., N query data) from the host device2000, and the interface230may receive a request message to process the predetermined data from the host device2000.

In addition, the NDP200may transfer an interrupt to the NDP core220by using the interface230. When the NDP core220receives the interrupt from the interface230, the NDP core220may read out the request message and the predetermined data (e.g., N query data) which the interface230and the memory controller210receive from the host device2000.

In addition, when the NDP core220completely reads the request message and the predetermined data (e.g., N query data) from the interface230and the memory controller210, the NDP core220may notify the interface230that the request message and the predetermined data are completely read.

In addition, the NDP core220may perform an operation on the predetermined data (e.g., N query data), transfer the predetermined data (e.g., N output data) on which the operation has been performed, and transmit a response message (e.g., N response message), which indicates that the operation has been completed, to the interface230.

When the interface230receives the response message (e.g., N response message) from the NDP core220, the interface230may transfer the interrupt and the response message to the host device2000. In addition, the memory controller210may transfer the predetermined data (e.g., N output data) on which the operation has been performed to the host device2000.

In addition, when the host device2000completely receives the response message (e.g., N response message) and the predetermined data (e.g., N output data), the host device2000may notify the interface230that the reception of the response message (e.g., N response message) and the predetermined data (e.g., N output data) has been completed.

FIG.6is a diagram illustrating a first embodiment according to the present disclosure.

Referring toFIG.6, according to an embodiment, the NDP200may perform a first operation and a second operation.

The host device2000may control the NDP200to perform a first operation and a second operation. More specifically, the host device2000may control the NDP200to perform the first operation and the second operation on the basis of a bandwidth ratio and a data size ratio. The bandwidth ratio may refer to a ratio of a second bandwidth of the NDP200and the host device2000to a first bandwidth of the memory device100and the NDP200. In addition, the data size ratio may refer to a ratio of the size of the final data to raw data. The first operation may be a processing operation to generate first data. The second operation may be a processing operation to generate second data. The NDP200may generate the second data by performing the first operation and the second operation on the raw data.

According to the first embodiment of the present disclosure, when the band ratio is smaller than or equal to the data size ratio, the host device2000may control the NDP200to perform the second operation. In addition, the NDP200may generate the second data (or the final data) by performing the second operation on the first data (or intermediate data), and may transfer the final data to the host device2000.

FIG.7is a diagram illustrating a second embodiment according to an embodiment of the present disclosure.

Referring toFIG.7, according to an embodiment, the NDP200may perform a first operation and the host device2000may perform a second operation.

The host device2000may control the NDP200to perform the first operation, and the host device2000may perform the second operation. More specifically, the host device2000may control the NDP200such that the first operation is performed by the NDP200on the basis of a bandwidth ratio and a data size ratio. In addition, the host device2000may perform the second operation on first data received from the NDP200. The NDP200may generate the first data by performing the first operation on the raw data, and the host device2000may generate second data by performing the second operation on the first data.

According to the second embodiment of the present disclosure, when the bandwidth ratio is greater than the data size ratio, the host device2000may control the memory system1000such that the second operation may be performed by the host device2000. The host device2000may control the memory system1000to perform the first operation only by the memory system1000. The NDP200may generate the first data (or intermediate data) by performing the first operation on the raw data and transfer the intermediate data to the host device2000. The host device2000may generate second data (or final data) by performing the second operation on the first data (or intermediate data).

FIG.8is a diagram illustrating a method of operating the host device2000according to an embodiment of the present disclosure.

Referring toFIG.8, the host device2000may transfer a query to request raw data to the NDP200at operation S810. More specifically, the host device2000may transfer the query to request the raw data stored in the plurality of memory devices100to the NDP200.

In addition, the host device2000may control the NDP200to perform a first operation on the raw data at operation S820. More specifically, when the NDP200receives the raw data from the plurality of memory devices100, the host device2000may control the NDP200to generate the first data by performing the first operation on the raw data.

In addition, the host device2000may determine a resource to perform a second operation S830. More specifically, based on a ratio of a size of second data to a size of the raw data and a ratio of a bandwidth between the NDP200and the host device2000to a bandwidth between the plurality of memory devices100and the NDP200, the host device2000may determine the resource to perform the second operation. That is, the host device2000may determine the resource to perform the second operation by comparing the data size ratio and the bandwidth ratio. The second data may be generated by performing the second operation on the first data. When the bandwidth ratio is greater than the data size ratio, the host device2000may determine the host device2000as the resource to perform the second operation. In addition, when the bandwidth ratio is smaller than or equal to the data size ratio, the host device2000may determine the NDP200as the resource to perform the second operation. After the host device2000determines the NDP200as the resource to perform the second operation, the host device2000may receive the second data from the NDP200.

FIG.9is a diagram illustrating a memory controller1300according to an embodiment of the present disclosure.

Referring toFIG.9, the memory controller1300may include a processor1310, a RAM1320, an error correction circuit1330, a ROM1360, a host interface1370, and a flash interface1380. The memory controller1300as shown inFIG.9may correspond to an embodiment of the memory controller210as shown inFIG.4.

The processor1310may communicate with the host device2000using the host interface1370and perform a logical operation to control operations of the memory controller1300. For example, the processor1310may load program commands, data files, data structures, etc. based on a request from the host2000or an external device, and may perform various types of operations, or generate commands or addresses. For example, the processor1310may generate various commands for a program operation, a read operation, an erase operation, a suspend operation, and a parameter setting operation.

In addition, the processor1310may function as a flash translation layer FTL. The processor1310may translate a logical block address LBA provided by the host device2000into a physical block address PBA through the flash translation layer FTL. The flash translation layer FTL may receive the logical block address LBA and translate the logical block address LBA into the physical block address PBA by using a mapping table. There may be various address mapping methods of the flash translation layer FTL, based on a mapping unit. Typical address mapping methods may include a page mapping method, a block mapping method, and a hybrid mapping method.

In addition, the processor1310may generate a command without a request from the host device2000. For example, the processor1310may generate a command for background operations such as wear leveling operations of the memory device100and garbage collection operations of the memory device100.

The RAM1320may serve as an operation memory, a cache memory or a buffer memory of the processor1310. The RAM1320may store codes and commands executed by the processor1310. The RAM1320may store data which is processed by the processor1310. In addition, the RAM1320may be realized with static RAM (SRAM) or dynamic RAM (DRAM).

The error correction circuit1330may detect errors during a program operation or a read operation and correct the detected errors. More specifically, the error correction circuit1330may perform an error correction operation according to an error correction code (ECC). In addition, the error correction circuit1330may perform ECC encoding based on data to be written to the memory device100. The ECC-encoded data may be transferred to the memory device100through the flash interface1380. In addition, the error correction circuit1330may perform ECC decoding based on data received from the memory device100through the flash interface1380.

The ROM1360may serve as a storage unit which stores various types of information for operations of the memory controller1300. More specifically, the ROM1360may include a map table, and the map table may include physical-logical address information and logical-physical address information. The ROM1360may be controlled by the processor1310.

The host interface1370may include a protocol for data exchange between the host device2000and the memory controller1300. More specifically, the host interface1370may communicate with the host device2000through one or more various communication standards or interfaces such as a universal serial bus (USB) protocol, a multimedia card (MMC) protocol, a peripheral component interconnection (PCI) protocol, a PCI-express (PCI-E) protocol, an advanced technology attachment (ATA) protocol, a serial-ATA protocol, a parallel-ATA protocol, a small computer system interface (SCSI) protocol, an enhanced small disk interface (ESDI) protocol, an integrated drive electronics (IDE) protocol, a private protocol, etc.

The processor1310may control the flash interface1380to communicate with the memory device100using a communication protocol. More specifically, the flash interface1380may perform communication with the memory device100for commands, addresses, and data through a channel. For example, the flash interface1380may include a NAND interface.

According to the present disclosure, a method of operating a storage device supporting improved near data processing (NDP) and a host device is provided.

While the present teachings have been illustrated and described with respect to specific embodiments, it will be apparent to those skilled in the art in light of the present disclosure that various changes and modifications may be made without departing from the spirit and scope of the disclosure as defined in the following claims. Furthermore, the embodiments may be combined to form additional embodiments.