Patent ID: 12197726

DETAILED DESCRIPTION

Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings, in which like reference numbers refer to like elements throughout. The present invention, however, may be embodied in various different forms, and should not be construed as being limited to only the illustrated embodiments herein. Rather, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present invention to those skilled in the art. Accordingly, processes, elements, and techniques that are not necessary to those having ordinary skill in the art for a complete understanding of the aspects and features of the present invention may not be described. Unless otherwise noted, like reference numerals denote like elements throughout the attached drawings and the written description, and thus, descriptions thereof may not be repeated.

In the drawings, the relative sizes of elements, layers, and regions may be exaggerated and/or simplified for clarity. Spatially relative terms, such as “beneath,” “below,” “lower,” “under,” “above,” “upper,” and the like, may be used herein for ease of explanation to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or in operation, in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as “below” or “beneath” or “under” other elements or features would then be oriented “above” the other elements or features. Thus, the example terms “below” and “under” can encompass both an orientation of above and below. The device may be otherwise oriented (e.g., rotated degrees or at other orientations) and the spatially relative descriptors used herein should be interpreted accordingly.

It will be understood that, although the terms “first,” “second,” “third,” etc., may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section described below could be termed a second element, component, region, layer or section, without departing from the spirit and scope of the present invention.

It will be understood that when an element or layer is referred to as being “on,” “connected to,” or “coupled to” another element or layer, it can be directly on, connected to, or coupled to the other element or layer, or one or more intervening elements or layers may be present. In addition, it will also be understood that when an element or layer is referred to as being “between” two elements or layers, it can be the only element or layer between the two elements or layers, or one or more intervening elements or layers may also be present.

The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting of the present invention. As used herein, the singular forms “a” and “an” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and “including,” when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.

As used herein, the term “substantially,” “about,” and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent variations in measured or calculated values that would be recognized by those of ordinary skill in the art. Further, the use of “may” when describing embodiments of the present invention refers to “one or more embodiments of the present invention.” As used herein, the terms “use,” “using,” and “used” may be considered synonymous with the terms “utilize,” “utilizing,” and “utilized,” respectively. Also, the term “exemplary” is intended to refer to an example or illustration.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and/or the present specification, and should not be interpreted in an idealized or overly formal sense, unless expressly so defined herein.

According to one or more embodiments of the present invention, reconfigurable compute logic may be implemented on a base die of a 3DS memory device. Generally, the 3DS memory device is a slave device, so compute is performed on the processor (e.g., CPU or GPU) side, and data is transferred back and forth between the processor and memory. The base die is generally an intermediate between a host (e.g., CPU or memory controller) and the memory dies. Its purpose is to provide the necessary interface and logic, such that the commands sent by the host can be understood by memory. In the case of HMC, the based die may also encode response from the memory die, such that information transferred back to the host can be understood.

However, 3DS memory generally has high bandwidth and low latency between the base die and the memory dies, and available semiconductor (e.g., silicon) real estate on the base die may be suitable to implement processor in memory (PIM) on the base die to compute data. By implementing PIM on the base die according to one or more embodiments of the present invention, the compute logic and memory are tightly coupled, and thus, data transfer time and/or energy consumption may be reduced.

FIG.1is a diagram illustrating a cross-sectional view of a 3D-stacked memory device, according to an example embodiment of the present invention.

Referring toFIG.1, a 3DS memory device100includes a base die102, a plurality of memory dies104stacked on the base die102, through silicon vias (TSVs)106, and an interface108between the 3DS memory device100and a host (e.g., an external host processor or controller such as a CPU or a GPU)110.

The memory dies104may be implemented as, for example, dynamic random access memory (DRAM). However, the present invention is not limited thereto, and the memory dies104may be implemented as any suitable memory that may be implemented in a 3D-stacked structure. The TSVs106connect (e.g., interconnect) the memory dies104and the base die102, and transfer data between the memory dies104and the base die102.

According to some embodiments of the present invention, the base die102includes a logic array, discussed in more detail below, including a plurality of switches and a plurality of arithmetic logic units (ALUs). As will be described in more detail with reference toFIG.2, the logic array may compute data, and may be reconfigured real time via PIM instructions received from the host110through the interface108. That is, the host110may communicate (e.g., directly communicate) with the base die102through the interface108, and the base die102may communicate (e.g., directly communicate) with the memory dies104to perform the compute logic.

FIG.2is a diagram illustrating further detail of the base die102of the 3DS memory device100shown inFIG.1, according to an example embodiment of the present invention.

Referring toFIG.2, the base die102includes program memory202and a logic array204. The logic array204includes a plurality of switches206and a plurality of ALUs208. The program memory202receives control instructions from the host110through the interface108, controls access to the memory dies104, configures the switches206, and instructs operation of the ALUs208. The switches206create data paths and direct data flow, and the ALUs208compute data.

According to an embodiment of the present invention, the program memory202may be implemented as, for example, static random access memory (SRAM). However, the present invention is not limited thereto, and the program memory202may be implemented as any suitable program memory on the base die102.

In more detail, the program memory202receives control instructions from the host110. The control instructions may include commands for configuring the switches206, accessing the memory dies104, and operating the ALUs208. The commands for accessing the memory dies104and operating the ALUs208may be similar to or the same as CPU instructions (e.g., LOAD, STORE, ADD, MULTIPLY, etc.).

The program memory202stores configuration data and instructions, and may store multiple configuration data and instructions at any given time to allow operations to pipeline efficiently. However, the present invention is not limited thereto. For example, the configuration data and/or the instructions may be stored in the memory dies as secondary storage (e.g., when the program memory202is full or substantially full).

For example, during system initialization, the host may read from the memory device to determine the size of the program memory202, and the capacity (e.g., maximum capacity) of the memory dies104that can be used for storing the configuration data and instructions. Because the capacity for storing the configuration data is determined by the memory device, there may be a case where none of the memory dies104can be used to store configuration data and instructions. However, in the case where the memory dies104store the configuration data and instructions (e.g., because the program memory202is full), the base die102may have logic that directs configuration from the host to the memory dies (e.g., bypassing the program memory202), and logic to fetch configuration from the memory dies104to the program memory202.

The size of the program memory202may depend on the number of the switches206, the number of the ALUs208, and/or the size of the memory dies104. Further, a control circuitry (e.g., a controller)203may be provided to read/write the program memory202, and to effectuate changes thereto.

The program memory202may utilize the configuration data to configure the switches206to create data paths and direct data flow. The program memory202may utilize memory instructions to access the memory dies104, and may utilize ALU instructions to compute data. The ALUs may support one or more arithmetic/logic operations, for example, ADD, MULTIPLE, DIVIDE, COMPARE, SHIFT, AND, OR, XOR, etc., to compute data.

While the switches206and the ALUs208are shown inFIG.2as having a one-to-one relationship, the present invention is not limited thereto. For example, in some embodiments, an ALU208may be associated with multiple switches206. In some embodiments, a plurality of ALUs208may be associated with a single or a plurality of switches206.

FIG.3is a diagram illustrating a switch shown inFIG.2, according to an example embodiment of the present invention.

Referring toFIGS.2and3, each of the switches206may direct data to flow from a direction to any of a plurality of directions. For example, assuming that a switch306from among the switches206may direct data to flow in a first direction, a second direction opposite the first direction, a third direction crossing the first direction, and a fourth direction opposite the third direction, the switch306may be implemented as a four-way switch including a plurality of multiplexers (MUXs). When the switch306is a four-way switch, the plurality of MUXs may include first to fourth MUXs, each of the MUXs corresponding to a direction. However, the present invention is not limited thereto, and the switches206may be implemented as any suitable switches capable of directing data to flow in any number of directions.

For convenience, the switch306shown inFIG.3is implemented as a four-way switch including first to fourth MUXs308,310,312, and314to direct data to flow in the first to fourth directions, but the present invention is not limited thereto. For example, a number of directions that the switch306may transmit the data may be more or less than four, and a number of the MUXs may correspond to the number of directions.

In more detail, the switch306inFIG.3includes the first to fourth MUXs308to314. Each of the MUXs308to314may direct data to flow in any one of the first to fourth directions. For example, the first MUX308may direct the data in the first direction via a first output terminal 1st_OUT, the second MUX310may direct the data in the second direction via a second output terminal 2nd_OUT, the third MUX312may direct the data in the third direction via a third output terminal 3rd_OUT, and the fourth MUX314may direct the data in the fourth direction via a fourth output terminal 4th OUT.

Each of the MUXs308to314may include a first input terminal 1st_IN to receive data from the first direction, a second input terminal 2nd_IN to receive data from the second direction, a third input terminal 3rd_IN to receive data from the third direction, and a fourth input terminal 4th_IN to receive data from the fourth direction. The first input terminals 1st_IN of each of the MUXs308to314may be electrically coupled to each other. The second input terminals 2nd_IN of each of the MUXs308to314may be electrically coupled to each other. The third input terminals 3rd_IN of each of the MUXs308to314may be electrically coupled to each other. The fourth input terminals 4th_IN of each of the MUXs308to314may be electrically coupled to each other.

In some embodiments, signals generated by a corresponding ALU may be looped back to the same ALU. In this case, the signals may loop back via a corresponding MUX, or the signal may loop back just inside the corresponding ALU. In the case where the signal loops back via the corresponding MUX, the MUX may further include a fifth input terminal, and the fifth input terminal may be electrically coupled to the output terminal of the MUX to receive data flowing in opposite directions. However, the present invention is not limited thereto.

Each of the MUXs308to314may further include a selection terminal SEL. The selection terminal SEL enables the corresponding MUX to direct the data towards the corresponding direction. For example, assuming that the switch306wants to direct data from the third direction to the first direction, the selection terminal SEL enables the first MUX308(and/or disables the second to fourth MUXs310to314), and the first MUX308outputs via its output terminal 1st_OUT the data received at its third input terminal 3rd_IN to the first direction.

FIGS.4A-4Care diagrams illustrating an example logic operation in a 4×4 logic array of the 3DS memory device, according to an embodiment of the present invention. The example shown inFIGS.4A to4Cassumes that the goal is to sum four numbers A, B, C, and D, and to store the sum in the memory dies404.FIG.4Aillustrates a first cycle of the operation,FIG.4Billustrates a second cycle of the operation, andFIG.4Cillustrates a third cycle of the operation. WhileFIGS.4A to4Cillustrate a simplified example operation of the 4×4 logic array, the present invention is not limited thereto.

InFIGS.4A to4C, the base die400includes a 4×4 logic array of switches SW11to SW44and ALUs A11to A44, and the program memory402to configure and instruct the switches SW11to SW44and the ALUs A11to A44.

Referring toFIG.4A, during the first cycle, the program memory402configures the switches SW11and SW12and the memory dies404to load the values of A and B from the memory dies404to the ALU A11, and configures the switches SW13and SW14and the memory dies404to load the values of C and D from the memory dies404into the ALU A13. The program memory402instructs the ALU A11to ADD the values of A and B, and instructs the ALU A13to ADD the values of C and D.

Referring toFIG.4B, during the second cycle, the program memory402configures the switches SW11and SW21to transmit a result X of the sum of A and B from the ALU A11to the ALU A21, and configures the switches SW13, SW21, SW22, and SW23to transmit a result Y of the sum of C and D from the ALU A13to the ALU A21. The program memory402instructs the ALU A21to ADD the values of X and Y.

Referring toFIG.4C, during the third cycle, the program memory402configures the switches SW12, SW21, and SW22and the memory dies404to store a result Z of the sum of X and Y from the ALU A21to the memory dies404.

Accordingly, the switches SW11to SW44may create data paths and direct data flow, and the ALUs A11to A44may compute data.

Referring again toFIGS.1and2, the host110may send configuration instructions and commands for the base die102(e.g., the switches206and the ALUs208) to compute data through the interface108between the 3DS memory device100and the host110. In some embodiments, the interface108may include response and request links (e.g., asynchronous response and request links). In some embodiments, the interface may include control and data buses. For example, according to one or more embodiments of the present invention, the interface108may include the interface of an HMC device or an HBM device that is modified to transmit PIM instructions for the base die102(e.g., the switches206and the ALUs208on the base die102) to compute data. However, the present invention is not limited thereto, and the interface108may include any suitable interface to configure and instruct the base die102to compute data.

FIGS.5A to5Fillustrate example packets for an interface including response and request links according to an embodiment of the present invention.FIG.5Aillustrates a packet with no data,FIG.5Billustrates a packet with data,FIG.5Cillustrates a request packet header layout,FIG.5Dillustrates a request packet tail layout,FIG.5Eillustrates a response packet header layout, andFIG.5Fillustrates a response packet tail layout.

When the interface includes the response and request links (e.g., asynchronous response and request links), such as the interface for an HMC device, the request link sends packets from the host to the 3DS memory device, and the response link sends packets from the 3DS memory device to the host. The host sends a packet including address information, commands, and/or data to the 3DS memory device via a request packet. The 3DS memory device responds to the host via a response packet. Each of the host and the 3DS memory device may include an encoder and a decoder to process the packets. Configuration instructions and commands may be embedded in the packets, and the 3DS memory device may decode the configuration instructions and the commands to configure the base die102(e.g., the switches206and the ALUs208on the base die102) to compute data.

Referring toFIGS.5A and5B, a packet generally includes a header and a tail. For example, when the packet (e.g., 128-bits) does not include data, the header may be allocated to bit 0 to bit 63, and the tail may be allocated to bit 64 to bit 127. When the packet includes data (e.g., 32B data payload), the packet is serialized into a transmission unit Flit (e.g., a Flit is 128-bits), and each numbered data field represents a byte with bit positions [7(MSB0: 0(LSB)].

Referring toFIG.5C, when the interface108, for example, is the same or substantially the same as that of the HMC device, the request packet header layout includes a plurality of fields. The fields include cube ID CUB (e.g., bit range [63:61]), reserved RES (e.g., bit range [60:58]), address ADRS (e.g., bit range [57:24]), reserved RES (e.g., bit range [23]), tag (e.g., bit range [22:12]), packet length LNG (e.g., bit range [11:7]), and command CMD (e.g., bit range [6:0]).

The request packet header fields may be defined, for example, in Table 1 below (e.g., for the HMC device).

TABLE 1FieldBitBitNameLabelCountRangeFunctionCube IDCUB3[63:61]CUB field used to match requestwith target cube. The internal CubeID Register defaults to the valueread on external CUB pins of eachHMC device.ReservedRES3[60:58]Reserved: These bits are reservedfor future address or Cube IDexpansion. The responder willignore bits in this field from therequester except for includingthem in the CRC calculation.The HMC can use portions ofthis field range internally.AddressADRS34[57:24]Request address. For somecommands, control fields areincluded within this range.ReservedRES1[23]TagTAG11[22.12]Tag number uniquely identifyingthis request.PacketLNG5[11:7]Length of packet in FLITs (1LengthFLIT is 128 bits). Includes header,any data payload, and tail.Com-CMD7[6:0]Packet command.mand

Referring toFIG.5D, when the interface108, for example, is the same or substantially the same as that of the HMC device, the request packet tail layout includes a plurality of fields. The fields include cyclic redundancy check CRC (e.g., bit range [63:32]), return token count RTC (e.g., bit range [31:29]), source ling ID SLID (e.g., bit range [28:26]), reserved RES (e.g., bit range [25:22]), poison bit Pb (e.g., bit range [21]), sequence number SEQ (e.g., bit range [20:18], forward retry pointer FRP (e.g., bit range [17:9]), and return retry pointer RRP (e.g., bit range [8:0]).

The request packet tail fields may be defined, for example, in Table 2 below (e.g., for the HMC device).

TABLE 2FieldBitBitNameLabelCountrangeFunctionCyclicCRC32[63:32]The error-detecting code field thatredun-covers the entire packet.dancycheckReturnRTC3[31:29]Return token count for transaction-tokenlayer flow control. In the requestcountpacket tail, the RTC contains theencoded value for tokens thatrepresent available space inthe requester's input buffer.SourceSLID3[28:26]Used to identify the source link forLink IDresponse routing. The incomingvalue of this field is ignored bythe HMC. Internally, HMCoverwrites this field and uses thevalue for response routing; refer tothe description of the SLID field inthe response header.ReservedRES4[25:22]Reserved: The responder will ignorebits in this field from the requesterexcept for including them in theCRC calculation. The HMC can useportions of this field rangeinternally.Poison bitPb121Poison bit: the DRAM addresses tobe written in this request will bepoisoned by writing a special poisoncode to alternating 16-byte blocksbeing written, starting with the firstrequested 16-byte block. The other16-byte blocks within this requestare written normally using thecorresponding write data includedin the data payload.SequenceSEQ3[20:18]Incrementing value for each packetnumbertransmitted, except for PRET andIRTRY packets.ForwardFRP9[17:9]Retry pointer representing thisretrypacket's position in the retry buffer.pointerReturnRRP9[8:0]Retry pointer being returned forretryother side of link.pointer

Referring toFIG.5E, when the interface108, for example, is the same or substantially the same as that of the HMC device, the response packet header layout includes a plurality of fields. The fields include cube ID CUB (e.g., bit range [63:61]), reserved RES (e.g., bit range [60:42]), source link ID SLID (e.g., bit range [41:39]), reserved RES (e.g., bit range [38:34]), Atomic flag AF (e.g., bit range [33]), reserved RES (e.g., bit range [32:23]), tag (e.g., bit range [22:12]), packet length LNG (e.g., bit range [11:7]), and command CMD (e.g., bit range [6:0]).

The response packet header fields may be defined, for example, in Table 3 below (e.g., for the HMC device).

TABLE 3BitNameField LabelCountBit rangeFunctionCUB IDCUB3[63:61]The target cube inserts its Cube ID numberinto this field. The requester can use thisfield for verification and for identifying uniquetags per the target cube.ReservedRES19[60:42]Reserved: The host will ignore bits in thisfield from the HMC except for including themin the CRC calculation.Source LinkSLID3[41:39]Used to identify the source link for responseIDrouting. This value is copied from thecorresponding Request header and used forresponse routing purposes.The host can ignore these bits (except forincluding them in the CRC calculation.)ReservedRES5[38:34]Reserved: The host will ignore bits in thisfield from the HMC except for including themin the CRC calculation.Atomic FlagAF1[33]Atomic flagReservedRES10[32:23]Reserved: The host will ignore bits in thisfield from the HMC except for including themin the CRC calculation.TagTAG11[22:12]Tag number uniquely associating thisresponse to a request.Packet lengthLNG5[11:7]Length of packet in 128-bit FLITs. Includesheader, any data payload, and tail.CommandCMD7[6:0]Packet command

Referring toFIG.5F, when the interface108, for example, is the same or substantially the same as that of the HMC device, the response packet tail layout includes a plurality of fields. The fields include cyclic redundancy check CRC (e.g., bit range [63:32]), return token counts RTC (e.g., bit range [31:29]), error status ERRSTAT (e.g., bit range [28:22]), data invalid DINV (e.g., bit range [21]), sequence number SEQ (e.g., bit range [20:18], forward retry pointer FRP (e.g., bit range [17:9]), and return retry pointer RRP (e.g., bit range [8:0]).

The response packet tail fields may be defined, for example, in Table 4 below (e.g., for the HMC device).

TABLE 4BitNameField LabelCountBit rangeFunctionCyclicCRC32[63:62]Error-detecting code field that covers theredundancyentire packet.checkReturn tokenRTC3[31:29]Return token count for transaction-layer flowcountscontrol. In the response packet tail, the RTCcontains an encoded value equaling thereturned tokens. The tokens representincremental available space in the HMC inputbuffer.Error statusERRSTAT7[28:22]Error status bitsData invalidDINV1[21]Indicates validity of packet payload. Data inpacket is valid if DINV = 0 and invalid ifDINV = 1.SequenceSEQ3[20:18]Incrementing value for each packetnumbertransmitted.Forward retryFRP9[17:9]Retry pointer representing this packet'spointerposition in the retry buffer.Return RetryRRP9[8:0]Retry pointer being returned for the otherPointerside of link.

According to an example embodiment of the present invention, the reserved bits RES in the request/response packets may be used to carry the PIM instructions (e.g., for the reconfigurable compute). Further, if the reserved bits RES in the request/response packets are not sufficient, the request/response packet definition may be expanded, such that the tail has more bits. The expanded tail bits may then be used to carry the PIM instructions. Accordingly, the response/request packets of, for example, the HMC interface may be modified to transmit the PIM instructions to configure and instruct the base die102(e.g., the switches206and the ALUs208) to compute data through the interface108between the 3DS memory device100and the host110.

For example, referring back to the example shown inFIGS.4A to4C, the steps may include:(1) Load A, B, C, D (e.g., four LOAD instructions);(2) X=A+B; Y=C+D (e.g., two ADD instructions);(3) Z=X+Y (e.g., one ADD instruction); and(4) Store Z (e.g., one STORE instruction).

The host may send these instructions in order. Each instruction may be embedded in one request packet. Each request packet may also contain the data and address associated with the instructions. Thus, in this example, as shown inFIG.6, the sequence may include LOAD A (at block610), LOAD B (at block620), LOAD C (at block630), LOAD D (at block640), ADD X A B (at block650), ADD Y C D (at block660), ADD Z X Y (at block670), and STORE Z (at block680).

At the end of the sequence, the memory device may send back a response informing the host that it has finished, and providing the address in which the end result Z has been stored (at block690).

Alternatively, there may be an instruction called ADD4, which may add four data together. In this case, the host will simply send one packet, for example, ADD4 Z A B C D. Here, there is less overhead, and many PIM operations are transparent to the CPU, so it may be undesirable for the CPU to manage every detail.

However, the present invention is not limited thereto, and there may be various additional definitions depending on the capabilities of the ALUs and/or of the memory device.

FIG.6illustrates a flow diagram for an interface including control and data buses, according to an embodiment of the present invention.

When the interface108includes the control and data buses, such as the interface for an HBM device, the control bus is unidirectional from the host to the 3DS memory device and the data bus is bidirectional between the host and the 3DS memory device. The control bus transfers address bits and commands from the host to the 3DS memory device. The data bus transfers data to the memory dies from the host (e.g., via WRITE) and transfers data from the memory dies to the host (e.g., via READ).

The commands may include traditional commands (e.g., DRAM access commands, such as READ, WRITE, ACTIVATE, PRECHARGE, REFRESH, etc.), and may include a mode register set command (e.g., MRS command) to configure the 3DS memory device. For example, the MRS command may configure registers of the 3DS memory device to operate in a PIM mode or a normal mode. In the normal mode, the 3DS memory device may operate like a traditional 3DS memory device (e.g., the HBM device) without compute, and interprets the traditional commands (e.g., RAS, CAS, CASW, etc.) as would be expected. On the other hand, in the PIM mode, the 3DS memory device may interpret the traditional commands to have a different meaning. In other words, when the MRS command is received to operate in the PIM mode, the traditional commands (e.g., RAS, CAS, CASW, etc.) may be interpreted by the 3DS memory device as instructions to compute data (e.g., ADD, MULTIPLY, DIVIDE, COMPARE, etc.) via the base die102(e.g., ALUs208).

For example, referring toFIGS.4A to4C and7, the process starts and at block700, the 3DS memory device receives the MRS command from the host via the interface to configure the 3DS memory device. At block705, the 3DS memory device determines whether the MRS command includes instructions to operate in the PIM mode. If the determination is no, the 3DS memory device operates in the normal mode at block710. If the determination is yes, then at block715, the 3DS memory device is configured to enter the PIM mode. At block720, the 3DS memory device receives traditional commands (e.g., RAS/CAS) for A, B, C, and D. At block725, the 3DS memory device determines that the traditional commands (e.g., RAS/CAS) should be interpreted as compute logic (e.g., ADD). At block730, the 3DS memory device sums the value of A, B, C, and D as X+Y=Z and stores the value of Z (e.g., seeFIG.4). Several traditional commands (e.g., RAS/CAS) may be received depending on the operation.

In some embodiments, the data bus may be repurposed when in the PIM mode to transfer not just data, but also command, address, status, etc. For example, assuming that the compute instructions include ADD4 Z A B C D, a RAS command could be used to represent ADD4, and data bus bits could be used to represent the addresses of A, B, C, and D. After the computation is completed, the data bus can be used to transfer the address of Z back to the host. Similarly, in some embodiments, the compute instructions may include, for example, ADD X A B, in which case, we could use a CAS command to represent ADD, and use the data bus to present the addresses of A and B.

While the examples herein are described with repurposed RAS/CAS commands, the present invention is not limited thereto. For example, other traditional commands (e.g., DRAM commands) could be repurposed as desired, or PIM instructions that are not defined by any of the traditional commands may be defined as desired.

In some embodiments, the MRS command may be utilized to configure the 3DS memory device to exit the PIM mode, and to act as a traditional 3DS memory device (e.g., a normal HBM device). Then, the 3DS memory device may operate as a traditional memory device (e.g., a HBM device) without computing data. However, the present invention is not limited thereto, and in some embodiments, the 3DS memory device may revert back to the normal mode, for example, as a default without further MRS commands, once the compute has completed.

According to one or more example embodiments of the present invention, reconfigurable compute logic may be implemented on the base die of a 3DS memory device. For example, the base die may include program memory and a logic array including a plurality of switches and a plurality of ALUs. The switches may direct data flow and create data paths, and the ALUs may compute the data. Further, an interface between the 3DS memory device and a host may be utilized to configure and command the base die (e.g., the switches and ALUs) to compute data. In some embodiments, the interface may include request/response links (e.g., an HMC interface) to send/receive packetized address/commands/data. In some embodiments, the interface may include control and data buses (e.g., an HBM interface) to configure the 3DS memory device in a PIM mode, and to interpret traditional commands as computation commands. In some embodiments, the interface may include the interface of the HMC device or the HBM device that is modified to transmit PIM instructions for the base die (e.g., the switches and the ALUs on the base die) to compute data.

The electronic or electric devices and/or any other relevant devices or components according to embodiments of the present invention described herein may be implemented utilizing any suitable hardware, firmware (e.g. an application-specific integrated circuit), software, or a combination of software, firmware, and hardware. For example, the various components of these devices may be formed on one integrated circuit (IC) chip or on separate IC chips. Further, the various components of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on one substrate. Further, the various components of these devices may be a process or thread, running on one or more processors, in one or more computing devices, executing computer program instructions and interacting with other system components for performing the various functionalities described herein. The computer program instructions are stored in a memory which may be implemented in a computing device using a standard memory device, such as, for example, a random access memory (RAM). The computer program instructions may also be stored in other non-transitory computer readable media such as, for example, a CD-ROM, flash drive, or the like. Also, a person of skill in the art should recognize that the functionality of various computing devices may be combined or integrated into a single computing device, or the functionality of a particular computing device may be distributed across one or more other computing devices without departing from the spirit and scope of the exemplary embodiments of the present invention.

Although the present invention has been described with reference to the example embodiments, those skilled in the art will recognize that various changes and modifications to the described embodiments may be performed, all without departing from the spirit and scope of the present invention. Furthermore, those skilled in the various arts will recognize that the present invention described herein will suggest solutions to other tasks and adaptations for other applications. It is the applicant's intention to cover by the claims herein, all such uses of the present invention, and those changes and modifications which could be made to the example embodiments of the present invention herein chosen for the purpose of disclosure, all without departing from the spirit and scope of the present invention. Thus, the example embodiments of the present invention should be considered in all respects as illustrative and not restrictive, with the spirit and scope of the present invention being indicated by the appended claims, and their equivalents.