Patent ID: 12189523

DETAILED DESCRIPTION

Memory devices having circuitry to perform data modification operations within the time interval generally required for data access are disclosed in various embodiments. In general, by hiding the data modification operation under or within the memory access time, exclusive access to the data is established by the memory access processes within the memory device itself, thus obviating costly and inefficient exclusivity control mechanisms within the processor(s), memory controller or other upstream circuitry. In several single integrated circuit embodiments presented below, an input-output pipeline of an integrated circuit memory device includes capability to perform multi-step operations and to write data back into the same or different memory cells; because these operations are performed by the input-output pipeline (which generally can only be used by one requestor at a time), a lock can be effectively established without using complicated software mechanisms or multi-processor communications.

A hypothetical illustration is presented with an example of two users, each having a separate workstation and desiring to update a shared financial account database having an entry of $100. Each user's workstation might read the entry in-question ($100), and each user might desire to update the account to add different increments (e.g., to add $20 in the case of the first user, and to add $50 in the case of the second user). The use of software locks in this situation would imply unavailability of the entry or of related processing to one user until the other user is finished; the failure to use locks might imply that the second user's access may read a stale entry ($100) and then overwrite that stale entry with an update (e.g., $150) of stale info, resulting in an incorrect entry (e.g., $150 overwriting the first user's entry of $120 when the correct value should be $170).

FIG.1Aillustrates an embodiment of a data processing system100capable of carrying out access-protected data modification operations—that is, data modification operations that are carried out concurrently or coextensively with a memory access operation and thus protected from undesired intervening access by timing restrictions imposed by the memory access itself. Such compound memory operations (i.e., involving data retrieval from one or more memory cores as well as logical, arithmetic, exchange or other operations with respect to the retrieved data) are referred to herein generally as “atomic” operations as they are indivisible from the standpoint of competing memory requestors. Accordingly, as the atomic character of such compound operations is enforced (effected) by circuitry within individual memory devices that populate the memory subsystem, and issuance of specialized “atomic” memory access commands (or requests or instructions) by a memory controller, such memory devices are referred to herein as “atomic” memory devices and the memory controller as an “atomic” memory controller. Thus, the memory subsystem ofFIG.1Aincludes an atomic memory controller101that responds to memory access requests (issued via a host request path, “HostReq”) from one or more processors102or other host devices running one or more threads; each “atomic operation” is effected by issuing corresponding command and address information to one or more atomic memory devices103via command/address path110(“C/A”). In the case of a host-write request, write data is output from the host processor102to the atomic memory controller101via a host data path (“HostData”) and then from the atomic memory controller101to the atomic memory devices103via memory data path112(“Data”). Conversely, in the case of a host-read request, read data is output from one or more of the atomic memory devices103to the atomic memory controller101via the memory data path112, and then from the atomic memory controller101to the requesting host102via the corresponding host data path.

With regard to the memory subsystem topology, any number of atomic memory devices103may be coupled to the atomic memory controller101in any combination of point-to-point and multi-drop signaling paths. In one embodiment, for example, the atomic memory devices103are organized into one or more memory ranks (i.e., selected as a unit via one or more chip-select lines or other selection mechanisms) with each atomic memory device103of the memory rank being coupled to the atomic memory controller101via a common command/address path110and via respective (dedicated-per-memory-device) memory data paths110. By this arrangement, the memory devices of a given rank may be selected as a unit (e.g., via a shared chip-select line or other device selection mechanism) to receive the same memory access command and memory address, and to respond to the common command/address by receiving or outputting respective portions of the overall data word (via dedicated memory data paths) being transferred between the rank of atomic memory devices103and the atomic memory controller101. In alternative embodiments, separate command/address paths110may be provided to enable selection of atomic memory devices either as a rank or individually (or in sub-groups within a rank), and/or multiple atomic memory devices103may be coupled to a memory data path110(or to each memory data path110in system100) in a multi-drop arrangement.

Still referring toFIG.1A, each of the atomic memory devices103includes core access circuitry107to enable access to a memory core105formed by one or more arrays of storage cells. The storage arrays of the memory core105may be populated by virtually any type of storage cells including for example and without limitation, volatile or non-volatile memory; for example, the storage cells may include static random access memory (static RAM or SRAM) cells, dynamic RAM (DRAM) cells, charge-trapping cells such NAND or NOR flash memory cells, phase-change memory cells, ferro-magnetic memory cells or any other storage cell technology that permits storage and retrieval of digital data. Further, the memory core105may include various signal lines and circuitry to enable access to the underlying storage cells such as, for example and without limitation, word lines to enable access to address-selected rows of storage cells, bit lines to convey data signals between word-line-selected storage cells and sense amplifiers or like circuitry, and sense amplifiers themselves for sensing (and/or latching) signals output from the selected cells during data retrieval (read) operations and for driving signals back to the selected cells during write or refresh operations. For purposes of clarity and definitiveness, atomic memory devices are presented in embodiments below as having a DRAM core and occasionally referred to as atomic DRAMs. In all such embodiments, data storage technologies other than DRAM may be used in accordance with the innovations and improvements disclosed herein, with commensurate changes in the memory core105, core access circuitry107, atomic memory controller101and interconnection topology.

Continuing with the exemplary atomic memory device103shown inFIG.1A, core access circuitry107includes control logic circuitry109coupled to receive command and address information via the command/address path110(which may be formed by separate command and address paths, or by a single signaling path that is time-multiplexed with command and address information) and data input/output (I/O) circuitry115to manage data transfer between the memory core105and the external data path (i.e., memory data path112). The control logic circuitry109(“control logic” for short) responds to incoming commands and addresses by controlling operation of the memory core105(initiating row and column decoding operations, sense operations, refresh operations, precharge operations, programming operations, erase operations, etc.) and also controlling operation of the data I/O circuitry115. In particular, the control logic circuitry109may manage the timing of data reception during memory write operations (and atomic memory operations that involve data storage), enabling the data I/O circuitry to begin data sampling incoming data (e.g., write data, swap data and/or operand data as discussed below) from the external data path at a predetermined time with respect to registration of a corresponding memory command and to latch the incoming data in one or more registers or buffers that form an interface to the memory core105. Similarly, control logic109may manage the timing of data transmission during memory read operations, enabling the data I/O circuitry115to begin unloading read data from the memory core interface and outputting the read data onto the external data path at a predetermined time with respect to registration of a memory read command (or a command to perform an atomic operation that returns data to the host requestor). Though not specifically shown, various timing signals including clock and/or strobe signals (i.e., signals that transition to indicate the presence of valid data) may be received or generated within the atomic memory device103and used to coordinate data sampling, transmission operations as well as internal operations within the control logic109, data I/O circuitry115and/or memory core.

In contrast to conventional memory devices, the atomic memory device103includes circuitry to execute the above-described data modification operations concurrently with data retrieval. More specifically, the control logic109includes circuitry that responds to atomic operation commands (i.e., commands to execute specified atomic operations) as well as non-atomic memory read and write commands (and row activation commands, precharge commands, refresh commands, program commands, erase commands, and so forth as necessary to manage the underlying memory technology). Further, as shown inFIG.1A, the data I/O circuitry115includes modify logic circuitry117(“modify logic” for short) to enable modification and write-back of retrieved data as it is en route to its external destination, if any. In one embodiment, for example, the modify logic117is coupled between internal serial data lines131,133used to convey outgoing (read) and incoming (write) data between data I/O sampler and driver circuits (121and123, respectively) and the memory core interface. As retrieved data is shifted bit by bit onto the outgoing serial data line133(i.e., away from the memory core105), the serial data bits may be received within and operated upon by the modify logic117in accordance with a specified modify-operation128to produce modified data which is, in turn, shifted bit by bit onto the incoming serial data line131(i.e., toward the memory core) and thus written back to the memory core105. Because the overall data modification and write-back time may be completely or at least substantially hidden under the data retrieval time itself (e.g., within the column access time or column access cycle time of a memory device), the inherent timing restrictions imposed by the memory core technology serve to prevent undesired access to the modified data prior to its storage within the memory core105(including storage within the sense amp bank of the memory core, if not within the more remote storage cells themselves) and thus ensure coherency without need for coherency mechanisms within the host requestor or memory controller. That is to say, a data read and write (e.g., read-modify-write) operation performed in the memory device is performed within the input-output path in a manner that it (a) is performed far more quickly than lock mechanisms (where a memory lock is established through software or hardware during processing by a remote controller) and (b) cannot be interfered with by another incoming operation. Returning to the hypothetical illustration presented above, relating to a financial account entry, the memory device may employ a single read-modify-write operation such that an operation to write an updated value into memory (e.g., $120) can be performed in a single command, such updates can never be commenced for data that is stale.

FIGS.1B-1Fillustrates the flow of data in a number atomic memory operations supported by the atomic memory device103and atomic memory controller101ofFIG.1A. Exemplary operations that can be performed include operations that combine a data access with one or more logical operations, for example, increment operations, decrement operations, inversion, shift and similar operations. Other operations may combine multiple memory access operations, such as for example a swap operation where data in one memory location is swapped with data from another memory location or with data provided by a memory command.

Starting withFIG.1B, in a read/modify operation, data is retrieved as part of a memory read operation, modified within the modify logic117in accordance with a modify-operation128(“op”) specified by the control logic (and thus by the atomic command from the atomic memory controller) and written back to the memory core105in place of the retrieved data.

InFIG.1C, a similar atomic operation is performed except that, instead of a unary operation in which the read data constitutes the sole operand, a binary (two-operand) operation is performed in accordance with the specified modify-operation128using the read data as a primary operand and an internally or externally sourced data value (shown inFIG.1Cas “operand”140) as the secondary operand. As discussed below, such a sourced operand may be a value previously retrieved from the memory core105and stored within an operand register, a value stored within an operand register as part of a memory-controller-instructed register-write operation, a carry-bit from another memory device (e.g., from an adjacent rank) or any other operand data source. An externally-sourced operand (e.g., a value to be loaded into an operand) may be provided, for example, via the external data path (i.e., memory data path112ofFIG.1A) in generally the same manner (though not necessarily the same command-relative timing) as write data. Alternatively, an externally-sourced operand may be provided via the command/address path (e.g., time-multiplexed with other information transmitted thereon) or any other signaling connection to the atomic memory device (e.g., an out-of-band transmission channel such as common mode signaling over a differential data link, low-speed signal channel used to initialize the memory system, etc.). As a specific example of an externally-sourced operand, an operand-load instruction and operand value may be provided from atomic memory controller101to atomic memory device103via the C/A and data paths, respectively (or via either path individually, or via any other in-band or out-of-band signaling path). The control logic109within the atomic memory device103responds to the operand-load instruction by enabling the specified operand register to be loaded with the incoming operand value. In addition to the techniques identified above, a carry-bit or other operand or result of an operation within modify logic117may be output from a memory device (or rank) as shown at141(“op result”) and provided to another memory device or to a memory controller, to indicate overflow/underflow or other results of such operations.

FIG.1Dillustrates an atomic data-exchange or data-swap operation that may be performed within the memory device architecture ofFIG.1A. Although similar to the binary operation shown inFIG.1C, instead of performing a modification of the read data value, the read data is conditionally or unconditionally swapped with a swap data value (“swap data”) via multiplexing circuitry151. That is, the swap data value is conditionally or unconditionally written back to the memory core105in place of the read data, and the swap data may also be conditionally or unconditionally returned to the memory controller (and thus the host requestor) to signify the swap result. As with the secondary operand in a binary operation, the swap data value may be internally or externally sourced (i.e., provided by a source within or outside the atomic memory device, respectively). In an unconditional swap, referred to herein simply as a swap operation, the swap data value is written back to the memory core in place of the read data value (i.e., overwriting the read data value) while the read data value is returned to the memory controller (and thus to the host requestor). In a conditional swap, modify logic117evaluates the read data and/or swap data and conditionally exchanges (swaps) the read data and swap data depending on the evaluation result. As an example, in a particular form of conditional swap referred to herein as a compare-and-swap, the modify logic compares the swap data and read data to determine which is more superlative (greater than, less than, higher magnitude, more ‘1’ or more ‘0’ bits, etc.), writing the swap data back to memory core105only if it is the more superlative value. Alternatively (or in response to a different type of conditional swap command), the read data alone or the swap data alone may be evaluated (or may be evaluated with respect to a register-sourced condition or compare value as shown by dashed arrow142inFIG.1D) to determine whether the exchange condition is satisfied (e.g., determining whether a predetermined characteristic of the read data or swap data is met (e.g., more ‘1’s than ‘0’s) or whether read data or swap data exceeds (in any sense) the register-sourced compare value). Whichever data evaluation is performed, if the swap condition is satisfied, the read data may be returned to the swap-data source (e.g., internal register or memory controller) with or without also writing back the read data to memory core105(e.g., if no data change will occur, write-back may be suppressed or otherwise omitted) and, conversely, if a swap does occur (i.e., exchange-condition satisfied), the swap data may be returned to the swap-data source. Alternatively, or as part of a different conditional swap command, the read data may be returned regardless of whether the exchange condition is satisfied.

Still referring toFIG.1D, in a more general conditional operation, the read data may be conditionally modified according to an evaluation of the read data, externally sourced data and/or internally-sourced (e.g., register-supplied) data, with the conditionally modified data written back to the memory core105and/or returned to the host requestor. Also, a combination of conditional modification and conditional swap may be carried out. As an example of a unary conditional operation, a read data value may be evaluated to determine whether it has more ‘0’ bits than ‘1’ bits (or vice-versa) and, if so, complemented by the modify logic to generate, as a modified data value, an inverted read data value that is written back to the memory core (and optionally transmitted back to the operation requestor). As an example of a binary conditional operation, the read data value may be compared with an externally sourced data value, with the more superlative value (read data value or externally sourced value) modified in some way (e.g., incrementing a counter field within the more superlative value to indicate the number of comparisons the more superlative value has won) before writing the more superlative value back to the memory core105. More generally, virtually any useful conditional exchange and/or conditional modification may be executed within the modify logic117with optional return of the original, modified and/or superlative data to the memory core or to the memory controller.

FIG.1Eillustrates a special case of a modification operation in which the modified data is returned to the memory controller instead of the read data value. As discussed, the read data value may be absolutely (i.e., in all cases) modified or conditionally modified within the modify logic117.

FIG.1Fillustrates another special case in which the read data is not returned to the memory controller in either its original or modified form, while the data modified or conditionally modified by modify logic117is written back to the memory core105.

Reflecting on the atomic operations described in reference toFIGS.1B-1F, it can be seen that each generally involves bi-directional data transfer with respect to the memory core105(including conditional bi-directional transfer as the write-back may be conditionally omitted or suppressed as discussed above). Accordingly, such operations are occasionally referred to herein as “duplex” operations to distinguish them from “simplex” operations in which data flow is uni-directional with respect to the memory core. While such duplex operations may be implemented with any underlying memory technology as discussed above, in memory technologies that exhibit a relatively long write latency (e.g., NAND-based flash memory, in which an entire physical page may be written at once), a number of implementation choices may be provided with regard to duplex operation timing. For example and without limitation, an internal write cache may be provided to buffer data to be written as part of a duplex operation, thereby enabling data write-back to be completed quickly within the write-cache. Transfer from the write-cache to the memory core may then occur over a longer time interval (e.g., as required by the underlying memory technology) and potentially at a later time, after multiple updates to the contents of the write cache.

Examples of simplex operations, which are also supported by the atomic memory device103, include memory read operations and memory write operations as illustrated inFIGS.1G and1H. As shown, operation of the modify logic117is disabled (an/or the internal read-data path/write-data path is decoupled from the modify logic117as indicated by the ‘X’) so that read data flows uni-directionally from the memory core105to the memory controller in a memory read operation (FIG.1G) and write data flows uni-directionally from the memory controller to the memory core105in a memory write operation (FIG.1H).

In the various embodiments described above, a memory device architecture supporting atomic operations within the device input-output path may receive a superset of commands including both commands for atomic operations as well as more traditional commands, such as those depicted with reference toFIGS.1G-1H.

FIG.2illustrates an embodiment of an atomic memory device180in greater detail. As with the generalized atomic memory device ofFIG.1A, atomic memory device180includes a memory core181, control logic circuitry183and data I/O circuitry185. For purposes of explanation only, the memory core105is assumed to be a DRAM core having one or more arrays of DRAM cells and corresponding sense amplifier banks191that are accessed in response to row and column commands received via command path214and corresponding row and column addresses received via address path216(the command and address paths collectively forming command/address path210). Incoming memory access commands are received within a command decoder197(e.g., a state machine, sequencer or other decode-and-control circuitry) which issues corresponding control signals to address decoding circuitry and to data I/O circuitry185to carry out the requested operation. Upon receiving a row activation command (i.e., command to transfer an address-selected row of data to the sense amp bank), for example, the command decoder197asserts a row-decode-enable signal (“rowdec_en”) to row decoder199to enable the row decoder to decode a row address received via the address path216and thereby activate a word line coupled to an address-selected row of cells within the memory core181. The activated word line enables the contents of the corresponding storage cells (i.e., the storage cells coupled to the word line) to drive respective bit lines (or pairs of bit lines) which are sensed by the bank(s) of sense amplifiers191. Through this “row activation” operation, the contents of a storage row may be sensed and latched within the sense amplifier bank(s)191, thus opening a “page” of data that may be accessed via column access (read and write) operations. Accordingly, upon receiving a column access command (i.e., command to read or write a column of data within a previously activated row, and thus a row of data within the sense amplifier bank), the command decoder197issues a column-decode-enable signal (“coldec_en”) to enable the column decoder201(or column multiplexer) to decode a column address received via the address path216and, by that operation, form a multiplexed signal conduction path between an address-selected column of data within the sense amplifier bank(s)191and a parallel data path referred to herein as the core data path260.

When an atomic command is received within the command decoder197, the command decoder issues decode-enable signals in accordance with the atomic operation requested (e.g., column-decode-enable if column data is to be retrieved as part of the atomic operation) and also outputs an operation-select (“opsel”) value to one or more modify logic circuits251included within the data I/O circuitry185. The command decoder197may additionally output numerous signals and commands to control data sample timing (i.e., data reception), data transmission timing, data buffering, internal data transfer between component circuit blocks, specialized program/erase operation (e.g., in the case of NAND or NOR flash or similar memory), maintenance operations (e.g., self-refresh, auto-refresh, signaling calibration, etc.) or any other control function within the atomic memory device181. Also, the command decoder197may include or enable access to various status registers, control registers and data registers to allow device configurability. In one embodiment, for example, support for atomic operations may be disabled through host-instructed programming of a mode register218within the command decoder, thus enabling the atomic memory device to mimic the behavior of legacy memory devices (i.e., in terms of operational timing and/or manner of decoding incoming commands, etc.). As another example, one or more operand registers216may be provided to provide operand(s) to the data modify logic251. In one implementation, for example, a solitary programmable operand register216is used to provide operand data to each of the modify logic circuits251within the data I/O circuitry185. In an alternative embodiment, a bank of programmable operand registers216are provided, with one or more of the operand registers216being selected in accordance with an incoming atomic memory command to provide operand data (“operand”) to the modify logic circuits251. All such mode registers218and operand registers216may be one-time or run-time programmable. In the case of run-time programmable registers, for example, the mode register218may be programmed in response to host instructions (e.g., provided via the memory controller) during system startup to establish an initial operating configuration, and the operand register(s)216may be programmed during startup and as needed thereafter to provide operands for use in atomic operations. Values programmed within the mode registers218and operand registers216may be transferred to the atomic memory device180via any or all of the signal paths shown (address216, command214, data212(DQ)), or via other signaling paths such as low-bandwidth control path, out-of-band signaling channel, etc.).

In one embodiment, the data I/O circuitry185includes a number of I/O bit-slice circuits225each coupled to a respective data link of the external data path via a pin (or pair of pins in a differential signaling implementation) or other integrated-circuit interconnect. Referring to the detail view of I/O bit-slice circuit2250(“I/O slice2250” for short), the on-chip portion of the incoming data link is coupled to a signal transceiver formed by sampling circuit231and output driver233. In one embodiment, data reception within the sampling circuit231is timed by transitions of a receive timing signal (which may be a strobe signal or clock signal received in association with the incoming data signal, or an internally synthesized signal) so that the sampling circuit outputs a serial stream of received data bits onto write-data-in (“wdi”) line232. As shown, the write-data-in line232extends into the modify logic251where it is coupled to one or more modify units259and also to a write-data-out (“wdo”) multiplexer255(or other signal switching or selection logic). As discussed below, the wdo multiplexer255selects either the wdi line232or an output of the modify units259to drive a serial write-data-out line234(“wdo line”), and thus enables passage of write data to the memory core181in a simplex write operation, or passage of swap data or modified data to the memory core181in a duplex (atomic) operation.

In one embodiment, data to be written to the memory core181is converted from serial to parallel form within each of the I/O bit-slices225, thus enabling the core cycle frequency (e.g., column-access cycle time (column cycle time), core clock cycle, or other cycle time of the memory core181) to be a fraction of the data I/O frequency. That is, a deserializing circuit241(“deser”) is provided at the interface between the core data path260and the data I/O circuitry185to convert serial data conveyed on the wdo line234to parallel data for conveyance on core data path260and storage within memory core181. In the particular implementation ofFIG.2, for example, serial data on the wdo line234is shifted bit by bit into deserializer241at the data I/O frequency (i.e., 1/(bit-time on data path)) and then framed and transferred out of the deserializer and onto core data path260at a word rate (e.g., the ratio of serial data frequency to core frequency), e.g., ⅛thor 1/16ththe data I/O frequency. For example, after serial shifting of each group of sixteen bits into a shift register of deserializer241, the core timing signal can be transitioned to transfer the 16-bit data slice within the shift register, in parallel, onto core data path260. In the exemplary embodiment ofFIG.2, the atomic memory device180has a 32-bit wide data interface (i.e., to interface to a 32-bit wide external data path) and enables operation of all the I/O bit slice circuits2250-22531simultaneously (i.e., each circuit receives data in parallel) so that a core data word formed by a total of 32*16=512 bits is transferred from the data I/O circuitry185to the core data path260at the conclusion of each core framing interval (core cycle) as marked by a transition of the core timing signal. The core data word is conveyed via the column decoder circuitry201to the appropriate 512 bit column of sense amplifiers within sense amplifier bank(s)191, overwriting the contents therein to complete a column write operation. Thereafter, after some number of memory write/read operations directed to the open page of data (i.e., contents of a storage row present in the sense amplifiers) is completed, a precharge operation may be carried out to close the open page. That is, if the page of data within the sense amplifier bank(s)191(which may include thousands or more 512-bit columns) has not already been written back to the corresponding row of storage cells, write-back to the storage cells is completed, the corresponding word line deactivated, and the bit lines and sense amplifiers conditioned in preparation for the next row activation operation.

Still referring toFIG.2, data flow in a simplex memory read operation is essentially the reverse of that described above in connection with a simplex memory write. That is, an address-selected column of data is output from the memory core181(i.e., from sense amplifier bank(s)191in a DRAM embodiment) to the core data path260via the column decoder201. Serializers243(“ser”) within respective I/O bit slice circuits225then operate in reverse-manner to the deserializers241described above, each converting a respective parallel set of 16 bits into a corresponding stream of sixteen serial bits that are output onto a read-data-in line236(the “rdi” line). The rdi line236is coupled to the modify units259within the modify logic251and to a read-data-out (“rdo”) multiplexer257. The rdo multiplexer257also receives a data output from the modify units259and operates in response to a control signal to pass either the serial data stream supplied via the rdi line236(i.e., the “retrieved data” or “read data”) or modified data from the modify units259to a read-data-out line238(the “rdo” line). The rdo line238conveys the serial stream of retrieved or modified data to output driver233which drives the data serially onto a respective one of the signaling links of the external data path212.

Still referring toFIG.2, the above-described relationship between the data I/O frequency and core cycle interval is shown at262and264. That is, during each core cycle interval in which data is being written into and/or retrieved from the memory core181, sixteen data bits are transmitted serially via the wdo line234and/or the rdi line236. During that same core cycle interval (though potentially offset to account for transfer delays within various circuits of the data I/O circuitry and/or core memory), a 512-bit core data word is transferred between the memory core and the core interface.

Reflecting on the atomic memory ofFIG.2, it should be noted that the specific numbers of bits, bit ratios, frequency ratios and so forth are provided for purposes of example only. In all such cases, different numbers of bits and ratios may apply. Further, while specific circuit blocks have been shown, numerous other circuit blocks may also be provided (and the functions of the circuit blocks shown and described organized differently with regard to such other circuit blocks) without departing from the scope of the present disclosure.

FIG.3illustrates an embodiment of a modify logic circuit280that may be used to implement modify logic251ofFIG.2. The modify logic circuit280includes a modify controller281, write-data-out (wdo) and read-data-out (rdo) multiplexers285and287, and a set of one or more modify units2930-293N-1(collectively, “293”). The modify controller281responds to incoming operation-select signals128(“opsel”) by issuing multiplexer-control signals, wdo_sel and rdo_sel, to the wdo and rdo multiplexers285and287, respectively, and by asserting enabling one or more modify-enable signals (en_0, en1, . . . , enN−1) to enable corresponding modify units293to perform evaluation and/or modification operations. The modify controller281may also receive one or more operation-result signals (res_0, res_1, res_N−1) from the modify units293and use those results in whole or part in generating the enable signals and multiplexer control signals. For example, in a compare-and-swap operation, one of the modify units293may perform a data comparison and provide the comparison result to the modify controller281to enable determination of the wdo multiplexer setting (and/or rdo multiplexer setting). Although not specifically shown, the modify controller281may be clocked or otherwise advanced between states in response to a clock signal (e.g., operating at the data I/O frequency or a subdivided frequency thereof such as the core clock cycle frequency) and may be implemented by any combination of combinatorial and state management logic. For example, in one embodiment the modify controller281is implemented as a finite state machine, though an instruction sequencer or even purely combinatorial implementation may be provided in alternative embodiments. In these embodiments, the modify controller281may include sets of parallel logic, one for each slice (i.e., for each modify unit2930-293N-1) for processing each slice in parallel.

Each of the modify units293or any subset thereof may be coupled to the read-data-in line236to enable receipt of retrieved serial data as necessary to carry out the operation specified by opsel signal128. Each of the modify units293or a subset thereof may also be coupled to the write-data-in line232to enable receipt of externally received serial data which may be write-data, swap data, an externally sourced operand or any other externally supplied information having useful application within the modify logic280. The modify units293or any subset thereof may be coupled to receive an operand from an operand register via operand path141as discussed above. Also, while a solitary operand path141is shown, multiple operand paths may be provided to provide multiple operands to a given modify unit293and/or to provide respective operands to different modify units. Each of the modify units293or any subset thereof may also include a select input (s0, s1, . . . , sN−1) coupled to receive a respective enable signal from the modify controller281, a result signal output (res_0, res_1, . . . , resN−1) to deliver an operation-result signal to the modify controller281, and a serial-data-output coupled to a modified-data line284to deliver modified data serially thereto. The modified-data line284conveys modified data to the wdo multiplexer285to enable the modified data to be written back to the memory core, and also to the rdo multiplexer287to enable the modified data to be output from the atomic memory device via the external data path, both as discussed above.

Still referring toFIG.3, any number of modify units293may be provided within modify logic280, each to perform respective modify functions or categories of modify functions. In the particular embodiment shown, modify units representative of three classes of modify-operations are depicted including a unary-operation unit2930, a binary-operation unit2931and an evaluation-operation unit293N-1.

The unary-operation unit2930demonstrates signal inputs and outputs representative of those used to enable unary operations with respect to data retrieved from the memory core. That is, the rdi line236is coupled to deliver the retrieved data to the unary operation unit which, when enabled by the modify controller281(i.e., en_0 asserted), carries out a specified unary operation (or unary operation for which the underlying circuitry is specifically designed) including for example and without limitation, increment/decrement, complement, absolute value, multiply or divide by fixed constant, exponent (raise to power), root (square root, cubed root or the like), logarithm, table lookup or any other single-argument function. Any result, res_0, generated as part of the unary operation may be returned to the modify controller281and/or stored within the modify unit2930or elsewhere within the atomic memory device for later use. As an example, a carry bit (i.e., overflow bit) or borrow bit (i.e., underflow bit) may be generated as part of an increment operation or decrement operation and summed with/subtracted from a subsequently retrieved data value to enable the increment/decrement operation to be extended to data values greater than 16 bits (i.e., enabling multiple retrieved data values to be processed as constituent parts of a larger data value).

The binary-operation unit2931demonstrates signal inputs and outputs representative of those used to enable binary operations with respect to data retrieved from the memory core. In the particular implementation shown, the binary-operation unit receives the retrieved data via rdi line236and an operand supplied via wdi line232as inputs. As discussed above, an operand may additionally or alternatively be supplied via from one or more operand registers within the atomic memory device via respective operand paths141. In any case, the binary operation unit2931carries out a binary operation (or ternary operation, quaternary operation, etc. according to the number of supplied operands) when enabled by the modify controller281and outputs resultant modified data onto the modified-data line284and, optionally, a result signal (e.g., borrow, carry, etc.) onto the result signal line, res_1. As with the unary-operation unit2930, the binary-operation unit2931may execute a selected, specified operation (e.g., specified by the modify controller) or an operation for which the underlying circuitry is specifically designed. Examples of the binary operations performed include, for example and without limitation, arithmetic operations (add, subtract, multiply, divide), bit-wise logical operations (e.g., a mask operation), Boolean operations (AND, OR, XOR, . . . ), two-dimensional table lookup, or any other multi-operand functions.

Still referring toFIG.3, the evaluation-operation unit293N-1may be viewed as a form of unary or binary operation unit (according to the number of operands delivered) but is presented separately to emphasize that at least some operations do not require data output onto the modified-data line284(hence the dashed interconnection of unit293N-1and modified-data line284). That is, in one embodiment, the evaluation-operation unit performs an evaluation of a retrieved data value alone (unary evaluation) or in combination with data received from an external source and/or one or more operands (binary evaluation) and outputs an evaluation-result signal on result line res_N−1 and/or an evaluation data value (e.g., comparison winner) on modified-data line284. The result of the evaluation may be signaled to the modify controller281, for example, to enable the modify controller281to responsively control the wdo and/or rdo multiplexers285,287. As an example, if an evaluation result resolves to “TRUE” (as signaled via res_N−1), then the wdi line232can be used to drive wdo line234, thereby enabling a data swap. Otherwise, if the evaluation result resolves to “FALSE”, a write-back operation can be enabled, for example, by passing the retrieved data through the evaluation-operation unit293N-1to the modified-data line284(e.g., as a comparison winner), and setting the wdo multiplexer285to couple the modified-data line284to the wdo line234, thereby routing the retrieved data onto the wdo line234for write-back to the memory core. An example of the foregoing operation is an operation to swap data only if the incoming value is greater than the resident value in memory.

As an example of a unary evaluation, retrieved data may be evaluated to determine whether a particular Boolean condition is met (e.g., retrieved value evaluates to TRUE or FALSE) or whether the retrieved data otherwise meets a predetermined condition, with data exchange or other modify operations being performed with respect to the same retrieved data value or a subsequently retrieved data value according to the evaluation result. For example, one conditional-increment function may increment data only if not at a maximum (e.g., either a defined maximum of data or an increment that avoids an overflow). In a binary evaluation, the retrieved data value may be compared with the incoming operand data (i.e., from register and/or external source) to generate an operation result (e.g., inequality, match, logical combination of retrieved value and operand satisfy a predetermined or register-specified condition, etc.), with the operation result again being used to enable conditional data exchange or other modify operations with respect to the retrieved data value or a subsequently retrieved data value.

As shown in detail view301, each of the modify units293may be implemented by a modify circuit305and one or more delay circuits,307,309,311. The modify circuit305may include any combinatorial or state-based logic for generating an operation result and modified data in response to an enable signal (which may be a multi-bit signal to instruct operation of one of multiple possible operations supported by the modify circuit305). In general, such logic may be synthesized using circuit design tools by specifying the operation to be performed (and thus the operation result and the modified data output) with respect to the incoming retrieved data and any operands. The delay circuits307,309,311may include hardwired or adjustable delay circuits (e.g., in response to a register-programmed value, or a dynamic value provide in connection with the operation-selection value) to delay propagation of any or all incoming operands (ingress delay circuits307,309) to the inputs of the modify circuit305, and/or to delay propagation of the modified data to the modified-data line284(egress delay circuit311) and/or to delay output of the operation result (result delay circuit313) to the modify controller. By this arrangement, incoming operands may be provided to the modify circuit305at an appropriate time or the modified data or an operation-result may be driven at a desired time, thus enabling coordination of various events within and external to the modify logic280as well as pipelining of atomic operations. A designer may utilize these features so as to tailor (e.g., optimize) traffic flow through the input-output path of the memory device for the atomic operations supported for the particular design.

FIG.4illustrates a table of operations (340) that may be initiated and controlled by the modify controller ofFIG.3in accordance with the operation specified by the command decoder or other control circuitry within the atomic memory device. Starting with simplex memory read and simplex memory write operations shown in the first two rows of table340, because no data is being conditionally or absolutely transmitted in a direction counter to the simplex data flow, no modify unit is enabled. Instead, the modify controller sets the rdo multiplexer to forward the retrieved data (“read data”) onto the read-data-out line in a simplex memory read operation, and sets the rdi multiplexer to forward the incoming write data onto the write-data-out line in a simplex memory write operation. Although only two simplex data operations are shown, other simplex operations may be performed, including masked write, masked read, etc.

Turning to the duplex (atomic) operations listed in table340, in a read/increment operation, the modify controller enables a unary modify unit to carry out an increment operation with respect to data retrieved from the memory core (the “read data), and sets the rdo and wdo multiplexers to output the read data from the atomic memory device and to deliver the modified data output from the enabled modify unit (i.e., the incremented read data in this example) to the memory core to be stored in place of the just-retrieved read data. Thus, a memory read is performed concurrently with incrementing the read data value, returning the read data to the host requestor concurrently (i.e., at least partly overlapping in time) with writing the incremented read data back to the memory core. The increment/read operation is similar (i.e., unary modify unit also selected), except that the modified (incremented) data is both written back to the memory core and returned to the host requester.

Other examples of unary operations specifically shown in table340include read/complement (read a data value and overwrite it with its complement (inverted data value)) and complement/read (overwrite the read data value with its complement and return the complement value to the host). In all such cases, the write-back to the memory core may be conditioned on evaluation of the retrieved data and/or one or more operands. A l's complement operation may also be used (as opposed to a straight complement). As should be apparent, although not specifically listed in table340, numerous other unary operations may be performed as discussed above.

Turning to examples of binary duplex operations that reference register-sourced operands, in a read/add-offset operation, the offset value within a register is added to a retrieved data value to establish a variable+constant result that may be unconditionally or conditionally written back to the memory core. More specifically, a binary-operation unit that performs the data+operand operation is enabled by the modify controller, and the rdo and wdo multiplexers are set to pass the retrieved data value to the rdo line and the modified data value to the wdo line, respectively. In an add-offset/read operation, similar results are obtained, but the modified data value (retrieved data value plus operand) is returned to the host requestor instead of the read data value. Read/subtract-offset and subtract-offset/read operations are presented as additional examples of binary, register-based operations. Though not specifically listed in table340, numerous other register-based binary operations may be performed.

The last set of exemplary operations presented in table340are binary operations that involve a host-supplied operand, that is, binary duplex operations in which an externally-sourced operand delivered via the wdi line is supplied to a modify unit together with a retrieved data value. The specific examples presented include swap, compare-and-swap, read/add-variable, and add-variable/read. In the swap operation, no modify unit need be enabled (as signified by the “N/A” or not-applicable designation) and instead the modify controller sets the rdo and wdo multiplexers to output the read data to the host requestor and to deliver the swap data to the memory core to overwrite the just-retrieved read data (thus effecting a swap operation). A compare-and-swap operation is carried out similarly, except that the modify controller enables a compare operation within an evaluation-operation unit, and then sets the rdo and wdo multiplexers in accordance with the compare result. In the embodiment shown, for example, the wdo multiplexer may deliver either the swap data or read data onto the wdo line (alternatively, no data may be driven onto the wdo line if the swap data is not to be written back to the memory core) and conversely deliver either the read data or the swap data onto the rdo line according to the comparison result. That is, if a swap is to be executed, the swap data is delivered to the memory core and the read data is returned to the host requestor. If a swap is not to be executed, the read data is delivered to the memory core (or no write back is executed) and the swap data is optionally returned to the memory requestor to enable the memory requestor to ascertain the comparison result.

Turning to the read/add-variable operation, retrieved data is returned to the host requestor and also added to an externally-supplied operand to generate a sum that is written back, as a modified data value, to the memory core. In the case of an add-variable/read operation, the sum is both written back to the memory core and returned to the host requestor. Again, though not specifically listed in table340, numerous other binary operations that involve host-supplied operands may be performed. Also, as described above, all such arithmetic operations, regardless of their operand source, may be extended to enable operation with respect to multiple retrieved data values through borrow or carry storage or other state information as appropriate for the operation performed.

FIG.5illustrates a generalized and exemplary operation of an atomic memory device in response to receipt of a memory access command as shown at351. If no memory read is required (determined in decision block353), then the requested memory access is a simplex write. Accordingly, the incoming write data value is stored in the memory core as shown at355. If a memory read is required, then read data is retrieved from the memory core in an operation shown generally at357. If the memory access command indicates a duplex operation (i.e., data flowing both into and out of the memory core in response to the memory access command), then execution proceeds to decision block359. Otherwise, the memory access command is simplex read command, and the read-data-out multiplexer is set to select the read-data-in line and thus output the read data to the host requestor as shown at361, thereby completing the simplex operation.

Continuing with the case of a duplex operation (i.e., affirmative determination at block359), if the atomic command indicates a binary operation (i.e., it is determined at363that the operation involves a data source other than the retrieved data value), then the operand is received from a register or external source (e.g., from an operand register or via the write-data-in line) at365and supplied to the appropriate modify unit. Thereafter, whether unary operation (negative determination at decision block363) or binary operation, the appropriate modify unit is selected in accordance with the specified atomic operation and enabled at367to generate a modified data value or evaluation result with respect to the retrieved data and any supplied operands. The read-data-out multiplexer and write-data-out multiplexer are set at369in accordance with the duplex operation being performed and any evaluation result, thus enabling concurrent output of the read data or modified data to the host requestor at371and/or storage of operand data (e.g., swap data) or modified data in the memory core at373.

FIG.6is a timing diagram illustrating signal timing during an exemplary duplex operation within the atomic memory device ofFIG.3. At the start of a core cycle ‘i,’ a memory access command385specifying a duplex (atomic) operation is received via the command/address path210(C/A) concurrently with receipt of an operand386to be applied within the duplex operation via the external data path212(D/Q). After a data sampling delay, the operand is presented on the write-data-in line232as shown at388and thus to the modify logic of the atomic memory device. Meanwhile, during the interval marked inFIG.6as “Read Data Retrieval,” read data is retrieved from the memory core in accordance with an address provided in association with the duplex operation, eventually becoming valid and presented to the modify logic on the read-data-in line236as shown at390. Assuming a duplex operation in which read data or modified data (including swap data) is to be returned to the host requestor, the output of the modify logic (read data or modified data or swap data, for example) is output onto the read-data-out line238as shown at392and, at approximately the same time (or shortly before or after data output onto rdo line238), modified data (including swap data) is output onto the write-data-out line to234be written back to the memory core as shown at394. Concurrently with write-back to the memory core, data output onto the read-data-out line is driven onto the external data path212as shown at396and thereby returned to the host requestor during the beginning of the succeeding core cycle (i.e., core cycle i+1). In the particular example shown, a simplex memory read operation is commanded at the start of core cycle i+1 (i.e., as shown at398), with the data retrieval operation being carried out with essentially the same timing as the data retrieval in the preceding duplex operation. That is, read data i+1 becomes available on the rdi line as shown at400(i.e., after the Read Data Retrieval interval), and is routed onto the rdo line shortly thereafter as shown at402. Read data i+1 is then output onto the external data path as shown at404, thus completing the simplex memory readjust as another read operation is received in the ensuing core cycle.

It should be noted that the various circuits disclosed herein may be described using computer aided design tools and expressed (or represented), as data and/or instructions embodied in various computer-readable media, in terms of their behavioral, register transfer, logic component, transistor, layout geometries, and/or other characteristics. Formats of files and other objects in which such circuit expressions may be implemented include, but are not limited to, formats supporting behavioral languages such as C, Verilog, and VHDL, formats supporting register level description languages like RTL, and formats supporting geometry description languages such as GDSII, GDSIII, GDSIV, CIF, MEBES and any other suitable formats and languages. Computer-readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, computer storage media in various forms (e.g., optical, magnetic or semiconductor storage media, whether independently distributed in that manner, or stored “in situ” in an operating system).

When received within a computer system via one or more computer-readable media, such data and/or instruction-based expressions of the above described circuits may be processed by a processing entity (e.g., one or more processors) within the computer system in conjunction with execution of one or more other computer programs including, without limitation, net-list generation programs, place and route programs and the like, to generate a representation or image of a physical manifestation of such circuits. Such representation or image may thereafter be used in device fabrication, for example, by enabling generation of one or more masks that are used to form various components of the circuits in a device fabrication process.

In the foregoing description and in the accompanying drawings, specific terminology and drawing symbols have been set forth to provide a thorough understanding of the present invention. In some instances, the terminology and symbols may imply specific details that are not required to practice the invention. For example, any of the specific numbers of bits, signal path widths, signaling or operating frequencies, component circuits or devices and the like may be different from those described above in alternative embodiments. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the present invention unnecessarily. Additionally, the interconnection between circuit elements or blocks may be shown as buses or as single signal lines. Each of the buses may alternatively be a single signal line, and each of the single signal lines may alternatively be buses. Signals and signaling paths shown or described as being single-ended may also be differential, and vice-versa. A signal driving circuit is said to “output” a signal to a signal receiving circuit when the signal driving circuit asserts (or deasserts, if explicitly stated or indicated by context) the signal on a signal line coupled between the signal driving and signal receiving circuits. The expression “timing signal” is used herein to refer to a signal that controls the timing of one or more actions within an integrated circuit device and includes clock signals, strobe signals and the like. “Clock signal” is used herein to refer to a periodic timing signal used to coordinate actions between circuits on one or more integrated circuit devices. “Strobe signal” is used herein to refer to a timing signal that transitions to mark the presence of data at the input to a device or circuit being strobed and thus that may exhibit periodicity during a burst data transmission, but otherwise (except for transition away from a parked condition or other limited pre-amble or post-amble transition) remains in a steady-state in the absence of data transmission. The term “coupled” is used herein to express a direct connection as well as a connection through one or more intervening circuits or structures. Integrated circuit device “programming” may include, for example and without limitation, loading a control value into a register or other storage circuit within the device in response to a host instruction and thus controlling an operational aspect of the device, establishing a device configuration or controlling an operational aspect of the device through a one-time programming operation (e.g., blowing fuses within a configuration circuit during device production), and/or connecting one or more selected pins or other contact structures of the device to reference voltage lines (also referred to as strapping) to establish a particular device configuration or operation aspect of the device. The terms “exemplary” and “embodiment” are used to express an example, not a preference or requirement.

While the invention has been described with reference to specific embodiments thereof, it will be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope. For example, features or aspects of any of the embodiments may be applied, at least where practicable, in combination with any other of the embodiments or in place of counterpart features or aspects thereof. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.