Patent ID: 12204478

DETAILED DESCRIPTION

As contemplated by this disclosure, commonly used processor cache hierarchy techniques may move data closer to a processor or cores of a multi-processor to reduce a cost of data movement. This movement of data that causes a retaining of data in a particular cache for subsequent reuse may work well for cache friendly applications or kernels that have data access characteristics such as a high level of spatial or temporal locality. But these types of processor cache hierarchy techniques may be ineffective for types of applications or kernels having data access characteristic in which there is no significant data reuse or has a low level of spatial or temporal locality. In addition, the inefficiencies of these types of cache hierarchy techniques further worsen performance for these types of applications or kernels. For types of applications or kernels having a data access characteristic that includes a low level of spatial or temporal locality, moving at least a portion of processing nearer to data may more efficiently reduce data movement in a computing system executing these applications or kernels and improve performance of these types of applications or kernels.

Existing multi-core architectures may not be able to realize bandwidth advantages of new memory technologies such as, but not limited to, high bandwidth memory (HBM) without some improvement in efficiency of data movement. For example, if a 16 tile die (i.e., 16 core processor) is built to include two to four HBM stacks the 16 tiles may not drive enough data access requests to utilize a relatively high amount of memory bandwidth provided by the two to four HBM stacks. Similarly, a mesh interconnect for this type of 16 tile die (e.g., an on-die interconnect (ODI)) may not be able to route data access requests at a rate fast enough to utilize this relatively high amount of memory bandwidth. Additionally, respective memory controllers for each HBM stack may further degrade the rate at which data access requests may occur. For example, applications or kernels having data access requests with a low level of spatial and temporal locality may cause cores to submit data access requests to separate HBM stacks and those data access requests may be processed by multiple memory controllers. Data access requests to multiple memory controllers may add a significant amount of memory access latency when executing these types of applications or kernels.

As described in more detail below, energy efficient, near data accelerators or processors with reduced compute and caching capabilities may be able to better utilize the high memory bandwidth provided by new memory technologies such as, but not limited to, HBM. These types of energy efficient, near data accelerators or processors may be capable of overcoming at least some of the above-mentioned cache hierarchy techniques and ODI data movement inefficiencies and better utilize the increased memory bandwidth provided by new memory technologies.

FIG.1illustrates an example system100. According to some examples, as shown inFIG.1, system100includes a processor101coupled with memory130-1to130-N via respective memory controllers120-1to120-N that are communicatively coupled with cores110-1to110-N via an on-die interconnect (ODI)105. For these examples, “N” represents any whole, positive integer greater than 3. Also, for these examples, ODI105may couple memory controllers120-1to120-N to cores110-1to110-N as part of a mesh interconnect architecture that includes a 2-dimensional array of half rings going in the vertical and horizontal directions which allow communication routes between cores and memory controllers to take a shortest path. This mesh interconnect architecture may also allow for at least some elements of cores110-1to110-N to couple with memory controllers120-1to120-N via ODI105. For example, cache home agents (CHAs)114-1to114-N that manage cache lines (e.g., including up to 64 bytes of data) maintained in respective caches112-1to112-N may be able to communicate with elements of memory controllers120-1to120N via ODI105. As described more below, memory controllers such as memory controllers120-1to120-N may include near data processors (NDPs) such as NDPs122-1to122-N to communicate with CHAs such as CHAs114-1to114-N via ODI105in order to facilitate efficient near data processing. These near data processing schemes may include NDPs122-1,122-2,122-3and122-N to directly access data maintained in respective memories130-1,130-2,130-3and130-4. The directly accessed data, for example, to be used to execute kernels or applications having data access characteristics to a memory (e.g., memory130-1,130-2,130-3or130-N) with a low level of spatial or temporal locality.

According to some examples, the elements of system100may be included on a system on a chip (SoC) or in a multi-chip package. For these examples, elements of processor101may be included in a first portion of an SoC or first chip of a multi-chip package. Meanwhile memory130-1to130-N may be positioned in different portions of the SoC or in separate chips of a multi-chip package. For example, memory130-1to130-N may be HBM types of memory separately included in respective chips of the multi-chip package.

In some examples, memory130-1to130-N may include volatile types of memory whose state (and therefore the data stored in it) is indeterminate if power is interrupted. Dynamic volatile memory requires refreshing of date stored to this type of volatile memory to maintain state. One example of dynamic volatile memory incudes DRAM (Dynamic Random Access Memory), or some variant such as Synchronous DRAM (SDRAM). A memory subsystem as described herein may be compatible with a number of memory technologies described in various standards or specifications, such as DDR3 (Double Data Rate version 3, original release by JEDEC (Joint Electronic Device Engineering Council) on Jun. 27, 2007). DDR4 (DDR version 4, originally published in September 2012 by JEDEC), DDR5 (DDR version 5, originally published in July 2020), LPDDR3 (Low Power DDR version 3, JESD209-3B, August 2013 by JEDEC), LPDDR4 (LPDDR version 4, JESD209-4, originally published by JEDEC in August 2014), LPDDR5 (LPDDR version 5, JESD209-5A, originally published by JEDEC in January 2020), WIO2 (Wide Input/output version 2, JESD229-2 originally published by JEDEC in August 2014), HBM (High Bandwidth Memory, JESD235, originally published by JEDEC in October 2013), HBM2 (HBM version 2, JESD235C, originally published by JEDEC in January 2020), or HBM3 (HBM version 3 currently in discussion by JEDEC), or others or combinations of memory technologies, and technologies based on derivatives or extensions of such specifications. The JEDEC standards or specifications are available at www.jedec.org.

According to some examples, memory130-1to130-N may include at least some non-volatile types of memory whose state is determinate even if power is interrupted. These types of non-volatile memory may include block or byte-addressable, write-in-place memories. Examples may include, but are not limited to, single or multi-level Phase Change Memory (PCM) or phase change memory with a switch (PCMS), non-volatile types of memory that include chalcogenide phase change material (for example, chalcogenide glass), resistive memory including metal oxide base, oxygen vacancy base and Conductive Bridge Random Access Memory (CB-RAM), nanowire memory, ferroelectric random access memory (FeRAM, FRAM), magneto resistive random access memory (MRAM) that incorporates memristor technology, spin transfer torque (STT)-MRAM, a spintronic magnetic junction memory based device, a magnetic tunneling junction (MTJ) based device, a DW (Domain Wall) and SOT (Spin Orbit Transfer) based device, a thyristor based memory device, or a combination of any of the above, or other types of block or byte-addressable, write-in-place memory.

FIG.2illustrates an example near data processor architecture200. In some examples, as shown inFIG.2, NDP122included in memory controller120may be configured according to NDP architecture200that has multiple processing engines (PEs)220-1to220-N and multiple memory buffers (MBs)210-1to210-N to facilitate direct access to a memory (e.g., HBM) via memory channels221. Memory channels221may configured to operate according to various JEDEC standards, for example, LPDDR4, LPDDR5, DDR4, DDR5, HBM, HBM2 or HBM3. As described more below, PEs220-1to220-N may be processing or accelerator circuitry configured to make direct, high bandwidth, low latency requests to access data maintained in memory managed by memory controller120. At least a portion of MBs210-1to210-N may serve as request buffers or queues for access requests submitted by PEs220-1to220-N to access data and/or maintain coherency with one or more cores of a processor. In some examples, MBs210-1to210-N may be composed of volatile types of memory such as, but not limited to, static RAM (SRAM) or DRAM. According to some examples, NDP122may be capable of using data maintained in coarse grained memory regions to execute types of kernels or applications having a low level of spatial and temporal locality. A host (e.g., core of processor110) may submit work requests to NDP122to execute these types of example loop or sub-routine applications. MBs210-1to210-N may also be utilized by PEs220-1to210-N as request or response buffers for work requests placed to NDP122and for results or responses by NDP122for those work requests.

According to some examples, NDP122may be configured as a matrix, vector or spatial accelerator that can drive a high amount of data bandwidth available to PEs220-1to220-N. For these examples, NDP122may receive a request to execute memory intensive kernels, applications, or loop routines with poor cache reuse. Hence, NDP122does not include a cache. Rather, as shown inFIG.2for NDP architecture200, NDP122may be configured as an array of PEs220-1to220-N and MBs210-1to210-N. Examples are not limited to the array structure shown inFIG.2for NDP architecture200. Other examples may include an array structure having different combinations of PEs and MBs than what is shown inFIG.2.

In some examples, a kernel, an application, or loop routine may be requested or offloaded to NDP122by a host (e.g., a core of processor101). For these examples, NDP122may have to consider implementing memory requests to an entire virtual address space for the host. Implementing a memory request to the entire virtual address space may make it important for NDP122to have its own address translation mechanism. According to some examples, NDP122may include a memory management unit (MMU)215to implement this address translation mechanism. A possible address translation mechanism may include using larger page sizes and configuring MMU215to work with either these larger page sizes or custom ranges of the virtual address space to reduce translation lookaside buffer (TLB) misses that may negatively impact the performance of NDP122(TLB not shown). For example, MMU215may implement an address translation mechanism that utilizes a TLB to map physical memory addresses of system memory (e.g., included in memory130-1to130-N) to custom ranges of the virtual address space allocated or assigned to individual cores of a processor such as processor101.

According to some examples, a challenge of using NDP122to move computing closer to memory is that there may be a risk that NDP122may only have access to a limited portion of system memory as opposed to a type of accelerator that is on the same die (on-die) in relation to the system memory. For example, NDP122-1is placed on memory controller120-1, as shown inFIG.1. For this this example, NDP122-1will have direct access to memory130-1. However, kernels or applications executed by NDP122-1may not be limited to data stored in only memory130-1. In some examples, to address data partitioning between memories, in order for NDP122-1to be functionally complete, NDP122-1needs to ensure that execution of kernels or applications does not fail if: (1) data used for that execution is not local to the physical memory region directly accessible to NDP122-1and yet to be energy efficient, NDP122-1should be selected for execution of kernels or applications that minimize requests to obtain data located in non-local memory (e.g., located in memory130-2,130-3or130-N); and (2) a single instance of kernel or application execution might need data located in multiple memories (e.g., memory130-1, memory130-2, memory130-3, etc.).

FIG.3illustrates an example pseudocode300andFIG.4illustrates an example OPCODE400. According to some examples, pseudocode300indicates how an NDP such as NDP122may execute a type of locality aware memory instruction such as OPCODE400to enable NDP122to be functionally complete and avoid possible execution fails due to data partitioning. For these examples, physical memory may be interleaved at a cache line or page size granularity. Since NDPs such as NDP122-1, NDP122-2, NDP122-3or NDP122-N have direct access to physical memory included in corresponding memories130-1,130-2,130-3and130-N, depending on the interleaving scheme used, if an address translation fails all addresses within the interleaved physical memory will fail. In order to maximize energy efficiency, NDPs should avoid unnecessary memory accesses to physical memory. For these examples, OPCODE400, when executed, serves as a type of locality aware memory instruction that may minimize unnecessary memory requests by issuing memory requests based on previous translation failure.

FIG.5illustrates an example system500. In some examples, as shown inFIG.5, system500includes a processor501coupled with high bandwidth memory (HBM)530-1and530-2via respective memory controllers520-1and520-2that couple with cores510-1,510-2and510-3via an on-die interconnect (ODI)505. For these examples, processor501may include additional cores and/or memory controllers coupled with ODI505, but only three cores and two memory controllers are shown for simplicity purposes. Similar to ODI105mentioned above for system100inFIG.1, ODI505may couple memory controllers520-1and520-2to cores of processor501as part of a mesh network. Also, similar to what was mentioned for ODI105, ODI505may allow for at least some elements of cores510-1,510-2and510-3to couple with memory controllers520-1and520-2. For example, cache home agents (CHAs)514-1,514-2and514-3that manage cache lines maintained in respective core-specific caches for respective cores510-1,510-2and510-3may be able to communicate with NDPs522-1and522-2via ODI505in order to facilitate efficient near data processing (e.g., to execute offloaded kernels or applications having data access characteristics with a low level of spatial or temporal locality).

According to some examples, NDP522-1and NDP522-2may separately have high bandwidth, low latency access to respective HBM530-1and HBM530-2. A user of processor501may cause a device driver (not shown) that manages/controls NDP522-1or NDP522-2to intelligently partition data to limit data access to NDPs not directly accessing an HBM. For example, an intelligent partition of data would limit NDP522-1's access to data maintained in HBM130-2and vice versa for NDP522-2's access to data maintained in HBM130-1. However, for functional completeness, some applications or kernels may require a low frequency of accesses by an NDP to other, non-directly accessible memory or HBM stack(s).

In some examples, an inter-memory controller network between memory controller520-1and memory controller520-2may be established using ODI505to enable NDPs to have at least a low frequency access to non-directly accessible HBMs. For these examples, during a system boot process, NDP522-1and NDP522-2would separately register (e.g., via a device driver of an operating system) as an agent communicatively coupled to or on ODI505. Once registered as an agent on ODI505, NDP522-1or NDP522-2may make requests to other memory controllers. For example, NDP522-1may make a request to memory controller520-2to access data maintained in HBM530-2. According to some examples, registration of NDPs522-1and522-2as agents on ODI505may be used as an inter-memory controller for other features like data transfers between HBM530-1and530-2without a need to engage with or involve any cores of processor501.

FIG.6illustrates an example process600. In some examples, process600may be an example process implemented to enable an NDP such as NDP522-1to avoid possible execution fails due to possible cache coherency issues with cores of processor501for data loaded to NDP522-1that is also maintained in shared memory regions of HBM530-1or530-2, but the cores rarely access the data maintained in these shared memory regions at the same time as NDP522-1. For these examples, process600shows a latency advantage for implementing a lazy coherence with speculation for loading data accessed from a shared memory region of HBM530-1compared to lazy coherence without speculation. Implementing lazy coherence with speculation may not always be an option or can be expensive in terms of performance impacts for some types of near data processors or accelerator architectures. For example, a significant fraction of data accessed by an NDP such as NDP522-1may be in HBM530-1. Therefore, latency costs to ensure coherency should be minimized. As described more below for process600, logic and/or features of NDP522-1may speculatively make requests to HBM530-1and CHA514-1simultaneously to minimize possible latency hits due to ensuring coherency.

Beginning at 6.1, logic and/or features of NDP522-1send a read request to CHA514-1to indicate a need to access data maintained in a shared region of HBM530-1for use to execute a kernel, an application, or a loop routine. The read request placed in 6.1 is a solid line to indicate that the read request is without speculation. In other words, no speculation is made that CHA514-1will not own or control a cache line or core510-1is not currently using the data maintained in the shared region of HBM530-1.

Moving to 6.2, the logic and/or features of NDP522-1may also send a read request to HBM530-1to read the data maintained in the shared region at substantially the same time as sending the read request to CHA514-1. The read request placed at 6.2 is a dashed line to indicate that a speculation is made that CHA514-1does not own or control a cache line or core510-1is using the data NDP522-1is requesting.

Moving to 6.3, CHA514-1indicates that the requested data is owned or controlled by CHA514-1and logic and/or features of NDP522-1may receive the data from core510-1cache line to load that cache line data for processing and the load is complete for use to execute the kernel, the application, or the loop routine. Process600may stop at this point and responses from HBM530-1for the data may be ignored or discarded by the logic and/or features of NDP522-1.

Moving to 6.4, CHA514-1indicates a cache line to the data maintained in the shared memory region of HBM530-1is not owned or controlled by CHA514-1or is not being used by core510-1. This indication by CHA514-1results in the logic and/or features of NDP522-1determining that no coherency issues exists for directly accessing the data maintained in the shared memory region of HBM530-1. The solid line for 6.4 indicates that logic and/or features of NDP522-1had to place a read request after first receiving an indication from CHA514-1that CHA514-1did not own or control the cache line and no coherency issues exist to access the data from HBM530-1.

Moving to 6.5, the logic and/or features of NDP522-1may receive the data requested from HBM530-1responsive to the lazy coherence with speculation request made at 6.2. Logic and/or features of NDP522-1may load that data received from HBM530-1for use to execute the kernel, the application, or the loop routine.

Moving to 6.6, data read from HBM530-1is received following a lazy coherence without speculation request. Process600may come to an end for lazy coherence without speculation.

In some examples, as shown inFIG.6, the speculation latency advantage is shown to indicate that some latency advantage may be gained when a lazy coherence with specification is implemented and CHA514-1does not own or control the cache line to the data maintained in the shared memory region of HBM530-1. In other words, speculation that CHA514-1would not own or control the cache line and simultaneously submitting a read request to HBM530-1helps to mitigate possible latency hits associated with maintaining coherency for data accessed from a shared memory region of HBM530-1.

According to some examples, logic and/or features of NDP522-1may default to always waiting for a response from CHA514-1before loading data for use to execute the kernel, the application, or the loop routine. That way, if the response from CHA514-1at 6.3 is received after a response from HBM530-1at 6.5, the logic and/or features of NDP522-1may ensure the data is correct or coherent before using the data to execute the kernel, the application, or the loop routine.

In some examples, logic and/or features of NDP522-1may perform an occasional or periodic calibration of speculation for data maintained in a shared region of memory for HBM530-1. For these examples, memory operations may be further optimized by tracking success of speculative loads from memory to reduce false speculations. If previous loads of the data received from HBM530-1fail, (e.g., CHA514-1owned the cache line or had a more up to date value for the data) then future speculations may fail as well. The logic and/or features of NDP522-1may stop lazy coherence with speculation if a number of fails reaches a threshold over a given period of time. Speculation may be stopped for a period of time and then started again and continue to be tracked for success or failure to determine whether to again stop lazy coherence with speculation, at least for the period of time.

FIG.7illustrates an example process700. In some examples, process700may be another example process implemented to enable an NDP such as NDP522-1to avoid possible execution fails due to cache coherency issues with cores of processor501for data to be stored in shared memory regions of HBM530-1or530-2but the cores rarely access data to be stored in these shared memory regions at the same time as NDP522-1. For these examples, process700may be implemented when a CHA for a core such as CHA514-1for core510has a full cache line (CL) ownership or control for data maintained in a shared memory region of HBM530-1, and new values are to be stored to HBM530-1at the shared memory region of HBM530-1. The new values, for example, generated by PEs of NDP522-1when executing an application, a kernel, or a loop routine.

Beginning at 7.1, logic and/or features of NDP522-1send a request to CHA514-1to indicate that a new value has been generated for data that is to be stored to a shared memory region of HBM530-1. For process700, only lazy coherence without speculation is shown to show a simplified example when CHA514-1has full CL ownership or control and new values are to be stored to the shared memory region of HBM530-1.

Moving to 7.2, CHA514-1indicates that it has full CL ownership or control. In some examples, CHA514-1may provide the data included in the CL to logic and/or features of NDP522-1to indicate that CL ownership or control.

Moving to 7.3, the logic and/or features of NDP522-1may send an indication to CHA514-1that the data in the CL owned or controlled by CHA514-1is out-of-date or invalid. For example, by indicating new values to be stored to the shared memory region of HBM530-1.

Moving to 7.4, the logic and/or features NDP522-1may also send the new values to the shared memory region of HBM530-1to store the new values. Process700then comes to an end.

FIG.8illustrates an example process800. In some examples, process800may be another example process implemented to enable an NDP such as NDP522-1to avoid possible execution fails due to cache coherency issues with cores of processor501for data to be stored in shared memory regions of HBM530-1or530-2but the cores rarely access data to be stored in these shared memory regions at the same time as NDP522-1. For these examples, process800may be implemented when a CHA for a core such as CHA514-1for core510has partial CL ownership or control pertaining to a CL used by CHA514-1to access data maintained in a shared memory region of HBM530-1and new values are to be stored to HBM530-1at the same shared memory region of HBM530-1accessed by the CL used by CHA514-1. The new values, for example, generated by PEs of NDP522-1when executing an application, kernel, or loop routine using data maintained in a shared memory region of HBM530-1.

Beginning at 8.1, logic and/or features of NDP522-1send a request to CHA514-1to indicate that a new value has been generated for data that is to be stored to a shared memory region of HBM530-1. For process800, both lazy coherence without speculation (solid line) and lazy coherence with speculation (dashed line) are shown to indicate a possible speculation latency advantage when CHA514-1only has a partial CL ownership or control and new values are to be stored to the shared memory region of HBM530-1.

Moving to 8.2, the logic and/or features of NDP522-1may also send a read request to HBM530-1to read the data maintained in the shared region at substantially the same time as sending the request to CHA514-1. The sending of a request to both CHA514-1and HBM530-1reflects a speculation that CHA514-1does not have a full CL ownership or control.

Moving to 8.3, CHA514-1responds to the request by indicating only partial CL ownership or control. In some examples, CHA514-1may provide the data in the partially owned or controlled CL to logic and/or features of NDP522-1to indicate the partially owned or controlled CL and for the logic and/or features to determine what values are included in the partially owned or controlled CL.

Moving to 8.4, the logic and/or features of NDP522-1may read data from HBM530-1and based on the portion of the CL indicated as not being owned or controlled by CHA514-1determine what values in the data stored to the shared memory region of HBM530-1are to be updated. In some examples, the full CL may be returned due to the speculative request sent to HBM530-1at 8.2 and the logic and/or features of NDP522-1only read the portion of the CL not owned or controlled by CHA514-1.

Moving to 8.5, the logic and/or features of NDP522-1may place a request to HBM530-1for the remaining portion of the CL not owned or controlled by CHA514-1. According to some examples, the request to HBM530-1is needed due to the lack of speculation. Store may be complete for process800at this point with and without speculation.

Moving to 8.6, the logic and/or features of NDP522-1may merge the data in the partially owned or controlled CL with the new values and then send the merged data to be stored in the shared memory region of HBM530-1. As a result, store is complete if CL is not owned or controlled by CHA514-1, with speculation. In some examples, the logic and/or features of NDP522-1may send an indication to CHA514-1that the data in the partially owned or controlled CL is out-of-date or invalid by indicating new values to be stored to the shared memory region of HBM530-1that are associated with the partially owned or controlled CL. Process700then comes to an end for a lazy coherence with speculation.

Moving to 8.7, the logic and/or features of NDP522-1may receive a response from HBM530-1that enable NDP522-1to read from HBM530-1. In some examples, the logic and/or features of NDP522-1may read the portion of the CL indicated as not being owned or controlled by CHA514-1to determine what values in the data stored to the shared memory region of HBM530-1are to be updated.

Moving to 8.8, the logic and/or features of NDP522-1merge the portion of the CL indicated as being owned or controlled by CHA514-1with the new values and then send the merged data to be stored in the shared memory region of HBM530-1. Store may be complete for process800if CL is not owned or controlled by CHA514-1, without speculation.

In some examples, as shown inFIG.8, the speculation latency advantage is shown to indicate that some latency advantage may be gained when a lazy coherence with specification is implemented and CHA514-1only partially owns or controls the cache line to the data maintained in the share memory region of HBM530-1. In other words, speculation that CHA514-1would not fully own or control the cache line and simultaneously submitting a read request to HBM530-1helps to mitigate possible latency hits associated with maintaining coherency for data accessed from a shared memory region of HBM530-1.

FIG.9illustrates an example logic flow900. In some examples, logic flow900may illustrate a logic flow to configure and manage an NDP such as NDP522-1shown inFIG.5. For these examples, logic flow may be implemented by elements of an operating system (OS) executed by one or more elements of a processor such as cores of processor501shown inFIG.5. The elements of the OS may include, for example, a device driver. Examples are not limited to an OS executed by processor501.

According to some examples, logic flow900at block910indicates a system boot. For example, system500shown inFIG.5may be booted or powered up.

In some examples, for logic flow900at block920an NDP such as NDP522-1may be discoverable by a device driver for an OS such as an OS executed by one or more elements of processor501. Discovering NDP522-1may include recognizing that NDP522-1is an accelerator resource of memory controller520-1to facilitate near data processing of data primary stored to HBM530-1.

According to some examples, logic flow900at block930may configure NDP522-1for operation. For these examples, configuration of NDP522-1may include the device driver registering NDP522-1as an agent communicatively coupled to ODI505that enables NDP522-1to communicate with CHAs for cores of processor501as well as communicate with other memory controllers such as memory controller520-2.

In some examples, logic flow900at block940may establish application interface (APIs) to allow applications or parallel language runtime libraries (e.g., OpenMP) to create and manage context/threads and/or request work to be scheduled to NDP522-1. For these examples, the device driver may enable the APIs for NDP522-1.

According to some examples, logic flow900a block950may manage/schedule work for NDP522-1. For these examples, the device driver may identify applications, kernels or loop routines that may have data access characteristics that may be the best candidates for offloading work to NDP522-1(e.g., low level of spatial or temporal locality for data maintained in HBM530-1). In some examples, cores of processor501may submit work to NDP522-1based on requested work placed by applications or parallel language runtime libraries via the established APIs. For these examples, the cores may submit work to NDP522-1by writing into a request queue. Results or responses generated by NDP522-1may be written to a response queue.

In some examples, a low latency interface path may be used for critical tasks (e.g., needed to meet performance requirements) such as offloading a kernel to NDP522-1. A higher latency interface path may be used for tasks that are not critical tasks (e.g., little or no relation to meeting at least some performance requirements). For these examples, a device driver may determine which interface path to use in order to maximize performance when a high level of performance is needed for an offloaded application, kernel, or loop routine.

FIG.10illustrates an example apparatus1000. Although apparatus1000shown inFIG.10has a limited number of elements in a certain topology, it may be appreciated that the apparatus1000may include more or less elements in alternate topologies as desired for a given implementation.

According to some examples, apparatus1000may be supported by circuitry1020and apparatus1010may be a near data processor included in a memory controller of a processor. For example, near data processors122-1to122-N included in respective memory controllers120-1to120-N of processor101as shown inFIG.1or near data processors522-1and522-1included in respective memory controllers520-1and520-2of processor501. Circuitry1020may be arranged to execute one or more software or firmware implemented logic, components, or modules1022-a(e.g., implemented, at least in part, by a controller of a memory device). It is worthy to note that “a” and “b” and “c” and similar designators as used herein are intended to be variables representing any positive integer. Thus, for example, if an implementation sets a value for a=5, then a complete set of software or firmware for logic, components or modules1022-amay include logic1022-1,1022-2,1022-3,1022-4or1022-5. Also, at least a portion of “logic” may be software/firmware stored in computer-readable media, or may be implemented, at least in part in hardware and although the logic is shown inFIG.10as discrete boxes, this does not limit logic to storage in distinct computer-readable media components (e.g., a separate memory, etc.) or implementation by distinct hardware components (e.g., separate application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs)).

According to some examples, circuitry1020may include at least a portion of one or more ASICs or programmable logic (e.g., FPGA) and, in some examples, at least some logic1022-aor processing resources (e.g., PEs) may be implemented as hardware elements of these ASICs or programmable logic.

In some examples, apparatus1000may include a work request logic1022-1. Work request logic1022-1may be a logic and/or feature executed by circuitry1020to receive a work request to execute a kernel, an application, or a loop routine using data maintained in a memory region of a first memory device coupled with apparatus1000via at least one memory channel. For these examples, the work request may be included in work request1005and the requester may be a core from among a plurality of cores. The core coupled with the memory controller that includes apparatus1000via an on-die interconnect.

According to some examples, apparatus1000may also include an access logic1022-2. Access logic1022-2may be a logic and/or feature executed by circuitry1020to access the data maintained in the memory region. For these examples, the access request may be included in access request1010and the data accessed may be included in access response1015.

In some examples, apparatus1000may also include an execute logic1022-3. Execute logic1022-3may be a logic and/or feature executed by circuitry1020to generate values responsive to execution of the kernel, the application, or the loop routine using the data included in access response1015.

According to some examples, the memory controller that includes apparatus1000may also couple with a second memory controller of the processor via the same on-die interconnect that couples the memory controller to the core as mentioned above. For these examples, the second memory controller may control access to a second memory device. Access logic1022-2may be arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller. Execute logic1022-3may execute at least a portion of the kernel, the application, or the loop routine using the accessed data maintained in the second memory device to generate values.

According to some examples, apparatus1000may also include indicate logic1022-4. Indicate logic1022-4may be a logic and/or feature executed by circuitry1020to indicate to a requester of the work request that the values have been generated. As mentioned above, the requestor may be a core coupled with the memory controller that includes apparatus1000via an on-die interconnect. Indicate logic1022-4may cause an indication to be provided the core via value indication1025.

In some examples, apparatus1000may also include a coherency logic1022-5. Coherency logic1022-5may be a logic and/or feature executed by circuitry1020to send coherency requests to a CHA of the core of the processor that placed the work request. For these examples, the CHA manages a cache hierarchy for the core based on a shared memory region of the first memory device via which data used to execute the kernel, the application, or the loop routine are obtained and/or result values generated by execute logic1022-3are stored. Coherency logic1022-5may send a coherency request1030to CHA of the core to determine whether the core has a cache hierarchy that includes data obtained from the shared memory region (e.g., via a cache line) where the result values are to be stored and/or the data used to execute the kernel, the application, or the loop routine is to be accessed. Concurrently, access logic1022-2may send access request1010to the first memory device to access the data maintained in the memory region that corresponds to the cache line. Coherency logic1022-5may receive, responsive to coherency request1010, an indication that the data is not included in the cache hierarchy for the core via coherency response1035. Access logic1022-2may then receive an access response1015from the first memory device that includes the data maintained in the shared memory region. Execute logic1022-3may then execute the kernel, the application, or the loop routine using the data included in access response1015. In an alternative example, coherency logic1022-5may have received a coherency response1035that indicates that the data is included in the cache for the core. For this alternative example, coherency response1035may include the data or allow access logic1022-2to obtain the data from the cache hierarchy for the core. Execute logic1022-3may then execute the kernel, the application, or the loop routine using the data obtained from the cache for the core. Access response1015from the first memory device, for this alternative example, may be discarded or ignored.

FIG.11illustrates an example of a logic flow1100. Logic flow1100may be representative of some or all of the operations executed by one or more logic, features, or devices described herein, such as logic and/or features included in apparatus800. More particularly, logic flow1000may be implemented by one or more of work request logic1022-1, access logic1022-2, execute logic1022-3or indicate logic1022-4.

According to some examples, as shown inFIG.11, logic flow1100at block1102may receive, at a near data processor of a memory controller of a processor, a work request to execute a kernel, an application, or a loop routine using data maintained in a memory region of a first memory device coupled with the near data processor via at least one memory channel. For these examples, work request logic1022-1may receive the work request.

In some examples, logic flow1100at block1104may access the data maintained in the memory region to generate values responsive to execution of the kernel, the application, or the loop routine. For these examples, access logic1022-2may access the data and execute logic1022-3may use the data to generate the values.

According to some examples, logic flow1100at block1106may indicate to a requester of the work request that the values have been generated. For these examples, indicate logic1022-4may make the indication to the requester.

The set of logic flows shown inFIGS.9and11may be representative of example methodologies for performing novel aspects described in this disclosure. While, for purposes of simplicity of explanation, the one or more methodologies shown herein are shown and described as a series of acts, those skilled in the art will understand and appreciate that the methodologies are not limited by the order of acts. Some acts may, in accordance therewith, occur in a different order and/or concurrently with other acts from that shown and described herein. For example, those skilled in the art will understand and appreciate that a methodology could alternatively be represented as a series of interrelated states or events, such as in a state diagram. Moreover, not all acts illustrated in a methodology may be required for a novel implementation.

A logic flow may be implemented in software, firmware, and/or hardware. In software and firmware embodiments, a logic flow may be implemented by computer executable instructions stored on at least one non-transitory computer readable medium or machine readable medium, such as an optical, magnetic or semiconductor storage. The embodiments are not limited in this context.

FIG.12illustrates an example of a first storage medium. As shown inFIG.12, the first storage medium includes a storage medium1200. The storage medium1200may comprise an article of manufacture. In some examples, storage medium1200may include any non-transitory computer readable medium or machine readable medium, such as an optical, magnetic or semiconductor storage. Storage medium1200may store various types of computer executable instructions, such as instructions to implement logic flow1100. Examples of a computer readable or machine readable storage medium may include any tangible media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and so forth. Examples of computer executable instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like. The examples are not limited in this context.

FIG.13illustrates an example computing platform1300. In some examples, as shown inFIG.13, computing platform1300may include a memory system1330, a processing component1340, other platform components1350or a communications interface1360. According to some examples, computing platform1300may be implemented in a computing device.

According to some examples, memory system1330may include a controller1332and a memory1334. For these examples, circuitry resident at or located at controller1332may be included in a near data processor and may execute at least some processing operations or logic for apparatus1000based on instructions included in a storage media that includes storage medium1200. Also, memory1334may include similar types of memory that are described above for system100shown inFIG.1. For example, types of memory included in memory130-1to130-N shown inFIG.1.

According to some examples, processing components1340may include various hardware elements, software elements, or a combination of both. Examples of hardware elements may include devices, logic devices, components, processors, microprocessors, management controllers, companion dice, circuits, processor circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, ASICs, programmable logic devices (PLDs), digital signal processors (DSPs), FPGAs, memory units, logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. Examples of software elements may include software components, programs, applications, computer programs, application programs, device drivers, system programs, software development programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an example is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints, as desired for a given example.

According to some examples, processing component1340may include and infrastructure processing unit (IPU) or data processing unit (DPU) or may be utilized by an IPU or DPU. An xPU may refer at least to an IPU, DPU, graphic processing unit (GPU), general-purpose GPU (GPGPU). An IPU or DPU may include a network interface with one or more programmable or fixed function processors to perform offload of operations that could have been performed by a CPU. The IPU or DPU can include one or more memory devices (not shown). In some examples, the IPU or DPU can perform virtual switch operations, manage storage transactions (e.g., compression, cryptography, virtualization), and manage operations performed on other IPUs, DPUs, servers, or devices.

In some examples, other platform components1350may include common computing elements, memory units (that include system memory), chipsets, controllers, peripherals, interfaces, oscillators, timing devices, video cards, audio cards, multimedia input/output (I/O) components (e.g., digital displays), power supplies, and so forth. Examples of memory units or memory devices included in other platform components1350may include without limitation various types of computer readable and machine readable storage media in the form of one or more higher speed memory units, such as read-only memory (ROM), random-access memory (RAM), dynamic RAM (DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), static RAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, polymer memory such as ferroelectric polymer memory, ovonic memory, phase change or ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS) memory, magnetic or optical cards, an array of devices such as Redundant Array of Independent Disks (RAID) drives, solid state memory devices (e.g., USB memory), solid state drives (SSD) and any other type of storage media suitable for storing information.

In some examples, communications interface1360may include logic and/or features to support a communication interface. For these examples, communications interface1360may include one or more communication interfaces that operate according to various communication protocols or standards to communicate over direct or network communication links. Direct communications may occur via use of communication protocols or standards described in one or more industry standards (including progenies and variants) such as those associated with the PCIe specification, the NVMe specification or the I3C specification. Network communications may occur via use of communication protocols or standards such those described in one or more Ethernet standards promulgated by the Institute of Electrical and Electronics Engineers (IEEE). For example, one such Ethernet standard promulgated by IEEE may include, but is not limited to, IEEE 802.3-2018, Carrier sense Multiple access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications, Published in August 2018 (hereinafter “IEEE 802.3 specification”). Network communication may also occur according to one or more OpenFlow specifications such as the OpenFlow Hardware Abstraction API Specification. Network communications may also occur according to one or more Infiniband Architecture specifications.

Computing platform1300may be part of a computing device that may be, for example, user equipment, a computer, a personal computer (PC), a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet, a smart phone, embedded electronics, a gaming console, a server, a server array or server farm, a web server, a network server, an Internet server, a work station, a mini-computer, a main frame computer, a supercomputer, a network appliance, a web appliance, a distributed computing system, multiprocessor systems, processor-based systems, or combination thereof. Accordingly, functions and/or specific configurations of computing platform1300described herein, may be included or omitted in various embodiments of computing platform1300, as suitably desired.

The components and features of computing platform1300may be implemented using any combination of discrete circuitry, ASICs, logic gates and/or single chip architectures. Further, the features of computing platform1300may be implemented using microcontrollers, programmable logic arrays and/or microprocessors or any combination of the foregoing where suitably appropriate. It is noted that hardware, firmware and/or software elements may be collectively or individually referred to herein as “logic”, “circuit” or “circuitry.”

It should be appreciated that the exemplary computing platform1300shown in the block diagram ofFIG.13may represent one functionally descriptive example of many potential implementations. Accordingly, division, omission or inclusion of block functions depicted in the accompanying figures does not infer that the hardware components, circuits, software and/or elements for implementing these functions would necessarily be divided, omitted, or included in embodiments.

Although not depicted, any system can include and use a power supply such as but not limited to a battery, AC-DC converter at least to receive alternating current and supply direct current, renewable energy source (e.g., solar power or motion based power), or the like.

One or more aspects of at least one example may be implemented by representative instructions stored on at least one machine-readable medium which represents various logic within the processor, which when read by a machine, computing device or system causes the machine, computing device or system to fabricate logic to perform the techniques described herein. Such representations may be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.

Various examples may be implemented using hardware elements, software elements, or a combination of both. In some examples, hardware elements may include devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, ASICs, PLDs, DSPs, FPGAs, memory units, logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. In some examples, software elements may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, APIs, instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an example is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints, as desired for a given implementation.

Some examples may include an article of manufacture or at least one computer-readable medium. A computer-readable medium may include a non-transitory storage medium to store logic. In some examples, the non-transitory storage medium may include one or more types of computer-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and so forth. In some examples, the logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, API, instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

According to some examples, a computer-readable medium may include a non-transitory storage medium to store or maintain instructions that when executed by a machine, computing device or system, cause the machine, computing device or system to perform methods and/or operations in accordance with the described examples. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. The instructions may be implemented according to a predefined computer language, manner or syntax, for instructing a machine, computing device or system to perform a certain function. The instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.

Some examples may be described using the expression “in one example” or “an example” along with their derivatives. These terms mean that a particular feature, structure, or characteristic described in connection with the example is included in at least one example. The appearances of the phrase “in one example” in various places in the specification are not necessarily all referring to the same example.

Some examples may be described using the expression “coupled” and “connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, descriptions using the terms “connected” and/or “coupled” may indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

The following examples pertain to additional examples of technologies disclosed herein.

Example 1. An example apparatus may include a memory controller of a processor to couple with multiple cores of the processor via an on-die interconnect. The memory controller may control access to a first memory device and may also include a near data processor. The near data processor may include circuitry to execute a kernel, an application, or a loop routine using data maintained in the first memory device. The data maintained in the first memory device may be directly accessible to the near data processor via at least one memory channel coupled with the first memory device. The near data processor may also include a plurality of memory buffers arranged to receive a work request from a core from among the multiple cores and also arranged to indicate that values have been generated by the circuitry responsive to the work request.

Example 2. The apparatus of example 1 may also include the memory controller to couple with a second memory controller of the processor via the on-die interconnect. The second memory controller may control access to a second memory device. For this example, the near data processor may be arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller. The circuitry to execute at least a portion of the kernel, the application, or the loop routine may use the accessed data maintained in the second memory device.

Example 3. The apparatus of example 1, the data maintained in the first memory device may include the data being maintained in a memory region of the first memory device shared with the core from among the multiple cores. For this example, the circuitry may further send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core. The circuitry may also concurrently send an access request to the first memory device to access the data maintained in the memory region. The circuitry may also receive, responsive to the coherency request, an indication that the data is not included in the cache for the core. The circuitry may also receive, responsive to the access request, the data maintained in the memory region and use the data maintained in the memory region to execute the kernel, the application, or the loop routine.

Example 4. The apparatus of example 1, the data maintained in the first memory device may include the data maintained in a memory region of the first memory device shared with the core from among the multiple cores. For this example, the circuitry may send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core. The circuitry may also concurrently send an access request to the first memory device to access the data maintained in the memory region. The circuitry may also receive, responsive to the coherency request, an indication that the data is included in the cache for the core and use the data included in the cache for the core to execute the kernel, the application, or the loop routine.

Example 5. The apparatus of example 1, the generated values may be stored to a memory region of the first memory device shared with the core from among the multiple cores. For this example, the circuitry may send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region. The circuitry may also receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes data from the memory region. The circuitry may also send an indication to the coherency agent that data included in the cache line is invalid and cause the values to be stored to the memory region.

Example 6. The apparatus of example 1, the generated values may be stored to a memory region of the first memory device shared with the core from among the multiple cores. For this example, the circuitry may send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region where the generated values are to be stored. The circuitry may also concurrently send an access request to the first memory device to access the data maintained in the memory region that corresponds to the cache line. The circuitry may also receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes only a portion of the data from the memory region. The circuitry may also receive, responsive to the access request, the data maintained in the memory region. The circuitry may also send an indication to the coherency agent that the cache line includes invalid data and cause the values to be stored to the memory region.

Example 7. The apparatus of example 1, the first memory device may include an HBM stack resident on a separate chip from the memory controller and the multiple cores of the processor. For this example, the HBM stack may include dynamic random access memory.

Example 8. The apparatus of example 7, the data may be directly accessible to the near data processor via the at least one memory channel coupled with the HBM stack. For this example, the at least one memory channel may be arranged to operate in compliance with a JEDEC specification to include HBM version 2, JESD235C.

Example 9. An example method may include receiving, at a near data processor of a memory controller of a processor, a work request to execute a kernel, an application, or a loop routine using data maintained in a memory region of a first memory device coupled with the near data processor via at least one memory channel. The method may also include accessing the data maintained in the memory region to generate values responsive to execution of the kernel, the application, or the loop routine. The method may also include indicating to a requester of the work request that the values have been generated.

Example 10. The method of example 9 may also include receiving the work request in a memory buffer of the near data processor. For this example, circuitry of the near data processor may be arranged to execute the kernel, the application or the loop routine using the data maintained in the memory region of the first memory device.

Example 11. The method of example 10, the requester of the work request may include a core of the processor, the core coupled with the memory controller via an on-die interconnect.

Example 12. The method of example 11, comprising the memory controller of the processor to couple with a second memory controller of the processor via the on-die interconnect. The second memory controller may control access to a second memory device. For this example, the near data processor may be arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller. Also, the circuitry may execute at least a portion of the kernel, the application, or the loop routine using the accessed data maintained in the second memory device.

Example 13. The method of example 11, the data maintained in the first memory device may include the data maintained in a memory region of the first memory device shared with the core of the processor. The method may further include the near data processor sending a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core. The method may also include the near data processor concurrently sending an access request to the first memory device to access the data maintained in the memory region. The method may also include the near data processor receiving, responsive to the coherency request, an indication that the data is not included in the cache for the core. The method may also include the near data processor receiving, responsive to the access request, the data maintained in the memory region; and using the data maintained in the memory region to execute the kernel, the application, or the loop routine.

Example 14. The method of example 11, the data maintained in the first memory device may include the data maintained in a memory region of the first memory device shared with the core of the processor. For this example, the method may further include the near data processor sending a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core. The method may also include the near data processor concurrently sending an access request to the first memory device to access the data maintained in the memory region. The method may also include the near data processor receiving, responsive to the coherency request, an indication that the data is included in the cache for the core and using the data included in the cache for the core to execute the kernel, the application, or the loop routine.

Example 15. The method of example 11, the values to be stored to a memory region of the first memory device may be shared with the core of the processor. For this example, the method may further include the near data processor sending a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region. The method may also include the near data processor receiving, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes data from the memory region. The method may also include the near data processor sending an indication to the coherency agent that data included in the cache line is invalid and causing the values to be stored to the memory region.

Example 16. The method of example 11, the values to be stored to a memory region of the first memory device may be shared with the core of the processor. For this example, the method may further include the near data processor sending a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region where the values are to be stored. The method may also include the near data processor concurrently sending an access request to the first memory device to access the data maintained in the memory region that corresponds to the cache line. The method may also include the near data processor receiving, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes only a portion of the data from the memory region. The method may also include the near data processor receiving, responsive to the access request, the data maintained in the memory region. The method may also include the near data processor sending an indication to the coherency agent that the cache line includes invalid data and causing the values to be stored to the memory region.

Example 17. The method of example 11, the first memory device may include an HBM stack resident on a separate chip from the memory controller and the core of the processor. For this example, the HBM stack includes dynamic random access memory.

Example 18. The method of example 17, the data may be directly accessible to the near data processor via the at least one memory channel coupled with the HBM stack. For this example, the at least one memory channel may be arranged to operate in compliance with a JEDEC specification to include HBM version 2, JESD235C.

Example 19. An example at least one machine readable medium may include a plurality of instructions that in response to being executed by a system may cause the system to carry out a method according to any one of examples 9 to 18.

Example 20. An example apparatus may include means for performing the methods of any one of examples 9 to 18.

Example 21. An example system may include a first memory device, a plurality of cores of a processor and a first memory controller of the processor to couple with the plurality of cores via an on-die interconnect. The first memory controller may control access to the first memory device. The first memory controller may include a near data processor. The near data processor may include circuitry to execute a kernel, an application, or a loop routine using data maintained in the first memory device, the data directly accessible to the near data processor via at least one memory channel coupled with the first memory device. The near data processor may also include a plurality of memory buffers arranged to receive a work request from a core from among the plurality cores and arranged to indicate that values have been generated by the circuitry responsive to the work request.

Example 22. The system of example 21 may also include a second memory device and a second memory controller of the processor to couple with the plurality of cores and the first memory controller via the on-die interconnect. The second memory controller may control access to the second memory device. For this example, the near data processor may be arranged to serve as an agent on the on-die interconnect to access data maintained in the second memory device through the second memory controller. The circuitry to execute at least a portion of the kernel, the application, or the loop routine using the accessed data maintained in the second memory device.

Example 23. The system of example 21, the data maintained in the first memory device may include the data maintained in a memory region of the first memory device shared with the core from the plurality of cores. For this example, the circuitry of the near data processor may also send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core. The circuitry may also concurrently send an access request to the first memory device to access the data maintained in the memory region. The circuitry may also receive, responsive to the coherency request, an indication that the data is not included in the cache for the core. The circuitry may also receive, responsive to the access request, the data maintained in the memory region and use the data maintained in the memory region to execute the kernel, the application, or the loop routine.

Example 24. The system of example 21, the data maintained in the first memory device includes the data maintained in a memory region of the first memory device may be shared with the core from the plurality of cores. For this example, the circuitry of the near data processor may also send a coherency request to a coherency agent of the core to determine whether the data for use to execute the kernel, the application, or the loop routine is included in a cache for the core. The circuitry may also concurrently send an access request to the first memory device to access the data maintained in the memory region. The circuitry may also receive, responsive to the coherency request, an indication that the data is included in the cache for the core and use the data included in the cache for the core to execute the kernel, the application, or the loop routine.

Example 25. The system of example 21, the generated values to be stored to a memory region of the first memory device may be shared with the core from the plurality of cores. For this example, the circuitry of the near data processor may also send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region. The circuitry may also receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes data from the memory region. The circuitry may also send an indication to the coherency agent that data included in the cache line is invalid and cause the values to be stored to the memory region.

Example 26. The system of example 21, the generated values may be stored to a memory region of the first memory device shared with the core from the plurality of cores. For this example, circuitry of the near data processor may also send a coherency request to a coherency agent of the core to determine whether the coherency agent controls a cache line that includes data from the memory region where the generated values are to be stored. The circuitry may also concurrently send an access request to the first memory device to access the data maintained in the memory region that corresponds to the cache line. The circuitry may also receive, responsive to the coherency request, an indication that the coherency agent controls a cache line that includes only a portion of the data from the memory region. The circuitry may also receive, responsive to the access request, the data maintained in the memory region. The circuitry may also send an indication to the coherency agent that the cache line includes invalid data and cause the values to be stored to the memory region.

Example 27. The system of example 21, the first memory device may include an HBM stack resident on a separate chip from the first memory controller and the plurality of cores of the processor, wherein the HBM stack includes dynamic random access memory.

Example 28. The system of example 27, the data may be directly accessible to the near data processor via the at least one memory channel coupled with the HBM stack. For this example, the at least one memory channel may be arranged to operate in compliance with a JEDEC specification to include HBM version 2, JESD235C.

It is emphasized that the Abstract of the Disclosure is provided to comply with 37 C.F.R. Section 1.72(b), requiring an abstract that will allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in a single example for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed examples require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed example. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate example. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein,” respectively. Moreover, the terms “first,” “second,” “third,” and so forth, are used merely as labels, and are not intended to impose numerical requirements on their objects.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.