Patent Description:
In computing environments that have multiple processor chips on the same drawers and/or processor chips in other drawers sharing one or more caches, those processor chips may have access to the same cache lines. A cache line or line is a portion of data of a specific size (e.g., <NUM> bytes), which fits into a single cache entry in a cache. Coherency is managed on a cache line granularity as data is transferred between memory and a cache. To ensure coherency of a cache line, snoop requests (also referred as snoops or fetch requests) are used. Snoop requests are processed by receiving a snoop request from a requesting cache, determining if this cache has a copy of the cache line in an appropriate state, sending the cache line to the requesting cache, and updating the state of the cache line in this cache.

A cache structure and its related services provide processor chips with data consistency of their shared data. Many processor chips on the same drawer and/or processor chips on different drawers may attempt to access the same cache line. Techniques are needed to improve access to a cache line of a shared cache.

<CIT>, <NPL>) discloses a distributed shared-memory system includes several nodes that each have one or more processor cores, caches, local main memory, and a directory. Each node further includes predictors that use historical memory access information to predict future coherence permission requirements and speculatively initiate coherence operations. In one embodiment, predictors are included at processor cores for monitoring a memory access stream (e.g., historical sequence of memory addresses referenced by a processor core) and predicting addresses of future accesses. In another embodiment, predictors are included at the directory of each node for monitoring memory access traffic and coherence-related activities for individual cache lines to predict future demands for particular cache lines. In other embodiments, predictors are included at both the processor cores and directory of each node. Predictions from the predictors are used to initiate coherence operations to speculatively request promotion or demotion of coherence permissions.

According to one aspect, there is provided a computer-implemented method comprising: requesting authority for a cache line in conjunction with querying for outstanding requests for the cache line; receiving one or more responses regarding the outstanding requests for the cache line; and in response to receiving the one or more responses regarding the outstanding requests and in advance of receiving the authority for the cache line, preemptively tracking the outstanding requests in a requested structure associated with the cache line.

According to another aspect, there is provided a computer program comprising program code means adapted to perform the method described above, when said program is executed on a computer.

According to another aspect, there is provided a system comprising: a shared cache; and a controller coupled to the shared cache, the controller configured to: request authority for a cache line in conjunction with querying for outstanding requests for the cache line; receive one or more responses regarding the outstanding requests for the cache line; and in response to receiving the one or more responses regarding the outstanding requests and in advance of receiving the authority for the cache line, preemptively track the outstanding requests in a requested structure associated with the cache line.

According to another aspect, there is provided a controller comprising logic, the logic executable to perform operations comprising: requesting authority for a cache line in conjunction with querying for outstanding requests for the cache line; receiving one or more responses regarding the outstanding requests for the cache line; and in response to receiving the one or more responses regarding the outstanding requests and in advance of receiving the authority for the cache line, preemptively tracking the outstanding requests in a requested structure associated with the cache line.

Embodiments of the present invention are directed to computer-implemented methods for providing preemptive tracking of remote requests for decentralized hot cache line fairness tracking. A non-limiting computer-implemented method includes requesting authority for a cache line in conjunction with querying for outstanding requests for the cache line and receiving one or more responses regarding the outstanding requests for the cache line. Also, the method includes, in response to receiving the one or more responses regarding the outstanding requests and in advance of receiving the authority for the cache line, preemptively tracking the outstanding requests in a requested structure associated with the cache line.

This can provide an improvement over known methods for processing cache requests by efficiently providing a fairness tracking mechanism to ensure that all requestors who need access to the cache line will gain access to the line in a timely and fair manner. One or more embodiments of the invention overcome the scalability issues of prior implementations by using a decentralized, hierarchical hot cache line fairness mechanism. One or more embodiments of the invention preemptively build the requested structure in advance of obtaining access of the cache line, to thereby provide an efficient technique of maintaining a complete picture of who is requesting access to the hot cache line, resulting in fairness to all requestors.

In addition to one or more of the features described above or below, or as an alternative, in further embodiments of the invention the method includes, in response to receiving the authority for the cache line along with receiving access to the cache line, receiving transfer of a serviced structure associated with the cache line and keeping bits set in the requested structure, wherein the bits were set in advance of receiving the authority. One or more embodiments advantageously use vectors that are allocated, deallocated, and associated with each cache line, and one or more of these vectors are passed around the system with the data of each cache line, eliminating the need for a large centralized physical space in the design. In addition, these vectors are structured hierarchically, reducing the storage space required.

In addition to one or more of the features described above or below, or as an alternative, in further embodiments of the invention the method includes, in response to not receiving the authority for the cache line, resetting the requested structure. One or more embodiments advantageously use vectors that are allocated, deallocated, and associated with each cache line, and one or more of these vectors are passed around the system with the data of each cache line, eliminating the need for a large centralized physical space in the design. In addition, these vectors are structured hierarchically, reducing the storage space required.

In addition to one or more of the features described above or below, or as an alternative, in further embodiments of the invention a serviced structure is received from a controller previously having the authority, the serviced structure being configured to track requests to access the cache line that were observed and have been granted access to the cache line. One or more embodiments advantageously provide vectors working concurrently to form encoded states of where the hot cache line has been requested and where it has already been granted access, thereby eliminating the need for a large centralized physical space in the design. A requestor who re-requests the same cache line (has already been granted access to the hot cache line once before) cannot receive the line again if there are other requestors who previously requested the cache line and have not been granted access yet.

In addition to one or more of the features described above or below, or as an alternative, in further embodiments of the invention the requested structure is built in anticipation of receiving the authority for a cache line. One or more embodiments of the invention preemptively build the requested structure in advance of obtaining access of the cache line, to thereby provide an efficient technique of maintaining a complete picture of who is requesting access to the hot cache line, resulting in fairness to all requestors. One or more embodiments of the invention overcome the scalability issues of prior implementations by using a decentralized, hierarchical hot cache line fairness mechanism.

In addition to one or more of the features described above or below, or as an alternative, in further embodiments of the invention the one or more responses regarding the outstanding requests correspond to an exclusive fetch request, or a fetch, if the authority of the snooped cache line is one authority level less than the authority of the requestor (i.e., the scope of the broadcast). One or more embodiments of the invention preemptively build the requested structure in advance of obtaining access of the cache line, to thereby provide an efficient technique of maintaining a complete picture of who is requesting access to the hot cache line, resulting in fairness to all requestors. One or more embodiments of the invention overcome the scalability issues of prior implementations by using a decentralized, hierarchical hot cache line fairness mechanism.

Other embodiments of the present invention implement features of the above-described methods in computer systems and computer program products.

One or more embodiments of the invention are configured to provide a decentralized hot cache line tracking fairness mechanism. One or more embodiments of the invention provide a technique for decentralized tracking of hot line (cache line) requests, using two vectors that are allocated, deallocated, and associated with each cache line. One vector is a requested vector, and the other vector is a serviced vector. The requested vector is a vector that tracks requests to access the cache line in which the requests were observed, but access has not been granted. The serviced vector is a vector that tracks requests to access the cache line in which the requests were observed and have been granted access to the cache line. Per cache line, the requested vectors and the serviced vectors are passed around the system with the data of each cache line, eliminating the need for a large centralized physical space in the design. In addition, the requested and serviced vectors are structured hierarchically, reducing the storage space required. The two vectors, requested vector and serviced vector, work concurrently to form encoded states indicating where the hot cache line has been requested and where (e.g., to whom) the hot cache line has already been granted access. A requestor who re-requests the same cache line (i.e., has already been granted access to the hot cache line once before) cannot receive the cache line again if there are other requestors who have previously requested the cache line and have not been granted access yet.

Further, one or more embodiments of the invention provide preemptive tracking of remote requests for decentralized hot cache line fairness tracking. A reverse compare decentralized hot cache line fairness tracking mechanism is disclosed which provides a technique that efficiently and preemptively gathers a more complete picture of which other requestors are requesting a hot cache line. Rather than waiting for snoops (which are requests) to arrive at the requestor after the requestor has gotten authority, the requestor is configured to preemptively query the remote caches (i.e., other requestors) at the same time as the requestor transmits its broadcast to gain authority, and requestor is configured to speculatively collect information on remote requestors that are fetching the cache line. If the authority is not achieved by the requestor, the requestor is configured to drop the speculative information gained from preemptively querying the other requestors (i.e., remote caches). This technique provides a more complete picture of which remote requestors are looking for the cache line, in order to provide efficiency and fairness to all requestors.

A hot line or hot cache line is a cache line that numerous operations, as requestors, are trying to access to observe and/or to modify. Cache lines can only be accessed by one operation a time. When there are multiple requests to access the cache line, there is a backlog of operations that are waiting to access the cache line as another operation is working on the cache line. Unsuccessful requestors, those who have requested the cache line and did not get access to the cache line, are rejected back to origin and must start the requesting process over again. Meanwhile, requestors who already accessed the cache line could request for the line again and their requests could arrive before the unsuccessful requestors come back asking for (i.e., requesting) the cache line. This could cause unsuccessful requestors to get starved or blocked out because other requestors continue getting access to the cache line repeatedly, and the unsuccessful requestors wait too long to get the cache line. This type of behavior negatively impacts performance and response times because the operation that requested the cache line is then stalled for long periods of time and forward progress is not made.

Accordingly, one or more embodiments of the invention provide a fairness tracking mechanism to ensure that all requestors who need access to the cache line will gain access to the cache line in a timely and fair manner. As technical solutions and benefits, one or more embodiments use the requested vectors and serviced vectors to improve performance and response times of cache requests from requestors by ensuring that forward progress is being made.

For the sake of brevity, conventional techniques related to semiconductor device and IC fabrication may or may not be described in detail herein. Moreover, the various tasks and process steps described herein can be incorporated into a more comprehensive procedure or process having additional steps or functionality not described in detail herein. In particular, various steps in the manufacture of semiconductor devices and semiconductor-based ICs are well known and so, in the interest of brevity, many conventional steps will only be mentioned briefly herein or will be omitted entirely without providing the well-known process details.

Turning now to <FIG>, a computer system <NUM> is generally shown in accordance with one or more embodiments of the invention. The computer system <NUM> can be an electronic, computer framework comprising and/or employing any number and combination of computing devices and networks utilizing various communication technologies, as described herein. The computer system <NUM> can be easily scalable, extensible, and modular, with the ability to change to different services or reconfigure some features independently of others. The computer system <NUM> may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer system <NUM> may be a cloud computing node. Computer system <NUM> may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system <NUM> may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

As shown in <FIG>, the computer system <NUM> has one or more central processing units (CPU(s)) 101a, 101b, 101c, etc., (collectively or generically referred to as processor(s) <NUM>). The processors <NUM> can be a single-core processor, multi-core processor, computing cluster, or any number of other configurations. The processors <NUM>, also referred to as processing circuits, are coupled via a system bus <NUM> to a system memory <NUM> and various other components. The system memory <NUM> can include a read only memory (ROM) <NUM> and a random access memory (RAM) <NUM>. The ROM <NUM> is coupled to the system bus <NUM> and may include a basic input/output system (BIOS) or its successors like Unified Extensible Firmware Interface (UEFI), which controls certain basic functions of the computer system <NUM>. The RAM is read-write memory coupled to the system bus <NUM> for use by the processors <NUM>. The system memory <NUM> provides temporary memory space for operations of said instructions during operation. The system memory <NUM> can include random access memory (RAM), read only memory, flash memory, or any other suitable memory systems.

The computer system <NUM> comprises an input/output (I/O) adapter <NUM> and a communications adapter <NUM> coupled to the system bus <NUM>. The I/O adapter <NUM> may be a small computer system interface (SCSI) adapter that communicates with a hard disk <NUM> and/or any other similar component. The I/O adapter <NUM> and the hard disk <NUM> are collectively referred to herein as a mass storage <NUM>.

Software <NUM> for execution on the computer system <NUM> may be stored in the mass storage <NUM>. The mass storage <NUM> is an example of a tangible storage medium readable by the processors <NUM>, where the software <NUM> is stored as instructions for execution by the processors <NUM> to cause the computer system <NUM> to operate, such as is described herein below with respect to the various Figures. Examples of computer program product and the execution of such instruction is discussed herein in more detail. The communications adapter <NUM> interconnects the system bus <NUM> with a network <NUM>, which may be an outside network, enabling the computer system <NUM> to communicate with other such systems. In one embodiment, a portion of the system memory <NUM> and the mass storage <NUM> collectively store an operating system, which may be any appropriate operating system to coordinate the functions of the various components shown in <FIG>.

Additional input/output devices are shown as connected to the system bus <NUM> via a display adapter <NUM> and an interface adapter <NUM>. In one embodiment, the adapters <NUM>, <NUM>, <NUM>, and <NUM> may be connected to one or more I/O buses that are connected to the system bus <NUM> via an intermediate bus bridge (not shown). A display <NUM> (e.g., a screen or a display monitor) is connected to the system bus <NUM> by the display adapter <NUM>, which may include a graphics controller to improve the performance of graphics intensive applications and a video controller. A keyboard <NUM>, a mouse <NUM>, a speaker <NUM>, etc., can be interconnected to the system bus <NUM> via the interface adapter <NUM>, which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit. Suitable I/O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI) and the Peripheral Component Interconnect Express (PCle). Thus, as configured in <FIG>, the computer system <NUM> includes processing capability in the form of the processors <NUM>, and, storage capability including the system memory <NUM> and the mass storage <NUM>, input means such as the keyboard <NUM> and the mouse <NUM>, and output capability including the speaker <NUM> and the display <NUM>.

In some embodiments, the communications adapter <NUM> can transmit data using any suitable interface or protocol, such as the internet small computer system interface, among others. The network <NUM> may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. An external computing device may connect to the computer system <NUM> through the network <NUM>. In some examples, an external computing device may be an external webserver or a cloud computing node.

It is to be understood that the block diagram of <FIG> is not intended to indicate that the computer system <NUM> is to include all of the components shown in <FIG>. Rather, the computer system <NUM> can include any appropriate fewer or additional components not illustrated in <FIG> (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Further, the embodiments described herein with respect to computer system <NUM> may be implemented with any appropriate logic, wherein the logic, as referred to herein, can include any suitable hardware (e.g., a processor, an embedded controller, or an application specific integrated circuit, among others), software (e.g., an application, among others), firmware, or any suitable combination of hardware, software, and firmware, in various embodiments.

<FIG> depicts a block diagram of an example system <NUM> configured to provide a decentralized hot cache line tracking fairness mechanism according to one or more embodiments of the invention. In system <NUM>, there can be many interconnected drawers <NUM>, such as drawer <NUM>, drawer <NUM>, and drawer <NUM>. Each of the drawers <NUM> includes processor chips <NUM>, such as processor chip <NUM>, processor chip <NUM>, and processor chip <NUM>. Each processor chip <NUM> is coupled to and/or includes shared caches <NUM> (e.g., level three (L3) caches), such as L30, L31, L32, and L33. Further details of an example shared cache <NUM> is depicted in <FIG>. Computer system <NUM> may be integrated with and/or use processor chips <NUM> in <FIG>. Many computer systems <NUM> and/or features of computer systems <NUM> may be integrated in system <NUM>. The system <NUM> may be representative of one or more portions of a cloud computing environment <NUM> depicted in <FIG>. One or more processors <NUM> may represent processor chips <NUM>. The processor chips <NUM> include all the standard processing circuitry and shared caches <NUM> include the standard memory and circuity as understood by one of ordinary skill in the art. Although not shown for conciseness, the processor chips can include higher-level caches, such as L2 caches and L1 caches, where the L1 cache is closest to the processor core as understood by one of ordinary skill in the art.

<FIG> depicts a block diagram of an example shared cache <NUM> (e.g., L30, L31, L32, L33) configured to provide a decentralized hot cache line tracking fairness mechanism according to one or more embodiments of the invention. Although details of a single shared cache <NUM> are illustrated in <FIG>, the discussion applies by analogy to the other shared caches <NUM>. Shared cache <NUM> can include and/or be coupled to various hardware controllers <NUM> in which the hardware controllers <NUM> are configured to control access to cache lines on the shared cache <NUM> such as for reading or writing, to send requests to access other cache lines on other shared caches <NUM>, and to implement a requested vector and serviced vector per cache line according to one or more embodiments. Particularly, each hardware controller <NUM> includes logic <NUM> configured to provide decentralized hot cache line tracking fairness using a requested vector and serviced vector. A hardware controller <NUM> may include the requested vector and serviced vector for a single cache line in memory <NUM>, and the requested vector and serviced vector for the cache line are passed along as metadata to the next hardware controller <NUM> along with the data in the cache line. As such, each cache line in a shared cache <NUM> has its own requested vector and serviced vector which are maintained and passed throughout the system <NUM> according to one or more embodiments.

Hardware controllers <NUM> can include the functionality of one or more known hardware controllers. Hardware controllers <NUM> can be and/or include functionality of local fetch address register controllers (LFARs), control store address register controllers (CSARs), central processor fetch controllers (CFARs), remote fetch controllers (RFARs), line local store address register controllers (L-LSARs), remote local store address register controllers (R-LSARs), remote store address controller (RSAR), and other hardware controllers understood by one of ordinary skill in the art. Logic <NUM> can include logic circuitry, firmware executable by circuits on hardware controller <NUM>, and/or a combination of logic circuitry and firmware. Moreover, logic <NUM> can be implemented as instructions stored on a computer-readable storage medium, as hardware modules, as special-purpose hardware (e.g., application specific hardware, application specific integrated circuits (ASICs), as embedded controllers, hardwired circuitry, etc.), or as some combination or combinations of these. In examples, logic <NUM> described herein can be a combination of hardware and programming. The programming can be processor executable instructions stored on a tangible memory, and the hardware can include processing circuitry for executing those instructions. Alternatively or additionally, logic <NUM> can include dedicated hardware, such as one or more integrated circuits, Application Specific Integrated Circuits (ASICs), Application Specific Special Processors (ASSPs), Field Programmable Gate Arrays (FPGAs), or any combination of the foregoing examples of dedicated hardware, for performing the techniques described herein. Logic <NUM> of hardware controllers <NUM> of the shared caches are configured to provide a decentralized hierarchical hot line fairness tracking mechanism. This mechanism is comparable to centralized fairness tracking in terms of performance, and it scales. The decentralized mechanism grows organically and does not need additional space for each address. As noted herein, the decentralized hierarchical fairness tracking mechanism uses a requested vector and a serviced vector, which are allocated, deallocated, and associated with a single cache line.

<FIG> depicts a block diagram illustrating an example requested vector and an example serviced vector used for each cache line in decentralized hierarchical hot line fairness tracking according to one or more embodiments. The requested vector and/or serviced vector are passed around the system <NUM> along with and at the same time as passing the data in a cache line, thereby eliminating the need for a large centralized physical space in the design. The requested vector and the serviced vector are built by the hardware controllers <NUM>. In one or more embodiments, a hardware controller <NUM> may pass the serviced vector but not pass the requested vector to the next hardware controller <NUM> that is receiving the cache line, because the next hardware controller <NUM> preemptively built the requested vector in advance of receiving access/authority to the cache line. Each cache line has its own requested vector and serviced vector. In addition, the requested and serviced vectors are structured hierarchically, reducing the storage space required as compared to a centralized queue. The serviced vector is a vector that tracks requests to access the cache line that were observed and have been granted access to the cache line. The requested vector is a vector that tracks requests to access the cache line that were observed, but access has not been granted. The requested and serviced vectors work concurrently to form encoded states of where the hot cache line has been requested and where it has already been granted access. A requestor who re-requests the same cache line (has already been granted access to the hot cache line once before) cannot receive the line again if there are other requestors who want the cache line and have not been granted access yet.

As further details of the structure of the serviced and requested vectors illustrated in <FIG>, each vector includes scopes, which can be defined to function as desired. In one or more embodiments, <FIG> illustrates an example in which the serviced and requested vectors each include three hierarchical scopes which are the chip scope, drawer scope, and system scope. In the hierarchy, the drawer scope is higher than the chip scope, and the system scope is higher than the drawer scope. Each vector is kept within the hardware controllers <NUM> instead of the directory <NUM>, thereby minimizing the number of bits needed, because there are a smaller number of hardware controllers <NUM> compared to number of cache entries in the directory <NUM>. Although the requested and serviced vectors are described as vectors, it should be appreciated that any structure could be utilized to maintain, store, and eventually pass/transfer the respective states having a respective state per requestor.

In <FIG>, the chip scope which is the lowest scope tracks requests from other shared caches <NUM> (e.g., other L3 caches) on the same chip as the current shared cache <NUM> that has access to the cache line. Each shared cache <NUM> (e.g., L3 cache) on a chip <NUM> gets a unique bit corresponding to the particular shared cache <NUM> in the service and requested vectors. <FIG> illustrates an example chip scope for an example hardware controller <NUM> according to one or more embodiments. In <FIG>, the chip scope of the vector could be representative of a requested vector and/or a serviced vector. Because each processor chip (e.g., chip <NUM>, chip <NUM>, chip <NUM>) has four shared caches (L30, L31, L32, L33), there are four bit places, one unique bit place (position) assigned to each shared cache <NUM> in the processor chip (such as, e.g., chip <NUM>), as depicted in <FIG>. It is noted that the designation of bit places is from left to right in this example, but the designation could be from right to left, or some other designation. From left to right, the bit place/position in the chip scope corresponds to L30, L31, L32, L33 in an individual chip <NUM>. Four bits are illustrated to represent the four example shared caches <NUM> in <FIG>, but more or fewer chip scope bits can be utilized according to the number of shared caches <NUM> in an induvial chip <NUM>.

In <FIG>, the drawer scope tracks requests from other chips <NUM> that are on the same drawer <NUM> as the controlling hardware controller <NUM> of current shared cache <NUM> that has access to the cache line. Each chip <NUM> on a drawer <NUM> gets a unique bit corresponding to the particular chip <NUM> in the service and requested vectors. <FIG> illustrates an example drawer scope for an example hardware controller <NUM> according to one or more embodiments. In <FIG>, the drawer scope of the vector could be representative of a requested vector and/or a serviced vector. Because each drawer <NUM> (e.g., drawer <NUM>, drawer <NUM>, drawer <NUM>) has three processor chips <NUM> (e.g., chip <NUM>, chip <NUM>, chip <NUM>), there are three bit places, one unique bit place (position) assigned to each processor chip (e.g., chip <NUM>, chip <NUM>, chip <NUM>) in a drawer <NUM> (such as, e.g., drawer <NUM>), as depicted in <FIG>. From left to right, the bit place/position in the drawer scope corresponds to chip <NUM>, chip <NUM>, chip <NUM> in an individual drawer <NUM>. Three bits are illustrated to represent the three example chips <NUM> in <FIG>, but more or fewer drawer scope bits can be utilized according to the number of chips <NUM> in an individual drawer <NUM>.

In <FIG>, the system scope tracks requests from other drawers <NUM>. Each drawer <NUM> on the system <NUM> gets a unique bit corresponding to a particular drawer <NUM> in the service and requested vectors. <FIG> illustrates an example system scope for an example hardware controller <NUM> according to one or more embodiments. In <FIG>, the system scope of the vector could be representative of a requested vector and/or a serviced vector. Because the system <NUM> has three drawers <NUM> (e.g., drawer <NUM>, drawer <NUM>, drawer <NUM>), there are three bit places, one unique bit place (position) assigned to each drawer <NUM> (e.g., drawer <NUM>, drawer <NUM>, drawer <NUM>) in the system <NUM>, as depicted in <FIG>. From left to right, the bit place/position in the system scope corresponds to drawer <NUM>, drawer <NUM>, drawer <NUM> in the system <NUM>. Three bits are illustrated to represent the three example drawers <NUM> in <FIG>, but more or fewer system scope bits can be utilized according to the number of drawers <NUM> in the system <NUM>.

The hardware controllers <NUM> use an authority level for a cache line to set and reset tracking bits in the requested and serviced vectors for that cache line. Each cache line has its own associated requested vectors and serviced vectors. There are three types of authority levels discussed below, which include Global/System Coherency Authority, Drawer Coherency Authority, and Chip Coherency Authority. Global/System Coherency Authority (Global Intervention Master, or GIM) for a cache line is the highest point of coherency for that cache line if a GIM copy of the cache line exists in a shared cache <NUM>. This authority is acquired by a hardware controller <NUM> by receiving a cache line and GIM authority for the cache line from the previous GIM (authority) for that cache line, and/or by determining that there are no other copies in the system <NUM> with GIM for the requested cache line in which case only the GIM then has authority to receive the cache line from memory. If an exclusive copy of the cache line exists in a shared cache, that cache line must also be the GIM. The GIM for a cache line is also the Drawer Intervention Master or DIM for its drawer and the Chip Intervention Master or CIM for its chip. In the <NUM>-level intervention master hierarchy, only the GIM is allowed to return data for requests from other drawers, according to one or more embodiments. For a given cache line, there can only be one GIM in a system <NUM>.

Next, turning to the Drawer Coherency Authority (Drawer Intervention Master, or DIM) for a cache line, the DIM is the highest level of authority for that cache line in a drawer <NUM>. This authority is acquired by a hardware controller <NUM> by receiving DIM authority for the cache line from the previous DIM on the drawer <NUM> for that cache line, and/or by determining that there are no other copies in the drawer <NUM> with DIM for the requested cache line in which case only the DIM then has authority to request the cache line from other drawers <NUM>. A GIM copy of the cache line may be downgraded to DIM when a shared copy of the cache line and GIM authority are passed to another drawer <NUM>. If a shared cache <NUM> controlled by a hardware controller <NUM> has a copy of the cache line with DIM authority but not GIM authority, the hardware controller <NUM> must have a read-only copy of the line. The DIM for a cache line is also the CIM for its chip <NUM>. In the <NUM>-level intervention master hierarchy, only the DIM is allowed to return data for requests from other chips on its drawer, according to one or more embodiments. For a given cache line, there can be as many DIMs in the system <NUM> as there are drawers <NUM>, but only one DIM per drawer.

Chip Coherency Authority (Chip Intervention Master, or CIM) for a cache line is the highest level of authority for that cache line in a chip. This authority is acquired by a hardware controller <NUM> by receiving CIM authority for the cache line from the previous CIM on the chip for that cache line, and/or by determining that there are no other copies on the chip with CIM for the requested cache line in which case only the CIM then has authority to request the cache line from other chips. A DIM copy of the line may be downgraded to CIM when a shared copy of the cache line and DIM authority are passed to another chip <NUM>. If a cache has a copy of the cache line with CIM authority but not GIM authority, the controlling hardware controller <NUM> must have a read-only copy of the line. In the <NUM>-level intervention master hierarchy, only the CIM is allowed to return data for requests from other shared caches <NUM> on its chip.

A non-intervention master copy of a cache line is a read-only copy of the cache line in a shared cache <NUM> (e.g., L3 cache) that does not have any authority to share the cache line with other requesters. This can sometimes be thought of as the lowest coherency scope.

Turning to further details regarding maintaining the requested and serviced vectors, an example is discussed for setting and resetting tracking bits in the requested and serviced vectors for a cache line. As noted herein, hardware controllers <NUM> are responsible for setting and maintaining the fairness vectors. Each hardware controller <NUM> updates both vectors (i.e., requested vector and serviced vector) based on the on-chip and off-chip requests that the particular hardware controller <NUM> observes, as well as based on the fairness the hardware controller <NUM> receives once it gets access to the cache line. For example, once a hardware controller <NUM> receives data that indicates there are un-serviced requestors, the hardware controller <NUM> does not retire as usual after the hardware controller <NUM> returns data to the processor core. Instead, the hardware controller <NUM> remains active to protect the cache line until an eligible requestor arrives. While waiting for an eligible requestor to arrive, the hardware controller <NUM> ensures that requests (also referred to as snoops) from non-eligible requestors get a coherency reject.

To set a requested bit in a scope, a snoop broadcast from a requestor must compare against a remote hardware controller <NUM> of a remote shared cache that has the authority level one level lower than the authority level that can service both the requestor and the remote shared cache <NUM> (e.g., L3 cache) on a broadcast where the requestor ends with the authority level that can service both the requestor and the remote shared cache. The remote hardware controller works for its shared cache and is the controlling hardware controller for a cache line of a given shared cache. The requestor requesting access to the cache line of the given shared cache is the requestor hardware controller of its shared cache. Because each shared cache has its own hardware controllers, sometimes a shared cache may be referred to as requesting a cache line. To set a bit in the requested vector for the chip scope, the requesting hardware controller <NUM> of the requesting shared cache <NUM> (e.g., shared cache L30) must establish itself as at least a chip coherency authority during or before the broadcast where hardware controller <NUM> can start tracking requests from other hardware controllers <NUM> corresponding to other shared caches <NUM> (e.g., other shared caches L31, L32, L33) on the processor chip <NUM> (such as, e.g., processor chip <NUM>).

To set a bit in the requested vector for drawer scope, the requesting hardware controller <NUM> of the requesting shared cache <NUM> must establish itself as at least a drawer coherency authority during or before the broadcast where hardware controller <NUM> can start tracking requests from other chips <NUM> on the drawer <NUM> for the cache line of a given shared cache. Requestors on remote chips must be for an Exclusive Fetch or have reached at least chip coherency authority for the requesting controller to set a bit in the requested vector.

To set a bit in the requested vector for system scope, the requesting hardware controller <NUM> of the requesting shared cache <NUM> must establish itself as the system coherency authority during or before the broadcast where the hardware controller <NUM> can start tracking requests from other drawers <NUM> on the system <NUM> for the cache line of a given shared cache. Requestors on remote drawers must be for an Exclusive Fetch or have reached at least drawer coherency authority for the requesting controller to set a bit in the requested vector.

These setting restrictions are utilized because otherwise, an observed request could be served by someone else (e.g., another hardware controller <NUM> for another shared cached <NUM> that does have proper authority) and should not be counted as a request that must be serviced by the local hardware controller <NUM> for a shared cache <NUM>.

Further details of maintaining the requested and serviced vectors are discussed below. For a serviced vector, serviced bits in a scope (e.g., such as chip, drawer, and system scopes) get set when a local hardware controller <NUM> for its corresponding shared cache <NUM> with access to the cache line data sets its own corresponding chip, drawer, and system bits. For a requested vector, requested bits in a scope (e.g., such as chip, drawer, and system scopes) get reset when the corresponding serviced bit of the serviced vector is set in the same scope, as well as when the local shared cache <NUM> via its hardware controller <NUM> invalidates its own copy, and/or a timeout limit has been reached. For a serviced vector, serviced bits in a scope (e.g., such as chip, drawer, and system scopes) get reset when all known requestors have been serviced, when the local shared cache <NUM> via its hardware controller <NUM> invalidates its copy, and/or when the timeout limit has been reached. Requested and serviced bits in a lower scope are reset when the cache line is passed to a requestor of a higher scope (i.e., drawer vector gets cleared if the cache line is passed to another drawer, but the drawer vector will keep its data if the cache line is passed to another chip on the same drawer). Moreover, the drawer scope/vector refers to the chips on this drawer, which is relative to the L3 cache maintaining the vector. If the cache line moves to another chip on this same drawer, the drawer vector values are still meaningful for the receiving chip since the drawer vectors of both the sending and receiving chip refer to chips on the same drawer. On the other hand, if the cache line moves to a chip on another drawer, then the new drawer scope/vector created on the receiving chip will refer to the receiving drawer, while the drawer scope/vector on the sending chip refers to chips on the sending drawer. The state in the drawer scope/vector from the sending drawer is meaningless to the receiving drawer, and therefore, requested and serviced bits in a lower scope are reset when the cache line is passed to a requestor of a higher scope.

Further details are discussed regarding passing the requested and serviced vectors around the system <NUM>, thereby providing a decentralized hot cache line tracking fairness mechanism. In one or more embodiments, the requested and serviced fairness vectors for each cache line are passed along (e.g., from one hardware controller <NUM> for a shared cache <NUM> to the next hardware controller <NUM> for a shared cache <NUM>) as part of the metadata for that cache line. In one or more embodiments, the method could pass only the serviced vector along with the cache line, because the requested vector can be rebuilt by each requestor (e.g., hardware controller <NUM>). In one or more embodiments, there can be a combination of where the requested vector is rebuilt by some requestors (e.g., hardware controller <NUM>) while the serviced vector is passed along with the cache line and where the requested and serviced vectors are passed along with the cache line. Only un-serviced requestors get access to the cache line next, based on checking serviced bits in the serviced vector. All serviced requestors are rejected and must come back to try again. The fairness vectors (i.e., the requested vector and serviced vector) will not be passed to a higher scope until there are no requested bits in the requested vector set for lower scopes. Serviced bits/states in the serviced vector are eventually reset based on the already established rules.

<FIG>, <FIG>, and <FIG> depict block diagrams illustrating an example of vector maintenance and passing for decentralized hot cache line tracking fairness using preemptive tracking of remote requests according to one or more embodiments of the invention. This example scenario is vector updating and passing on chip, for example, passing the serviced vector on the same chip <NUM>. Accordingly, the example scenario illustrates chip scope and drawer scope but does not illustrate system scope, for ease of understanding. It should be appreciated that system scope applies by analogy. Although one shared cache and its hardware controller are discussed at a time, it should be appreciated that analogous actions are being performed simultaneously or nearly simultaneously for other shared caches for the same cache line.

At action <NUM>, shared cache L30 (which is a shared cache <NUM>) via its controlling hardware controller <NUM> has access to and/or control of the cache line of a given shared cache <NUM>. For explanation purposes, the controlling hardware controller <NUM> is a designation used to illustrate that the hardware controller has control of the cache line of the given shared cache. In this example, there are four shared caches <NUM> (e.g., L30, L31, L32, L33) depicted by four bits in the chip scope for both the requested vector and the serviced vector. Also, in this example, there are two chips <NUM> (e.g., chip <NUM>, chip <NUM>) depicted by two bits in the drawer scope. As can be seen at action <NUM>, the hardware controller state for the requested vector is all zeros. In the requested vector, the four bits of the chip scope are all zeros because no other shared cache <NUM> on the same chip <NUM> (e.g., chip <NUM>) has requested access to the cache line (yet). Also, in the requested vector, the two bits of the drawer scope are all zeros because no other chip has requested access to the cache line.

However, in the chip scope of the serviced vector, the first bit (i.e., the most left bit) corresponding to shared cache L30 is set to one ("<NUM>") because the shared cache L30 is being serviced (i.e., accessing the cache line of the given shared cache) in action <NUM>, which means the hardware controller <NUM> for the shared cache L30 is controlling accessing to the cache line. The remaining serviced bits in the chip scope of the serviced vector are zero because the other shared caches <NUM> on the same chip <NUM> (e.g., chip <NUM>) have not previously been serviced, which means the other shared caches <NUM> (e.g., L31, L32, L33) have not had access to the cache line via their respective hardware controllers <NUM>. In the drawer scope of the serviced vector, the first bit which is the most left bit is set to one ("<NUM>") because the corresponding chip <NUM> which is chip <NUM> is being serviced. The remaining serviced bit in the drawer scope of the serviced vector is set to zero because the other chip <NUM> (e.g., chip <NUM>) on the same drawer <NUM> (e.g., drawer <NUM>) has not been serviced. If there were more than one drawer in this example, then a system scope would have a bit set to identify the drawer currently being serviced, thereby identifying the drawer having a chip currently serviced. Since this example shows the first round, the other serviced bits are zero but if other caches had been serviced, then their respective bits would be one.

At action <NUM>, shared cache L32 via its requestor hardware controller <NUM> requests the cache line (fetches) from the requesting hardware controller <NUM> of shared cache L33. It is noted that actions <NUM> and <NUM> occur concurrently and/or nearly concurrently. At this point, the shared cache L32 is a requestor via its requestor hardware controller <NUM>. Shared cache L32 via its requestor hardware controller <NUM> snoops on shared cache L33 and gets a response that shared cache L33 has an outstanding fetch to the cache line of the given shared cache. The snoop (also referred to as bus snooping or bus sniffing) by the requestor hardware controller <NUM> of shared cache L32 checks the other shared caches <NUM> and monitors the bus, which includes the interconnections connecting the shared caches <NUM>, for any outstanding fetch requests from other shared caches <NUM>. After receiving the response from the hardware controller <NUM> of the shared cache L33, the shared cache L32 via its requestor hardware controller <NUM> speculatively sets the requested bit for shared cache L33 in its requested vector, as depicted in action <NUM>. Particularly, the hardware controller <NUM> of shared cache L32 records the response from shared cache L33 in the corresponding (L33) bit place in the chip scope of requested vector. As seen in the requested vector, the furthest right bit in the chip scope is now set to one ("<NUM>") which indicates the corresponding shared cache L33 at that bit position has also requested the cache line of the given shared cache.

At action <NUM> in <FIG>, the shared cache L32 via its requestor hardware controller <NUM> requests the cache line from the controlling hardware controller <NUM> of the shared cache L30, and the shared cache L30 via its controlling hardware controller <NUM> accepts the request. Therefore, the shared cache L32 is given a copy of the cache line and the serviced vector of shared cache L30. For example, the shared cache L30 via its controlling hardware controller <NUM> passes the cache line to the shared cache L32 via its hardware controller <NUM>. In addition to passing access and/or control of the cache line, the controlling hardware controller <NUM> of shared cache line L30 also passes the serviced vector for the cache line of the given shared cache <NUM> to the requestor hardware controller <NUM> of the shared cache L32. At this point the hardware controller <NUM> of shared cache L32 becomes the new controlling hardware controller <NUM>. As the controlling (L32) hardware controller <NUM> of shared cache L32, the controlling (L32) hardware controller <NUM> sets its own serviced bits by updating the serviced bit corresponding to the shared cache L32 in the chip scope of the serviced vector from zero to one because the shared cache L32 is being serviced, which means the shared cache L32 via its controlling (L32) hardware controller <NUM> has access/control of the cache line in the given shared cache. Also, the controlling (L32) hardware controller <NUM> can keep the requested bit that it preemptively set ("<NUM>") where the requested bit corresponds to shared cache L33 in the chip scope of the requested. It is further noted that shared cache L32 launched a broadcast to all other shared caches <NUM>, asking for the cache line. Around the same time, shared cache L33 also launched a broadcast to all other shared caches <NUM>, asking for the cache line. In this example scenario, the shared cache L32's broadcast request arrived at shared cache L30 first, while the shared cache L33's broadcast request was still enroute to shared cache L30. In other examples, shared cache L33's broadcast could have arrived at shared cache L30 first, while shared cache L32's broadcast request was still enroute to shared cache L30.

At action <NUM> in <FIG>, in response to shared cache L33 via its requestor hardware controller <NUM> having requested the cache line (fetches) from the requesting hardware controller <NUM> of shared cache L32 while the request of shared cache L32 was still outstanding, the cache line request of shared cache L33 has reached shared cache L32. Action <NUM> may occur concurrently with and/or near the same time as action <NUM>, as noted herein. Shared cache L33 via its requestor hardware controller <NUM> snoops on shared cache L32 and gets a response that shared cache L32 has an outstanding fetch to the cache line of the given shared cache. After receiving the response from the hardware controller <NUM> of the shared cache L32, the shared cache L33 via its requestor hardware controller <NUM> speculatively sets the requested bit for shared cache L32 in its requested vector, as depicted in action <NUM>. Particularly, the hardware controller <NUM> of shared cache L33 records the response from shared cache L32 in the corresponding (L32) bit place in the chip scope of requested vector. As seen in the requested vector, the second from the right bit in the chip scope is now set to one ("<NUM>") which indicates the corresponding shared cache L32 at that bit position has also requested the cache line of the given shared cache.

At action <NUM> in <FIG>, in response to shared cache L33 via its requestor hardware controller <NUM> having requested the cache line (fetches) from the controlling hardware controller <NUM> of shared cache L30, the cache line request of shared cache L33 has reached shared cache L30 and is rejected because the cache line is being moved to shared cache L32. Accordingly, shared cache L33 via its requestor hardware controller <NUM> has to reset the requested bit for shared cache L32 that it set speculatively on this broadcast. As seen in action <NUM>, the second from the right most bit corresponding to the bit place for shared cache L32 is rest to zero in the chip scope of the requested vector maintained by shared cache L33 because the shared cache L32 is being serviced. It is noted that the serviced vector is not preemptively maintained by the shared cache L32 because the controlling shared cache maintains its own serviced vector.

At action <NUM> in <FIG>, the shared cache L33 via its requestor hardware controller <NUM> re-requests the cache line of the given shared cache again, and the controlling hardware controller <NUM> of the shared cache L32 passes the cache line to shared cache L33. In addition to passing access and/or control of the cache line, the controlling hardware controller <NUM> of shared cache line L33 also passed the serviced vector for the cache line of the given shared cache <NUM> to the requestor hardware controller <NUM> of the shared cache L33. At this point the hardware controller <NUM> of shared cache L33 becomes the new controlling hardware controller <NUM>. As the controlling (L33) hardware controller <NUM> of shared cache L33, the controlling (L33) hardware controller <NUM> sets its own serviced bits by updating the serviced bit corresponding to the shared cache L33 in the chip scope of the serviced vector from zero to one because the shared cache L33 is being serviced, which means the shared cache L33 via its controlling (L33) hardware controller <NUM> has access/control of the cache line in the given shared cache.

Although some example scenarios are discussed herein in which the previous (controlling) hardware controller <NUM> passes both the serviced vector and requested vector for a cache line to the next (controlling) hardware controller <NUM>, some embodiments of the invention may not pass the requested vector because each hardware controller <NUM> is configured with logic <NUM> to build the requested vector (as well as the serviced vector) using responses from other hardware controllers. Further details of maintaining the requested and serviced vectors are discussed below for setting request tracking bits in the vectors. As noted herein, the hardware controllers <NUM> are responsible for setting and maintaining the remote fairness vectors. After receiving a broadcast sent out on the system <NUM> by the querying hardware controller <NUM>, each queried hardware controller <NUM> of a remote cache <NUM> having received the query determines if that queried hardware controller <NUM> will give a response, given the following are true: <NUM>) the queried hardware controller has an outstanding fetch request; and <NUM>) the outstanding fetch is an exclusive fetch request, or the outstanding fetch request of the queried hardware controller is one coherency authority level lower than the scope of the broadcast. If the hardware controller does not have sufficient authority, the outstanding fetch request of the remote cache might be serviced by another cache instead of by the cache doing the broadcast. Further regarding one authority level lower than the scope of the broadcast is discussed below.

One scope lower than system scope: if the cache line request originated from an L3 on Drawer <NUM> and if the remote cache being queried is an L3 on Drawer <NUM>, this makes the broadcast scope a System scope. Therefore, to respond with an outstanding fetch request, the queried remote cache on Drawer <NUM> must have the coherency Drawer coherency authority (DIM). If the queried remote cache has only Chip (CIM) or non-IM authority, queried remote cache cannot respond that it has an outstanding fetch request.

One scope lower than drawer scope: if the cache line request originated from an L3 on Chip <NUM> and if the remote cache being queried is an L3 on Chip <NUM>, this makes the broadcast scope a Drawer scope. Therefore, to respond with an outstanding fetch request, the queried remote cache on Chip <NUM> must have the coherency Chip coherency authority (CIM). If the queried remote cache has only non-IM authority, the queried remote cache cannot respond that it has an outstanding fetch request.

One scope lower than chip scope: if the cache line request originated from an L30 on Chip <NUM> and if the remote cache being queried is an L31 on Chip <NUM>, this makes the broadcast scope a Chip scope. Therefore, to respond with an outstanding fetch request, the queried remote cache L31 must have non-IM coherency.

More particularly, a querying hardware controller <NUM> (of its shared cache <NUM>) requesting a copy of the cache line of a given shared cache <NUM> broadcasts its fetch request to all other shared caches <NUM> having hardware controllers <NUM> in the system <NUM> at the same time as and/or nearly the same time as sending a query to other hardware controllers <NUM> of shared caches <NUM> to inquire if the queried hardware controllers <NUM> have outstanding fetch requests for the (same) cache line of the given shared cache <NUM>. Each queried hardware controller <NUM> of other shared caches provides an indication if it has the cache line and an indication if that queried hardware controller <NUM> has an outstanding fetch request that meets the previous criteria.

If a queried hardware controller <NUM> having made a request is on chip (i.e., on the same chip <NUM> as the querying hardware controller <NUM>), then a requested bit is set in the chip scope of the (preemptive) requested vector of the querying hardware controller <NUM>, for each queried hardware controller <NUM> (of its shared cache <NUM>) that returned an outstanding fetch response. If the queried hardware controller <NUM> having made a request is off chip and on drawer (i.e., on the same drawer <NUM> as but a different chip <NUM> than the querying hardware controller <NUM>), then the outstanding fetch responses on the same chip <NUM> are logically ORed together by the querying hardware controller <NUM>, which means an OR logic operation returns true if either of its inputs are true and false otherwise. That OR result is then sent to the chip <NUM> of the querying hardware controller <NUM> having made the query, which uses the result to set the corresponding requested bit in the drawer scope in the requested vector. In one or more embodiments, a queried hardware controller <NUM> (on the same chip as other queried hardware controllers <NUM>) can perform the OR logic operation for responses on its same chip <NUM> and then send OR result to the querying hardware controller <NUM>. In one or more embodiments, the querying hardware controller <NUM> can perform the OR logic operation for the response from off-chip queried hardware controllers <NUM>. Also, if a queried hardware controller <NUM> (having made a request) has achieved the appropriate level of authority and has an outstanding request, this can also be ORed into the OR result that is utilized by the querying hardware controller <NUM> to set requested bit of the drawer scope of the requested vector.

If the queried hardware controller <NUM> having made a request is off drawer, then the outstanding fetch responses on the drawer are ORed together. That OR result is then sent to the drawer <NUM> of the queried hardware controller <NUM>, which uses the OR result to set the corresponding request bit in the system scope in the requested vector. In one or more embodiments, a queried hardware controller <NUM> (on the same drawer as other queried hardware controllers <NUM>) can perform the OR logic operation for responses on its same drawer <NUM> and then send OR result to the querying hardware controller <NUM>. In one or more embodiments, the querying hardware controller <NUM> can perform the OR logic operation for all the responses from off-drawer queried hardware controllers <NUM>. If there is a rebroadcast, any remote fairness data in the requested vector that was collected outside of the authority level achieved by the queried hardware controller <NUM> must be cleared. The following example scenario regarding authority level is provided for explanation purposes. The requesting cache with CIM authority broadcasts to get GIM authority for the cache line, and the following occurs on that broadcast. The requesting cache finds outstanding on-chip misses (i.e., outstanding requests), and this causes requested bits to be set in the chip scope of the requested vector. The requesting cache finds outstanding on-drawer misses (i.e., outstanding requests), this causes requested bits to be set in the drawer scope of the requested vector. The requesting cache finds outstanding off-drawer misses (i.e., outstanding requests), and this causes requested bits to be set in the system scope of the requested vector. Now, it is assumed that the request by the requesting cache successfully acquired DIM authority but is rejected by the GIM. The requesting cache can keep the requested bits in the chip scope in the requested vector because the requesting cache has CIM or higher authority at the completion of the broadcast, i.e., both because it started as CIM, and because it acquired DIM, which is higher than CIM. The requesting cache can keep the requested bits in the drawer scope of the requested vector because it acquired DIM authority on this broadcast. The requested bits in the system scope of the requested vector must be reset because GIM authority was not achieved.

<FIG> depicts a block diagram illustrating preemptive tracking of remote requests decentralized hot cache line fairness tracking according to one or more embodiments. <FIG> is a simplified version of system <NUM> that omits some details, and it should be appreciated that any omitted details are included in system <NUM>. In the example scenario in <FIG>, a hardware controller <NUM> is designated as the querying hardware controller <NUM> for explanation purposes, where the querying hardware controller <NUM> includes logic <NUM> to broadcast its request for the cache line of a given shared cache <NUM> while simultaneously or nearly simultaneously broadcasting a query to all other hardware controllers <NUM> (for their respective shared caches <NUM>) in order to determine which other hardware controllers <NUM> have also requested the cache line of the given shared cache <NUM>. For ease of understanding, the other hardware controllers <NUM> having received the query may be referred to as the queried hardware controllers <NUM>. The queried hardware controllers <NUM> have logic <NUM> configured to respond back to the query with a response according to criteria discussed herein, in accordance with embodiments of the invention. The querying hardware controller <NUM> is configured to preemptively build a requested vector in memory <NUM> for the responses received from the queried hardware controllers <NUM>, which is in anticipation of the querying hardware controlled <NUM> gaining access/control of the cache line and receiving the serviced vector from the previous controlling hardware controller <NUM>. If the querying hardware controller <NUM> does not get authority to access/control the cache line of the given shared cache <NUM>, the requested vector is dropped or discarded from memory <NUM>. It should be appreciated that, although <FIG> depicts a single hardware controller <NUM> making a query, the other hardware controllers <NUM> make queries while broadcasting their respective (fetch) requests for the cache line of the given shared cache <NUM>, such that each of the other hardware controllers <NUM> have their own preemptively built requested vector. A querying hardware controller <NUM> may preemptively build the requested vector before, during, and/or after sending its request for the cache line of the given shared cache <NUM>.

<FIG> is a flowchart of a process <NUM> for preemptive tracking of remote requests for decentralized hot cache line fairness tracking according to one or more embodiments. <FIG> illustrates an example from the perspective of a querying hardware controller <NUM>, which is sometimes referred to as the local cache, sending queries to and receiving responses from various (queried) hardware controllers <NUM>, which are sometimes referred to as remote caches because each cache has its own hardware controllers.

At block <NUM>, the querying hardware controller <NUM> is configured to build a remote vector in memory <NUM> and set all its requested bits to zero, in advance of receiving the cache line. At block <NUM>, the querying hardware controller <NUM> is configured to broadcast around the system <NUM> the fetch request for the cache line of the given shared cache <NUM> and a query of other hardware controllers <NUM> (i.e., other requestors) of shared caches <NUM> requesting the cache line.

At block <NUM>, the querying hardware controller <NUM> is configured to check whether a queried hardware controller <NUM> of a remote cache responded with an indication that the queried hardware controller <NUM> has an outstanding fetch request for the cache line of the given shared cache <NUM>. At block <NUM>, if ("YES") the queried hardware controller <NUM> responds back with a response that it has an outstanding fetch request for the cache line of the given shared cache <NUM>, the querying hardware controller <NUM> is configured to update the requested vector to reflect the request for the cache line from the queried hardware controller <NUM>. For example, the requested bit is set to one ("<NUM>") in the chip scope, drawer scope, and/or system scope of the requested vector according to the response from the queried hardware controller <NUM>. The chip scope, drawer scope, and system scope are relative to the shared cache <NUM> of the querying hardware controller <NUM>. It should be appreciated that blocks <NUM> and <NUM> are repeated for each of the queried hardware controllers <NUM>. As noted herein, some responses from the same off chip can be ORed together before setting the corresponding requested bit in the drawer scope of the requested vector. Also, some responses from the same off drawer can be ORed together before setting the corresponding requested bit in system scope of the requested vector.

At block <NUM>, if ("NO") the queried hardware controller <NUM> does not respond back with a response that it has an outstanding fetch request for the cache line of the given shared cache <NUM>, the querying hardware controller <NUM> is configured to check did the querying hardware controller <NUM> get authority for the cache line. Authority is given from a previous hardware controller <NUM> having the desired authority. Authority for the cache line of the shared cache <NUM> is typically given before access/control of the cache line is given. In one or more embodiments, authority to the cache line and access/control of the cache line may be given together or nearly at the same time.

At block <NUM>, if ("NO") the queried hardware controller <NUM> does not get authority for the cache line of the given shared cache <NUM>, the querying hardware controller <NUM> is configured to reset the requested bits in the requested vector, and flow proceeds to block <NUM>.

At block <NUM>, if ("YES") the queried hardware controller <NUM> does get authority for the cache line of the given shared cache <NUM>, the querying hardware controller <NUM> is configured to gain access to the cache line, set serviced bits in serviced vector from fetch response, set its own serviced bits in the chip scope of the serviced vector that correspond to the querying hardware controller <NUM> (i.e., set the serviced bits that correspond to bit place/position of the shared cache <NUM> for the querying hardware controller <NUM>), and reset requested bits (to zero) in the requested vector corresponding to the service bits that were just set in the serviced vector for the querying hardware controller <NUM>. The querying hardware controller <NUM> now becomes the controlling hardware controller <NUM>. The querying hardware controller <NUM> receives the serviced vector from the previous controlling hardware controller <NUM> along with the cache line and may receive the requested vector from the previous controlling hardware controller <NUM>. When the querying hardware controller <NUM> also receives transfer of the requested vector from the previous controlling hardware controller <NUM>, the querying hardware controller <NUM> is configured to reconcile the set requested bits in preemptively built requested vector with the transferred-in requested vector. In an example scenario, the serviced vector is passed along with the cache line metadata, and when a new fetch controller gains access to the cache line, the fetch controller must update its own serviced vector to reflect the serviced vector that came along with the cache line metadata. For example, shared cache L30 is passing the cache line to shared cache L33. Shared cache L33 must update its serviced vector to reflect the serviced vector that shared cache L30 had.

<FIG> is a flowchart of a process <NUM> for preemptive tracking of remote requests for decentralized hot cache line fairness tracking according to one or more embodiments. <FIG> illustrates an example from the perspective of queried hardware controllers <NUM>, which are sometimes referred to as remote caches. The queried hardware controllers <NUM> are receiving a query from and sending responses to the querying hardware controller <NUM>, which is sometimes referred to as the local cache.

At block <NUM>, a queried hardware controller <NUM> is configured to receive a fetch broadcast with a query about a cache line for a given shared cache <NUM> from a querying hardware controller <NUM>. At block <NUM>, the queried hardware controller <NUM> is configured to check whether it has an outstanding fetch request for the cache line of the given shared cache <NUM>. At block <NUM>, if ("NO") the queried hardware controller <NUM> does not have an outstanding fetch request for the cache line of the given shared cache, the queried hardware controller <NUM> is configured to respond no it does not have an outstanding fetch request for the cache line.

At block <NUM>, if ("YES") the queried hardware controller <NUM> does have an outstanding fetch request for the cache line of the given shared cache <NUM>, the queried hardware controller <NUM> is configured to check with the outstanding fetch request is an exclusive fetch. An exclusive fetch request is a request for exclusive ownership of a cache line.

At block <NUM>, if ("YES") the queried hardware controller <NUM> does have an exclusive fetch request for the cache line of the given shared cache <NUM>, the queried hardware controller <NUM> is configured to respond back yes it has an outstanding fetch request to the querying hardware controller <NUM>.

At block <NUM>, if ("NO") the queried hardware controller <NUM> does not have an exclusive fetch request for the cache line of the given shared cache <NUM>, the queried hardware controller <NUM> is configured to check if it has authority exactly one level less that the lowest authority level that can service both the requesting broadcaster (e.g., the querying hardware controller <NUM>) and the remote cache (i.e., the queried hardware controller <NUM>). There are varying levels of "authority" of which a hardware controller can own a copy of a cache line. In this scenario, the authority determines whether or not the queried cache has enough visibility to respond accurately to the request. For example, there can be multiple hardware controllers that have read-only (non-IM authority) copies of the cache line. The scopes and authorities have been discussed herein. If the answer is negative ("NO") at block <NUM>, flow proceeds to block <NUM>. If answer is positive ("YES") at block <NUM>, flow proceeds to block <NUM>.

It should be appreciated various operations described in <FIG> and <FIG> can be performed cooperatively for the preemptive tracking of remote requests for decentralized hot cache line fairness tracking according to one or more embodiments. Additionally, various hardware controllers <NUM> can be concurrently performing operations described in <FIG> and <FIG>, although a single hardware controller <NUM> may have been identified at times.

Further regarding passing the vector around the system <NUM>, as discussed herein the remote fairness serviced vector is passed along with the data of the cache line (as a fetch response) as part of the metadata for the cache line. In one or more embodiments, the requested vector can be rebuilt by each (fetch) hardware controller <NUM> and therefore may not have to be passed around the system <NUM>. As an optional enhancement, the requested vector could be passed along with the serviced vector and fetch response, even if a hardware controller preemptively builds its own requested vector.

Further, in one or more embodiments, rather than accumulating information on which requestors are outstanding (i.e., have outstanding fetch requests), the querying hardware controller <NUM> may only collect a summary, which indicates that some outstanding requests exists. For instance, an indication may be collected that there is an outstanding request for the cache line on this chip, but not which shared cache <NUM> (L3 cache) on the chip is making the request. In this case, a requesting shared cache <NUM> (L3 cache) will wait a fixed interval, after achieving coherency, to determine if there are still any un-serviced requests from other caches (L3 caches) on this chip. After the interval expires, the requested bits in the chip scope, drawer scope, and system scope are reset in the requested vector. As a further option, requests from higher scopes would be rejected during this interval.

Further technical solutions and benefits are provided for preemptive tracking of remote requests for decentralized hot cache line fairness tracking according to one or more embodiments. A method of ensuring cache line fairness may include the following: broadcasting a request for a cache line; querying other caches to determine if they have an outstanding fetch request for that cache line; having a remote cache respond with an indication it has an outstanding fetch request; collecting the responses to those queries in a hardware controller; and maintaining a serviced vector of requestors that have received a copy of the cache line. Additionally, the method includes storing the collected responses to those queries in a requested vector, retiring the hardware controller when all outstanding requests have been serviced, transferring the serviced vector to another cache's hardware controller in response to a fetch request, and invalidating this cache's hardware controller. To preemptively build the requested vector, technical solutions and benefits include collecting all the lower scope responses into a single response to a request at a higher scope and dropping the collected responses if the broadcast does not achieve authority for the cache line. There can be different criteria for a remote cache (via its hardware controller) to respond: <NUM>) the remote cache responds that it has an outstanding fetch request if that request is an exclusive fetch; <NUM>) the remote cache responds that it has an outstanding fetch request if it is processing a non-exclusive fetch and it has an authority level exactly one level less than the lowest authority level that can service both the requestor and the remote cache; and/or <NUM>) a fetch hardware controller on a remote chip responds if it has an outstanding fetch request and if it has an authority level exactly one level less than the lowest authority level that can service both the requestor and the remote chip.

<FIG> is a flowchart of a computer-implemented method <NUM> for preemptive tracking of remote requests for decentralized hot cache line fairness tracking according to one or more embodiments. Reference can be made to any of the figures discussed herein.

At block <NUM>, the querying hardware controller <NUM> is configured to request authority for a cache line (of a shared cache <NUM>) in conjunction with querying (queried hardware controllers <NUM>) for outstanding requests for the cache line of a given shared cache <NUM>. The querying hardware controller <NUM> sends the queries prior to receiving the cache line. The querying hardware controller <NUM> can request authority from the controlling hardware controller <NUM> that currently owns/controls the cache line of the given shared cache <NUM>. At block <NUM>, the querying hardware controller <NUM> is configured to receive one or more responses regarding the outstanding requests having been made (by queried hardware controllers <NUM>) for the cache line. At block <NUM>, the querying hardware controller <NUM> is configured to, in response to receiving the one or more responses regarding the outstanding requests and in advance of receiving the authority for the cache line, preemptively track the outstanding requests in a requested structure associated with the cache line.

Preemptively tracking the outstanding requests in the requested structure associated with the cache line comprises setting a bit corresponding to the one or more responses. The querying hardware controller <NUM> is configured to receive the authority for the cache line along with receiving access to the cache line. The querying hardware controller <NUM> is configured to, in response to receiving the authority for the cache line along with receiving access to the cache line, receive transfer of a serviced structure associated with the cache line and keep any requested bits preemptively set in the requested structure, wherein the bits were set in advance of receiving the authority. The querying hardware controller <NUM> is configured to, in response to not receiving the authority for the cache line, reset the requested structure.

A serviced structure is received from a controller (e.g., previous hardware controller <NUM>) previously having the authority, the serviced structure being configured to track requests to access the cache line that are observed and have been granted access to the cache line. The requested structure is built (e.g., by the querying hardware controller <NUM>) in anticipation of receiving the authority for a cache line. The one or more responses regarding the outstanding requests correspond to an exclusive fetch request (e.g., made by queried hardware controllers <NUM> for their respective shared caches <NUM>).

The one or more responses regarding the outstanding requests are based on a determination that a responding cache (e.g., queried hardware controllers <NUM> for their respective shared caches <NUM>) has an authority level for the cache line in which the authority level is one level lower than a lowest authority level capable of servicing both a requesting cache (e.g., querying hardware controller <NUM>) and the responding cache (e.g., queried hardware controller <NUM> for its shared cache <NUM>).

They may be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described herein above, or a combination thereof.

In one or more embodiments, the hardware components can include system <NUM> discussed herein including the hardware controllers <NUM>.

Workloads layer <NUM> provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation <NUM>; software development and lifecycle management <NUM>; virtual classroom education delivery <NUM>; data analytics processing <NUM>; transaction processing <NUM>; and workloads and functions <NUM>.

<FIG> is a block diagram of a system <NUM> according to embodiments of the invention. The system <NUM> includes processing circuitry <NUM> used to generate the design <NUM> that is ultimately fabricated into an integrated circuit <NUM> (e.g., chips <NUM>), which use the decentralized hot cache line tracking fairness mechanism. The steps involved in the fabrication of the integrated circuit <NUM> are well-known and briefly described herein. Once the physical layout <NUM> is finalized, based, in part, on being configured to use the decentralized hot cache line tracking fairness mechanism according to embodiments of the invention, the finalized physical layout <NUM> is provided to a foundry. Masks are generated for each layer of the integrated circuit based on the finalized physical layout. Then, the wafer is processed in the sequence of the mask order. The processing includes photolithography and etch. This is further discussed with reference to <FIG>.

<FIG> is a process flow of a method of fabricating the integrated circuit according to exemplary embodiments of the invention. Once the physical design data is obtained, based, in part, on using the decentralized hot cache line tracking fairness mechanism in the chips <NUM>, the integrated circuit <NUM> can be fabricated according to known processes that are generally described with reference to <FIG>. Generally, a wafer with multiple copies of the final design is fabricated and cut (i.e., diced) such that each die is one copy of the integrated circuit <NUM>. At block <NUM>, the processes include fabricating masks for lithography based on the finalized physical layout. At block <NUM>, fabricating the wafer includes using the masks to perform photolithography and etching. Once the wafer is diced, testing and sorting each die is performed, at block <NUM>, to filter out any faulty die.

Various embodiments of the present invention are described herein with reference to the related drawings. Alternative embodiments can be devised without departing from the scope of this invention. Although various connections and positional relationships (e.g., over, below, adjacent, etc.) are set forth between elements in the following description and in the drawings, persons skilled in the art will recognize that many of the positional relationships described herein are orientation-independent when the described functionality is maintained even though the orientation is changed. These connections and/or positional relationships, unless specified otherwise, can be direct or indirect, and the present invention is not intended to be limiting in this respect. Accordingly, a coupling of entities can refer to either a direct or an indirect coupling, and a positional relationship between entities can be a direct or indirect positional relationship. As an example of an indirect positional relationship, references in the present description to forming layer "A" over layer "B" include situations in which one or more intermediate layers (e.g., layer "C") is between layer "A" and layer "B" as long as the relevant characteristics and functionalities of layer "A" and layer "B" are not substantially changed by the intermediate layer(s).

The phrase "selective to," such as, for example, "a first element selective to a second element," means that the first element can be etched and the second element can act as an etch stop.

As used herein, "p-type" refers to the addition of impurities to an intrinsic semiconductor that creates deficiencies of valence electrons. In a silicon-containing substrate, examples of p-type dopants, i.e., impurities, include but are not limited to: boron, aluminum, gallium and indium.

As used herein, "n-type" refers to the addition of impurities that contributes free electrons to an intrinsic semiconductor. In a silicon containing substrate examples of n-type dopants, i.e., impurities, include but are not limited to antimony, arsenic and phosphorus.

As previously noted herein, for the sake of brevity, conventional techniques related to semiconductor device and integrated circuit (IC) fabrication may or may not be described in detail herein. By way of background, however, a more general description of the semiconductor device fabrication processes that can be utilized in implementing one or more embodiments of the present invention will now be provided. Although specific fabrication operations used in implementing one or more embodiments of the present invention can be individually known, the described combination of operations and/or resulting structures of the present invention are unique. Thus, the unique combination of the operations described in connection with the fabrication of a semiconductor device according to the present invention utilize a variety of individually known physical and chemical processes performed on a semiconductor (e.g., silicon) substrate, some of which are described in the immediately following paragraphs.

In general, the various processes used to form a micro-chip that will be packaged into an IC fall into four general categories, namely, film deposition, removal/etching, semiconductor doping and patterning/lithography. Deposition is any process that grows, coats, or otherwise transfers a material onto the wafer. Available technologies include physical vapor deposition (PVD), chemical vapor deposition (CVD), electrochemical deposition (ECD), molecular beam epitaxy (MBE) and more recently, atomic layer deposition (ALD) among others. Removal/etching is any process that removes material from the wafer. Examples include etch processes (either wet or dry), and chemical-mechanical planarization (CMP), and the like. Semiconductor doping is the modification of electrical properties by doping, for example, transistor sources and drains, generally by diffusion and/or by ion implantation. These doping processes are followed by furnace annealing or by rapid thermal annealing (RTA). Annealing serves to activate the implanted dopants. Films of both conductors (e.g., poly-silicon, aluminum, copper, etc.) and insulators (e.g., various forms of silicon dioxide, silicon nitride, etc.) are used to connect and isolate transistors and their components. Selective doping of various regions of the semiconductor substrate allows the conductivity of the substrate to be changed with the application of voltage. By creating structures of these various components, millions of transistors can be built and wired together to form the complex circuitry of a modern microelectronic device.

As noted above, atomic layer etching processes can be used in the present invention for via residue removal, such as can be caused by via misalignment. The atomic layer etch process provide precise etching of metals using a plasma-based approach or an electrochemical approach. The atomic layer etching processes are generally defined by two well-defined, sequential, self-limiting reaction steps that can be independently controlled. The process generally includes passivation followed selective removal of the passivation layer and can be used to remove thin metal layers on the order of nanometers. An exemplary plasma-based approach generally includes a two-step process that generally includes exposing a metal such a copper to chlorine and hydrogen plasmas at low temperature (below <NUM>∘C). This process generates a volatile etch product that minimizes surface contamination. In another example, cyclic exposure to an oxidant and hexafluoroacetylacetone (Hhfac) at an elevated temperature such as at <NUM> can be used to selectively etch a metal such as copper. An exemplary electrochemical approach also can include two steps. A first step includes surface-limited sulfidization of the metal such as copper to form a metal sulfide, e.g., Cu<NUM>S, followed by selective wet etching of the metal sulfide, e.g., etching of Cu<NUM>S in HCl. Atomic layer etching is relatively recent technology and optimization for a specific metal is well within the skill of those in the art. The reactions at the surface provide high selectivity and minimal or no attack of exposed dielectric surfaces.

Semiconductor lithography is the formation of three-dimensional relief images or patterns on the semiconductor substrate for subsequent transfer of the pattern to the substrate. In semiconductor lithography, the patterns are formed by a light sensitive polymer called a photoresist. To build the complex structures that make up a transistor and the many wires that connect the millions of transistors of a circuit, lithography and etch pattern transfer steps are repeated multiple times. Each pattern being printed on the wafer is aligned to the previously formed patterns and slowly the conductors, insulators and selectively doped regions are built up to form the final device.

The photoresist can be formed using conventional deposition techniques such chemical vapor deposition, plasma vapor deposition, sputtering, dip coating, spin-on coating, brushing, spraying and other like deposition techniques can be employed. Following formation of the photoresist, the photoresist is exposed to a desired pattern of radiation such as X-ray radiation, extreme ultraviolet (EUV) radiation, electron beam radiation or the like. Next, the exposed photoresist is developed utilizing a conventional resist development process.

After the development step, the etching step can be performed to transfer the pattern from the patterned photoresist into the interlayer dielectric. The etching step used in forming the at least one opening can include a dry etching process (including, for example, reactive ion etching, ion beam etching, plasma etching or laser ablation), a wet chemical etching process or any combination thereof.

For the sake of brevity, conventional techniques related to making and using aspects of the invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly herein or are omitted entirely without providing the well-known system and/or process details.

In some embodiments, various functions or acts can take place at a given location and/or in connection with the operation of one or more apparatuses or systems. In some embodiments, a portion of a given function or act can be performed at a first device or location, and the remainder of the function or act can be performed at one or more additional devices or locations.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the disclosure. The embodiments were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

The diagrams depicted herein are illustrative. There can be many variations to the diagram or the steps (or operations) described therein without departing from the scope of the disclosure. For instance, the actions can be performed in a differing order or actions can be added, deleted or modified. Also, the term "coupled" describes having a signal path between two elements and does not imply a direct connection between the elements with no intervening elements/connections therebetween. All of these variations are considered a part of the present disclosure.

Additionally, the term "exemplary" is used herein to mean "serving as an example, instance or illustration. " Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" are understood to include any integer number greater than or equal to one, i.e., one, two, three, four, etc. The terms "a plurality" are understood to include any integer number greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include both an indirect "connection" and a direct "connection.

In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instruction by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

Claim 1:
A computer-implemented method comprising:
requesting (<NUM>) authority for a cache line in conjunction with querying for outstanding requests for the cache line;
receiving (<NUM>) one or more responses regarding the outstanding requests for the cache line; and
in response to receiving the one or more responses regarding the outstanding requests and in advance of receiving the authority for the cache line, preemptively tracking (<NUM>) the outstanding requests in a requested vector associated with the cache line, characterised in that a requested vector is a vector that tracks requests to access the cache line in which requests were observed, but access has not been granted, wherein a serviced vector is received from a controller previously having the authority, the serviced vector being configured to track requests to access the cache line that are observed and have been granted access to the cache line.