Patent Description:
Computing devices have become increasingly common in modern society and have evolved from large cumbersome devices that performed only the most rudimentary functions to small, portable devices that operate as full multimedia platforms while providing telephony functions and the like. The increase in the functionality of such devices is attributable in part to the continued progress in shrinking more and more transistors into an individual integrated circuit (IC) to allow improved processing power in an increasingly small space. While ICs have continued to shrink in size, most computing devices continue to employ multiple ICs within the computing device to effectuate different functions. Such plural ICs necessitate that the ICs be able to communicate with one another. Various protocols and standards have been promulgated that set forth ways in which the ICs may communicate with one another.

One popular protocol is the protocol set forth in the Peripheral Component Interconnect (PCI) standard. PCI has gone through multiple permutations, but as of this writing, one of the more popular versions is PCI express (PCIe). One popular use of PCIe is for memory read/write commands from a peripheral element to a memory element through a host device. Such read/write commands are typically associated with an address within the memory to which the read/write command applies. Early iterations required that the peripheral use a physical address of the memory. As memory sizes increased, the address range exceeded the addressing capability of the peripherals, and the concept of a virtual address was introduced.

While the use of virtual addresses helped allow the peripheral to use all of the memory, the use of virtual addresses imposed a processing burden on the host as the virtual address had to be translated back to a physical address. Such processing also added latency to the read/write process. Accordingly, address translation was pushed back to the peripheral through the use of address translation services (ATS) and the use of an address translation cache (ATC) at the peripheral. The peripheral would use virtual addresses internally, but use the ATC to find a physical address. The physical address located in the ATC would be sent with the read/write command from the peripheral to the host for use with the memory.

The ATC is populated by a memory management unit in the host. Periodically, the host may determine that a particular address translation has aged out, expired, or is otherwise no longer to be used. The traditional technique through which use of the expired address was handled was through an invalidation command from the host to the peripheral. Up to thirty-two (<NUM>) such commands could be pending at one time. When the peripheral finished certain clean-up tasks associated with the invalidation command, the peripheral would acknowledge the invalidation. These acknowledgements imposed a signaling overhead on the communication between the peripheral and the host. Accordingly, there remains a need to improve the address translation invalidation process.

Aspects disclosed in the detailed description include systems and methods for fast invalidation in Peripheral Component Interconnect (PCI) express (PCIe) address translation services (ATS). Exemplary aspects of the present disclosure initially utilize a fast invalidation request to alert endpoints that an address is being invalidated. Instead of requiring immediate action on the part of the endpoints, a host allows the endpoints to accumulate such invalidation requests until a fast invalidation synchronization command is sent from the host to the endpoints. On receipt of such a synchronization command, the endpoints flush through any residual read/write commands associated with any invalidated address and delete any associated address entries in an address translation cache (ATC). After these tasks are completed, each endpoint may send a synchronization complete acknowledgement to the host. Exemplary aspects of the present disclosure further provide a tag with a numerically incrementing identifier for each invalidation request. When a synchronization command is sent, the synchronization command includes a current value for the incrementing identifier. If an endpoint has not received requests having each of the tags between the last synchronized tag number and the tag in the synchronization command, the endpoint may take action to secure a missing invalidation request. Use of the fast invalidation commands, the synchronization commands, and the tags helps reduce overhead in the system as well as helps insure that invalidation commands are received and acted on so that the legacy address translations do not remain to clutter up the communication between the host and endpoints.

In this regard in one aspect, a method of communicating over a communication link is disclosed. The method includes inserting a tag including an incrementing identifier into each of a series of fast invalidation request messages relating to an address translation table. The method also includes detecting a missing or corrupted fast invalidation request message based on a missing incrementing identifier. In another aspect, a communication system is disclosed. The communication system includes a communication link. The communication link also includes a root complex. The root complex includes a link interface configured to be coupled to the communication link. The root complex also includes a control system. The control system is configured insert a tag including an incrementing identifier into each of a series of fast invalidation request messages relating to an address translation table. The control system is also configured to send the series of fast invalidation request messages over the communication link. The communication system also includes an endpoint. The endpoint includes an endpoint link interface configured to be coupled to the communication link. The endpoint also includes an endpoint control system. The endpoint control system is configured to receive the series of fast invalidation request messages over the communication link. The endpoint control system is also configured to detect a missing or corrupted fast invalidation request message within the series of fast invalidation request messages based on a missing incrementing identifier.

In another aspect, an apparatus is disclosed. The apparatus includes a root complex. The root complex includes a link interface configured to be coupled to a communication link. The root complex also includes a control system. The control system is configured to insert a tag including an incrementing identifier into each of a series of fast invalidation request messages relating to an address translation table. The control system is also configured to send the series of fast invalidation request messages over the communication link.

In another aspect, an apparatus is disclosed. The apparatus includes an endpoint. The endpoint includes an endpoint link interface configured to be coupled to a communication link. The endpoint also includes an endpoint control system. The endpoint control system is configured to receive a series of fast invalidation request messages over the communication link. The endpoint control system is also configured to detect a missing or corrupted fast invalidation request message within the series of fast invalidation request messages based on a missing incrementing identifier.

In another aspect, a method for communicating over a communication link is disclosed. The method includes inserting a tag including an incrementing identifier into each of a series of fast invalidation request messages relating to an address translation table. The method also includes sending the series of fast invalidation request messages over the communication link.

In another aspect, a method for communicating over a communication link is disclosed. The method includes receiving a series of fast invalidation request messages over the communication link. The method also includes detecting a missing or corrupted fast invalidation request message within the series of fast invalidation request messages based on a missing incrementing identifier.

Before addressing particulars of the present disclosure, a brief discussion of ATS is provided in conventional PCIe systems with reference to <FIG> followed by a discussion of a multi-endpoint PCIe system with reference to <FIG>. Discussion of exemplary aspects of the present disclosure begins below with reference to <FIG>.

In this regard, <FIG> is a block diagram of a computing device <NUM> having a first IC <NUM> coupled to a second IC <NUM> through a PCIe link <NUM>. While the PCIe connections described herein (both for <FIG> and for the other Figures) may act as a bus, the nomenclature of the PCIe standard and the general point-to-point nature makes it more appropriate to refer to the connections as links. However, it should be appreciated that such a connection may also be referred to as a bus. The first IC <NUM> may be an endpoint as that term is used by the PCle specification and may functionally correspond to a modem, an accelerator, or the like. The first IC <NUM> may include a link interface <NUM> which may correspond to a physical layer (PHY) configured to communicate over the PCIe link <NUM>. The first IC <NUM> may further include a control system (referenced as CS in the drawings) <NUM>.

With continued reference to <FIG>, the second IC <NUM> may be a host as that term is used by the PCIe specification and may functionally be an application processor, a system on a chip (SoC), or the like. As part of being a host, the second IC <NUM> includes logic elements that form a root complex <NUM> which includes a PHY <NUM> configured to communicate over the PCIe link <NUM>. The second IC <NUM> further includes an input/output memory management unit (IOMMU) <NUM> that communicates with a memory module <NUM> over an internal coherent bus <NUM>. The memory module <NUM> may contain a cache memory <NUM>, a memory controller <NUM>, and a memory element <NUM>. The second IC <NUM> may further include a control system <NUM> which may be a processing core or the like.

With continued reference to <FIG>, in the original PCIe systems such as the computing device <NUM>, the first IC <NUM> may initiate a read/write command to the memory element <NUM> through the PCIe link <NUM>. To make sure that the appropriate portion of the memory element <NUM> was addressed, the first IC <NUM> would use a physical address corresponding directly to the address of the relevant portion of the memory element <NUM>. As such, in the early systems there was no address translation.

As the size of the memory element <NUM> increased, and as the inconvenience of not being able to fragment use of the memory element <NUM> increased, the ICIe specification allowed virtual addresses to be used.

In this regard, <FIG> illustrates a computing device <NUM> that is similar to the computing device <NUM> of <FIG>. The computing device <NUM> has a first IC <NUM> coupled to a second IC <NUM> through a PCIe link <NUM>. The first IC <NUM> may be an endpoint as that term is used by the PCIe specification and may functionally correspond to a modem, an accelerator, or the like. The first IC <NUM> may include a link interface <NUM> which may correspond to a PHY configured to communicate over the PCIe link <NUM>. The first IC <NUM> may further include a control system (referenced as CS in the drawings) <NUM>.

With continued reference to <FIG>, the second IC <NUM> may be a host as that term is used by the PCIe specification and may functionally be an application processor, a SoC, or the like. As part of being a host, the second IC <NUM> includes logic elements that form a root complex <NUM> which includes a link interface or PHY <NUM> configured to communicate over the PCIe link <NUM>. The second IC <NUM> further includes an IOMMU <NUM> that communicates with a memory module <NUM> over an internal coherent bus <NUM>. The memory module <NUM> may contain a cache memory <NUM>, a memory controller <NUM>, and a memory element <NUM>. The second IC <NUM> may further include a control system <NUM> corresponding to a processing core or the like.

With continued reference to <FIG>, the first IC <NUM> may initiate a read/write command to the memory element <NUM> through the PCle link <NUM>. However, in contrast to the computing device <NUM> of <FIG>, the first IC <NUM> may use a virtual address when it addresses the memory element <NUM>. The virtual address is assigned by the IOMMU <NUM>, and the IOMMU <NUM> receives the read/write command with the virtual address. On receipt, the IOMMU <NUM> translates the virtual address to a physical address before sending the request with the physical address to the memory module <NUM> over the internal coherent bus <NUM>. While this approach allows use of larger memory elements and allows for fragmentary memory use, this approach also creates a bottleneck at the IOMMU <NUM> as multiple endpoints may require address translation concurrently.

To alleviate the bottleneck at the IOMMU <NUM>, further evolutions of PCIe moved the address translation to an ATC in the endpoint. This arrangement is illustrated in <FIG>. In this regard, <FIG> illustrates a computing device <NUM> that is similar to the computing devices <NUM> and <NUM> of <FIG> and <FIG>. Specifically, the computing device <NUM> has a first IC <NUM> coupled to a second IC <NUM> through a PCIe link <NUM>. The first IC <NUM> may be an endpoint as that term is used by the PCIe specification and may functionally correspond to a modem, an accelerator, or the like. The first IC <NUM> may include a link interface <NUM> which may correspond to a PHY configured to communicate over the PCle link <NUM>. The first IC <NUM> may further include a control system (referenced as CS in the drawings) <NUM>.

With continued reference to <FIG>, the second IC <NUM> may be a host as that term is used by the PCIe specification and may functionally be an application processor, a SoC, or the like. As part of being a host, the second IC <NUM> includes logic elements that form a root complex <NUM> which includes a link interface or PHY <NUM> configured to communicate over the PCIe link <NUM>. The second IC <NUM> further includes an IOMMU <NUM> that communicates with a memory module <NUM> over an internal coherent bus <NUM>. The memory module <NUM> may contain a cache memory <NUM>, a memory controller <NUM>, and a memory element <NUM>. The second IC <NUM> may include a control system <NUM> which may be a processing core or the like.

In contrast to the computing device <NUM> of <FIG> or the computing device <NUM> of <FIG>, the first IC <NUM> of the computing device <NUM> includes an ATC <NUM>, which may include necessary and sufficient memory elements to store a look-up table or the like to hold address translations. In particular, when the first IC <NUM> has an initial memory access request (i.e., a read or write), the first IC <NUM> communicates with the IOMMU <NUM> through a translation request. The IOMMU <NUM> assigns a physical address to virtual address translation and communicates this translation to the first IC <NUM> for storage in the ATC <NUM>. Then when the first IC <NUM> has a memory access request, the first IC <NUM> initially uses the virtual address, but sends it to the ATC <NUM>. The ATC <NUM> translates the virtual address to the physical address provided by the IOMMU <NUM> and the first IC <NUM> sends the memory access request with the physical address to the second IC <NUM>. On receipt, the IOMMU <NUM> sends the request with the physical address to the memory module <NUM> over the internal coherent bus <NUM>.

After some period of non-use or expiration of a predefined amount of time or other event, the IOMMU <NUM> or translation agent within the IOMMU <NUM> may determine that the initially provided virtual address to physical address mapping has expired. Accordingly, the IOMMU <NUM> causes the root complex <NUM> to send an invalidation request message to the first IC <NUM>. In traditional PCIe systems, receipt of an invalidation request message caused the first IC <NUM> to delete the entry in the ATC <NUM>, making sure that all pending non-posted transactions that use the invalidated address are completed, and flush all pending posted transactions which utilized the invalidated address. On completion of these activities, the first IC <NUM> would send an invalidation completion message to the second IC <NUM>. The invalidation completion message may be sent on each traffic class for which the invalidated address was used. The root complex <NUM><NUM> then collects all invalidation completion messages for each traffic class, waits for all previous posted and non-posted transactions using the invalidated address to complete, and indicates to software that the invalidation completion messages were received and executed. Up to thirty-two (<NUM>) invalidation request messages may be active and pending at any given time. The sending and receiving of the invalidation request and the invalidation completion messages imposes a signaling overhead penalty on the PCle link <NUM>.

<FIG> provides an exemplary PCIe system <NUM> where a root complex <NUM> in a host <NUM> may communicate to multiple endpoints <NUM>(<NUM>)-<NUM>(N) directly and to additional endpoints <NUM>(<NUM>)-<NUM>(M) through a switch <NUM>. Each connection between respective elements is a high-speed serial point-to-point PCIe link <NUM>(<NUM>)-<NUM>(M+N+<NUM>).

Recent proposals contemplate a fast invalidation request. As before, software in the root complex <NUM> of <FIG> may determine that it is appropriate to invalidate an earlier translation entry. The root complex <NUM> sends a fast invalidation request message to the first IC <NUM>. When the first IC <NUM> receives the fast invalidation request message, the first IC <NUM> performs several action, including: invalidate the entry in the ATC <NUM> and cease using the invalidated address; continue processing all pending non-posted transactions using the invalidated address to completion; and flush all pending posted transactions using the invalidated address. However, the first IC <NUM> does not need to send an invalidation completion message.

Under the fast invalidation proposal, when the software in the root complex <NUM> wants to flush all previous fast invalidation request messages, the root complex <NUM> sends a fast invalidation synchronization message to the first IC <NUM>. The first IC <NUM> completes the tasks that were initiated by receipt of the fast invalidation request and also waits for all current non-posted transactions utilizing invalidated addresses to complete (if they have not already completed). When the last transaction completes, the first IC <NUM> sends a fast invalidation synchronization completion message to the root complex <NUM>. The root complex <NUM> waits for all previous posted and non-posted transactions to complete or flush before notifying the software that the fast invalidation synchronization completion message was received and executed.

Current proposals assume that the endpoints will receive all fast invalidation requests, and thus, when the endpoints respond to the fast invalidation synchronization message, the root complex assumes that all fast invalidation messages have been processed when the endpoint sends a fast invalidation synchronization complete message. However, if an endpoint did not receive a fast invalidation request message, then it will not perform the clean-up tasks for one or more invalidated addresses corresponding to the messages not received and the endpoint will continue to have transactions that use the invalidated address.

Exemplary aspects of the present disclosure introduce a tag that has an incrementing identifier that helps identify corrupted or missing fast invalidation request messages. When the endpoint detects a missing tag, the endpoint may take remedial action.

<FIG> is a simplified signal flow diagram <NUM> of fast invalidation request messages between the root complex <NUM> of <FIG> and an endpoint such as endpoint <NUM>(M) through the switch <NUM> with fast invalidation synchronization (sync) messages, all having tags with incrementing identifiers (sometimes referred to as incremental tags) therein for missing message detection. <FIG> assumes a system where there are no lost or corrupted signals between the root complex <NUM> and the endpoint <NUM>(M). Thus, the root complex <NUM> sends a fast invalidation synchronization message <NUM> having a tag with an incrementing identifier (e.g., S-Tag = <NUM>). This causes the endpoint <NUM>(M) to complete all its clean-up tasks (e.g., clearing the ATC, pushing all non-published and published tasks to the root complex, and the like) for any previously received invalidation request messages. On completion of the clean-up tasks for all outstanding fast invalidation request messages since the last synchronization (or if there has been no previous synchronization, then from S-Tag = <NUM>), the endpoint <NUM>(M) sends a fast invalidation synchronization completion message <NUM> using the same incrementing identifier as was used in the synchronization message <NUM> (i.e., S-Tag = <NUM>). Subsequently, the root complex <NUM> sends an additional fast invalidation request message <NUM> with a tag having an incremented identifier (e.g., S-Tag = <NUM>), a fast invalidation request message <NUM> with an incremented identifier (e.g., S-Tag = <NUM>), and a fast invalidation request message <NUM> with an incremented identifier (e.g., S-Tag = <NUM>). The root complex <NUM> then sends a fast invalidation synchronization message <NUM> using the last identifier (e.g., S-Tag = <NUM>). In the signal flow <NUM> all the invalidation request messages are received at the endpoint <NUM>(M), and the endpoint <NUM>(M) sends a fast invalidation synchronization completion message <NUM> with an identifier corresponding to the synchronization message (e.g., S-Tag = <NUM>). Note that these identifiers are in tags in the messages.

In exemplary aspects of the present disclosure, the endpoint <NUM>(M) may store identifiers from the tags in registers, set a flag that they have been received, increment a counter, or otherwise determine that a given S-Tag has been received.

While the signal flow <NUM> assumes that all invalidation request messages were received, exemplary aspects of the present disclosure provide a mechanism for the endpoint <NUM>(M) to indicate to the root complex <NUM> that not all of the invalidation requests were received. In this regard, <FIG> is a simplified signal flow <NUM> of fast invalidation messages where one of the fast invalidation messages is lost or corrupted before arriving at an endpoint, thereby precluding a fast invalidation synchronization message from the endpoint and triggering resending of all intervening fast invalidation request messages. The signal flow <NUM> begins much like the signal flow <NUM> of <FIG> with the messages <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> sent by the root complex <NUM>. However, the request message <NUM> is not received by the endpoint <NUM>(M). Thus, when the synchronization message <NUM> is sent, the endpoint <NUM>(M) determines that not every message was received. For example, a register may show that only two requests were received but the incremented identifier shows that three messages were sent. A comparison of identifiers to flags in a register may be made. Regardless of how the determination is made, the endpoint <NUM>(M) does not send a synchronization completion message (e.g., the synchronization completion message <NUM> of the signal flow <NUM>). After a predetermined amount of time without having received the synchronization completion message, the root complex <NUM> resends the fast invalidation request messages with incrementing identifiers in tags in the form of messages <NUM>, <NUM>, and <NUM> along with another synchronization message <NUM>. For the sake of example, the messages <NUM>, <NUM>, and <NUM> were received by the endpoint <NUM>(M), and the endpoint <NUM>(M) sends a synchronization completion message <NUM> with an identifier corresponding to the last incremented identifier (S-Tag = <NUM>). If one or more of the messages <NUM>, <NUM>, and <NUM> were not received, another time out would occur.

The signal flow <NUM> where every fast invalidation request message is resent imposes a signaling penalty on the computing device. Additionally, the period spent waiting for a time out after sending the synchronization message <NUM> slows operation. Accordingly, an alternate exemplary aspect of the present disclosure sends an alert on detection of the missing invalidation request message. In this regard, <FIG> is a simplified signal flow <NUM> of fast invalidation messages where one of the fast invalidation messages is lost or corrupted before arriving at an endpoint <NUM>(M) causing the endpoint <NUM>(M) to signal loss of the message to the root complex <NUM>. The signal flow <NUM> begins identically to the signal flow <NUM> of <FIG> with the messages <NUM>, <NUM>, <NUM>, <NUM> and <NUM>. However, as with the signal flow <NUM> of <FIG>, the message <NUM> is lost or corrupted before reaching the endpoint <NUM>(M). On receipt of the message <NUM> with the incremented identifier in a tag more than one higher than the last identifier received (e.g., S-Tag = <NUM> in the message <NUM> reflects the missing S-Tag <NUM> from the message <NUM>), the endpoint <NUM>(M) generates and sends an error message <NUM> to the root complex <NUM>. In an exemplary aspect, the error message <NUM> indicates which message was missing through use of the identifier in a tag. In response to receiving the error message <NUM>, the root complex <NUM> resends at least the missing fast invalidation request message <NUM> with the known missing identifier (S-Tag = <NUM>), and may optionally resend the fast invalidation request message <NUM> (S-Tag = <NUM>). The root complex <NUM> then sends the fast invalidation synchronization message (S-Tag = <NUM>) <NUM>, and the endpoint <NUM>(M), having now received all the invalidation request messages, sends the synchronization completion message <NUM>.

As an alternative to the signal flow <NUM>, the endpoint <NUM>(M) may wait until the synchronization message <NUM> before sending the error message. This approach is similar to the signal flow <NUM> of <FIG>, but does not wait for a time out to resend messages. In this regard, <FIG> is a simplified signal flow <NUM> of fast invalidation messages where one of the fast invalidation messages is lost or corrupted before arriving at an endpoint causing the endpoint to signal that it is missing a specific message after receipt of a fast invalidation synchronization message from the root complex. The signal flow <NUM> begins like the signal flow <NUM>, with the messages <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>. However, for example, the message <NUM> is not received by the endpoint <NUM>(M). On receipt of the synchronization message <NUM>, the endpoint <NUM>(M) sends an error message <NUM>. The error message <NUM> may indicate which identifiers have not been received since the last synchronization message <NUM>. The root complex <NUM> then, depending on what information was in the error message <NUM>, may resend the fast invalidation requests for all the requests since the last successful synchronization as messages <NUM>, <NUM>, and <NUM>. Alternatively, if the error message <NUM> indicates with sufficient precision which requests have not been received then only that request (e.g., the message <NUM>) is resent. Or, as another option, the missing message and all subsequent messages are resent (e.g., messages <NUM> and <NUM>). Another synchronization message <NUM> is sent and the endpoint <NUM>(M) now sends the synchronization completion message <NUM>.

While there are numerous ways that the tag having the incrementing identifier can be implemented, an exemplary PCIe frame having an incremental identifier therein is provided with reference to <FIG>. In particular, <FIG> illustrates a PCIe packet <NUM> with a field <NUM> into which the identifier forming the tag of the present disclosure may be placed. Note that while the field <NUM> is considered appropriate for insertion of the identifier, other locations could be used without departing from the scope of the present disclosure.

<FIG> provide flowcharts corresponding to the signal flows of <FIG>, respectively. In this regard, <FIG> illustrates a process 1000A where the RC initiates ATS and populates the ATC (block <NUM>). The RC sends messages to one or more EPs (block <NUM>). The RC sends a first fast invalidation request message with S-Tag = <NUM> (block <NUM>). The RC then sends a next fast invalidation request message with the S-Tag (i.e., the incrementable identifier) incremented (block <NUM>). The RC determines whether it is time to synchronize (block <NUM>). As long as the answer is no, the RC will continue to send next fast invalidation request messages, each with an incremented S-Tag.

With continued reference to <FIG>, once block <NUM> is answered yes, then the RC sends a fast invalidation sync message with an S-Tag equal to the last message S-Tag value (block <NUM>) (message <NUM>). The EP completes clean up and sends the fast invalidation synchronization completion message with an S-Tag equal to the S-Tag of the synchronization message (block <NUM>) (message <NUM>). The RC sends a next fast invalidation request message with the S-Tag incremented (block <NUM>) (messages <NUM>, <NUM>, and <NUM>), while checking to see if it is time to synchronize (block <NUM>). If the answer is no at block <NUM>, it is possible that the EP does not receive one or more of the request messages (block <NUM>). However, the RC still sends more fast invalidation request messages. Once block <NUM> is answered affirmatively, the RC sends the fast invalidation sync message with an S-Tag equal to the last message S-Tag value (block <NUM>) (message <NUM>). The EP does not respond and a time out occurs (block <NUM>). After the RC detects the time out, the RC resends the messages (block <NUM>).

The process 1000B of <FIG> starts similarly up through block <NUM> after which the endpoint may not receive a request (block <NUM>). The endpoint compares the received S-Tag to the expected incremented S-Tag to determine if the requests have been received or if there is a discontinuity in the S-Tags (block <NUM>). When there is a discontinuity, the EP sends an error message (block <NUM>) that indicates that one or more requests have not been received. The root complex then resends one or more requests (block <NUM>). As noted above, depending on the details of the error message and whether the root complex has sent further requests (and whether the endpoint can store requests received out of order), the number of resent requests may vary.

Similarly, the process 1000C of <FIG> operates similar to the process 1000A of <FIG> until block <NUM> after which time, the endpoint sends an error message (block <NUM>). Depending on the details of the error message, the root complex resends the appropriate messages (block <NUM>).

The fast invalidation in PCIe ATS according to aspects disclosed herein may be provided in or integrated into any processor-based device. Examples, without limitation, include a set top box, an entertainment unit, a navigation device, a communications device, a fixed location data unit, a mobile location data unit, a global positioning system (GPS) device, a mobile phone, a cellular phone, a smart phone, a session initiation protocol (SIP) phone, a tablet, a phablet, a server, a computer, a portable computer, a mobile computing device, a wearable computing device (e.g., a smart watch, a health or fitness tracker, eyewear, etc.), a desktop computer, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio, a satellite radio, a music player, a digital music player, a portable music player, a digital video player, a video player, a digital video disc (DVD) player, a portable digital video player, an automobile, a vehicle component, avionics systems, a drone, and a multicopter.

In this regard, <FIG> is system-level block diagram of an exemplary mobile terminal <NUM> such as a smart phone, mobile computing device tablet, or the like. While a mobile terminal is particularly contemplated as being capable of benefiting from exemplary aspects of the present disclosure, it should be appreciated that the present disclosure is not so limited and may be useful in any system having a PCIe link.

With continued reference to <FIG>, the mobile terminal <NUM> includes an application processor <NUM><NUM> (sometimes referred to as a host) that communicates with a mass storage element <NUM> through a universal flash storage (UFS) bus <NUM>. The application processor <NUM> may further be connected to a display <NUM> through a display serial interface (DSI) bus <NUM> and a camera <NUM> through a camera serial interface (CSI) bus <NUM>. Various audio elements such as a microphone <NUM>, a speaker <NUM>, and an audio codec <NUM> may be coupled to the application processor <NUM> through a serial low power interchip multimedia bus (SLIMbus) <NUM>. Additionally, the audio elements may communicate with each other through a SOUNDWIRE NEXT bus <NUM>. A modem <NUM> may also be coupled to the SLIMbus <NUM>. The modem <NUM> may further be connected to the application processor <NUM> through a peripheral component interconnect (PCI) or PCI express (PCIe) bus <NUM> and/or a system power management interface (SPMI) bus <NUM>. The PCIe bus <NUM> may benefit from the exemplary aspects of the present disclosure provided above.

With continued reference to <FIG>, the SPMI bus <NUM> may also be coupled to a local area network (LAN) integrated circuit (IC) (LAN IC) <NUM>, a power management integrated circuit (PMIC) <NUM>, a companion integrated circuit (sometimes referred to as a bridge chip) <NUM>, and a radio frequency integrated circuit (RFIC) <NUM>. It should be appreciated that separate PCI buses <NUM> and <NUM> may also couple the application processor <NUM> to the companion integrated circuit <NUM> and the LAN IC <NUM>. The application processor <NUM> may further be connected to sensors <NUM> through a sensor bus <NUM>. The modem <NUM> and the RFIC <NUM> may communicate using a bus <NUM>.

With continued reference to <FIG>, the RFIC <NUM> may couple to one or more radio frequency front end (RFFE) elements, such as an antenna tuner <NUM>, a switch <NUM>, and a power amplifier <NUM> through an RFFE bus <NUM>. Additionally, the RFIC <NUM> may couple to an envelope tracking power supply (ETPS) <NUM> through a bus <NUM>, and the ETPS <NUM> may communicate with the power amplifier <NUM>. Collectively, the RFFE elements, including the RFIC <NUM>, may be considered an RFFE system <NUM>. It should be appreciated that the RFFE bus <NUM> may be formed from a clock line and a data line (not illustrated).

Those of skill in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithms described in connection with the aspects disclosed herein may be implemented as electronic hardware, instructions stored in memory or in another computer readable medium and executed by a processor or other processing device, or combinations of both. The devices described herein may be employed in any circuit, hardware component, IC, or IC chip, as examples. Memory disclosed herein may be any type and size of memory and may be configured to store any type of information desired. To clearly illustrate this interchangeability, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. How such functionality is implemented depends upon the particular application, design choices, and/or design constraints imposed on the overall system.

It is also noted that the operational steps described in any of the exemplary aspects herein are described to provide examples and discussion. Additionally, one or more operational steps discussed in the exemplary aspects may be combined. It is to be understood that the operational steps illustrated in the flowchart diagrams may be subject to numerous different modifications as will be readily apparent to one of skill in the art. Those of skill in the art will also understand that information and signals may be represented using any of a variety of different technologies and techniques.

Claim 1:
A method for communicating over a communication link, comprising:
inserting a tag comprising an incrementing identifier into each of a series of fast invalidation request messages relating to an address translation table;
sending (<NUM>,<NUM>) the series of fast invalidation request messages over the communication link; and characterized by:
sending (<NUM>, <NUM>) a fast invalidation synchronisation message with a tag equal to the tag inserted in the last of the series of fast invalidation request messages sent over the communication link.