Abstract:
A wireless communication system comprises a root component connected in a host device and directly coupled to an upstream bus unit, wherein the upstream bus unit is configured to maintain a first configuration space and a copy of a second configuration space, the first configuration space bridge includes at least hot-plug registers specifying at least capabilities and status of a slot of the upstream bus unit; and at least one endpoint component connected to at least one peripheral device and directly coupled to a downstream bus unit that communicates with the host bridge over a distributed link established over a distributed peripheral component interconnect express (PCIe) bus, wherein the downstream bus unit is configured to maintain the second configuration space; the second configuration space includes at least hot-plug registers specifying at least capabilities and status of a slot of the downstream bus unit.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is a continuation of a U.S. patent application Ser. No. 12/887,833 filed on Sep. 22, 2010, now allowed as U.S. Pat. No. 8,443,126. The contents of which are herein incorporated by reference. 
     
    
     TECHNICAL FIELD 
       [0002]    The present invention generally relates to hot-plug processes, and more particularly, for enabling a hot-plug process in a distributed interconnect bus. 
       BACKGROUND 
       [0003]    Peripheral Component Interconnect Express (PCI Express or PCIe) is a high performance, generic and scalable system interconnect for a wide variety of applications ranging from personal computers to embedded applications. PCIe implements a serial, full duplex, multi-lane, point-to-point interconnect, packet-based, and switch based technology. Current versions of PCIe buses allow for a transfer rate of 2.5 Giga bit per second (Gbps) or 5 Gbps, per lane, with up to 32 lanes. The PCIe bus protocol communication is encapsulated in packets. The packetizing and depacketizing data and status-message traffic is handled by the transaction layer of a PCIe port. 
         [0004]    PCIe is used as a motherboard-level interconnect and an expansion board interface for add-in cards. For example, as illustrated in  FIG. 1 , a PCIe bus  100  interconnects the card  110  to the motherboard  120  and further connects expansion cards  130  and  140  through a host  160 . The host  160  is connected to the motherboard  120 . Thus, the PCIe bus  100  allows connectivity between the various cards to a CPU sub system  170  of the computing device. An expansion card is typically inserted into a slot. Usually, the host and/or the motherboard are referred to as PCIe roots and the cards are PCIe endpoints. An internal memory  180  is also coupled to the motherboard  120 . 
         [0005]    As illustrated in  FIG. 2 , the PCIe is a layered protocol bus, consisting of a transaction layer  210 , a data link layer  220 , and a physical layer  230 . The PCIe implements split transactions, i.e., transactions with request and response separated by time, allowing the link to carry other traffic while the target device gathers data for the response. With this aim, the primary function of the transaction layer  210  is to assemble and disassemble transaction layer packets (TLPs). TLPs are used to carry transactions, where each TLP has a unique identifier that enables a response directed at the originator. The data link layer  220  acts as an intermediate between the transaction layer  210  and the physical layer  230  and provides a reliable mechanism for exchanging TLPs. The data link layer  220  implements error checking (known as “LCRC”) and retransmission mechanisms. LCRC and sequencing are applied on received TLPs and if an error is detected, a data link retry is activated. The physical layer  230  consists of an electrical sub-layer  234  and logical sub-layer  232 . The logical sub-layer  232  is a transmitter and receiver pair implementing symbol mapping, serialization, and de-serialization of data. At the electrical sub-layer  234 , each lane utilizes two unidirectional low-voltage differential signaling (LVDS) pairs for transmitting and receiving symbols from the logical sub-block  232 . 
         [0006]    Although the cards are physically connected to the motherboard, the PCIe bus protocol supports a hot-plug process, i.e., replacing system components without shutting down the power. This feature is highly important in blade servers where cards are frequently removed and inserted without powering on/off the server. A hot-plug process is supported in a current implementation of the PCIe buses and controllers. The PCIe standard defines a Slot Capabilities Register, Slot Control Register, and Slot Status Register to support the hot-plug process. These registers and the standard hot-plug process are described in detail in the PCI Express™ base Specification reversion 1.0a, sections 6.7, 7.8.9, 7.8.10, and 7.8.11 published on Apr. 15, 2003, by the PCI-SIG. 
         [0007]    Generally, the Slot Capabilities Register identifies specific capabilities of a PCIe slot. With regard to a hot-plug, the Slot Capabilities Register includes several bits, two of which are: Hot-Plug Surprise bit that indicates that a card in a designated slot can be removed without any prior notification and Hot-Plug Capable bit that indicates that a designated slot is capable of supporting hot-plug operations. The Slot Control Register includes bits, that when set, define if a hot-plug interrupt can be asserted, e.g., if a power fault or a presence of card in the slot is detected. The Slot Status Register provides information about slot specific parameters, e.g., if a power fault or a presence of the card is detected. These registers will be referred to hereinafter as “Hot-Plug Registers” or “HP R”. 
         [0008]    The hot-plug operation as currently implemented includes generating an interrupt when the slot status changes, i.e., from connected to disconnected and vice versa. The operating system (OS) captures the interrupt and allocates/reallocates resources to the inserted/removed device. Typically, when a card is inserted the OS enumerates the card in the order it appears on the PCIe bus. 
         [0009]    Another type of interconnect bus that recently has been developed is a distributed interconnect bus, for example, a distributed PCIe bus. A distributed interconnect bus connects the root to endpoints over a distributed medium, e.g., a wireless medium, a computer network, and the like. The distributed interconnect bus includes two bridges that implement the PCIe protocol. A first bridge is coupled to the root and a second bridge is connected to an endpoint. The first and second bridge communicate over the distributed medium. An example of a distributed bus can be found in US Patent Application Publication No. 2009/0024782, entitled “Distributed Interconnect Bus Apparatus,” assigned to the common assignee, and incorporated herein by reference in its entirety merely for the useful understanding of the background of the invention. 
         [0010]    Due to the physical nature of the distributed medium, the connectivity between the endpoints and the root is unreliable. For example, the wireless link may frequently be idle for a short period of time, and then operational again. Such an event may be treated as a hot-plug event (i.e., a card is removed and inserted). However, the above-referenced standard defines the hot-plug process in a standard PCIe bus where the root and endpoints are physically coupled to the bus, and the connectivity medium is entirely integrated in the computing device (e.g., server or PC). There is no a solution in the related art that provides a hot-plug process in computing systems that include distributed interconnect buses. Data transfers between the root and endpoint(s) are performed by encapsulating the TLPs in data structures compliant with the distributed medium. Further, the signaling definitions and protocol of a standard PCIe do not apply for communication over the distributed medium. Thus, if a bridge coupled to the endpoint generates a hot-plug interrupt signal, the signal cannot be transferred to the root (which informs the OS). 
         [0011]    Therefore, it would be advantageous to provide a solution to support a hot-plug process in distributed interconnect buses. 
       SUMMARY 
       [0012]    Certain embodiments disclosed herein include a method for performing a hot-plug removal process in a distributed peripheral component interconnect express (PCIe) bus. The method comprises receiving a hot-plug interrupt asserted by a shadow configuration space when a distributed link established over the distributed PCIe is unavailable, wherein the shadow configuration space is a copy of a configuration space of at least one endpoint component wirelessly connected to a host device over the distributed PCIe; reading a hot-plug status from the shadow configuration space; writing to the shadow configuration space to turn off at least one peripheral device connected to the at least one endpoint component; and de-allocating resources allocated to a driver of the at least one peripheral. 
         [0013]    Certain embodiments disclosed herein also include a method for performing a hot-plug insertion process in a distributed peripheral component interconnect express (PCIe) bus. The method comprises receiving a hot-plug interrupt asserted by a shadow configuration space when a distributed link established over the distributed PCIe is available, the hot-plug interrupt informing on detection of a hot-plug insertion event; reading a hot-plug status from a configuration space of at least one endpoint component wirelessly connected to a host device over the distributed PCIe, wherein the configuration space is a copy of the shadow configuration space updated to designate the hot-plug insertion event; writing to the configuration space to turn on at least one peripheral device connected to the at least one endpoint component; and asserting a message indicating that the host device and the at least one peripheral can communicate over the distributed PCIe bus. 
         [0014]    Certain embodiments disclosed herein also include a wireless communication system. The system comprises a root component connected in a host device and directly coupled to an upstream bus unit, wherein the upstream bus unit is configured to maintain a first configuration space and a copy of a second configuration space, the first configuration space bridge includes at least hot-plug registers specifying at least capabilities and status of a slot of the upstream bus unit; and at least one endpoint component connected to at least one peripheral device and directly coupled to a downstream bus unit that communicates with the host bridge over a distributed link established over a distributed peripheral component interconnect express (PCIe) bus, wherein the downstream bus unit is configured to maintain the second configuration space; the second configuration space includes at least hot-plug registers specifying at least capabilities and status of a slot of the downstream bus unit. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]    The subject matter that is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention will be apparent from the following detailed description taken in conjunction with the accompanying drawings. 
           [0016]      FIG. 1  is a diagram illustrating a PCIe bus connectivity; 
           [0017]      FIG. 2  is a schematic diagram illustrating the operation of a PCIe protocol; 
           [0018]      FIG. 3  is a block diagram of a distributed interconnect bus apparatus utilized to describe the embodiments of the invention; 
           [0019]      FIG. 4  is a flowchart illustrating a hot-plug removal method in a distributed PCIe bus in accordance with an embodiment of the invention; and 
           [0020]      FIG. 5  a flowchart illustrating a hot-plug insertion method in a distributed PCIe bus in accordance with an embodiment of the invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0021]    The embodiments disclosed by the invention are only examples of the many possible advantageous uses and implementations of the innovative teachings presented herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed inventions. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality. In the drawings, like numerals refer to like parts through several views. 
         [0022]      FIG. 3  shows a non-limiting and exemplary diagram of a distributed interconnect bus apparatus  300  utilized to describe the embodiments of the invention. The apparatus  300  comprises an upstream bus unit  305  that includes a first bridge  310  connected to a root component  320  and a second bridge  330  connected to an endpoint component  340 . The upstream bus unit  305  and bridge  330  communicate over a link  370  which is the distributed medium used to transfer the data between the components  320  and  340 . The medium may be, but is not limited to, a wireless medium, a copper cable, a fiber optic, and so on. 
         [0023]    The interconnect bus apparatus  300  forms a distributed bus for transferring data between remote peripheral devices connected to endpoint component  340  and a motherboard connected to the root component  320 . The transport protocol used to carry data over the link  370  is defined according to the type of the medium. For example, the transport protocol may be, but is not limited to, IEEE 802.11x (Wi-Fi), Ethernet, Infiniband, and the like. 
         [0024]    In accordance with an embodiment of the invention, the root component  320  may be either a PCIe root or a PCIe switch; the endpoint component  340  is a PCIe endpoint, and the bridges  310  and  330  are PCIe bridges. Thus, according to this embodiment the root component  320  and first bridge  310  communicate by means of the PCIe protocol, and the communication between the endpoint component  340  and the second bridge  330  is similar. However, as mentioned above the communication over the link  370  is not compliant with the PCIe protocol. 
         [0025]    In accordance with certain disclosed embodiments, a hot-plug process should be performed when the link  370  transits from an operational state to an idle state (up/down) and vice versa. The root component  320  should receive a hot-plug indication on such events to inform the OS of the changes that occurred at the other end of the bus. The OS correctly handles the hot-plug events to avoid crashes in the computing device and the software which is associated with it. 
         [0026]    The second bridge  330  implements a configuration space  332  that includes the hot-plug registers (HPR). A configuration space includes a setting for performing auto configuration of an endpoint connected to a slot of a bridge. Each of bridges  310  and  330  maintains its configuration space that includes at least the HPR described above, i.e., the capabilities and status of the slot. It should be noted that the structure of the configuration space is similar to all bridges, but the content of which may be different. In an embodiment of the invention, the HPR includes Slot Control Register, Slot Status Register, and Slot Capabilities Register described above. 
         [0027]    In accordance with an embodiment of the invention, the configuration space  332  of the second bridge  330  is shadowed in the upstream bus unit  305 . That is, an exact copy of the configuration space  332  is copied and saved in the upstream bus unit  305 . The shadow configuration space in the upstream bus unit  305  is labeled as 332-S. Whenever the content of the configuration space  332 -S and more specifically the status of the HPR is changed, a newer version of the shadow configuration space  332 -S is saved in the upstream bus unit  305 . In an embodiment of the invention, the shadow configuration space of 332-S is updated using a configuration write packet generated by the second bridge  330 . This provides the root component  320 , and hence the OS an access the most updated configuration space of a device connected to the endpoint component  340 . The first bridge  310  also maintains its configuration space  312 . 
         [0028]    When the link  370  becomes unavailable or unreliable, and transmission over the link  370  (for example, due to high bit error rate) cannot be guaranteed, the shadow configuration space  332 -S asserts a hot-plug interrupt indicating the endpoint component  340  has been removed. Further, the root component  320  updates the HPR in configuration space  332 -S on the status of the hot-plug events. The OS executed over the CPU accesses the HPR to read the hot-plug events and deallocates resources of a device connected to the endpoint component  340 . For example, the values of the “presence detect status” and “presence detect changed” in the Slot Status Register is read. 
         [0029]    It should be appreciated that as the OS can read status information from the shadowed configuration space  332 -S, actions related to a card removal can be performed without any errors. Further, the OS always has a complete and updated status of the distributed PCIe bus. Without having the shadow configuration space  332 -S, the OS would have tried accessing to the second bridge  330  to write/read to the configuration space. However, as the link  370  is idle such information would not be accessible. This would result in an OS error and a crash of the computing device. 
         [0030]    When the link  370  is reconnected and the PCIe connection is reestablished, it is considered a hot-plug event of a hot card insertion. In such an event, the upstream bus unit  305  copies the shadow configuration space  332 -S to the bridge  330 , updates the HPR in the configuration space  312  of the first bridge  310  and the shadow configuration space  332 -S. In addition, the shadow configuration space  332 -S asserts a hot-plug interrupt to the OS. Specifically, a signal is transferred from the configuration space  332 -S to the root component  320  through the upstream bus unit  305 . It should be appreciated that copying the shadow configuration space  332 -S provides coherency between the two bridges and allows the OS to resume communication with the endpoint component  340  without the need to reconfigure the fields of the configuration space  332  of the second bridge  330 . 
         [0031]    It should be noted that for the sake of simplicity and without limiting the scope of the invention, the operation of the hot-plug process has been described with a reference to an embodiment where only one endpoint component is connected to the second bridge  330 . The teachings of the invention are similar when multiple endpoint components are connected. In such an embodiment, the configuration space respective of each bridge that supports each endpoint component is shadowed to the shadow configuration space  332 -S. 
         [0032]      FIG. 4  shows an exemplary and non-limiting flowchart  400  illustrating the process of a hot-plug removal in a distributed PCIe bus in accordance with an embodiment of the invention. A hot-plug removal is performed when the data cannot be transferred over the link of the distributed medium (e.g., link  370 ). At S 410 , as long as the link is available, a second bridge (e.g., bridge  330 ) is connected to an endpoint of the distributed PCIe, and the configuration space of the second bridge is shadowed to an upstream bus unit  305  connected to the root component  320 . That is, the configuration space from the second bridge is copied to the upstream bus unit  305  forming a shadow configuration at the upstream bus unit  305  and any subsequent changes are written to the shadow configuration space. As mentioned above, the configuration space includes HPR that provide at least the bridge&#39;s slot status and capabilities. 
         [0033]    At S 420 , a check is made to determine if data can be transferred over the link, i.e., if the link is available. In an embodiment of the invention, this check is performed by the root component. If S 420  results with an affirmative answer, execution returns to S 410 ; otherwise, execution continues with S 430 , where a hot-plug interrupt is asserted by the shadow configuration space  332 -S. The interrupt indicates a hot card removal event. At S 440 , the HPR in the shadow configuration space  332 -S is updated with the status of the hot-plug event. At S 450 , the OS performs a hot-plug removal process during which, the OS reads and writes to the shadow configuration space  332 -S. The process executed by the OS is based on the type of the OS. For example, a hot-plug process performed by Windows based OS includes, in part, reading the hot-plug status bits from the Slot Status Register, requesting a plug-and-play system to eject the device connected to the endpoint, querying drivers for functions of the device, unloading the drivers of the device, writing to the shadow configuration space to turn off the device, and de-allocating resources used by driver(s). 
         [0034]    As can be understood from the above example, if the OS cannot access the configuration space of the second bridge, a system error would be generated. Thus, providing an updated shadow configuration space allows performing a hot-plug removal in a distributed PCIe bus. 
         [0035]      FIG. 5  shows an exemplary and non-limiting flowchart  500  illustrating the hot-plug insertion method in a distributed PCIe bus in accordance with an embodiment of the invention. The hot-plug insertion process is initiated when the data can be transferred over the link of the distributed medium (e.g., link  370 ), and the link is established. At S 510 , upon establishment of the distributed link, the second bridge  330  and the endpoint  340  connected thereon are reset. Then, at S 520 , execution waits for the initialization of the PCIe PHY and link layers of the second bridge and endpoint. 
         [0036]    At S 530 , the shadow configuration space  332 -S is copied from the upstream bus unit  305  connected to the root to the second bridge  330 . At S 540 , each bridge updates the HPR to represent a hot-plug insertion event. At S 550 , a hot-plug interrupt is asserted by the shadow configuration space  332 -S informing the OS of the hot-plug event. To start the communication over the distributed PCIe bus, the bridges  310  and  320  wait for the OS to complete the handling of the interrupt. At S 560 , the OS performs a hot-plug insertion process during which it reads and writes to the configuration space of the second bridge (i.e., to a recent copy of the shadow configuration space). The process executed by the OS is based on the type of the OS. For example, a hot-plug insertion process in a Windows based OS includes, in part, reading hot-plug status bits from the Slot Status Register to determine the type of the hot-plug event, enumerating the PCIe bus to include the endpoint, reading from the configuration space  332  of the second bridge  330  in order to identify the device connected to the endpoint and perform the proper initialization actions (e.g., memory allocation, driver loading, etc.), writing to configuration space at the second bridge to turn on the device, and asserting a message that communication with the device connected to the endpoint can start. 
         [0037]    As can be understood from above example, the ability to provide an updated copy of the configuration space at the second bridge allows immediately establishing the distributed PCIe bus. Thus, there is no need to re-configure the configuration space at the second bridge according to the device specification. It should be noted that as the process  500  is performed when the distributed link backs up, it is assumed that the device is still connected to the endpoint. 
         [0038]    The various embodiments disclosed herein can be implemented as hardware, firmware, software or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit, a non-transitory computer readable medium, or a non-transitory machine-readable storage medium that can be in a form of a digital circuit, an analogy circuit, a magnetic medium, or combination thereof. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPUs”), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit. Furthermore, a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal. 
         [0039]    The foregoing detailed description has set forth a few of a few of the many forms that different embodiments of the invention can take. It is intended that the foregoing detailed description be understood as an illustration of selected forms that the invention can take and not as a limitation to the definition of the invention. It is only the claims, including all equivalents that are intended to define the scope of this invention.