I/O device sharing system and I/O device sharing method

An I/O device sharing system characterized by comprising: an I/O device (50) shared by a plurality of hosts (20-1 to 20-N); a system manager (10) which sets the I/O device (50); a virtual bridge (40) which virtualizes the I/O device (50); and a network (3) which connects the I/O device (50), the system manager (10), the plurality of hosts (20-1 to 20-N) and the virtual bridge (40) to each other, wherein the virtual bridge (40) includes a connection virtualization unit (41) by which it is detected that an address setting of a plurality of virtual functions provided in the I/O device (50) that is set by the system manager (10) is performed, the virtual function is enabled, or both of them are performed and each host is permitted to access each virtual function.

This application is a National Stage Entry of PCT/JP2013/002316 filed on Apr. 3, 2013, which claims priority from Japanese Patent Application 2012-087888 filed on Apr. 6, 2012, the contents of all of which are incorporated herein by reference, in their entirety.

TECHNICAL FIELD

The present invention relates to an Input/Output (I/O) device sharing system and an I/O device sharing method and in particular, relates to an I/O device sharing system which can simplify management of an I/O device and an I/O device sharing method.

BACKGROUND ART

One example of the I/O device sharing system is described in PTL 1. The I/O device sharing system shown inFIG. 7includes a plurality of hosts (hereinafter, these hosts are described as a host2-1(a host 1), a host2-2(a host 2), . . . , and a host2-N (a host N)), an I/O device5corresponding to SR-IOV, an I/O virtual bridge6, and a network3. The I/O device5corresponding to SR-IOV conforms to the Single-Root I/O Virtualization (SR-IOV) standard shared by the hosts2-1to2-N. The I/O virtual bridge6virtualizes the I/O device5corresponding to SR-IOV. The network3connects the hosts2-1to2-N and the I/O virtual bridge6.

The host2-1includes a CPU201-1, a memory203-1, a host PCIe bridge204-1, and a route complex202-1. The CPU201-1operates according to a program. The memory203-1stores the program and data. The host PCIe bridge204-1connects a PCI Express (PCIe) bus that is an I/O bus of the host2-1and the network3. The route complex202-1connects the CPU201-1, the memory203-1, and the host PCIe bridge204-1to each other. The host PCIe bridge204-1encapsulates a PCIe packet in a network packet and decapsulates the PCIe packet encapsulated in the network packet. All the hosts2-1to2-N have the same configuration.

The I/O device5corresponding to SR-IOV holds a Physical Function (PF)501that is an interface for controlling the device and a plurality of Virtual Functions (VFs) that are interfaces to provide the function of the I/O device. Hereinafter, a plurality of the VFs are described as the VF502-1(VF1) to the VF502-N (VFN). The PF501holds a PF configuration register (PF_CFG_REG)5011for setting the PF501. The VF502-1is used by the host2-1. Similarly, the VFs502-2to502-N are used by the hosts2-2to2-N, respectively. Accordingly, the I/O device5corresponding to SR-IOV is shared by the hosts2-1to2-N.

The I/O virtual bridge6includes a network connection unit601which transmits/receives the PCIe packet to/from the network3, a packet transfer unit602which transfers the PCIe packet, an address conversion unit603which performs conversion of a destination address of the PCIe packet, and a connection virtualization unit604which performs a setting of the I/O device5corresponding to SR-IOV.

The network connection unit601decapsulates the PCIe packet that is encapsulated in the packet of the network3. The network connection unit601encapsulates the PCIe packet in the packet of the network3and transmits it to the network3.

The connection virtualization unit604accesses the PF_CFG_REG5011and performs the setting of the I/O device5corresponding to SR-IOV before the I/O device5corresponding to SR-IOV is used by the hosts2-1to2-N. At this time, the address of the I/O device5corresponding to SR-IOV is set.

The address conversion unit603converts the address of the PCIe packet with respect to the access from the hosts2-1to2-N to the I/O device5corresponding to SR-IOV. Specifically, the address conversion unit603performs the address conversion in which the addresses of the VFs502-1to502-N that are set by the hosts2-1to2-N are converted into the addresses of the VFs502-1to502-N that are set by the connection virtualization unit604, respectively.

The I/O device sharing system having the above-mentioned configuration operates as follows.

In the following explanation, a case in which the host2-1accesses the VF502-1will be explained as an example. The following explanation can be applied to a case in which the hosts2-2to2-N access the VFs502-2to502-N, respectively. When the CPU201-1which operates according to software in the host 1 issues an instruction to access the VF502-1, the route complex202-1creates the PCIe packet addressed to a memory address to which the VF502-1is mapped and transmits it. The host PCIe bridge204-1encapsulates the PCIe packet in the network packet and transmits it to the network3. The network connection unit601receives the network packet, decapsulates the PCIe packet, and transfers it to the packet transfer unit602.

The packet transfer unit602transfers the received PCIe packet to the address conversion unit603. The address conversion unit603converts the destination address designated by the host2-1into the address of the VF502-1that is set to the PF501by the connection virtualization unit604and transmits the PCIe packet to the VF502-1. As a result, the VF502-1receives the PCIe packet issued by the host2-1. Further, when the host2-1receives the PCIe packet transmitted by the VF502-1, the above-mentioned processes are performed in reverse order.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

However, the system described in PTL 1 has a problem in which management of the I/O device that is performed by the I/O virtual bridge is complicated. The reason is because each I/O device has a specific setting process and in order to respond to the various I/O devices, the specific setting process of each I/O device needs to be realized by the I/O virtual bridge.

An object of the present invention is to provide an I/O device sharing system which can simplify management of an I/O device that is performed by an I/O virtual bridge and an I/O device sharing method.

Solution to Problem

According to the present invention, there is provided an I/O device sharing system characterized by comprising: an I/O device shared by a plurality of hosts; a system manager which sets the I/O device; a virtual bridge which virtualizes the I/O device; and a network which connects the I/O device, the system manager, the plurality of hosts, and the virtual bridge to each other, wherein the virtual bridge includes a connection virtualization unit by which it is detected that an address setting of a plurality of virtual functions provided in the I/O device that is set by the system manager is performed, the virtual function is enabled, or both of them are performed and each host is permitted to access each virtual function.

According to the present invention, there is provided an I/O device sharing method characterized by comprising: connecting an I/O device and a network via a virtual bridge which virtualizes the I/O device, connecting a system manager which sets the I/O device and a plurality of hosts which share the I/O device with the network, and detecting that an address setting of a plurality of virtual functions held by the I/O device that is set by the system manager is performed, the virtual function is enabled, or both of them are performed and permitting each host to access each virtual function.

Advantageous Effect of Invention

The present invention has an effect in which management of the I/O device that is performed by the I/O virtual bridge can be simplified. The reason is because in the management of the I/O device, only the management applied to all the I/O devices is performed by the I/O virtual bridge and the other management that is used for a process individually performed by each I/O device is performed by a system manager. Namely, the process performed by each I/O device excluding the common process is performed by the system manager connected through a network by using a software provided by a manufacturer of the I/O device.

DESCRIPTION OF EMBODIMENTS

FIG. 1is a block diagram showing a configuration of one exemplary embodiment of an I/O device sharing system according to the present invention. The I/O device sharing system shown inFIG. 1includes a system manager10, hosts2-1to2-N, an I/O device5corresponding to SR-IOV, an I/O virtual bridge4, and a network3. The system manager10controls the I/O device5corresponding to SR-IOV. The hosts2-1to2-N share the I/O device5corresponding to SR-IOV. The I/O virtual bridge4connects the I/O device corresponding to SR-IOV and the network3. The network3connects the system manager10, the hosts2-1to2-N, and the I/O virtual bridge4to each other.

The system manager10includes a CPU101, a memory103, a host PCIe bridge104, a route complex102, and a PF setting software105. The CPU101operates according to software. The memory103stores a program and data. The host PCIe bridge104connects a PCIe bus and the network3. The route complex102connects the CPU101, the memory103, and the host PCIe bridge104to each other. The PF setting software105controls a PF501of the I/O device5corresponding to SR-IOV.

Further, the PF501corresponds to one example of a control function for controlling a virtual function.

The PF setting software105includes a device driver which operates in kernel space, an OS, and a PF setting user software which operates in user space. The PF setting user software is generally provided by a manufacturer which provides the I/O device5corresponding to SR-IOV as a series of software programs which control the device together with the device. The PF setting user software may be provided by the operating system which operates in the system manager10.

The I/O device5corresponding to SR-IOV holds the PF501that is an interface which controls the device and VFs502-1to502-N that are the interfaces which provide a function of the I/O device. The PF501holds a PF_CFG_REG5011for setting the PF501. The VF502-1holds a VF configuration register (VF CFG REG)5021-1(VF1 CFG REG) for setting the VF502-1. All the VFs502-1to502-N have the same configuration. The VF502-1is used by the host2-1. Similarly, the VFs502-2to502-N are used by the hosts2-2to2-N, respectively. Accordingly, the I/O device5corresponding to SR-IOV is shared by the hosts2-1to2-N.

The I/O virtual bridge4includes a network connection unit401, a packet transfer unit402, an address conversion unit403, a connection virtualization unit404, and a bridge memory405. The network connection unit401transmits/receives the PCIe packet to/from the network3. The packet transfer unit402transfers the PCIe packet. The address conversion unit403performs conversion of the destination address of the PCIe packet. The connection virtualization unit404performs a setting of the I/O device5corresponding to SR-IOV. The bridge memory405stores the setting of the I/O device5corresponding to SR-IOV.

The network connection unit401decapsulates the PCIe packet encapsulated in the packet of the network3. The network connection unit401encapsulates the PCIe packet in the packet of the network3and transmits it to the network3.

The bridge memory405includes a virtual PF_CFG_REG4051, a virtual VF1 CFG REG4052-1to a virtual VFN CFG REG4052-N, and an I/O setting storage unit4053. The virtual PF_CFG_REG4051simulates the PF_CFG_REG5011. The virtual VF1 CFG REG4052-1to the virtual VFN CFG REG4052-N simulate the VF1 CFG REG5021-1to the VFN CFG REG5021-N, respectively. The I/O setting storage unit4053stores the setting of the I/O device5corresponding to SR-IOV that is performed by the connection virtualization unit404.

The connection virtualization unit404operates according to a state machine shown inFIG. 2.

An activation initial state S1 is a first state in which the I/O virtual bridge4is activated. At this time, the I/O virtual bridge4reads a parameter which specifies the operation of the connection virtualization unit404from a setting storage medium. The state of the I/O virtual bridge4changes to an I/O device setting state S2.

In the I/O device setting state S2, the connection virtualization unit404accesses the PF_CFG_REG5011of the I/O device5corresponding to SR-IOV that is set by the system manager10, sets the addresses of the PF501and the VFs502-1to502-N, and enables the VFs502-1to502-N.

The address setting performed by the connection virtualization unit404will be described with reference toFIG. 3. The connection virtualization unit404creates an I/O device memory space M1. In the space M1, the connection virtualization unit404maps the PF501in a PF map area M101. Further, the connection virtualization unit404maps the VFs502-1to502-N in a VF1 map area M102-1to a VFN map area M102-N, respectively. Namely, the connection virtualization unit404sets the address to the virtual function and the control function. The register used for setting this map is held in the PF_CFG_REG5011. The connection virtualization unit404stores the setting value of the PF501in the I/O setting storage unit4053. The state of the connection virtualization unit404changes to a system manager connection state S3.

In the system manager connection state S3, the access to the PF501that is performed by the system manager10is permitted. Specifically, the connection virtualization unit404makes the network connection unit401establish the connection between the host PCIe bridge104and the network connection unit401so that the system manager10can access the PF501.

When the connection between the system manager10and the I/O device5corresponding to SR-IOV is established, software operated by the system manager10issues the configuration access to the I/O device5corresponding to SR-IOV and performs the configuration. The configuration access includes a configuration write access and a configuration reed access by the PCIe packet. The configuration access of the system manager10is transferred to the connection virtualization unit404by the packet transfer unit402.

Next, the connection virtualization unit404writes the setting of the system manager10to the virtual PF_CFG_REG4051. By this access, the system manager10maps the PF501in a PF virtual map area M201of a system manager physical memory space M2. Namely, the system manager10sets the address of the control function for controlling the virtual function. In the configuration access to the virtual PF_CFG_REG4051that is performed by the system manager10, when it is detected by the connection virtualization unit404that the VFs502-1to502-N are enabled, the state of the connection virtualization unit404changes to a host connection possible state S4. When it is detected by the connection virtualization unit404that the address setting of the VFs502-1to502-N is performed or the address setting of them is performed and the VFs502-1to502-N are enabled, the state of the connection virtualization unit404may change to the host connection possible state S4.

The host connection possible state S4 is a state in which the hosts2-1to2-N can use the VF502-1to502-N, respectively. The function with respect to the host2-1of the connection virtualization unit404will be described below. Similarly, the function of the hosts2-2to2-N can be described.

The connection virtualization unit404establishes the connection between the network connection unit401and the host PCIe bridge204-1and controls the network connection unit401so that the host2-1can communicate with the VF502-1. After the connection is established, first, the host2-1issues the configuration access to the VF1 CFG REG5021-1and performs the configuration of the VF502-1(VF1). The configuration access issued by the host2-1is transferred to the connection virtualization unit404by the packet transfer unit402. The connection virtualization unit404writes the setting of the host2-1to the virtual VF1 CFG REG4052-1. By this setting, as shown inFIG. 3, the host2-1maps the VF502-1in a VF1 virtual map area M301-1of a host 1 physical memory space M3-1. Namely, the host2-1performs the address setting to the virtual function. As a result, the host2-1can perform the memory access to the VF502-1. Further, the host2-1can use the VF502-1.

The host connection possible state S4 is a steady state for providing a service in which the I/O device5corresponding to SR-IOV is shared by the hosts2-1to2-N. Unless an event such as a system error, a release of the connection between the host PCIe bridge104and the network connection unit401, or the like occurs, the state of the connection virtualization unit404is maintained to the host connection possible state S4. When the connection between the host PCIe bridge104and the network connection unit401is released, the state of the connection virtualization unit404changes to an I/O device reset state S5.

In the I/O device reset state S5, the connection virtualization unit404controls the network connection unit401so that the connections between the network connection unit401and the hosts2-1to2-N are released (the connections are cut off). The connection virtualization unit404resets the I/O device5corresponding to SR-IOV. After the reset, the state of the connection virtualization unit404changes to the I/O device setting status S2, the process is returned to the process in which the setting of the I/O device5corresponding to SR-IOV is performed, and the above-mentioned processes are performed again.

The address conversion unit403performs conversion of the address described in the header of the PCIe packet transmitted and received between the system manager10and the PF501. The address conversion unit403performs conversion of the address described in the header of the PCIe packet transmitted and received between the hosts2-1to2-N and the VFs502-1to502-N used by the hosts2-1to2-N, respectively. For example, as shown inFIG. 3, the PF501is mapped in the PF map area M101of the I/O device memory space M1. Further, the PF501is mapped in the PF virtual map area M201of the system manager physical memory space M2. In that case, the address conversion unit403performs the address conversion corresponding to a difference between map values of these two address spaces. Similarly, the VF502-1is mapped in the VF1 map area M102-1of the I/O device memory space M1. Further, the VF502-1is mapped in the VF1 virtual map area M301-1of the host 1 physical memory space M3-1. In this case, the address conversion unit403performs the address conversion of these two address spaces. Similarly, the address conversion unit403performs the address conversion with respect to the VFs502-2to502-N.

When the address conversion unit403performs the address conversion of the PCIe packet passing through the address conversion unit403, the address conversion unit403refers to information of the I/O device memory space M1, the system manager physical memory space M2, and the host 1 physical memory space M3-1 to the host N physical memory space M3-N that are stored in the bridge memory405. These information are obtained from the information stored in the virtual PF_CFG_REG4051, the virtual VF1 CFG REG4052-1to the virtual VFN CFG REG4052-N, and the I/O setting storage unit4053.

The I/O device sharing system according to this exemplary embodiment operates as follows based on the above-mentioned configuration.

First, operation of a case in which the system manager10issues the configuration access to the PF501will be described with reference toFIG. 1andFIG. 4.

The configuration access is the configuration write request or the configuration read request. When the PF setting software105operated by the system manager10issues the configuration access to the PF501(Step A1), the PCIe packet is created by the route complex102. The PCIe packet is transmitted to the host PCIe bridge104.

The host PCIe bridge104encapsulates the PCIe packet in the packet of the network3and transmits it to the network3(Step A2). The network connection unit401receives the packet in which the PCIe packet is encapsulated. The network connection unit401decapsulates the network packet and transmits it to the packet transfer unit402(Step A3). The packet transfer unit402confirms that the PCIe packet is the configuration access by the system manager10and transfers it to the connection virtualization unit404(Step A4). When it is determined that the request is the write request based on the received configuration access, the connection virtualization unit404writes the setting value into the virtual PF_CFG_REG4051. When it is determined that the request is the read request, the connection virtualization unit404reads the setting value designated from the virtual PF_CFG_REG4051and transmits a reply to the system manager10(Step A5).

Next, operation of a case in which the system manager10performs the memory access to the area in which the PF501is memory-mapped will be described with reference toFIG. 1andFIG. 5.

The memory access is the memory write request or the memory read request. When the PF setting software105which operates by the system manager10issues the memory access to the PF501(Step B1), the PCIe packet addressed to the PF virtual map area M201is created by the route complex102. The PCIe packet is transmitted to the host PCIe bridge104. The host PCIe bridge104encapsulates the PCIe packet in the packet of the network3and transmits it to the network3(Step B2). The network connection unit401receives the network packet in which the PCIe packet is encapsulated, decapsulates the PCIe packet, and transmits it to the packet transfer unit402(Step B3).

The packet transfer unit402confirms that the PCIe packet is the memory access by the system manager10and transfers the PCIe packet to the address conversion unit403(Step B4).

The address conversion unit403performs conversion of the destination address of the PCIe packet from the PF virtual map area M201to the PF map area M101by using the value stored in the virtual PF_CFG_REG4051which holds the setting value of the PF virtual map area M201and the value stored in the I/O setting storage unit4053which holds the setting value of the PF map area M101in the bridge memory405. The address conversion unit403transmits the PCIe packet to the I/O device5corresponding to SR-IOV (Step B5). As a result, the PF501receives the memory access issued by the system manager10. When the memory access is the memory write request, the PF501writes the designated data and when it is the memory read request, the PF501reads the designated data and transmits a reply to the system manager10(Step B6).

FIG. 6is a block diagram showing a main portion of the I/O device sharing system according to the present invention. As shown inFIG. 6, the I/O device sharing system includes an I/O device50(for example, the I/O device5corresponding to SR-IOV shown inFIG. 1), a system manager100(for example, the system manager10shown inFIG. 1), a virtual bridge40(for example, the I/O virtual bridge4shown inFIG. 1), and the network3.

The I/O device50is shared by a plurality of hosts20-1to20-N (for example, the hosts2-1to2-N shown inFIG. 1). The system manager100sets the I/O device50. The virtual bridge40virtualizes the I/O device50. The network3connects the I/O device50, the system manager100, a plurality of the hosts20-1to20-N, and the virtual bridge40to each other. The virtual bridge40includes a connection virtualization unit44(for example, the connection virtualization unit404shown inFIG. 1). When it is detected by the connection virtualization unit44that the address setting of a plurality of the virtual functions provided in the I/O device50that is set by the system manager100is performed, the virtual function is enabled, or both of them are performed, the connection virtualization unit44permits each of the hosts20-1to20-N to access each virtual function.

The virtual bridge40further includes an address conversion unit43. With respect to the address described in the packet of the I/O bus that is transmitted and received between each of the hosts20-1to20-N and each of the virtual functions, the address conversion unit43performs the address conversion by referring to the address that is set to each virtual function by each of the hosts20-1to20-N and the address that is actually set to each virtual function.

As described above, this exemplary embodiment has a configuration in which the I/O virtual bridge controls the PF_CFG_REG that is common to all the I/O devices and the system manager connected to the I/O device through the network controls the memory map area of the PF that depends on the individual I/O device by using the PF setting software provided by the manufacturer of the I/O device. Therefore, the management of the I/O device that is performed by the I/O virtual bridge can be simplified. Further, a configuration of a portion related to the management of the I/O device that is performed by the I/O virtual bridge can be simplified.

Further, in this exemplary embodiment, the bus based on the standard of the bus which conforms to the PCIe is used as the I/O bus. However, a bus based on a new I/O bus standard that will be developed by expanding the function of the PCIe in the future or a bus based on another I/O bus standard which operates in a similar manner can be used for the present invention.

In this exemplary embodiment, the number of the hosts2-1to2-N is equal to the number of the VFs502-1to502-N of the I/O device5corresponding to SR-IOV. However, the number of the hosts may not be necessarily equal to the number of the VFs. For example, an exemplary embodiment in which the number of VFs is greater than the number of the hosts and each host uses one of the VFs may be used.

The I/O device5corresponding to SR-IOV according to this exemplary embodiment is a device which conforms to the PCIe SR-IOV standard such as a network interface card, a storage device, a storage network card, a USB controller, and an I/O device for built-in apparatus.

The invention of the present application has been described above with reference to the exemplary embodiment and the example. However, the invention of the present application is not limited to the above mentioned exemplary embodiment and example. Various changes in the configuration or details of the invention of the present application that can be understood by those skilled in the art can be made without departing from the scope of the invention.

This application claims priority based on Japanese Patent Application 2012-087888 filed on Apr. 6, 2012, the disclosure of which is hereby incorporated by reference in its entirety.

INDUSTRIAL APPLICABILITY

The present invention can be applied to a use in which a single I/O device is shared by a plurality of computers in a computer system. Further, the present invention can be applied to a use in which a single I/O device is shared by a plurality of apparatuses in a built-in system.

REFERENCE SIGNS LIST