Host device passthrough: List all types of host devices

The URI /host/devices only presents scsi_host (particularly fc_host)
device information. To implement host PCI pass through, we should list all
types of host devices. This patch adds support for parsing various host
devices information, and listing them on /host/devices. So the user is free
to choose any listed PCI device to pass through to guest. Since the
patch changes the device information dictionary format, the existing code
consuming the device information is also changed accordingly.

To get all types of host device, access the following URL.

curl -k -u root -H "Content-Type: application/json" \
  -H  "Accept: application/json" \
  https://127.0.0.1:8001/host/devices

To get only fc_host devices, change the URL to
  "https://127.0.0.1:8001/host/devices?_cap=fc_host"

To get only pci device, change the URL to
  "https://127.0.0.1:8001/host/devices?_cap=pci"

Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
Host device passthrough: List eligible device to passthrough

This patch adds a '_passthrough=true' filter to /host/devices, so it can
filter and shows all devices eligible to passthrough to guest.
Theoretically, all PCI, USB and SCSI devices can be assigned to guest
directly.

Linux kernel is able to recognize the host IOMMU group layout. If two
PCI devices are in the same IOMMU group, it means there are possible
interconnections between the devices, and the devices can talk to each
other bypassing IOMMU. This implies isolation is not pefect between those
devices, so all devices in a IOMMU group must be assigned to guest
together. On host that recognizes IOMMU groups, by accessing the URI
/host/devices?_passthrough_affected_by=DEVICE_NAME, it returns a list
containing the devices in the same IOMMU group as DEVICE_NAME, and all
of the children devices of them. So the front-end can show all the
affected devices to user, and it helps the user to determine which
host devices are to be assigned to guest.

How to test:

List all types of devices to passthrough
  curl -k -u root -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    'https://127.0.0.1:8001/host/devices?_passthrough=true'

List all eligible PCI devices to passthrough
  /host/devices?_passthrough=true&_cap=pci

List all USB devices to passthrough
  /host/devices?_passthrough=true&_cap=usb_device

List all SCSI devices to passthrough
  /host/devices?_passthrough=true&_cap=scsi

List devices in the same IOMMU group as pci_0000_00_19_0
  /host/devices?_passthrough_affected_by=pci_0000_00_19_0

Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
Host device passthrough: Directly assign and dissmis host device from VM

This patch enbales Kimchi's VM to use host devices directly, and it
greatly improves the related device performance. The user can assign
PCI, USB and SCSI LUN directly to VM, as long as the host supports one
of Intel VT-d, AMD IOMMU or POWER sPAPR technology and runs a recent
release of Linux kernel.

This patch adds a sub-collection "hostdevs" to the URI vms/vm-name/.
The front-end can GET
  vms/vm-name/hostdevs
and
  vms/vm-name/hostdevs/dev-name
or POST (assign)
  vms/vm-name/hostdevs
and DELETE (dismiss)
  vms/vm-name/hostdevs/dev-name

The eligible devices to assign are the devices listed by the URI
  host/devices?_passthrough=1
When assigning a host PCI device to VM, all the eligible PCI devices in
the same IOMMU group are also automatically assigned, and vice versa
when dismissing a host PIC device from the VM.

Some examples:

Assign a USB device:
curl -k -u root -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -X POST -d '{"name": "usb_1_1_6"}' \
  'https://127.0.0.1:8001/vms/rhel65/hostdevs'

Assign a PCI device:
  -d '{"name": "pci_0000_0d_00_0"}'

Assign a SCSI LUN:
  -d '{"name": "scsi_1_0_0_0"}'

List assigned devices:
curl -k -u root -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  'https://127.0.0.1:8001/vms/rhel65/hostdevs'
The above command should print following.
  [
    {
      "type":"scsi",
      "name":"scsi_1_0_0_0"
    },
    {
      "type":"usb",
      "name":"usb_1_1_6"
    },
    {
      "type":"pci",
      "name":"pci_0000_0d_00_0"
    },
    {
      "type":"pci",
      "name":"pci_0000_03_00_0"
    }
  ]
Notice that the device pci_0000_03_00_0 is also assigned automatically.

The assigned devices are hot-plugged to VM and also written to the
domain XML. When it's possible, it enables VFIO for PCI device
assignment.

On distribution with old Linux kernel, there are many limitations with
PCI passthrough and it's hardly useful. This patch tries to adapt to old
kernel but it's better to use a newer kernel with vfio support. Thus
this patch also provide a new capability in /config/capabilities. The
front-end can disable or freeze the related web UI if back-end reports
host does not support vfio.

curl -k -u root -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    'https://127.0.0.1:8001/config/capabilities'

The above command should print following.
{
  "blah": "blah",
  ...
  "kernel_vfio":true
}

Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
Host device passthrough: List VMs that are holding a host device

Add a "vm_holders" sub-collection under host device resource, so the
front-end can determine if a device is busy or not, and the user can
know which VMs are holding the device.

This patch scans all VM XML to check if a device is hold by a VM.

Example
  curl -k -u root -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    'https://127.0.0.1:8001/host/devices/usb_1_1_6/vm_holders'
Should output a list like following.
  [
    {
      "state":"shutoff",
      "name":"fedora20"
    },
    {
      "state":"running",
      "name":"f20xfce-slave"
    }
  ]

If there is no VM holding the device, it prints an empty list [].

Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
Host device passthrough: Add unit tests and documents

Add some basice unit tests for fetching host device information. Update
MockModel as well. Update "API.json" and "API.md" to reflect the change
in API.

Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
Host device passthrough (Front-end): Add PCI Devices to VM

Signed-off-by: Yu Xin Huo <huoyuxin@linux.vnet.ibm.com>
Signed-off-by: Wen Wang <wenwang@linux.vnet.ibm.com>
Import the latest spice-html5 code into Kimchi

All the imported files are a real copy of spice-html5 project
(http://cgit.freedesktop.org/spice/spice-html5) based on commit 097e2c2e

The spice_auto.html also supports wss:// protocol used by Kimchi.

The files are imported under ui/spice-html5 to make clear the files in
that directory are imported from a external project.

Signed-off-by: Aline Manera <alinefm@linux.vnet.ibm.com>
Modify spice_auto.html for Kimchi proposals

The sources were updated according to Kimchi configuration to get the
Javascript and CSS files from the installed spice-html5 package.
Kimchi is not using the default spice_auto.html because Kimchi uses
wss:// for all connections and it is only supported by recent versions of
spice-html5.
In addition to it, Kimchi points user to the right token on URL which
requires a change on spice_auto.html

Signed-off-by: Aline Manera <alinefm@linux.vnet.ibm.com>
Add new spice-html5 code to Kimchi build process

By default Kimchi will not install the new spice-html5 code as
almost all the supported Linux distributions already has it available in a
package.
If the system you are running Kimchi does not have a spice-html5, you
can run: ./autogen.sh --with-spice-html5 to get it.

Signed-off-by: Aline Manera <alinefm@linux.vnet.ibm.com>
Update Kimchi to use the installed spice-html5

spice-html5 is present in almost all supported Linux distributions
(RHEL6.5, Fedora20 and Ubuntu 14.04). So in those cases we should not package
the spice-html5 code into Kimchi and instead of that uses the spice-html5 package
as dependency.

This patch change the build process to check the spice-html5 package
availability in order to decide to package or not the spice-html5 code.

spice-html5 package is not available for RHEL7 and openSUSE 13.1. So
for those distributions, you must run the following command to Kimchi
includes the spice-html5 into its package.

./autogen.sh --with-spice-html5

Signed-off-by: Aline Manera <alinefm@linux.vnet.ibm.com>
Delete former imported spice code

Now all the spice-html5 code is centralized under ui/spice-html5.
That way we can properly point which code is imported.

Signed-off-by: Aline Manera <alinefm@linux.vnet.ibm.com>
Update COPYING content to expose the imported code

Signed-off-by: Aline Manera <alinefm@linux.vnet.ibm.com>