nyuuzyou/wikis huggingface dataset:
VAAI support

We want to speed up common VM disk operations such as:
# deleting disks
# creating empty disks
# cloning disks
Ideally the host would send the commands to the storage server/target and allow it to Do The Right Thing. Many arrays are 'smart' in the sense that they maintain block allocation maps and can create trees of disk clones without explicitly copying all the blocks. Even when an array doesn't have such smarts it is still better to copy blocks across the internal array fabric than across the host to array network link.
The VMware vStorage APIs for Array Integration (VAAI) are a defacto standard which allow common VM disk operations to be sped up including
# Atomic Test and Set (ATC): for fast LV locking
# Block-level Zero: for blanking freshly-created LVs
# Block-level Clone: for getting the array to copy LVs
# Block-level Delete: to clear unused space
(for more information see http://linux-iscsi.org/wiki/VStorage_APIs_for_Array_Integration )
How can we take advantage of storage arrays which support these?
= Notes on VAAI =
The APIs are presented as SCSI commands and have been implemented in the Linux-IO software target which has achieved VMware certification in several hardware products.
* Linux has a BLKZEROOUT ioctl
* Linux 3.12 has EXTENDED COPY which you can run via sg_xcopy
* Linux 3.16-rc4 has a fixed EXTENDED COPY which is enough for a proof-of-concept device mapper implementation (Jul 2014). This is automatically used if you use mirror, snapshot, thin or cache dm targets.
* Windows uses ODX (Offloaded Data Transfer) rather than EXTENDED COPY
* A comparison of file vs block primitives: http://cormachogan.com/2012/11/08/vaai-comparison-block-versus-nas/
* A draft of NFS v4.2 has server-side copy
VM Feature Identification

When you control your own hypervisor, you are aware of the features you are using, since you have configured the system and the VMs. However, if you are using a service provider for your hosting or cloud, you need to do some detective work to determine the hypervisor mode and features employed.
== MODE DETECTION ==
The output of the dmesg command contains many useful bits of information about the processor mode:
=== HVM-type: ===
If your VM is either HVM or PVHVM, look for:
[ 0.000000] Hypervisor detected: Xen HVM
and:
[ 0.000000] Booting paravirtualized kernel on Xen HVM
If it is actually PVHVM, you could see messages like:
[ 0.127240] Xen: using vcpuop timer interface
or:
[ 1.424040] Initialising Xen virtual ethernet driver.
or:
[ 0.000000] Xen HVM callback vector for event delivery is enabled
indicating that PV drivers are being utilized within the HVM machine.
Also, some newer versions might have:
[ 0.000000] xen: PV spinlocks enabled
indicating that PV spinlocks are used, so an HVM VM with this message must be PVHVM.
=== PV-type: ===
If your VM is PV or PVH, look for:
[ 0.000000] Booting paravirtualized kernel on Xen
Note the lack of reference to HVM, as seen in the messages above.
This message indicates that PV timers are in use:
[ 0.000000] Xen: using vcpuop timer interface
If PV network drivers are in use, you might see:
[ 0.432488] Initialising Xen virtual ethernet driver.
=== OTHER HINTS ===
* Disk names: names in the format xvdXN (e.g., xvda1, or xvdc3) indicates that the PV disk driver is in use.
* Network drivers: a network driver of vif is a PV driver
=== DECODE INDIVIDUAL FEATURES ===
Brendan Gregg has written a Perl-based script called xen-features.pl which can be retrieved from git here.
It takes the value stored in /sys/hypervisor/properties/features and decodes it into output like:
Xen features: 00000705
enabled: writable_page_tables
enabled: auto_translated_physmap
enabled: hvm_callback_vector
enabled: hvm_safe_pvclock
enabled: hvm_pirqs
=== REFERENCES ===
* A truly great article by Brendan Gregg
Video: Xen VGA Passthrough to Windows 8 Consumer Preview HVM Virtual Machine with Xen 4.2-unstable Changeset 25099 and Linux Kernel 3.3.0 in Ubuntu 11.10 oneiric ocelot amd64 Final Release Dom0

#REDIRECT Video: Xen VGA Passthrough to Windows 8 and Ubuntu 11.10 with Xen 4.2 and Kernel 3.3
Video: Xen VGA Passthrough to Windows 8 with Xen 4.2-unstable

This is a video presentation by Mr. Teo En Ming (Zhang Enming) in Singapore. It shows:
* Xen VGA Passthrough
* to a Windows 8 Consumer Preview HVM Virtual Machine
* with Xen 4.2-unstable Changeset 25099 and Linux Kernel 3.3.0 in Ubuntu 11.10 oneiric ocelot amd64 Final Release Dom0
----
Xen VGA Passthrough to Windows 8 Consumer Preview HVM Virtual Machine with Xen 4.2-unstable Demo
Category:Xen
Category:PVOPS
Category:Beginners
Category:Users
Category:Developers
Category:HowTo
Category:Tutorial
Category:Example
Category:VGA
Video: Xen VGA Passthrough with AMD Radeon HD 6450

This is a video presentation by Mr. Teo En Ming (Zhang Enming) in Singapore. 
Software Configuration:
*Host operating system dom0: Ubuntu 14.04 LTS
*Linux kernel: 3.16.1 (self-compiled)
*Xen hypervisor: 4.4.1 (self-compiled)
*Guest operating system domU: Windows 8.1 Enterprise HVM domU
*Display card driver installed: Catalyst 14.4
Hardware Configuration:
*Processor: Intel Core i5-4430 CPU @ 3.00GHz (Quad Core)
*Motherboard: Asrock B85M Pro4 LGA 1150 Motherboard
*Memory: 32 GB DDR3-1600
*PCI-E x16 Display Card: Sapphire AMD Radeon HD 6450 1 GB DDR3
*VT-x: Enabled in UEFI BIOS
*VT-d: Enabled in UEFI BIOS
----
Xen VGA Passthrough with AMD Radeon HD 6450 Demo
Category:Xen
Category:PVOPS
Category:Beginners
Category:Users
Category:Developers
Category:HowTo
Category:Tutorial
Category:Example
Category:VGA
Virsh Commands

The libvirt toolkit is focused primarily on KVM virtual machines, but it is intended to be usable for multiple virtualization technologies, including the Xen Project Hypervisor. As a result, some commands are essential when using Xen Project software, but others do not apply as they are targeted for KVM concepts.
Some of the essential virsh commands for manipulating Xen Project VMs include:
Common VIRSH commands:	
help 	print help
list 	list domains
create 	create a domain from an XML file
start 	start a previously created inactive domain
destroy 	destroy a domain
define 	define (but do not start) a domain from an XML file
domid 	convert a domain name or UUID to domain id
domuuid 	convert a domain name or id to domain UUID
dominfo 	domain information
domname 	convert a domain id or UUID to domain name
domstate 	domain state
quit 	quit this interactive terminal
reboot 	reboot a domain
restore 	restore a domain from a saved state in a file
resume 	resume a domain
save 	save a domain state to a file
shutdown 	gracefully shutdown a domain
suspend 	suspend a domain
undefine 	undefine an inactive domain
setmem 	changes the allocated memory.
setmaxmem 	changes maximum memory limit.
setvcpus 	changes number of virtual CPUs.
vcpuinfo 	domain vcpu information.
vcpupin 	control the domain vcpu affinity.
version 	show version
dumpxml 	domain information in XML
nodeinfo 	node information
Unused VIRSH commands:	
connect 	connect or reconnect to a hypervisor
Category:Beginners
Category:libvirt
Virtio On Xen - Legacy: GSoC

This page is about a legacy project. For current work being done for VirtIO on Xen, see Virtio On Xen.
== VirtIO-GSOC: 2011 Google Summer of Code Project ==
A Google Summer of Code project by Wei Liu investigated enabling VirtIO on Xen.
A working prototype was produced for both PV and HVM guests, using XenBus and the Qemu VirtIO backends. PV guests require a guest kernel patch to translate guest physical addresses to machine addresses in VirtIO rings.
Status: project completed.
=== Using the prototype ===
This section is intended to guide people who might be interested giving it a try.
Note: 
* Make sure you have the latest Xen unstable source (at least CS23728).
* Checkout Stefano Stabellini's QEMU repository: git://xenbits.xen.org/people/sstabellini/qemu-dm.git xen-stable-0.15
==== Virtio for HVM guest ====
In fact, there is nothing much to do.
* With this patch, we can enable libxl to support configuration for Virtio disk and nic.
** For disk configuration syntax, use 'vd*' as virtual identifier, see docs/misc/vbd-interface.txt for details.
** For nic configuration syntax, use 'model=virtio' in vif configuration. Latest QEMU may require 'model=virtio-net'. Just choose the one that works.
* Disable MSI in guest kernel, with 'pci=nomsi' option in guest kernel command line. Currently, MSI delivery is not supported by HVM guest (though Xen has one HVMOP to do this, QEMU is not yet ready for this).
* Also you might want to add 'xen_emul_unplug=never' in the guest kernel command line.
Use 'xl' to start the guest, not 'xm'.
==== Virtio for PV guest ====
First of all, you need the patch in the previous section to enable libxl to support configuration syntax for Virtio disk and nic.
Then apply the following patches to upstream QEMU and Linux kernel.
* QEMU
** qemu-01-xenpv-exec.patch
** qemu-02-virtio-for-pv.patch
* Linux kernel
** linux-01-virtio-xenbus.patch
** linux-02-virtio-ring.patch. N.B This patch breaks existing virtio ring implementation, so please compile it out of tree and load it manually.
* VM config file
** Make sure you add device_model_args_pv=['-net', 'tap,vlan=0,script=no'] to make QEMU add a tap for you, or you won't be able to send packets via Virtio net.
* Linux kernel config
** Make sure you have 'iommu=soft' in your guest kernel command line.
Run 'xl' to startup VM, 'xm' doesn't understand virtio configuration. After the VM is up, load modified virtio_ring.ko and all the other modules manually in the following order:
# virtio.ko
# virtio_ring.ko
# virtio_net.ko
# virtio_blk.ko
# virtio_xenbus.ko
Hopefully you will get Virtio devices without much trouble.
==== TODO ====
* Enable xen-mapcache for Virtio for PV, improve performance.
* Squash two evtchns into one, hopefully we can eliminate locking in transport layer and improve performance.
* Enable Virtio device DMA capability
* Bug fixes, transport layer crashes sometimes under heavy workload.
==== Related References ====
* http://wiki.xen.org/xenwiki/QEMUUpstream
For those who encounter problems, please contact Wei Liu <liuw AT liuw SPAMFREE dot name>
Category:Xen
Category:Project
Category:GSoC
Category:Internships
Virtual Machine

__NOTOC__
In virtualization, a virtual machine (or VM for short) is something that pretends to be an entire machine but really exists only in the imagination of the hypervisor. It has all the facilities of a real machine but there is no physical box associated with it. A single physical machine can be virtualized to create several virtual machines.
In interpreted languages, such as Java bytecode, virtual machine refers to the made-up InstructionSetArchitecture presented by the environment. The Java Virtual Machine is a stack-based ISA with knowledge of objects and number types, and Java applications are compiled to run on this architecture.
When talking about Xen, we often refer to a domain as a virtual machine. It shares the qualities of both of the definitions given above. Like in a fully virtualized system, it is an environment presented to operating systems where they would normally expect to see a real machine. Unlike in a fully virtualized system, it does not have all the facilities of a real machine: Xen provides alternatives to some of the machine's capabilities instead of emulating them. As in the second definition, Xen is a made-up InstructionSetArchitecture. Although it is very similar to the native machine (i.e. Xen x86 has all of the user-mode features of native x86, and most of the kernel-mode features as well), it is not exactly the same, so operating systems that wish to be run under Xen have to have their kernels ported to its architecture. This is a small cost, but it provides great performance gains over fully-virtualized systems.
Current developments in hardware support for virtualization mean that it will soon be possible to also run native OSes on Xen without modification.
Category:Xen
Category:Glossary
Virtual Trusted Platform Module (vTPM)

Xen provides support for Trusted Platform Module (TPM). You must have an x86 machine with a TPM on the motherboard, to make use of vTPM. The reader is assumed to have familiarity with building and installing Xen, Linux, and a basic understanding of the TPM and vTPM concepts.
== vTPM Support in XL ==
 vtpm-attach domain-id vtpm-device
Creates a new vtpm device in the domain specified by domain-id. vtpm-device describes the device to attach, using the same format as the vtpm string in the domain config file. See xl.cfg for more information.
 vtpm-detach domain-id devid|uuid
Removes the vtpm device from the domain specified by domain-id. devid is the numeric device id given to the virtual trusted platform module device. You will need to run xl vtpm-list to determine that number. Alternatively the uuid of the vtpm can be used to select the virtual device to detach.
 vtpm-list domain-id
List virtual trusted platform modules for a domain.
== vTPM Extensions in Xen 4.3 ==
Xen 4.3 exposes TPM functionality to virtual guest operating system (a DomU). This allows programs to interact with a TPM in a virtual system the same way they interact with a TPM on the physical system. Each guest gets its own unique, emulated, software TPM. However, each of the vTPM's secrets (Keys, NVRAM, etc) are managed by a vTPM Manager domain, which seals the secrets to the Physical TPM. If the process of creating each of these domains (manager, vTPM, and guest) is trusted, the vTPM subsystem extends the chain of trust rooted in the hardware TPM to virtual machines in Xen. Each major component of vTPM is implemented as a separate domain, providing secure separation guaranteed by the hypervisor. The vTPM domains are implemented in
mini-os to reduce memory and processor overhead.
== vTPM 2.0 in Xen 4.6 ==
Support for vTPM 2.0 was added in Xen 4.6: for more information, see section Appendix B: vtpmmgr on TPM 2.0.
== References ==
* Virtual TPM support
* Virtual Trusted Platform Module (research note)
* Virtual TPM Example Configurations
* Virtual TPM Manager CLI
* Configuring Virtual TPM (vTPM) for Xen 4.3 Guest Virtual Machines
* Extensive USENIX Research Paper on vTPM (2006)
* TrustedComputingGroup which defines TPM
Category:Xen
Category:Xen 4.3
Category:Xen 4.4
Category:Xen 4.3
Category:Xen 4.5
Category:Xen 4.6
Category:Security
Website Wall of Fame

This pages honors individuals who have identified and clearly documented problems with the website, or who have otherwise provided a particularly useful service toward the well-being of XenProject.org.
We thank them for protecting our website and helping to make it better!
== Wall Members ==
* Muhammad Uwais, May 2016, documented an exploit
* Shawar Khan, July 2015, documented an exploit
* Mohammad Yaseen, August 2014, documented an exploit
* Hritik Vijay, April 2015, documented an exploit
Why and how fake ARP works

netfront driver should send a fake arp package to encourage switches to learn the MAC. This wiki page will show you why should we do that and how to get it.
= Why? =
This feature is mainly for live migration. In the following example, if the VM migrated from Server 1 to Server 2, Switch 1 still forward ICMP packages from Workstation 1 to Switch 2; until a ARP timeout (may take long time) in Switch 1 or the VM send some package out. 
= How? =
We force the netfront drive to send a fake ARP package on every initializing time. Then all the switches will be flushed about the MAC <-> Port association.
The fake ARP package detail:
HTYPE
PTYPE
HLEN
PLEN
OPER
SHA
SPA
THA
TPA
The Ethernet transmission layer header:
dst_hw
src_hw
type
= TODO? =
* This feature is not implemented in the upstream linux kernel (which is using pv_ops): http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=drivers/net/xen-netfront.c;hb=HEAD (as of 20090703)
* For HVM guest using the ioemued vif, the solution is already in qemu upstream. So xen 3.4+ doesn't have this issue.
= Reference =
* http://wiki.wireshark.org/Gratuitous_ARP
* Address Resolution Protocol on Wikipedia.
* Linux 2.6.18-xen implement.
Category:Xen
Category:Users
Category:Beginners
Category:Overview
Windows PV Drivers/Building

== Getting ready ==
First of all, you have to get the driver source.
Then you will need a device driver development environment. You can use a Windows 8 or 8.1 WDK, which will involve installing either Visual Studio 2012 or 2013 respectively. Note that you cannot use Visual Studio 2012 Express edition for building drivers. (You may find it useful to install VirtualCloneDrive as Visual Studio is generally supplied in ISO form).
Install Visual Studio first (you only need install MFC for C++) and then the WDK. Set an environment variable called VS to the base
of the Visual Studio Installation (e.g. C:\Program Files\Microsoft Visual Studio 12.0) and a variable called KIT to the base of the
WDK (e.g. C:\Program Files\Windows Kits\8.1). Also set an environment variable called SYMBOL_SERVER to point at a location where
driver symbols can be stored. This can be local directory e.g. C:\Symbols.
Next you'll need a 3.x version of Python (which you can get from http://www.python.org). Make sure python.exe is somewhere on your
default path.
== Building a driver package ==
Now fire up a Command Prompt and navigate to the base of your git repository.
At the prompt type:
build.py checked
This will create a debug build of the driver. To create a non-debug build
type:
build.py free
Note that Static Driver Verifier is run by default as part of the build
process. This can be very time consuming. If you don't want to run the
verifier then you can add the 'nosdv' keyword to the end of your command
e.g.:
build.py free nosdv
Category:Windows PV Drivers
Windows PV Drivers/Repositories

= Version Control =
The Windows PV Drivers are maintained using the GIT version control system.
= Repositories =
There are currently five driver packages. The source for each one is hosted in a separate repository.
* XenBus: Xen PV Bus Driver
* XenVif: Xen PV Network Class Driver
* XenNet: Xen PV Network Device Driver
* XenVbd: Xen PV Storage Class Driver
* XenIface: Xen Interface Driver
Category:Windows PV Drivers
Writing good FAQs

== Conventions: Naming FAQs ==
Aim to name FAQs, such that they can easily found in an index
* Prefix the FAQ with the main scope, e.g. Xen Project, DomU Installation, Xen Project on ARM, etc.
* Then add FAQ
* It is good to group FAQs related to a topic, e.g. Linux, Installation, etc.
* If the topic is too large qualify, e.g. Linux, Running on, Linux, Building for Xen Project, etc.
For examples, see :Category:FAQ
== Attributes ==
Good FAQs typically have the following attributes:
* The question is clearly and concisely covered at the beginning of the FAQ
* They use simple and clear language
* They avoid the excessive use of technical language or add a definition of a new term
* They tend to be self containing, i.e. they limit the number of external links
* They contain examples, scripts, etc. 
Category:FAQ
Category:Wiki Management
XAPI

== What is the XAPI Project? ==
The XAPI project is a sub-project (or team) of the Xen Project effort that develops the enterprise ready XAPI toolstack. The hypervisor used with the XAPI toolstack consolidates server workloads, enables savings in power, cooling, and management costs and thus contributing to environmentally sustainable computing, an increased ability to adapt to ever-changing IT environments, an optimized use of existing hardware, and an improved level of IT reliability.
The XAPI team also develops tooling, agents and libraries that are needed to operate a XAPI based system.
== What is XAPI? ==
The Xen Project Management API (XAPI) is:
* A Xen Project Toolstack that exposes the XAPI interface. When we refer to XAPI as a toolstack, we typically include all dependencies and components that are needed for XAPI to operate (e.g. xenopsd).
* An interface for remotely configuring and controlling virtualised guests running on a Xen-enabled host. XAPI is the core component of XenServer and XCP.
XAPI adds additional functionality compared to other Xen Project toolstacks, including:
* Extending the software to cover multiple hosts
* Enhancing the VM lifecycle, including live snapshots, VM checkpointing, and VM migration
* Enabling resource pools to include live migration, auto configuration, and disaster recovery
* Allowing flexible storage and networking including integrated Open vSwitch support and storage XenMotion® live Migration (cross-pool migration, VDI migration)
* Enabling event tracking, with progress and notification
* Creating upgrade and patching capabilities
* Facilitating real-time performance monitoring and alerting
* Integrations with cloud orchestration stacks
* Built-in support and templates for Windows and Linux guests
== How Do I Use XAPI? ==
The "xe" syntax is fairly straightforward.
An "xe" command executed from Domain 0 would look like:
xe <command-name> <argument=values...>
From a user domain, the syntax will look like:
xe -s <host> -u <username> -pw <password> <command-name> <argument=values...>
So if you wanted to see a list of VMs on a host from Domain 0, you would use the command:
xe vm-list 
* Information about the "xe" commands can be found on the Command_Line_Interface page
* Read details on the XAPI_Management_Tools page
== Further Information ==
* See Category:XAPI and Category:XCP
== 3rd party Integrations and tools ==
* Open Source
* Commercial
== Tutorials and presentations ==
* XCP: Nuts & Bolts
* XCP: From the Enterprise to the Data Center
Category:XAPI
Category:Glossary
Category:Users
Category:Cloud
XAPI Design and Code Review

The Xapi code repositories are on Github in the xapi-project organisation.
If you want to make a change to Xapi then you should:
# if it's a bug: make a pull request with a code fix (maybe simply an experimental one for comments-- don't be shy!)
# if it's a new feature or impacts multiple components: describe your proposed changes in markdown first in a design doc pull request. This allows the general approach to be checked before you spend too much time coding everything up. Assuming everything is ok, you should also produce updates to the relevant technical documentation too, so they don't get out-of-sync.
To understand the level of detail required of a design doc, have a look at the Design Docs Index.
In both cases (code changes and doc changes) you should expect prompt feedback. If nothing has happened for a week or so, feel free to comment on the pull request to ask if there has been any progress and/or send an email on the xen-api mailing list
If at any point you aren't sure what to do then ask on the xen-api mailing list
Category:XAPI Devel‎