XCP Building Instructions

__NOTOC__
== Building xapi on CentOS 6.3 ==
In this guide we look at an experimental way to get xapi running on CentOS 6.3
This is not Zeus, there are no packages yet. This is a developer install using: http://opam.ocamlpro.com/
=== Using XCP-DevInstall to build XAPI ===
Rather than following the steps below, you can use the following set of scripts:
https://github.com/JohnGarbutt/xcp-devinstall
See the README in the above repo for instructions.
=== Setting up Xen ===
While not strictly required for the build, to help us test it, lets install Xen.
Minimal install of CentOS 6.3 x86_64. Use network install iso on VMware fusion, or full iso, minimal confuses things.
This is based on http://wiki.centos.org/QaWiki/Xen4
yum install wget
cd /etc/yum.repos.d/
wget http://dev.centos.org/centos/6/xen-c6/xen-c6.repo
yum repolist
yum install kernel kernel-firmware
yum install xen
Now need to edit /etc/grub.conf
Note: check the paths and the kernel version, also check dom0 memory.
Also: The above 3.4.x kernel should the second in the list, xen should be first.
title xen
 root (hd0,0)
 kernel /xen.gz dom0_mem=1024M,max:1024M loglvl=all guest_loglvl=all
 module /vmlinuz-3.4.32-6.el6.centos.alt.x86_64 ro root=/dev/mapper/vg_jgarbuttcentos-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_LVM_LV=vg_jgarbuttcentos/lv_root rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto rd_NO_DM rd_LVM_LV=vg_jgarbuttcentos/lv_swap
 module /initramfs-3.4.32-6.el6.centos.alt.x86_64.img
Reboot, then check all is well:
uname -a
xl info
=== Install OCAML 3.12.1 ===
Currently this is not part of CentOS 6.3
One way is to build the following Fedora 17 srpm:
http://dl.fedoraproject.org/pub/fedora/linux/releases/17/Everything/source/SRPMS/o/ocaml-3.12.1-1.fc17.src.rpm
Instructions can be found here, including the required setup:
http://wiki.centos.org/HowTos/RebuildSRPM
You can download some very rough packages, and install them like this:
mkdir ocaml
cd ocaml
wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-3.12.1-12.el6.x86_64.rpm
wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-camlp4-3.12.1-12.el6.x86_64.rpm
wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-camlp4-devel-3.12.1-12.el6.x86_64.rpm
wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-runtime-3.12.1-12.el6.x86_64.rpm
wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-docs-3.12.1-12.el6.x86_64.rpm
wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-ocamldoc-3.12.1-12.el6.x86_64.rpm
yum install gdbm-devel ncurses-devel rpm-build
rpm -i ocaml-*
The were created using this source rpm:
* http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-3.12.1.ocamlspotter-unknown.src.rpm
=== Install OPAM (from source) ===
Before you build, you will need:
* ocaml-3.12.1 ocaml-runtime ocaml-camlp4 ocaml-camlp4-dev (see above)
And a few other tools:
yum install make gcc
Build instructions are here:
http://opam.ocamlpro.com/doc/Advanced_Install.html
Download the latest stable, and build it:
wget https://github.com/OCamlPro/opam/archive/latest.tar.gz -O opam-latest.tgz
tar -xf opam-latest.tgz
cd opam-latest
./configure
make
make install
Now you have OPAM you can initialise it:
opam init
eval `opam config env`
To make sure you don't need the above line all the time, add the following into .bash_profile before the PATH is set:
eval `opam config env`
=== Install OCAML 4.00.1 ===
It is now possible to install the latest OCAML
opam switch 4.00.1
Now setup the environment again:
eval `opam config env`
=== Install External Dependencies ===
==== Yum ====
There are a few dependencies we need:
yum install git xen-devel 
yum install libuuid-devel time pam-devel tk-devel libvirt-devel zlib-devel # you may not actually need tk-devel, try without to avoid X dependencies
==== Unpackaged ====
XCP has a custom version on vncterm:
wget http://a94cd2de16980073c274-9e5915cce229bfd373f03bf01a9a7c85.r57.cf3.rackcdn.com/vncterm-1.6.10-251.x86_64.rpm
rpm -i vncterm-1.6.10-251.x86_64.rpm
This was built for CentOS 6.3 from the XCP srpm:
* http://a94cd2de16980073c274-9e5915cce229bfd373f03bf01a9a7c85.r57.cf3.rackcdn.com/vncterm-1.6.10-251.src.rpm
There is also an important python script:
wget http://a94cd2de16980073c274-9e5915cce229bfd373f03bf01a9a7c85.r57.cf3.rackcdn.com/eliloader -O /usr/bin/eliloader
chmod +x /usr/bin/eliloader
This was taken from the debian version:
* https://github.com/xapi-project/xcp-eliloader
==== OPAM ====
There are quite a few XCP libraries we can install using OPAM:
opam install ocamlfind omake
opam remote add xen-dev git://github.com/xapi-project/opam-repo-dev
opam install ssl xen-api-libs-transitional stdext nbd tapctl libvhd oclock cdrom netdev xenopsd
=== Build XAPI ===
There is work going on to improve the build and moving it towards OPAM. But XAPI is still not packaged using OPAM. You can find this work here:
https://github.com/jonludlam/xen-api/tree/ea-1254
A tree with additional fixes for building on CentOS 6.3 can be found here:
https://github.com/JohnGarbutt/xen-api/tree/centos63
Now lets try and build this:
git clone https://github.com/JohnGarbutt/xen-api.git
cd xen-api
git checkout centos63
make
set DEST_DIR=/tmp/xen-api
mkdir $DEST_DIR
make install
if you need to start again, try:
find -name "*.omc" | xargs rm -f
make clean
=== Configuration ===
Now we need to configure things correctly. Append the following settings at the end of the appropriate file.
./.opam/system/etc/xenopsd.conf
sockets-path=/var/lib/xcp # use the legacy path for old xapi
sockets-group=root
hvmloader="/usr/lib/xen/boot/hvmloader"
pygrub=/usr/lib64/xen/bin/pygrub
vncterm=/usr/lib64/xen/bin/vncterm
/etc/xensource/network.conf
openvswitch
=== Building and Running OVS ===
While not needed, this is handy, and assumed below.
The info below is a modified version of these docs:
* http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL;hb=HEAD
Some build prerequs:
yum install openssl-devel
autoconf 2.64 is required by 2.63 is packaged, so we try and get a more recent version from here:
* http://ftp.gnu.org/gnu/autoconf/autoconf-latest.tar.gz
* http://ftp.gnu.org/gnu/automake/automake-1.13.tar.xz
Due to issues with pkgconfig rpm, you then need to (see http://openvswitch.org/pipermail/discuss/2012-August/008064.html):
cp /usr/share/aclocal/pkg.m4 /usr/local/share/aclocal-1.13/
Now you can pull the latest ovs code and start to build it:
wget http://openvswitch.org/releases/openvswitch-1.4.5.tar.gz
tar -xf openvswitch-1.4.5.tar.gz
cd openvswitch-1.4.5
./boot.sh
./configure
make
Edit the Makefile so that you have the following setting, to match those expected by XenAPI:
prefix=/usr
Now we can install the binaries:
make install
==== Running OVS ====
First we need to turn off the bridge module, and check it is not loaded:
rmmod bridge
lsmod | grep bridge
We can make it stop loading in future by:
echo >> /etc/modprobe.d/blacklist.conf <<EOF
# Stop using bridge, using openvswitch instead
blacklist bridge
EOF
Now we need to load the kernel module for ovs:
insmod /lib/modules/3.4.32-6.el6.centos.alt.x86_64/kernel/net/openvswitch/openvswitch.ko
First time, we create the ovs db:
mkdir -p /usr/etc/openvswitch
ovsdb-tool create /usr/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
Now we can start the ovs config server (at the moment, this is without ssl):
ovsdb-server --remote=punix:/usr/var/run/openvswitch/db.sock \
 --remote=db:Open_vSwitch,manager_options \
 --pidfile --detach
First time, you need to init the db:
ovs-vsctl --no-wait init
Now start main ovs deamon listening to same port:
ovs-vswitchd --pidfile --detach
=== Running xapi ===
There are a few more runtime dependencies:
yum install dmidecode
We can now run all the services we need:
./.opam/system/bin/xcp-fe # used to fork process service
./.opam/system/bin/xenopsd -config ./.opam/system/etc/xenopsd.conf # the component that talks to xen
/opt/xensource/libexec/v6d #the license server
/opt/xensource/libexec/xcp-networkd
Now we can run xapi:
/opt/xensource/bin/xapi # the api server, and database, and others things too
=== Running XAPI and OVS after restart ===
After you restart, by default, non of the services will restart. To help development, we can run theses services in a screen by doing:
git clone https://github.com/JohnGarbutt/xcp-devinstall.git
cd xcp-devinstall
./run.sh
screen -x xcp
=== Configure XenAPI Networking ===
No we have xapi running, we can ask it to detect all the physical interfaces:
host_uuid=`xe host-list --minimal`
xe pif-scan host-uuid=$host_uuid
# assuming a single pif
pif_uuid=`xe pif-list --minimal`
xe pif-reconfigure-ip uuid=$pif_uuid mode=dhcp
xe host-management-reconfigure pif-uuid=$pif_uuid
To update OVS config we can do the following (TODO - find plugin, configure inventory):
host_uuid=`xe host-list --minimal`
xe host-call-plugin host-uuid="$host_uuid" plugin="openvswitch-cfg-update" fn="update"
=== Configure XenAPI Storage ===
We first need to instal the storage SRs:
cd ~
git clone https://github.com/JohnGarbutt/xcp-storage-managers.git
cd xcp-storage-managers
yum install swig python-devel
export DESTDIR=/
export PYTHON_INCLUDE=/usr/include/python2.6/
make
We need an SR to store the VM disks. It is best to add a new disk or partion, then just use that:
sr_uuid=`xe sr-create type=file device-config:location=/root/vhds name-label=localstorage sm-config:type=vhd`
pool_uuid=`xe pool-list --minimal`
xe pool-param-set uuid=$pool_uuid default-SR=$sr_uuid
== Xen Cloud Platform Building Instructions (CentOS 5.6) ==
=== Building the xapi toolstack ===
The recommended way to build the xapi toolstack is in an environment similar to that in which it will be installed - therefore you'll need a CentOS 5.5 32-bit OS installed - we tend to use VMs for this.
For convenience, there are YUM repositories hosted on downloads.xen.org that contain the build dependencies of xapi. To enable these, create a a file /etc/yum.repos.d/xcp.repo with the following contents:
[xcp]
name = xcp
baseurl = http://downloads.xen.org/XCP/xcp-1.0-repo
gpgcheck = 0
If you're building XCP unstable, replace '1.0' in the URL with 'unstable'.
Then execute the following commands:
rpm -e libaio
yum install gcc autoconf automake tetex ghostscript java-1.6.0-openjdk java-1.6.0-openjdk-devel ant pam-devel python-devel zlib-devel openssl-devel dev86 rpm-build texinfo flex bison sharutils elfutils-devel ncurses-devel e4fsprogs-devel ocaml ocaml-findlib ocaml-getopt ocaml-type-conv ocaml-xmlm omake ocaml-xmlm-devel xen-devel xapi-libs-devel xapi-libs-utils
You'll also need mercurial or git depending upon whether you're building the unstable version of XCP or 1.0. These are both available in the EPEL repository (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/)
Check out the xapi source:
hg clone http://xenbits.xen.org/XCP/xen-api.hg
or
git clone git@github.com:xapi-project/xen-api.git
and then build it:
cd xen-api.hg
make
or 
cd xen-api
make
It is also possible to build xapi in a non-CentOS 5.5 32-bit environment with the help of mock. Instructions for this are coming.
Category:XAPI Devel
Category:HowTo
Category:XCP