Xen On Funtoo From Scratch

== STUB - WORK IN PROGRESS ==
= Hardware Configuration =
* VGA: Nvidia GeForce 9800 GT
* CPU: Intel Core i7 3770
* Motherboard: DQ77MK
= Used Software =
* Dom0 OS: Funtoo GNU/Linux distro (derrived from gentoo).
* Kernel: 3.4.4
* Xen: 4.2.0-unstable rev 25099
* DomU OS: Windows XP SP3
= Reference documentation =
* Creating bootable USB-stick from gentoo livecd: http://www.gentoo.org/doc/en/liveusb.xml
* Funtoo installation: http://www.funtoo.org/wiki/Funtoo_Linux_Installation
* Xen 4.1 installation on gentoo: http://en.gentoo-wiki.com/wiki/Xen4.1
* Xen 4.2 installation on ubuntu: http://www.davidgis.fr/blog/index.php?2011/12/07/860-xen-%2042unstable-patches-for-vga-pass-through
* LVM installation guide on funtoo: http://www.funtoo.org/wiki/Rootfs_over_encrypted_lvm
= Make bootable USB =
== Preparation ==
 # emerge -av dosfstools syslinux
 
== Make partitions ==
 # fdisk -l /dev/sdb 
 
 Disk /dev/sdb: 8422 MB, 8422162432 bytes
 239 heads, 47 sectors/track, 1464 cylinders, total 16449536 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
 Disk identifier: 0x0008aec7
 
 Device Boot Start End Blocks Id System
 /dev/sdb1 * 2048 16449535 8223744 b W95 FAT32
== Install MBR ==
 # dd if=/usr/share/syslinux/mbr.bin of=/dev/sdb
 0+1 records in
 0+1 records out
 440 bytes (440 B) copied, 0.0190098 s, 23.1 kB/s
== Make fat32 filesystem ==
 # mkdosfs -F32 /dev/sdb1 
 mkdosfs 3.0.9 (31 Jan 2010)
== Download Gentoo LiveCD iso ==
Go to http://www.gentoo.org/main/en/where.xml ; pick mirror you like and download install-amd64-minimal-[0-9]*.iso, for example - this one: 
 wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-iso/install-amd64-minimal-20120621.iso
== Mount gentoo LiveCD ==
 # mkdir -p /mnt/cdrom/
 # mount -o loop install-amd64-minimal-20120621.iso /mnt/cdrom/
 mount: warning: /mnt/cdrom/ seems to be mounted read-only.
== Mount usb stick ==
 # mkdir -p /mnt/usb/
 # mount /dev/sdb1 /mnt/usb/
== Copy contents of LiveCD to usb stick ==
 # cp -aR /mnt/cdrom/* /mnt/usb/
 # mv /mnt/usb/isolinux/* /mnt/usb
 # mv /mnt/usb/isolinux.cfg /mnt/usb/syslinux.cfg
 # rm -rf /mnt/usb/isolinux*
 # mv /mnt/usb/memtest86 /mnt/usb/memtest
 # sed -i \
 -e "s:cdroot:cdroot slowusb:" \
 -e "s:kernel memtest86:kernel memtest:" \
 /mnt/usb/syslinux.cfg
== Unmount medias ==
 # umount /mnt/cdrom/
 # umount /mnt/usb/
== Install syslinux ==
 
 note guide # syslinux /dev/sdb1
= Boot from USB-stick =
If in trouble, try following gentoo handbook
= Install Funtoo =
Once booted, do this:
== Partition hard drive ==
 # gdisk -l /dev/sda 
 GPT fdisk (gdisk) version 0.8.4
 
 Partition table scan:
 MBR: protective
 BSD: not present
 APM: not present
 GPT: present
 
 Found valid GPT with protective MBR; using GPT.
 Disk /dev/sda: 1953525168 sectors, 931.5 GiB
 Logical sector size: 512 bytes
 Disk identifier (GUID): DDF92951-E1C2-4138-B01C-71051D08D1FB
 Partition table holds up to 128 entries
 First usable sector is 34, last usable sector is 1953525134
 Partitions will be aligned on 2048-sector boundaries
 Total free space is 2014 sectors (1007.0 KiB)
 
 Number Start (sector) End (sector) Size Code Name
 1 2048 67583 32.0 MiB EF02 BIOS boot partition
 2 67584 1091583 500.0 MiB 8300 Linux filesystem
 3 1091584 17868799 8.0 GiB 8200 Linux swap
 4 17868800 1953525134 923.0 GiB 8E00 Linux LVM
== Setup LVM ==
 # pvcreate /dev/sda4 
 Writing physical volume data to disk "/dev/sda4"
 Physical volume "/dev/sda4" successfully created
 # vgcreate data /dev/sda4 
 Volume group "data" successfully created
 # lvcreate -L40G -n root data
 Logical volume "root" created
 # lvcreate -L40G -n winxp data
 Logical volume "winxp" created
== Setup Filesystems ==
 # mkfs.ext3 /dev/sda2
 # mkswap /dev/sda3 
 # mkfs.ext4 /dev/data/root 
== Mount filesystems ==
 # mkdir /mnt/funtoo
 # mount /dev/data/root /mnt/funtoo/
 # mkdir /mnt/funtoo/boot
 # mount /dev/sda2 /mnt/funtoo/boot/
 # cd /mnt/funtoo/
== Setup networking inside of LiveUSB OS ==
 # /etc/init.d/dhcpcd start
== Download Funtoo stage3 distribution ==
 # wget http://ftp.osuosl.org/pub/funtoo/funtoo-current/x86-64bit/corei7/stage3-current.tar.xz
== Unpack Funtoo distribution ==
 # tar xJpf stage3-current.tar.xz
== Chrooting ==
 # mount --bind /dev/ /mnt/funtoo/dev/
 # mount --bind /proc/ /mnt/funtoo/proc 
 # cp /etc/resolv.conf /mnt/funtoo/etc/
 # chroot /mnt/funtoo/
== Syncing ==
 # emerge --sync
= Configure Funtoo =
=== /etc/fstab ===
 # <fs>			<mountpoint>	<type>		<opts>		<dump/pass>
 
 /dev/sda2		/boot		ext3		defaults	1 2
 /dev/sda3		none		swap		sw		0 0
 /dev/data/root		/		ext4		noatime		0 1
=== /etc/localtime ===
 # cp /usr/share/zoneinfo/Europe/Moscow /etc/localtime 
=== /etc/make.conf ===
 ACCEPT_KEYWORDS="~amd64"
 CHOST="x86_64-pc-linux-gnu"
 CFLAGS="-march=corei7 -O2 -pipe"
 CXXFLAGS="-march=corei7 -O2 -pipe"
 SYNC="git://github.com/funtoo/ports-2012.git"
 
 USE="X xinerama"
 
 VIDEO_CARDS="nvidia"
 
 MAKEOPTS="-j10"
== Install system-wide services ==
 # echo "sys-boot/boot-update" >> /etc/portage/package.unmask
 # echo "sys-devel/automake" >> /etc/portage/package.unmask
 # emerge -av eix lvm2 grub bridge-utils vixie-cron syslog-ng logrotate =sys-boot/boot-update-1.6.2
 # eix-update
 # rc-update add lvm default
 # rc-update add vixie-cron default
 # rc-update add syslog-ng default
 # rc-update add sshd default
 # rc-update add dhcpcd default
 # passwd
= Install Kernel =
 # emerge -av gentoo-sources 
= Configure kernel =
 # cd /usr/src/linux
 # make menuconfig
 
 General setup --->
 <*> Kernel .config support
 [*] Enable access to .config through /proc/config.gz
 
 Bus options (PCI etc.) --->
 <*> PCI Stub driver
 
 Processor type and features ---> 
 [*] Paravirtualized guest support ---> 
 [*] Xen guest support
 
 [*] Networking support --->
 Networking options --->
 <*> 802.1d Ethernet Bridging 
 
 Device Drivers ---> 
 [*] Block devices (NEW) --->
 <*> Xen virtual block device support
 <*> Xen block-device backend driver
 
 Device Drivers ---> 
 [*] Network device support --->
 [*] Ethernet driver support (NEW) ---> 
 [*] Intel devices (NEW)
 < > Intel(R) PRO/100+ support
 < > Intel(R) PRO/1000 Gigabit Ethernet support
 <*> Intel(R) PRO/1000 PCI-Express Gigabit Ethernet support
 < > Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support
 < > Intel(R) 82576 Virtual Function Ethernet support
 < > Intel(R) PRO/10GbE support
 < > Intel(R) 10GbE PCI Express adapters support
 < > Intel(R) 82599 Virtual Function Ethernet support
 [ ] Intel (82586/82593/82596) devices
 <*> NetXen Multi port (1/10) Gigabit Ethernet NIC
 
 Device Drivers ---> 
 [*] Network device support --->
 <*> Universal TUN/TAP device driver support 
 <*> Xen network device frontend driver
 <*> Xen backend network device
 
 Device Drivers ---> 
 Xen driver support ---> 
 [*] Xen memory balloon driver (NEW) 
 [*] Scrub pages before returning them to system (NEW) 
 <*> Xen /dev/xen/evtchn device (NEW) 
 [*] Backend driver support (NEW) 
 <*> Xen filesystem (NEW) 
 [*] Create compatibility mount point /proc/xen (NEW) 
 [*] Create xen entries under /sys/hypervisor (NEW) 
 <*> userspace grant access device driver (NEW) 
 <*> User-space grant reference allocator driver (NEW) 
 <*> xen platform pci device driver (NEW)
 <*> Xen ACPI processor
 
 File systems --->
 <*> Ext3 journalling file system support
 <*> The Extended 4 (ext4) filesystem
 [*] Use ext4 for ext2/ext3 file systems (NEW)
 [*] Ext4 extended attributes (NEW)
 # make -j 10
 # make install
 # make modules_install
== Install initramfs ==
 # mkdir /root/initramfs
 # cd /root/initramfs/
 # git clone git://github.com/slashbeast/better-initramfs.git
 # cd better-initramfs
 # bootstrap/bootstrap-all
 # make prepare
 # make image 
 # cp output/initramfs.cpio.gz /boot/
= Configure bootloader =
 # grub-install --no-floppy /dev/sda
 
=== /etc/boot.cfg ===
 boot {
 generate grub
 default "Funtoo Linux XEN" 
 timeout 3 
 }
 
 "Funtoo Linux" {
 kernel vmlinuz[-v]
 initrd /initramfs.cpio.gz
 params += root=/dev/data/root lvm
 }
 
 "Funtoo Linux XEN" {
 type xen
 xenkernel xen.gz
 kernel vmlinuz[-v]
 initrd /initramfs.cpio.gz
 params += root=/dev/data/root lvm
 }
=== Update ===
 # boot-update
= Booting in new system =
When booting, select regular (non-XEN) kernel at the first time. It is because we don't have XEN userspace tools to operate with yet.
 # reboot
= Install xen 4.2-unsable =
== Get current Nvidia flash ROM ==
 # wget http://www.davidgis.fr/download/nvflash_5.100.1_usb.iso.tar.bz2
 # tar xvf nvflash_5.100.1_usb.iso.tar.bz2
 # dd if=nvflash_5.100.1_usb.iso of=/dev/sdb
 
 # reboot
=== Boot from USB stick ===
 C:\> nvflash.exe --save vgabios.rom
=== Boot back to the system ===
Choose non-xen kernel again
== Install XEN 4.2 ==
 # emerge mercurial yajl dev86 bin86 iasl
 # rev=25099; hg clone -r $rev http://xenbits.xensource.com/staging/xen-unstable.hg/ xen-unstable.hg-rev-${rev}
 # cd xen-unstable.hg-rev-25099/tools/
 # ./configure
 # sed '/Werror/d' -i tests/mce-test/tools/Makefile
 # make FLEX=flex -j 10
 # make clean
 # cd ../
 # wget http://www.davidgis.fr/download/xen-4.2_rev24798_gfx-passthrough-patchs.tar.bz2
 # tar -xvf xen-4.2_rev24798_gfx-passthrough-patchs.tar.bz2
 # for file in xen-4.2_rev24798_gfx-passthrough-patchs/*; do patch -p1 < $file; done
Now the tricky part. You have to tune one of asl-files to support your very own VGA card memory ranges.
It goes like there:
 # lspci | grep VGA
 01:00.0 VGA compatible controller: NVIDIA Corporation G92 [GeForce 9800 GT] (rev a2)
 # dmesg | grep 01:00.0 | grep mem | head -n3
 [ 0.281769] pci 0000:01:00.0: reg 10: [mem 0xf6000000-0xf6ffffff]
 [ 0.281776] pci 0000:01:00.0: reg 14: [mem 0xe0000000-0xefffffff 64bit pref]
 [ 0.281783] pci 0000:01:00.0: reg 1c: [mem 0xf4000000-0xf5ffffff 64bit]
These three ranges goes to tools/firmware/hvmloader/acpi/dsdt.asl by formula 
 MaxRange - MinRange + 1
(consider using some hex calculator (emerge wcalc), for example - `wcalc -h`)
In this example this would be:
* 0xf6ffffff - 0xf6000000 + 1 = 0x01000000
* 0xefffffff - 0xe0000000 + 1 = 0x10000000
* 0xf5ffffff - 0xf4000000 + 1 = 0x02000000
Then edit tools/firmware/hvmloader/acpi/dsdt.asl file as follows:
 /* reserve MMIO BARs of gfx for 1:1 mapping */
 DWordMemory(
 ResourceProducer, PosDecode, MinFixed, MaxFixed,
 Cacheable, ReadWrite,
 0x00000000,
 0xf6000000,
 0xf6ffffff,
 0x00000000,
 0x01000000)
 
 DWordMemory(
 ResourceProducer, PosDecode, MinFixed, MaxFixed,
 NonCacheable, ReadWrite,
 0x00000000,
 0xe0000000,
 0xefffffff,
 0x00000000,
 0x10000000)
 
 DWordMemory(
 ResourceProducer, PosDecode, MinFixed, MaxFixed,
 Cacheable, ReadWrite,
 0x00000000,
 0xf4000000,
 0xf5ffffff,
 0x00000000,
 0x02000000)
Remember USB stick with vgabios.rom on it we dumped before? Now we need it.
 # mkdir /mnt/usb
 # mount /dev/sdb1 /mnt/usb/
 # cp /mnt/usb/VGABIOS.ROM tools/firmware/vgabios/vgabios-pt.bin
Thats it. Now simply (and dirty) make && make install all needed components:
 # make xen && make tools && make stubdom 
 # make install-xen && make install-tools PYTHON_PREFIX_ARG= && make install-stubdom
Add following script to /etc/init.d/xen
 #!/sbin/runscript
 # Copyright 1999-2011 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 # $Header: /var/cvsroot/gentoo-x86/net-misc/openssh/files/sshd.rc6.3,v 1.2 2011/09/14 21:46:19 polynomial-c Exp $
 
 start() {
 /etc/init.d/xencommons start
 /etc/init.d/xend start
 }
 
 stop() {
 /etc/init.d/xend stop
 /etc/init.d/xencommons stop
 }
 # chmod +x /etc/init.d/xen
 # rc-update add xen default
Now you can safely choose XEN kernel - it should be bootable.
 # reboot
* install winxp
* integreate
Category:Xen
Category:Beginners
Category:Tutorial
Category:Funtoo