Xen Document Days/TODO

----
Go back to Xen Project Document Days
== Requested Documentation ==
== THEME OF THE MONTH: "Beginner Information" ==
We've seen an increase of interest from potential new users recently and we need to make sure that people can successfully get started with 4.6 using our current set of Wiki pages. We have been slowly replacing the xm references with xl, but we could use people to read over many of the existing pages to make sure they still make sense with the latest release.
Pages of special interest include:
* Xen Project Beginners Guide (recently revised, but could use a read-through)
* Getting Started
* Xen Project Best Practices
* Xen Common Problems
And, of course, any other page which needs to reflect the realities of working with 4.6.
== Other Items of Interest ==
The following topics also need addressing:
=== Items Requested by Users ===
The following requests have appeared in our IRC channel and UserVoice for additional documentation. Recent requests include the following (feel free to add additional topics as they appear):
* Development Documentation: An overview of the driver architecture. This may already exist, but it needs to be more prominently visible. If it's out there, it is not as easy to find as it should be.
* User Documentation: Simple example of moving a VM from one machine to another offline and possibly changing toolsets as it goes. Wei recently gave the summary below in ##xen. We need a page with more meat on the bones, particularly for migrating between toolkits and across multiple versions of the hypervisor (e.g., moving from 4.1 to 4.4).
 xm save domain image; scp image user@remote:~; scp domu.cfg user@remote:~; 
 modify domu.cfg on remote host; xl restore
* User Documentation: Update to the Xen_Windows_GplPv doc. The drivers are still used, but the information on the page is dated. It needs a refresh.
* Build a NEW HOW-TO describing how to create a Xen 4.5.1 AMD64 environment. This should include sub-HOWTOs for networking and DomU creation.
 From Uservoice:
 All the examples for installing and using Xen are currently antiquated.
 Can someone just record how they set up a new system using 
 Ubuntu 14/15, Xen 4.5.1, and XAPI/xl? Sections on setting up 
 networking, passthru, BIOS bugs and *NIX/Win/OSX would be 
 exactly what is needed.
=== New HowTos Needed ===
We've got a lot of good information in the documentation, but it isn't always in a form that will help someone use a particular feature for the first time. Or, the information is present, but it might need to be brought up-to-date.
Some of these possible topics include:
* Installing on ARM (need a basic HowTo)
* Using Xen Project with common applications (see this Q&A post about Citrix Netscaler)
* Get up and running with MirageOS
* HowTo use smbios_firmware option. See this thread for background.
* Any other niche topic which doesn't have a good HowTo-type doc page
=== Man pages loose ends ===
* move the xenstat/xentrace manpages into docs
* there are a few misc documents scattered through tools/ which would be better moved into docs/
=== Other Pages loose ends and questions ===
* Talk:Setting_boot_order_for_domUs
=== Remnants of the Great 4.5 Clean-Up ===
There are a lot of pages which rely on xm which need to be updated to xl. If you notice one and don't have time to fix it, add it here for others to do:
* Xen_Project_Beginners_Guide (should default to xl, and give a nod to xm for old releases)
** Lars.kurth 12:00, 27 February 2015 (UTC) : This may not be entirely straightforward and may need to wait until we have a Debian version shipping Xen 4.5 - although jessie ships 4.4. Some of the text also refers to quite old xen versions, which should be looked at. For example it does use xen-tools and other external tools are in use and someone will need to go through the page and check whether everything works as intended.
** For example xen-tools, xt-create-xen-config seems to have something which writes a sudo stanza for you, involving xm. xm-create-image has the ability to start images with xm. It does NOT have a config option (--xen-toolstack=xl) for changing the command, but contains a heuristic to try and figure our what to use (see ). Thus, it may be easy to fix, although AFAICT it isn't used everywhere. Information from "dpkg -L xen-tools | xargs grep -e '\bxm\b' |less". Overall conclusion: there are some nits to pick, but by and large it will work just fine with xl. The package ought to be updated. 
* XenCommonProblems
* Xen_FAQ_High_Availability
* Xen_Project_Best_Practices
=== Performance-related Documentation ===
The documentation on Performance Tuning seems thin, given the importance of the subject:
* :Category:Performance Lists the Performance docs currently available. It seems rather sparse.
* Network_Throughput_and_Performance_Guide Has lots of interesting information, but it is oriented for XenServer/XCP. This needs to be recast as a true Xen Project document.
* Tuning_Xen_for_Performance Is potentially the most important document, but it has only a few topics and not that many details. There is some discussion about this page in section 3.12
=== Integration Documentation ===
These documents are for projects which users need to use in conjunction with Xen Project. The goal of these documents is not to repeat the documentation found in other projects, but to give people the essential information required to integrate them into their Xen Project installation.
Most of these documents are still quite new and can benefit from improvements by people who have used these projects.
* Using Ceph storage with Xen Project
* Using Gluster file system with Xen Project
* Using ConVirt virtualization management platform with Xen Project
* Using Xen Project in OpenStack clouds
* Using Xen Project in CloudStack clouds
* Using Xen Project in OpenNebula clouds
* Unikernels (e.g., Mirage OS, LING, HaLVM, OSv, etc.) and how they relate to Xen Project
=== Missing bits in Beginners Guide ===
Xen_Project_Beginners_Guide has a few TODOs in it. In particular, it is missing some diagrams
=== (IMPORTANT) Linux Distros ===
It seems that the list of Linux Distros which support Xen is a lot larger than listed on this wiki. 
* Funtoo
* Mageia
* Univention Corporate Server (UCS)
* Are there others?
=== Easy tasks for Beginners ===
You may want to read Wiki Help if you do not have an account.
==== Migrate useful blog posts to wiki ====
Sift through blog.xenproject.org and copy useful posts to wiki. This will mainly be a reformatting exercise. Before you copy an article, perform a search on the wiki (headline or key word) to see whether the post exists already. If in doubt, ask on the IRC channel.
==== Fix articles that need attention ====
* :Category:Contains_Needs_Formatting - Pages that need formatting
* :Category:Contains_Needs_Action - Pages that need fixing. Go through these, and if you get stuck ask people on the IRC channel for help, or go to the next page.
* :Category:Contains_TODO - Pages that have a TODO item on the page. Go through these, and if you get stuck ask people on the IRC channel for help, or go to the next page.
==== Review Categorization ====
* Sometimes new pages on the wiki are not created with categories. You can find these by going to Special:UncategorizedPages. Documentation about which category is used where can be found in Categories for Authors. If in doubt ask on IRC.
==== Pages With Comments or Improvement Suggestions ====
* We ask users to highlight issues using talk pages, following the conventions outlined in Help:Talk Pages. Comments made using these conventions can be identified using the :Category:Pages with comments.
==== Links to Developer Documents on XenBits ====
There are a number of developer-related documents which are hosted on XenBits which should be locatable through this Wiki. Ideally, someone should work backwards from the XenBits document index to make sure each document exists in a link from the appropriate Wiki page.
If no appropriate Wiki page exists, one should be created or listed here for creation. For function-specific docs which don't necessarily deserve a Wiki page, we should have a catch-all page to let people know that these docs exist.
==== Missing Pages ====
* Special:WantedPages - You can find missing pages by going to this link. There are several courses of action that can be taken:
** The page is needed : Add {{TODO|With a comment}} to the top of the page
** The page is not needed : Remove the reference to the page (follow the link in the bracket besides the page)
** Note that the list of Wanted pages contains lots of needed translations and categories, which you can ignore
** If in doubt, ask for help on the IRC channel
 There are a gazillion Template-Whatever including a TODO, really needed? 
 See #Special:WantedPages_broken
==== Update links to defunct gentoo-wiki.com site ====
According to it is lost and not coming back. That site has archives but it seems like is the new home. Pages referring to gentoo-wiki.info ought to be updated.
=== Xen : Open work items ===
==== Documentation for bootloader (grub) integration ====
Need to document available `/etc/default/grub` settings such as `GRUB_CMDLINE_XEN`, `GRUB_CMDLINE_XEN_DEFAULT``, and `GRUB_CMDLINE_LINUX_XEN_REPLACE` + `GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT` (which override the default `GRUB_CMDLINE_LINUX` and `GRUB_CMDLINE_LINUX_DEFAULT` for Xen entries. This could be improved both in upstream grub and in our docs.
==== Review and Update: xenpm usage recommendations ====
There is Xen power management and Xenpm command which may all be outdated. These should probably be reviewed and updated.
* There has been a bit of activity on Talk:Xen power management which needs to be merged
==== User focused guide for setting up host EFI boot ====
We have http://xenbits.xenproject.org/docs/unstable/misc/efi.html but could use something a bit more user focused, e.g. a walkthrough type guide.
==== UEFI Secure Boot ====
Write a document on how to sign the xen binary for UEFI Secure Boot. Potential reference here: Dealing With Secure Boot
==== Mirage OS ====
While much documentation is available at their website, we need to have pages here which describe Mirage OS and give useful pointers to their docs.
Ideally, it would be good to have extensive documentation for this project on this Wiki, as Mirage OS is being fostered by Xen Project.
==== Documentation on lib(xen)vchan ====
The header has some stuff, but a more complete guide would be useful.
The usage of the example code is also not documented. http://lists.xen.org/archives/html/xen-users/2014-01/msg00162.html has some (incomplete) info.
==== Update Xen Windows GplPv page ====
The page is sorely out of date (e.g., the download pages don't appear to exist anymore). Even though there are other Windows PV drivers, this set is still in demand.
==== Describe the different ways, and necessary components, to create PVs using xl ====
Raised by mail by Matthias Blankenhaus (who is prepared to take the lead, but needs help):
This document would describe the different ways, and necessary components that each way requires, to create PVs using xl. Right now there are at least the following cases:
* loopbacked vhds
** single partition format
** disk format
* qemu backed vhds
** qcow / qcow2
* LVS backed image
This is not a complete list. Also the mileage of each case differs depending on the kernel features enabled in dom0/domu and also depending on the Xen version. Furthermore, certain format require additional infrastructure like the tap device. 
All in all, this a complex topic that needs clarification. Doing this based on the lowest common denominator, xl, would clarify many things. In addition, baseline performance data on all of the above would help a great deal. Matthias volunteered to run the majority of needed perf tests
==== HOWTO : using xl trigger to poweroff or firing off NMIs ====
Howto on using xl trigger to poweroff or firing off NMIs.
==== HOWTO: Chain pypxeboot and pygrub====
How to chain pypxeboot and pygrub 
* Is pypxeboot popular at all?
* I added that. It is somewhat common, OVM 2 and OVM 3 rely on it for PXE installs. I'm not sure if xenpvnetboot is the same, or works. The use case is "you want to use the same installation server for Xen VMs and phys. servers. You want to be able to reinstall them based on PXE reply.
* There is documentation for pypxeboot, but there is no (afaik) documentation for how pygrub is used as a fallback loader. We wasted many days just to get some hack that can't even load a newer kernel. Which sucks, in the end you spend more time/money making PV PXE boot work than you save by using virtualized servers.
* How could it be popular given the state of documentation. <= Key issue to the left of this sentence.
* wondering whether this video contains what is needed
* Although I've not watched the whole thing I stepped through that video enough to see all the slides (I think) and I didn't see anything about this topic or anything like it.
* I've downloaded pypxeboot 0.0.3 and as far as I can tell the fallback to pygrub is supposed to happen automatically if pxelinyx.cfg says to do a local boot. If this doesn't Just Work then I think that is a code bug rather than a documentation issue. I'd recommend retiring this docs TODO in favour of a bug report to the pypxeboot authors and/or xen-devel. I googled around a bit and didn't find any existing bug reports anywhere.
==== Refactor/refresh ARM h/w information ====
* (DONE) Xen ARM with Virtualization Extensions effectively duplicates the list of supported h/w in #Hardware and #Platform specific configuration, make it a single list.
* Update Arndale info to be less Linaro specific now that mainline is usable.
* Refactor Xen ARM with Virtualization Extensions/OdroidXU and Xen ARM with Virtualization Extensions/Arndale into a generic Eynos page + board specific notes. (and check other platform vs. board combinations).
=== Visually distinguish Xen, MirageOS and Community Documentation better ===
 Users are getting confused when looking at wiki pages. If there was a visual way to distinguish Xen, MirageOS and Other content better, this would help. We could use colors and skinning when it comes to various elements, as well as mascots/logos:
* For example: Help:Contents has a different feel (and colour scheme)
* This could be replicated in trail boxes and other elements
* We can also use mascots
But first, I need to investigate what is possible.
 Looks as if the only way this can be done is via Custom Namespaces. The only way to do this properly is to 
* Have different namespaces for Xen, MirageOS, ... and Community stuff and keep all the common stuff in Xen
* The name spaces could then be styled differently, for example the Help namespace includes the ns-12 style into the body attributes, whereas the Main (default) namespace uses the ns-14 style. 
* Custom namespaces can be created by changing the config and are linked to custom styles
* The styles are then defined in MediaWiki:Common.css
Implications of using custom namespaces:
* Search: Pages in a custom namespace do not show up in default search results. So if we had a custom MirageOS namespace, it would mean that search results for MirageOS would not show up. One would need to go to Advanced Search and select MirageOS
* Rename: Renaming pages is possible and as straightforward as renaming pages
* Xen Namespace: The Xen namespace exists already, this may need to be renamed
* Categories and Namespaces: Categories cross namespaces. In other words, Category:XXX would go across all namespaces. As an example, see :Category:Wiki_Management
=== Mediwiki Issues ===
==== Special:WantedPages broken ====
===== Issue 1: Pages that need translation are spamming the list =====
* Solution: fix language templates and mark with 
* There is something wrong with the language templates, essentially spamming the wanted pages. Do ignore these instructions until fixed.
* Fixed language template issues by removing most languages from the templates
* Started marking pages with 
===== Issue 2: Missing templates are spamming the list =====
* Solution: mark missing templates with 
* Started marking all missing templates with 
===== Issue 3: Missing categories are spamming the list ===== 
* Solution: mark with 
* Started marking pages with 
==== Categories & Indexes ====
* Combined categories such as list all XCP FAQs have been asked for sevceral times
* Investigate plug-ins
== Developer Docs ==
=== Docs that need attention or are missing ===
* Make XAPI docs (built from source) available on XenProject.org
* some more xenstore docs (i.e. recent python bindings that were sent to the -users list)
** I don't think these are included in Xen? Need a reference.
* Using libxenstat python bindings (imo they're broken)
** Is this a bug report? If they are broken we should fix them (as well as potentially documenting them)
* More PV protocol docs (see recent patches to blkif.h for example)
** blkif.h and netif.h are perhaps the most critical ones and those are now pretty good. But the rest (fbif.h, kbdif.h, pciif.h, usbif.h, tmpif.h and vscsiif.h) could all stand to be improved.
* Document xenstore paths used by guests and toolstack etc
** This is now in tree: http://xenbits.xen.org/docs/unstable/misc/xenstore-paths.html
== Done Recently ==
=== In progress (some just need double checking whether finished) ===
* FreeBSD_Dom0 - confirmed up-to-date (Feb, 2015)
==== Draft a list of Questions for a User Survey ====
* Lars.kurth 10:09, 30 April 2014 (UTC) : See Xen Project User Survey
* Lars.kurth 4 Feb 2015 (UTC) : Russell and Sarah are driving this now
==== Xen 4.3 / 4.4 Documentation : Spice Config Example for upstream QEMU ====
Take this thread and convert into an Example
 Created SPICE_support_in_Xen for now
Category:Community
Category:Events