Mwlib

=== Notes ===
====Notes on building mwlib 'offline collection generator' hosts . . .====
ocg1 installed - testing cherrypicked-to-production puppet work here
ocg2 installed and are awaiting puppetization
ocg3 has a firmware issue that needs resolution
http://mwlib.readthedocs.org/en/latest/renderserver.html
http://mwlib.readthedocs.org/en/latest/installation.html#ubuntu-install
http://hynek.me/articles/python-deployment-anti-patterns/
Ref. http://wiki.debian.org/Python
* use site-packages?
* use dh_python2?
Should these be isolated from stock python packages, i.e. in an alternate library path, etc?
dpkg-buildpackage -rfakeroot -b
====Labs pediapress project====
bob - build machine, ip 10.4.0.90
pediapress-ocg1 - ubuntu precise test host
pediapress-ocg2 - ubuntu lucid test host
pediapress-packager - ubuntu lucid host for packaging
==== To Be Determined ====
1) daemonizing: runit, init, upstart?
2) package individual modules or entire homedir heap as a single package?
3) depending on #2, install modules system-wide or under a user? 
4) other user and filesystem layout specifics not covered in 1-3
=== Build process ===
==== puppet does this already ====
 apt-get install -y dvipng g++ gcc git-core imagemagick libfreetype6-dev libjpeg-dev liblcms-dev \
 libxml2-dev libxslt-dev libz-dev make ocaml-nox pdftk ploticus python python-dev python-docutils \
 python-greenlet python-imaging python-lxml python-pyparsing python-pypdf python-virtualenv \
 texlive-latex-recommended
 groupadd pp
 useradd -m -c "pediapress" -g pp -d /opt/pp pp
==== this part needs puppetification ====
 su - pp
 virtualenv --distribute ~/local; export PATH=~/local/bin:$PATH; hash -r; export PIP_INDEX_URL=http://pypi.pediapress.com/simple/
 pip install pil # provided by python-imaging
 pip install pyfribidi # no additional dependencies
 pip install mwlib # pulls odfpy-0.9.4, gevent-1.0b2, timelib-0.2.3, bottle-0.10.9, apipkg-1.0, qserve-0.2.7, roman-1.4.0, py-1.4.7, sqlite3dbm-0.1.4
 pip install mwlib.rl # pulls mwlib.ext, pygments-1.5
 pip install mwlib.zim # pulls pyzim
 set up daemons
=== Packages to Build ===
==== Stock Packages That Should Work ====
 apipkg-1.0
 ubuntu: python-apipkg (lucid:n/a precise:1.0-1.1build1)
 bottle-0.10.9
 ubuntu: python-bottle (lucid:0.6.4-1 precise:0.10.6-1)
 source package: git://git.debian.org/collab-maint/python-bottle.git
 http://bottlepy.org/docs/0.10/changelog.html
 stock package looks tricky to backport due to debian/control build dependencies
 pygments-1.5
 ubuntu: python-pygments (lucid:1.2.2+dfsg-1ubuntu1 precise:1.4+dfsg-2)
 http://pygments.org/docs/changelog/#version-1-5
 source: http://pypi.python.org/packages/source/P/Pygments/Pygments-1.5.tar.gz
 on precise, it's pulled in as a python-docutils dependency
 c-ares
 ubuntu: libc-ares-dev (lucid:1.7.0-1-1 precise:1.7.5-1)
 ubuntu: libc-ares2 (lucid:1.7.0-1 precise:1.7.5-1)
 libev
 ubuntu: libev-dev (lucid:1:3.8-1 precise:1:4.11-1)
 ubuntu: libev3 (lucid:1:3.8-1)
 ubuntu: libev4 (precise:1:4.11-1)
 reportlab
 ubuntu: python-reportlab (lucid:2.4-1 precise:2.5-1.1build1)
 reportlab-accel
 ubuntu: python-reportlab-accel (lucid:2.4-1 precise:2.5-1.1build1)
 cython
 ubuntu: cython (lucid:0.11.2-1 precise:0.15.1-1ubuntu1)
 python-lxml
 ubuntu: python-lxml (lucid:2.2.4-1 precise:2.3.2-1)
 cxxtools
 ubuntu: libcxxtools-dev (lucid:1.4.8-3 precise:2.0-4ubuntu2)
 tntdb
 ubuntu: libtntdb-dev (lucid:1.0.1-3build1 precise:1.1-1)
 xz
 ubuntu: (?) xz-utils (lucid:4.999.9beta+20091116-1 precise:5.1.1alpha+20110809-3)
==== New Packages That Build Easily ====
 odfpy-0.9.4
 * built for lucid & precise
 py-1.4.7
 * built for lucid & precise
 qserve-0.2.7
 * built for lucid & precise
 sqlite3dbm-0.1.4
 * built for lucid & precise
 pyfribidi-0.11.0
 ubuntu: python-pyfribidi (lucid:0.10.0-2 precise:0.10.0-4), both patched for buffer overflow
 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=663189
 http://pypi.python.org/pypi/pyfribidi/
 upstream appears dead, pediapress released 0.11.0 but changelog ends at 0.10.1
 comment: 0.11 and 0.10.1 have the same API. 0.10.0 won't work and is still broken on ubuntu precise
 notes: remove fribidi-src before packaging
 * built for lucid & precise but needs to either replace stock package or be packaged with a different name
==== New Packages That Require Significant Refactoring Before Build ====
 pyzim (& zimlib, zimwriter)
 source: http://www.openzim.org/Releases
 http://pypi.python.org/pypi/pyzim
 build requires: cxxtools =2.0, tntdb =1.1 (for zimwriter), tntnet =2.0 (for zimreader), libxz-dev, autoconf, automake, libtool
 uses a Makefile to build cxxtools, tntdb, xz, zimlib and zimwriter so you'll want remove the vendor directory
 recommend:
 - fulfill requires from stock cxxtools, tntdb, xz
 - build zimlib, pyzim, zimwriter as necessary as discrete packages
 gevent-1.0b2
 backporting 1.0b2 to fix bugs Ralf patched and sent upstream in previous versions 
 ubuntu: python-gevent (lucid:n/a precise:0.13.6-1ubuntu1)
 stable: gevent-0.13.7
 beta: gevent-1.0b2 requires greenlet, libev & c-ares, release tarball includes dependencies
 significant changes between 0.13.7 and 1.0b2
 remove included libev and c-ares
 http://packages.debian.org/experimental/python/python-gevent
 recommend:
 - fulfill requires from stock packages
 - create python-pediapress-gevent from 1.0b2 source
 mwlib-0.13.6
 "mwlib itself also includes a slightly patched apipkg (v1.1) module. I think I already
 got the changes merged upstream. mwlib also includes a patched EasyTimeline.pl script."
 recommend:
 - determine whether stock apipkg will suffice
 - otherwise backport as a replacement for stock mwlib 
 - possibly package EasyTimeline.pl separately
 mwlib.ext-0.12.4
 provides external dependencies needed by the mwlib library
 essentially reportlab 2.5 with some patches
 http://pypi.python.org/pypi/mwlib.ext/0.12.4 (no Makefile)
 - determine whether stock reportlab will suffice
 - otherwise backport as a replacement for stock reportlab 
 mwlib.rl-0.12.10
 check with Pediapress re. font install
 mwlib.zim-0.2.0
 requires cython, lxml, mwlib, pyzim (pyzim requires zimlib, zimwriter)
 http://pypi.python.org/packages/source/m/mwlib.zim/mwlib.zim-0.2.0.zip (no Makefile)
 recommend:
 - fulfill requirements with stock packages
 - build mwlib.zim itself as new package
 timelib-0.2.3
 ./ext-date-lib is an import of php-5.2.10/ext/date/lib for use in static linking
 "timelib includes unmodified sources from php. you should also break those out
 and link against libphp (if there such a thing in ubuntu)."
 * first attempt package failed, won't build as fakeroot
 recommend:
 - preferred: reimplement in python
 - good: rewrite dynamically linked against stock php
 - acceptable: rewrite statically linked against stock php
 texvc
 - pediapress way
 git clone https://github.com/pediapress/texvc
 cd texvc; make; make install PREFIX=~/local
 - mediawiki way (?)
 http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/Math/math
 version #? use mediawiki version #? what?
 is debian/copyright sane?
 is version sane for pediapress folks?
 recommend: consult mw and pediapress devs for instructions on what to package