Git-buildpackage

= Using git-buildpackage with source that already has a debian/ directory =
(extracted from #Experiments below.)
== Initially cloning the git repository from Upstream ==
* Clone the repository from upstream if it doesn't already exist in Gerrit, e.g. from git.debian.org:
 $ cd ~/git/debs/
 $ git clone git://git.debian.org/git/pkg-puppet/repo.git
* Set git username/email
 $ git config --global user.name "Your Name"
 $ git config --global user.email you@example.com
* Set up the local branches upstream and pristine-tar from the remote branches (but don't switch to them):
 $ git branch upstream origin/upstream
 $ git branch pristine-tar origin/pristine-tar
* Build a source package:
 $ gbp buildpackage --git-export-dir=/var/tmp/build-area/packagename -S
* Build the binary packages with Pbuilder:
 # pbuilder build --basetgz /var/cache/pbuilder/lucid.tgz /var/tmp/build-area/packagename/packagename_version.dsc
Note: Applying role::package::builder on a labs VM will pre-populate a pbuilder/cowbuilder environment for you. Documentation on it is kept up to date at: https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/modules/package_builder/
== Making changes and testing snapshot builds ==
* Make changes to the package files
* Make commits or amended commits (uncommitted changes are not exported to the build directory)
* Build snapshot builds for testing with:
 $ git-dch --snapshot
 $ gbp buildpackage --git-export-dir=/var/tmp/build-area/packagename -S
 # pbuilder --build --basetgz /var/cache/pbuilder/lucid.tgz /var/tmp/build-area/packagename/packagename_version.dsc
== Doing a release ==
When a package is ready to be released, use the following steps:
 $ git-dch -R -N version
* Commit the debian/changelog file (or amend a previous commit)
* Build the final source package with --git-tag:
 $ gbp buildpackage --git-tag --git-export-dir=/var/tmp/build-area/packagename -S 
* Build the binaries with Pbuilder as usual
=== Pushing changes into Gerrit ===
Push changes for each of the three branches (if applicable):
 $ git push gerrit master:refs/for/master
 $ git push gerrit upstream:refs/for/upstream
 $ git push gerrit pristine-tar:refs/for/pristine-tar
== Pushing the repository into Gerrit ==
You should push any repository with local changes into Gerrit.
* Create a new repository, operations/debs/packagename using the following Gerrit command:
 $ ssh username@gerrit.wikimedia.org -p 29418 'gerrit create-project -d "Package packagename" operations/debs/packagename -o ldap/ops -p operations/debs'
* Create a new remote called gerrit:
 $ git remote add gerrit ssh://username@gerrit.wikimedia.org:29418/operations/debs/packagename
* Push the three local branches (master, upstream and pristine-tar) into the fresh remote repository:
 $ git push gerrit --all
= How to build a Python deb package using git-buildpackage =
First, it might be good to familiarize yourself with git-buildpackage.
This manual is actually very helpful:
http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html
Particularly, read the bit about different ways to import packages. The package
that you are importing may vary in its own packaging. Do they release tarballs?
Are you importing from an upstream git repository? Etc. etc.
== Building a python package from a release tarball ==
I just built python-jsonschema. I'll use it as an example case.
=== Prep your git repository ===
Download the release tarball:
 wget https://pypi.python.org/packages/source/j/jsonschema/jsonschema-1.1.0.tar.gz
If you don't yet have a repository in gerrit, you'll need to create one:
 ssh username@gerrit.wikimedia.org -p 29418 'gerrit create-project -d "Package python-jsonschema" operations/debs/python-jsonschema -o ldap/ops -p operations/debs'
 git remote add gerrit ssh://username@gerrit.wikimedia.org:29418/operations/debs/python-jsonschema
The gerrit repository we just created is empty. Let's start a new one locally.
 mkdir -p python-jsonschema
 cd python-jsonschema
 git init .
We are importing from a tarball, and not a git repository and git-import-orig
wants an empty 'upstream' branch to exist on our repository.
See: http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.html#AEN212
 git checkout --orphan upstream
 git commit --allow-empty -m 'Initial upstream branch.'
 # switch back to master
 git checkout -f master
NOTE: master will be the name of the git-buildpackage specific 'debian-branch'.
IF you want to use a different branch for this name, you will have to tell
git-buildpackage via a config file, or via the CLI options. git-buildpackage
suggests using master, so we'll stick with it.
=== Import the tarball ===
Since we're building a python module, be sure to name the package with the
'python-' prefix. We're also using --pristine-tar as git-buildpackage recommends.
 git-import-orig --pristine-tar ../jsonschema-1.1.0.tar.gz
 What will be the source package name? [jsonschema] python-jsonschema
 What is the upstream version? [1.1.0]
 gbp:info: Importing '../jsonschema-1.1.0.tar.gz' to branch 'upstream'...
 gbp:info: Source package is python-jsonschema
 gbp:info: Upstream version is 1.1.0
 gbp:info: Merging to 'master'
 Merge made by the 'recursive' strategy.
 gbp:info: Successfully imported version 1.1.0 of ../jsonschema-1.1.0.tar.gz
=== Use stdeb to create the debian/ directory ===
Faidon says not to use stdeb, but it creates a decent debian/ directory
that gives a good starting point. You can create your debian/ directory
however you like. See python-jsonschema/debian
for a decent version of a python package debian/ directory that will use dh_python2.
For our example, we'll use stdeb and then edit the debian/ files and use dh_python2
to build the package.
See: https://pypi.python.org/pypi/stdeb
 python setup.py --command-packages=stdeb.command debianize
Since we don't like stdeb's output, follow these instructions to modify debian/ files
to work with dh_python2:
http://wiki.debian.org/Python/TransitionToDHPython2
In summary:
* changelog:
** Change maintainer to you.
** Change changelog message to 'Initial release.'
* compat:
** change to 10 (9 if you need to support jessie)
* control:
** Change maintainer to you.
** debhelper (>= 10) (again, 9 for jessie)
** Put relevant dependencies (from setup.py / requirements.txt) in Depends
*** If you are building for python3 as well, put those dependencies also in build depends.
** Change Standards-Version to 3.9.5
** Modify description to remove any special formatting (markdown, etc.)
*rules
** remove comments about stdeb
** Change to use pybuild, e.g.:
#!/usr/bin/make -f
# The deb package will be named python3?-$PYBUILD_NAME
export PYBUILD_NAME=google-api
## Uncomment these if you need to disable tests during deb build.
## Disable tests so we don't need test dependencies at build time.
#export PYBUILD_DISABLE_python2.7=test
#export PYBUILD_DISABLE_python3=test
%:
	dh $@ --with python2,python3 --buildsystem=pybuild
( --with whatever versions of python you are building for.)
Add the debian/ directory to git:
 git add debian
 git commit -m 'Initial deb packaging'
=== Build the package ===
Now try to build the package, yay!
 mkdir -p ../build-area/python-jsonschema
 gbp buildpackage --git-export-dir=../build-area/python-jsonschema
Check to see that the .deb looks like it should:
 dpkg -c ../build-area/python-jsonschema/*.deb
If that looks good, push everything to gerrit, minus your most recent commit
(that one needs review).
=== Push to gerrit for review ===
Directly push the upstream and pristine-tar branches, those don't need review:
 git push gerrit upstream
 git push gerrit pristine-tar
Directly push everything in master except for the commit in which you added the
debian/ directory:
 git push gerrit HEAD~1:refs/heads/master
 
Finally git-review (or whatever) to push the last unpushed commit for review:
 git review
=== Add the reviewed package to apt.wikimedia.org ===
Once your review has been approved and merged, copy the files in
build-area/python-jsonschema to install1002, and add the .changes file with reprepro.
We need to add the --ignore=wrongdistribution flag, since we didn't build the package
with a special wikimedia distribution name.
 reprepro --ignore=wrongdistribution -C main include precise-wikimedia jsonschema_1.1.0-1_amd64.changes
== Building a python package from an existing git repository ==
Based on 
$ git clone https://github.com/sripathikrishnan/redis-rdb-tools
$ cd redis-rdb-tools
$ vim .git/config
 # hack out the branch "master" upstream tracking
# upstream doesn't have tags, hack these in
$ git reset --hard d2792069291484af1d9d64abd849aaa44ddf6211
$ git tag upstream/0.1.6
then follow the instructions above from 'Use stdeb to create the debian/ directory'.
Ottomata TODO: understand and write this section.
= FAQ =
== dpkg-source complaining about local changes ==
You might have dpkg-source complains about local change. You can get them ignore by editing debian/source/options and add the file to ignore list:
 $ cat debian/source/options
 extend-diff-ignore = '^\.gitreview$'
Reference: http://www.debian.org/doc/manuals/maint-guide/dother.en.html#sourceopt
== Investigating build failures ==
If you're invoking git-buildpackage with --git-builder=pbuilder, the chroot where the package is built will be removed at the end of the process, leaving you without the chance of investigating the build failure:
 -> Cleaning COW directory
 forking: rm -rf /var/cache/pbuilder/build//cow.25824
 
The following pbuilder hook can be installed under /var/cache/pbuilder/hooks/$DIST/ to run a shell in the chroot after a build failure:
 #!/bin/bash
 # example file to be used with --hookdir
 #
 # invoke shell if build fails.
 apt-get install -y "${APTGETOPT[@]}" vim less 
 cd /tmp/buildd/*/debian/..
 /bin/bash < /dev/tty > /dev/tty 2> /dev/tty
= Experiments =
== experiment build mathoid debian package on vagrant and publish to ppa ==
The result https://launchpad.net/~physikerwelt/+archive/ubuntu/mathoid/+packages
 sudo apt-get install git-buildpackage
 sudo apt-get install libdistro-info-perl
without the libdistro-info-perl the changes between the last release could not be calculated
 cd
it does not work in a folder that is mounted to the windows host machine (i.e. /vagrant/srv/mathoid can not be used)
 mkdir container
 ls
 cd container/
 git clone https://github.com/physikerwelt/mathoid-server
 cd mathoid-server/
 npm install
 git-dch -R -N 0.2.8
 git commit -a -m "Update debian release notes"
 sudo apt-get install dh-make
 sudo apt-get install gnupg2
 gpg2 --gen-key
 apt-get install haveged
this command was executed in a second terminal window because it was hard to generate entropy
 gpg2 --send-keys --keyserver=keyserver.ubuntu.com 7271B791
 vi test.msg
you get an encrypted email that contains a link that you have to click... you need to find some way to read the contents... probably there is also a plugin for the gmail webui... but I could not find it
 gpg -d test.msg
 rm test.msg
 gbp buildpackage --git-tag -S -k7271B791
normally the k argument should not be required
 sudo apt-get install dput
 dput ppa:physikerwelt/mathoid ../mathoid_0.2.8_source.changes
 git push origin debian/0.2.8
= External links =
* Git-buildpackage documentation
* a nice guide on git-buildpackage
* a howto beginning from start
Category:Ubuntu Category:Package management