TipsAndTricks/OnTargetWorkFlowLeveragingRPMPackagefeeds

= Leveraging Pkg Feeds or How I got Tired of Burning Images =
I was trying to build a sw repo on target and found that I lacked many of the tools and libraries in order to accomplish this. I tend to do this first, if I want to make a recipe from scratch for software that I am not too familiar with. This way, I can follow the instructions for building w/o reinterpretation, verify that I have or can build the necessary dependencies, and check that the sw works properly; all before diving into recipes. I have found reflashing and rebooting the image on the target to be quite slow. Luckily YP supports a much better workflow for this process.
 Note: This works as well with kernel modules as it does with libraries/binaries
In this example, we will be building znc 1.6.5, an irc bouncer. This is just a good complex example. In order to use this guide, you should set up your bitbake environment and your target as outlined in PkgFeeds the EasiestWay
== Steps ==
* It's typically easies to make an image that has *most* of what you think you'll need. In this example, I added the following to my local.conf:
 EXTRA_IMAGE_FEATURES += " ssh-server-openssh package-management dev-pkgs tools-sdk tools-debug tools-profile "
* Now I'll grab the src code we will try to build Note: this is an ON TARGET build the build host side is just for the dependencies...
 git clone https://github.com/znc/znc.git
 cd znc
 git checkout -b 1.6.5 znc-1.6.5
 * follow the build instructions
 ./autogen.sh
 ./configure
This worked, but we decide we want to have cyrus support as well.
* First we try 
 ./configure --enable-cyrus
but we get
 configure: error: could not find libsasl2. Try --disable-cyrus.
* So we need the cyrus-sals-dev pkg. If we aren't sure where that is, we can use openembedded layer index and search for the recipe cyrus (this finds cyrus-sasl2 in meta-networking). So, we add meta-networking to bblayers (and meta-oe and meta-python, as its dependencies) then build cyrus-sasl.
 bitbake cyrus-sasl
and remake the package index
 bitbake package-index
* target side, pull in the new package. Since I am building stuff, I need the -dev pkg for configure to be happy.
 $dnf install cyrus-sasl-dev (note this will also pull in the cyrus-sasl rpm as a dependency.)
If this fails, make sure you followed the instructions in PkgFeeds the EasiestWay. You should have set up the on target repo in /etc/yum.repos.d/myrepo.repo and have a web server of some type pointed to tmp/deploy/rpm (like twistd -n web --path tmp/deploy/rpm -p 5678) on your build host.
* now we rerun configure and get:
 ./configure --enable-cyrus
 ZNC 1.6.5 configured
 prefix: /usr/local
 debug: no
 ipv6: yes
 openssl: yes
 dns: threads
 perl: no
 python: no
 swig: not needed
 cyrus: yes
 tcl: no
 charset: yes
 zlib: yes
 run from src: no
Pretty Easy, you can see how this cycle may be repeated a number of times for more complex projects. 
* Let's build it
 ~/znc# make
 Packing man page znc.1.gz...
 Packing man page znc-buildmod.1.gz...
 It looks like git submodules are not initialized. Run: git submodule update --init --recursive
 make: *** [Makefile:168: third_party/Csocket/Csocket.h] Error 1
 ~/znc# git submodule update --init --recursive
 git: 'submodule' is not a git command. See 'git --help'.
Whoops, we need git submodule... This one is harder to find. After verifying that we already have git-dev installed, I ended up searching the poky tree for git-submodule
 $ git grep git-submodule
 bitbake/lib/bb/tests/fetch.py: fetcher = bb.fetch.Fetch(["gitsm://git.yoctoproject.org/git-submodule-test;rev=f12e57f2edf0aa534cf1616fa983d165
 meta/recipes-devtools/git/git.inc: ${libexecdir}/git-core/git-submodule \
If we look in meta/recipes-devtools/git/git.inc we can see that the git-submodule is included in git-perltools:
 PACKAGES =+ "${PN}-perltools".
So we can pull in the git-perltools as well on the target:
 # dnf install git-perltools
and rerun the git submodule init
 # git submodule update --init --recursive
 Submodule 'Csocket' (https://github.com/jimloco/Csocket.git) registered for path 'third_party/Csocket'
 Cloning into '/home/root/znc/third_party/Csocket'...
 Submodule path 'third_party/Csocket': checked out '448e18a29ed383451db3cb648a72da4fcbb3f8e5'
It worked :), and now we can successfully type make. Having to reflash the image for each of these steps would have been much slower!
 # make
 Building module autocycle...
 Linking module autocycle...
 ...
 Building core object SSLVerifyHost...
 Linking znc...
 ZNC was successfully compiled.
 Use 'make install' to install ZNC to '/usr/local'.
* I prefer emacs, so I tend to to this a lot as zile is a small emacs analogue:
 $ bitbake zile
 $ bitbake package-index
Target ->
 # dnf install zile
== Ways to Serve up the rpm repo == 
* very simple - use python twistd.
** let's suppose you have an rpm repo on the host that is in dir /home/me/mrrepo. 
** 
 cd /home/me/mrrepo;
 twistd -n web --path . -p 5678
 
** similarly, if you are just serving up the build directories rpm's you can just do 
*** 
 cd /home/me/mrrepo;
 twistd -n web --path tmp/deploy/rpm -p 5678
 
*** Note , only 1 process can own a given port.
* less simple, but better response and better stability- use lighthttpd. 
** I advocate using this from a container as it is simpler to maintain.
** 
 docker run -it --rm --name user-rpm-server -v /home/me/mrrepo:/var/www/localhost/htdocs -v /home/me/lighttpd:/etc/lighttpd -p 5678:80 sebp/lighttpd 
 
** What goes in /home/me/lighttpd? the config for the lighthttpd container you are running.
** Where did they come from? Copied from inside the container 
*** docker run -it --rm --user=root --entrypoint=/bin/sh -v /home/me/lighttpd:/frog sebp/lighttpd 
*** # cp -a /etc/lighttpd/* /frog 
*** Now your /home/me/lightppd has the necessary config files.
** The files in /home/me/lighthttpd:
*** lighttpd.conf lighttpd.conf~ mime-types.conf mod_cgi.conf mod_fastcgi.conf mod_fastcgi_fpm.conf
**** From this you can see I run emacs. The only change I did was to turn on directory listing .
**** diff lighttpd.conf lighttpd.conf~
***** 
 114c114
 < dir-listing.activate = "enable"
 ---
 > # dir-listing.activate = "enable"
 
** the -p 5678:80 exposes the default port the container runs it's service (aka 80) on to the host port 5678.