Application Development with Extensible SDK

[WORK IN PROGRESS]. Back to the Cookbook home page. 
== Introduction ==
Welcome to the Yocto Project Test Kitchen! It can be a bit overwhelming to know where to start with the Yocto Project. You know you want to build an application that runs on embedded hardware, but where and how do you get started? A development environment can be an abstract, somewhat hard-to-imagine thing. This part of the wiki is intended to fill the knowledge gap between the Quick Start and the myriad of ways to build an image and run it on real (or emulated) hardware. So what better way to do that than with a "Test Kitchen" analogy? 
Our Test Kitchen must be Linux-powered, of course. And using one of the supported distributions in our Linux-powered kitchen means that we'll already either have or have a way to easily get everything we need to follow the recipes in our Cookbook.
=== What devtool can do ===
One of the most powerful tools in our test kitchen is devtool. We can think of devtool as something that can be used to "mix" customization into our Yocto Project image. Some of the common ways to get mixing are:
* To add new applications and libraries to a Yocto Project image, 
* To modify the source for an existing component, 
* To test changes on target hardware, and 
* To integrate that change into the rest of the OpenEmbedded (OE) build system. 
In short, we can think of devtool as a single point of entry for creating, modifying and testing recipes. Read this section of the SDK manual to find out more on the types of projects devtool can support].
=== What devtool cannot do ===
Although devtool will make a good start at generating a recipe for your project, the reciope will probably need some manual tweaking before it does eaxctly what you want. Also devtool cannot publish your recipe to the layer index.
(From the Yocto Project documentation: Here's a presentation by Trevor Woerner that, while somewhat dated, provides detailed background information and a complete working 
Yocto Project Developer Workflow Tutorial.)
== How to get devtool ==
=== Standard build environment ===
If you have set up the Yocto Project build environment using the Quick Start Guide, you already have access to devtool for qemux86 image. 
=== Extensible SDK ===
Alternatively you can install an Extensible SDK (eSDK). These come in two flavors; "minimal" and "full". The "minimal" eSDK is the preferred option as it comes as a small 30Mb installer and downloads the components it needs depending on requirements of application project. The installer contains the URL of an sstate mirror so it does not no need to build all dependencies from source code. The "full" eSDK contains all the components already built so is much larger (similar in size to traditional SDK, i.e. at least 1GB). The Yocto SDK Manual contains more details on installing and using the eSDK.
Currently the Yocto Project autobuilder only builds "full" eSDKs. The qemux86 full eSDK can be found here. There is a feature enhancement request for the autobuilder to create minimal eSDKs.
In the meantime, an experimental minimal qemux86 eSDK can be found here.
To install the extensible SDK, make sure it is executable and then run
 poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-2.2.sh -d /path/to/esdk
Note that installation will take 7-10 minutes.
The configure build environment, run 
 source /path/to/esdk/environment-setup-i586-poky-linux
== Create a recipe from existing source ==
Before generating your recipe, it's best to build the base image first, then you will only have to build your application after adding the recipe. This can be done using devtool:
 devtool build-image
=== Generate recipe ===
Let's say you have a C application that you want to add to your OS image. Assume repo is https://github.com/whbruce/bbexample.git and you want to name recipe (and thus the executable it creates) bbexample. The best way of doing this is use devtool
 devtool add bbexample https://github.com/whbruce/bbexample.git
You will see the following output
 NOTE: Creating workspace layer in /path/to/build/workspace
 NOTE: Enabling workspace layer in bblayers.conf
 NOTE: Using default source tree path /path/to/build/workspace/sources/bbexample
 NOTE: Recipe /path/to/build/workspace/recipes/bbexample/bbexample_git.bb has been automatically created; further editing may be required to make it fully functional
Key entries here are the locations of the source code in /path/to/build/workspace/sources/bbexample. Note that git will create a branch named devtool for your work. The recipe is in /path/to/build/workspace/recipes/bbexample/bbexample_git.bb
You can now look at the generated recipe with
 devtool edit-recipe bbexample
We'll take a look at what's in the recipe. Comments and whitespace have been removed for easier reading
 LICENSE = "Unknown"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=96af5705d6f64a88e035781ef00e98a8"
 SRC_URI = "git://github.com/whbruce/bbexample.git;protocol=https"
 # Modify these as desired
 PV = "0.1+git${SRCPV}"
 SRCREV = "${AUTOREV}"
 S = "${WORKDIR}/git"
 inherit autotools
 EXTRA_OECONF = ""
Let's go through the bitbake commands and variables in the recipe and explain what they do:
;LICENSE
:The type of license. This is set to unknown because devtool cannot recognize the old MIT text. 
;LIC_FILES_CHKSUM
:devtool looks in the root of the source folder for a file called LICENSE, and generates a checksum to ensure checkout has been successful. 
;SRC_URI
:The git repo given in the 'devtool add' command. This is where the source will be pulled from. 
;PV
:The package version. Usually taken from recipe filename (e.g. bbexample_0.1.bb would automatically set PV = "0.1")
;SRCREV
:the git revision to checkout. ${AUTOREV} means HEAD and must be used during app development. It should be set a specific commit before being added to a layer.
;S
:Where the source can be found after being fetched and unpacked.
;inherit autotools
:Use a bitbake class that can configure, build and install an autotools project
;EXTRA_OECONF
:Options to be passed to configure script. In this case it is empty, so is not necessary.
=== Build ===
 $ devtool build bbexample
 Parsing recipes: 100% |#########################################| ETA: 00:00:00
 Parsing of 872 .bb files complete (0 cached, 872 parsed). 1302 targets, 47 skipped, 0 masked, 0 errors.
 NOTE: Resolving any missing task queue dependencies
 NOTE: Preparing RunQueue
 NOTE: Checking sstate mirror object availability (for 51 objects)
 NOTE: Executing SetScene Tasks
 NOTE: Executing RunQueue Tasks
 NOTE: bbexample: compiling from external source tree /path/to/esdk/workspace/sources/bbexample
 NOTE: Tasks Summary: Attempted 377 tasks of which 371 didn't need to be rerun and all succeeded.
This build command runs bitbake just as in a full Yocto Project OS image build. The key line in the output is Checking sstate mirror object availability (for 51 objects). As the extensible SDK installer gives you a minimal build environment it has to download all the dependencies (e.g. native build tools and necessary target libraries). Here 51 objects have been found and are downloaded in their pre-built state greatly speeding up the build process (i.e. we just run 377 tasks of which 371 didn't need to be rerun). This is the extensible part of the SDK in action.
=== Deploy ===
Before deploying, the target must be running, have a network connection, and have an ssh server running. In this case, we'll start the qemux86 target as follows. 
 $ runqemu qemux86
Then we'll deploy the package
 $ devtool deploy-target -s bbexample root@192.168.7.2
 Parsing recipes..done.
 devtool_deploy.list 100% 167 0.2KB/s 00:00 
 devtool_deploy.sh 100% 1002 1.0KB/s 00:00 
 ./
 ./usr/
 ./usr/bin/
 ./usr/bin/bbexample
 ./usr/include/
 ./usr/include/bbexample.h
 ./usr/lib/
 ./usr/lib/libbbexample.so.1
 ./usr/lib/libbbexample.so
 ./usr/lib/libbbexample.so.1.0.0
 ./usr/lib/libbbexample.la
 NOTE: Successfully deployed /path/to/esdk/sdk/tmp/work/i586-poky-linux/bbexample/0.1+git999-r0/image
=== Run ===
 root@qemux86:~# bbexample
 Hello Yocto World...
 Hello World (from a shared library!)
=== Update ===
Now the application is packaged, you'll probably want to make some changes. This examples assumes that you cannot push you changes upstream so have to apply them as a patch. Let's simply update the app to output 
 Hello Yocto World v2...
Make the change to bbexample.c and commit with message "Changed bbexample message" and check that bbexample operates as expected. Note that if you are using devtool from the eSDK it will create temporary folders oe-logs and oe-workdir so you should configure git to ignore them.
=== Publish ===
Now you'll want to add your new app to a layer. In this example, we'll create a new layer called meta-example in folder /path/to, but an existing layer can also be used. First create the layer and it add it BBLAYERS. We use the yocto-layer utility. When you run it, accept all default values. The bitbake-layers script will add the later to BBLAYERS.
 $ layers/poky/scripts/yocto-layer create /path/to/meta-example
 $ layers/poky/bitbake/bin/bitbake-layers add-layer /path/to/meta-example
Now we move over the recipe we created, along with the patch for the source code. The devtool publish command first appeared in Yocto 2.2 which grearly simplifies this task.
==== Yocto 2.2 and later ====
 $ devtool finish bbexample meta-example
This will create a recipes folder called recipes-bbexample, with a recipe called bbexample_git.bb in it. You should change the recipe name to reflect the version (e.g. bbexample_0.1.bb)
==== Yocto 2.1 ====
Manually create patches then copy over recipe and patches. More details to come.
== Update Existing Recipe ==
Updating a recipe is similar to creating one; however, rather than using devtool add we use the devtool modify command. 
 devtool modify bbexample
The edit/build/test cycle is then the same as for adding a recipe but note that the recipe is not copied into devtool's workspace but remains in its layer.
== Build Image ==
To test the updated OS image that now includes your application, you can build it as follows. If you are in a eSDK environment, devtool will automatically build the image used to create the SDK. If in the standard build environment you must specify an image.
 devtool build-image [image-name]
Output will be as follows:
Parsing recipes..done.
NOTE: Building image core-image-minimal with the following additional packages: bbexample
Loading cache: 100% |###########################################| ETA: 00:00:00
Loaded 1302 entries from dependency cache.
Parsing recipes: 100% |#########################################| Time: 00:00:00
Parsing of 872 .bb files complete (870 cached, 2 parsed). 1302 targets, 49 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.30.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.1"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta 
meta-poky 
meta-yocto-bsp = "krogoth:f7b994b75261550faa3ccf9005fc174950c7bee9"
workspace = "<unknown>:<unknown>"
NOTE: Preparing RunQueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
bbexample-0.1+git999-r0 do_compile: NOTE: bbexample: compiling from external source tree /path/to/build/workspace/sources/bbexample
NOTE: Tasks Summary: Attempted 2062 tasks of which 2046 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/build/tmp/deploy/images/qemux86