Transcript: Using the Yocto BSP tools to create a meta-intel BSP

Here's a cut-and-paste shell session showing how to have the Yocto BSP tools create a new BSP from scratch. It starts with a fresh git checkout, then invokes yocto-bsp to create a new x86_64-based BSP, and finally builds a sato image and boots into a sato desktop in a qemu session (commands also shown for that).
Once you've successfully followed the steps in this transcript, you can use the result along with the 'yocto-kernel' BSP tool to add, remove, and list Yocto kernel patches and config items by following the steps outlined in the companion transcript Transcript: Using the Yocto BSP tools to manage kernel patches and config items
NOTE: this transcript was from a build done on a Ubuntu 11.10 system. Please see the 'Yocto Project Quickstart' for the packages required to be installed on the host system for building 
One thing to note with this example is that I'm checking out from poky/master.
 trz@elmorro:/usr/local/dev$ mkdir Yocto
 trz@elmorro:/usr/local/dev$ cd Yocto/
 trz@elmorro:/usr/local/dev/Yocto$ git init
 Initialized empty Git repository in /usr/local/dev/Yocto/.git/
 trz@elmorro:/usr/local/dev/Yocto$ git remote add yocto git://git.yoctoproject.org/poky.git
 trz@elmorro:/usr/local/dev/Yocto$ git remote update
 Fetching yocto
 remote: Counting objects: 154283, done.
 remote: Compressing objects: 100% (42482/42482), done.
 remote: Total 154283 (delta 109346), reused 152606 (delta 108055)
 Receiving objects: 100% (154283/154283), 78.80 MiB | 80 KiB/s, done.
 Resolving deltas: 100% (109346/109346), done.
 From git://git.yoctoproject.org/poky
 * [new branch] 1.1_M1 -> yocto/1.1_M1
 * [new branch] 1.1_M2 -> yocto/1.1_M2
 * [new branch] 1.1_M3 -> yocto/1.1_M3
 * [new branch] 1.1_M4 -> yocto/1.1_M4
 * [new branch] 1.2_M1 -> yocto/1.2_M1
 * [new branch] 1.2_M2 -> yocto/1.2_M2
 * [new branch] 1.2_M3 -> yocto/1.2_M3
 * [new branch] 1.2_M4 -> yocto/1.2_M4
 * [new branch] 1.3_M1 -> yocto/1.3_M1
 * [new branch] 1.3_M2 -> yocto/1.3_M2
 * [new branch] bernard -> yocto/bernard
 * [new branch] blinky -> yocto/blinky
 * [new branch] clyde -> yocto/clyde
 * [new branch] denzil -> yocto/denzil
 * [new branch] edison -> yocto/edison
 * [new branch] elroy -> yocto/elroy
 * [new branch] green -> yocto/green
 * [new branch] laverne -> yocto/laverne
 * [new branch] master -> yocto/master
 * [new branch] master-next -> yocto/master-next
 * [new branch] pinky -> yocto/pinky
 * [new branch] purple -> yocto/purple
 * [new tag] 1.1_M1.final -> 1.1_M1.final
 * [new tag] 1.1_M2.final -> 1.1_M2.final
 * [new tag] 1.1_M2.rc3 -> 1.1_M2.rc3
 * [new tag] 1.1_M3.final -> 1.1_M3.final
 * [new tag] 1.1_M3.rc3 -> 1.1_M3.rc3
 * [new tag] 1.2_M1.final -> 1.2_M1.final
 * [new tag] 1.2_M1.rc2 -> 1.2_M1.rc2
 * [new tag] 1.2_M2.final -> 1.2_M2.final
 * [new tag] 1.2_M2.rc1 -> 1.2_M2.rc1
 * [new tag] 1.2_M3.final -> 1.2_M3.final
 * [new tag] 1.2_M3.rc1 -> 1.2_M3.rc1
 * [new tag] 1.3_M1.final -> 1.3_M1.final
 * [new tag] 1.3_M1.rc1 -> 1.3_M1.rc1
 * [new tag] 1.3_M2.final -> 1.3_M2.final
 * [new tag] bernard-5.0.2+docs -> bernard-5.0.2+docs
 * [new tag] denzil-7.0.1 -> denzil-7.0.1
 * [new tag] denzil-7.0.1_rc2 -> denzil-7.0.1_rc2
 * [new tag] edison-6.0.2 -> edison-6.0.2
 * [new tag] edison-6.0.2_rc1 -> edison-6.0.2_rc1
 * [new tag] pinky-3.1.2 -> pinky-3.1.2
Because we're going to be creating an Intel BSP, we'll also need to clone the meta-intel repository in order to be able to add the common meta-intel layer to BBLAYERS in a subsequent step.
 trz@elmorro:/usr/local/dev/Yocto$ mkdir meta-intel
 trz@elmorro:/usr/local/dev/Yocto$ cd meta-intel
 trz@elmorro:/usr/local/dev/Yocto/meta-intel$ git init
 Initialized empty Git repository in /usr/local/dev/Yocto/meta-intel/.git/
 trz@elmorro:/usr/local/dev/Yocto/meta-intel$ git remote add meta-intel git://git.yoctoproject.org/meta-intel.git
 trz@elmorro:/usr/local/dev/Yocto/meta-intel$ git remote update
 Fetching meta-intel
 remote: Counting objects: 4297, done.
 remote: Compressing objects: 100% (3461/3461), done.
 remote: Total 4297 (delta 2188), reused 432 (delta 212)
 Receiving objects: 100% (4297/4297), 1.88 MiB | 154 KiB/s, done.
 Resolving deltas: 100% (2188/2188), done.
 From git://git.yoctoproject.org/meta-intel
 * [new branch] 1.1_M1 -> meta-intel/1.1_M1
 * [new branch] 1.1_M2 -> meta-intel/1.1_M2
 * [new branch] 1.1_M3 -> meta-intel/1.1_M3
 * [new branch] 1.1_M4 -> meta-intel/1.1_M4
 * [new branch] 1.2_M1 -> meta-intel/1.2_M1
 * [new branch] 1.2_M2 -> meta-intel/1.2_M2
 * [new branch] 1.2_M3 -> meta-intel/1.2_M3
 * [new branch] 1.3_M1 -> meta-intel/1.3_M1
 * [new branch] 1.3_M2 -> meta-intel/1.3_M2
 * [new branch] bernard -> meta-intel/bernard
 * [new branch] denzil -> meta-intel/denzil
 * [new branch] edison -> meta-intel/edison
 * [new branch] laverne -> meta-intel/laverne
 * [new branch] master -> meta-intel/master
 * [new branch] sgw/tlk -> meta-intel/sgw/tlk
 * [new tag] 1.1_M1.rc2 -> 1.1_M1.rc2
 * [new tag] 1.1_M2.final -> 1.1_M2.final
 * [new tag] 1.1_M2.rc3 -> 1.1_M2.rc3
 * [new tag] 1.1_M3.final -> 1.1_M3.final
 * [new tag] 1.1_M3.rc3 -> 1.1_M3.rc3
 * [new tag] 1.2_M1.final -> 1.2_M1.final
 * [new tag] 1.2_M1.rc2 -> 1.2_M1.rc2
 * [new tag] 1.2_M3.rc1 -> 1.2_M3.rc1
 * [new tag] 1.3_M2.final -> 1.3_M2.final
 * [new tag] 1.3_M2.rc1 -> 1.3_M2.rc1
 From git://git.yoctoproject.org/meta-intel
 * [new tag] 1.1_M2.rc1 -> 1.1_M2.rc1
 * [new tag] 1.1_M3.rc2 -> 1.1_M3.rc2
 * [new tag] 1.1_M4.rc2+ -> 1.1_M4.rc2+
 * [new tag] 1.1_M4.rc4 -> 1.1_M4.rc4
 * [new tag] 1.2 -> 1.2
 * [new tag] 1.2_M2.final -> 1.2_M2.final
 * [new tag] 1.2_M2.rc1 -> 1.2_M2.rc1
 * [new tag] bernard-1.0rc1 -> bernard-1.0rc1
 * [new tag] bernard-5.0 -> bernard-5.0
 * [new tag] bernard-5.0rc1 -> bernard-5.0rc1
 * [new tag] bernard-5.0rc2 -> bernard-5.0rc2
 * [new tag] edison-6.0 -> edison-6.0
 
 trz@elmorro:/usr/local/dev/Yocto/meta-intel$ git checkout -b cur meta-intel/master
 Branch cur set up to track remote branch master from meta-intel.
 Switched to a new branch 'cur'
Now that we've cloned the yocto and meta-intel repos, we need to source the environment in order to create or build a BSP:
 trz@elmorro:/usr/local/dev/Yocto$ source oe-init-build-env 
 You had no conf/local.conf file. This configuration file has therefore been
 created for you with some default values. You may wish to edit it to use a 
 different MACHINE (target hardware) or enable parallel build options to take 
 advantage of multiple cores for example. See the file for more information as 
 common configuration options are commented.
 
 The Yocto Project has extensive documentation about OE including a reference manual
 which can be found at:
 http://yoctoproject.org/documentation
 
 For more information about OpenEmbedded see their website:
 http://www.openembedded.org/
 
 You had no conf/bblayers.conf file. The configuration file has been created for
 you with some default values. To add additional metadata layers into your
 configuration please add entries to this file.
 
 The Yocto Project has extensive documentation about OE including a reference manual
 which can be found at:
 http://yoctoproject.org/documentation
 
 For more information about OpenEmbedded see their website:
 http://www.openembedded.org/
 
 
 
 ### Shell environment set up for builds. ###
 
 You can now run 'bitbake <target>'
 
 Common targets are:
 core-image-minimal
 core-image-sato
 meta-toolchain
 meta-toolchain-sdk
 adt-installer
 meta-ide-support 
 
 You can also run generated qemu images with a command like 'runqemu qemux86'
 
We're now ready to generate a BSP using the 'yocto-bsp' tool. The 'yocto-bsp' tool will step through a series of questions about how the new BSP should be configured. The Yocto BSP tools all have extensive help and usage screens built in, which can be easily displayed at any time to remind yourself of the command syntax and descriptions of what the commands and sub-commands do. The tools have a git-like interface, meaning that there's a top-level command which essentially invokes multiple lower-level subcommands.
To display help for a top-level command, simply invoke the command without any parameters:
 trz@elmorro:/usr/local/dev/Yocto/build$ yocto-bsp
 
 Usage: 
 
 Create a customized Yocto BSP layer.
 
 usage: yocto-bsp [--version] [--help] COMMAND [ARGS]
 
 The most commonly used 'yocto-bsp' commands are:
 create Create a new Yocto BSP
 list List available values for options and BSP properties
 
 See 'yocto-bsp help COMMAND' for more information on a specific command.
 
 
 Options:
 --version show program's version number and exit
 -h, --help show this help message and exit
 -D, --debug output debug information
In order to create a new BSP, we'll want to use the 'yocto-bsp create' command.
To display usage help for a sub-command like 'yocto-bsp create', simply invoke it without any parameters:
 trz@elmorro:/usr/local/dev/Yocto/build$ yocto-bsp create
 
 Usage: 
 
 Create a new Yocto BSP
 
 usage: yocto-bsp create <bsp-name> <karch> [-o <DIRNAME> | --outdir <DIRNAME>]
 [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
 
 This command creates a Yocto BSP based on the specified parameters.
 The new BSP will be a new Yocto BSP layer contained by default within
 the top-level directory specified as 'meta-bsp-name'. The -o option
 can be used to place the BSP layer in a directory with a different
 name and location.
 
 The value of the 'karch' parameter determines the set of files that
 will be generated for the BSP, along with the specific set of
 'properties' that will be used to fill out the BSP-specific portions
 of the BSP. The possible values for the 'karch' paramter can be
 listed via 'yocto-bsp list karch'. 
 
 
 Options:
 -h, --help show this help message and exit
 -o OUTDIR, --outdir=OUTDIR
 name of BSP dir to create
 -i PROPERTIES_FILE, --infile=PROPERTIES_FILE
 name of file containing the values for BSP properties
 as a JSON file
 -c, --codedump dump the generated code to bspgen.out
You can also get more extensive help for a sub-command by adding 'help' before the sub-command name:
 trz@elmorro:/usr/local/dev/Yocto/build$ yocto-bsp help create
 
 NAME
 yocto-bsp create - Create a new Yocto BSP
 
 SYNOPSIS
 yocto-bsp create <bsp-name> <karch> [-o <DIRNAME> | --outdir <DIRNAME>]
 [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
 
 DESCRIPTION
 This command creates a Yocto BSP based on the specified
 parameters. The new BSP will be a new Yocto BSP layer contained
 by default within the top-level directory specified as
 'meta-bsp-name'. The -o option can be used to place the BSP layer
 in a directory with a different name and location. 
 
 The value of the 'karch' parameter determines the set of files
 that will be generated for the BSP, along with the specific set of
 'properties' that will be used to fill out the BSP-specific
 portions of the BSP. The possible values for the 'karch' paramter
 can be listed via 'yocto-bsp list karch'.
 
 The BSP-specific properties that define the values that will be
 used to generate a particular BSP can be specified on the
 command-line using the -i option and supplying a JSON object
 consisting of the set of name:value pairs needed by the BSP. 
 
 If the -i option is not used, the user will be interactively
 prompted for each of the required property values, which will then
 be used as values for BSP generation.
 
 The set of properties available for a given architecture can be
 listed using the 'yocto-bsp list' command.
 
 Specifying -c causes the Python code generated and executed to
 create the BSP to be dumped to the 'bspgen.out' file in the
 current directory, and is useful for debugging.
 
 NOTE: Once created, you should add your new layer to your
 bblayers.conf file in order for it to be subsquently seen and
 modified by the yocto-kernel tool.
 
 NOTE for x86- and x86_64-based BSPs: The generated BSP assumes the
 presence of the of the meta-intel layer, so you should also have a
 meta-intel layer present and added to your bblayers.conf as well.
The other sub-command of 'yocto-bsp' is 'yocto-bsp list', which we can use to list all of the architectures we can create a BSP for:
 trz@elmorro:/usr/local/dev/Yocto/build$ yocto-bsp list karch
 Architectures available:
 arm
 powerpc
 i386
 mips
 x86_64
 qemu
For this BSP, we'll choose the 'x86_64' architecture.
To create the BSP we'll invoke 'yocto-bsp create', but first let's change directories out of the 'build' directory (we could stay there, or use the -o option to generate the BSP somewhere else, but for this example we'll just cd into the Yocto dir and generate our BSP there).
We'll call our machine 'myintelbsp'; 'yocto-bsp create' will create our BSP layer in meta-myintelbsp in the current directory. We'll use the default for everything in this BSP, so just press 'enter' at each prompt:
 trz@elmorro:/usr/local/dev/Yocto/build$ cd ..
 trz@elmorro:/usr/local/dev/Yocto$ yocto-bsp create myintelbsp x86_64
 Would you like to use the default (3.4) kernel? (y/n) [default: y] 
 Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [y/n] [default: y] 
 Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.4.git...
 Please choose a machine branch to base this BSP on: [default: standard/common-pc-64/base]
 	1) standard/arm-versatile-926ejs
 	2) standard/base
 	3) standard/beagleboard
 	4) standard/cedartrail
 	5) standard/common-pc-64/base
 	6) standard/common-pc-64/chiefriver
 	7) standard/common-pc-64/jasperforest
 	8) standard/common-pc-64/romley
 	9) standard/common-pc-64/sugarbay
 	10) standard/crownbay
 	11) standard/emenlow
 	12) standard/fishriver
 	13) standard/fri2
 	14) standard/fsl-mpc8315e-rdb
 	15) standard/mti-malta32
 	16) standard/mti-malta64
 	17) standard/qemuppc
 	18) standard/routerstationpro
 	19) standard/sys940x
 
 Do you need SMP support? (y/n) [default: y] 
 Do you need support for X? (y/n) [default: y] 
 Please select an xserver for this machine: [default: xserver_i915] 
 	1) VESA xserver support
 	2) i915 xserver support
 
 Does your BSP have a touchscreen? (y/n) [default: n] 
 Does your BSP have a keyboard? (y/n) [default: y] 
 New x86_64 BSP created in meta-myintelbsp
The message states that a new BSP was created for us, and indeed it appears in our directory listing:
 total 104
 drwxrwxr-x 17 trz trz 4096 2012-08-09 10:52 .
 drwxr-xr-x 42 trz trz 4096 2012-08-09 10:16 ..
 drwxrwxr-x 6 trz trz 4096 2012-08-06 16:54 bitbake
 drwxrwxr-x 9 trz trz 4096 2012-08-09 09:22 build
 drwxrwxr-x 10 trz trz 4096 2012-08-06 16:54 documentation
 drwxrwxr-x 8 trz trz 4096 2012-08-09 10:36 .git
 -rw-rw-r-- 1 trz trz 156 2012-08-06 16:54 .gitignore
 -rw-rw-r-- 1 trz trz 545 2012-08-06 16:54 LICENSE
 drwxrwxr-x 20 trz trz 4096 2012-08-06 16:54 meta
 drwxrwxr-x 4 trz trz 4096 2012-08-06 16:54 meta-hob
 drwxrwxr-x 17 trz trz 4096 2012-08-06 16:54 meta-intel
 drwxrwxr-x 8 trz trz 4096 2012-08-09 10:52 meta-myintelbsp
 drwxrwxr-x 8 trz trz 4096 2012-08-09 00:34 meta-myqemuarm
 drwxrwxr-x 5 trz trz 4096 2012-08-06 16:54 meta-skeleton
 drwxrwxr-x 9 trz trz 4096 2012-08-06 16:54 meta-yocto
 -rwxrwxr-x 1 trz trz 1531 2012-08-06 16:54 oe-init-build-env
 drwxrwxr-x 5 trz trz 4096 2012-08-06 16:54 poky-extras
 -rw-rw-r-- 1 trz trz 1365 2012-08-06 16:54 README
 -rw-rw-r-- 1 trz trz 17245 2012-08-06 16:54 README.hardware
 drwxrwxr-x 8 trz trz 4096 2012-08-06 16:54 scripts
 
In order to build or use the other Yocto BSP tools such as 'yocto-kernel', you need to add the new BSP layer to BBLAYERS in bblayers.conf:
 trz@elmorro:/usr/local/dev/Yocto$ emacs -nw build/conf/bblayers.conf
Change:
 BBLAYERS ?= " \ 
 /usr/local/dev/Yocto/meta \ 
 /usr/local/dev/Yocto/meta-yocto \ 
 "
To:
BBLAYERS ?= " \ 
 /usr/local/dev/Yocto/meta \ 
 /usr/local/dev/Yocto/meta-yocto \ 
 /usr/local/dev/Yocto/meta-intel \ 
 /usr/local/dev/Yocto/meta-myintelbsp \ 
 "
You also need to edit local.conf set MACHINE to your new machine:
 trz@elmorro:/usr/local/dev/Yocto$ emacs -nw build/conf/local.conf
Change:
 # This sets the default machine to be qemux86 if no other machine is selected: 
 MACHINE ??= "qemux86"
To:
 # This sets the default machine to be qemux86 if no other machine is selected: 
 MACHINE ??= "qemux86"
 
 MACHINE ??= "myintelbsp"
You should also change anything else you need to in local.conf, such as BB_NUMER_THREADS values, etc. Once you've done that, you should be able to build and boot the sato image: 
 trz@elmorro:/usr/local/dev/Yocto$ cd build
 trz@elmorro:/usr/local/dev/Yocto/build$ bitbake core-image-sato
 Loading cache: 100% |####################################################################| ETA: 00:00:00
 Loaded 1109 entries from dependency cache.
 
 OE Build Configuration:
 BB_VERSION = "1.15.1"
 TARGET_ARCH = "x86_64"
 TARGET_OS = "linux"
 MACHINE = "myintelbsp"
 DISTRO = "poky"
 DISTRO_VERSION = "1.1+snapshot-20120317"
 TUNE_FEATURES = "m64"
 TARGET_FPU = ""
 meta 
 meta-yocto = "bd2d8904181aa67258b6be8a7bbeaf8d12a8dd5a:bd2d8904181aa67258b6be8a7bbeaf8d12a8dd5a"
 meta-intel = "master0:d9132cc66316be45f44beeea6eba734bb3ab337d"
 meta-myintelbsp = "bd2d8904181aa67258b6be8a7bbeaf8d12a8dd5a:bd2d8904181aa67258b6be8a7bbeaf8d12a8dd5a"
 
 NOTE: Resolving any missing task queue dependencies
 NOTE: Preparing runqueue
 NOTE: Executing SetScene Tasks
 NOTE: Executing RunQueue Tasks
 NOTE: Running task 4369 of 4825 (ID: 8, /usr/local/dev/Yocto/meta/recipes-sato/images/core-image-sato.bb, do_rootfs)
 NOTE: Running task 4815 of 4825 (ID: 86, /usr/local/dev/Yocto/meta/recipes-core/images/core-image-minimal-initramfs.bb, do_rootfs)
 NOTE: package core-image-sato-1.0-r0: task do_rootfs: Started
 NOTE: package core-image-minimal-initramfs-1.0-r0: task do_rootfs: Started
 NOTE: package core-image-sato-1.0-r0: task do_rootfs: Succeeded
 NOTE: package core-image-minimal-initramfs-1.0-r0: task do_rootfs: Succeeded
 NOTE: Running task 4824 of 4825 (ID: 9, /usr/local/dev/Yocto/meta/recipes-sato/images/core-image-sato.bb, do_bootimg)
 NOTE: package core-image-sato-1.0-r0: task do_bootimg: Started
 NOTE: package core-image-sato-1.0-r0: task do_bootimg: Succeeded
 NOTE: Running noexec task 4825 of 4825 (ID: 5, /usr/local/dev/Yocto/meta/recipes-sato/images/core-image-sato.bb, do_build)
 NOTE: Tasks Summary: Attempted 4825 tasks of which 4821 didn't need to be rerun and all succeeded.
 
We had a successful build, and can now burn the image and boot into a sato desktop:
 root@elmorro:/usr/local/dev/Yocto/build/tmp/deploy/images# dd if=core-image-sato-myintelbsp-20120317164555.hddimg of=/dev/sdg
 869920+0 records in
 869920+0 records out
 445399040 bytes (445 MB) copied, 2.45513 s, 181 MB/s
 root@elmorro:/usr/local/dev/Yocto/build/tmp/deploy/images# eject /dev/sdg
 root@elmorro:/usr/local/dev/Yocto/build/tmp/deploy/images# dd if=/dev/zero of=/dev/sdg bs=1M count=1024
 1024+0 records in
 1024+0 records out
 1073741824 bytes (1.1 GB) copied, 106.58 s, 10.1 MB/s
 root@elmorro:/usr/local/dev/Yocto/build/tmp/deploy/images# dd if=core-image-sato-myintelbsp-20120317164555.hddimg of=/dev/sdg
 869920+0 records in
 869920+0 records out
 445399040 bytes (445 MB) copied, 179.825 s, 2.5 MB/s
 root@elmorro:/usr/local/dev/Yocto/build/tmp/deploy/images# eject /dev/sdg