Binary configuration support

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries to generate custom image, without a need to rebuild from source 
Features that can be supported includes, however the list is tentative.
# Selects/deselects and integrates binary packages, i,e., OS packages & Various Applications packages from package pool, to generate the final image ready to deploy.
# Binary level offline configuration includes 
#:Configure the users/passwords
#:Configure the network type etc
#:Configure the host name
#:Configure the serial console
#:Select the services to be started by default
#:Select the modules to be loaded by default
#:Security related configuration
#:configure exportfs format in ramfs/ext3/... partitioning etc
#:More...
It has following benefits
* User need not modify the poky layers content or build configuration for few configuration requirements.
* Makes packages and Linux image more configurable, gives platform for managing and extending binary configuration. 
* Provides clean configuration interface to user.
* Allows selection of binary packages to be installed and supports package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom Linux image.
* Can be further extended to make configuration graphical user interface.
Refer bug [3252]
== How it works? ==
Binary configuration can be used in two ways
# Binary configuration integrated to current poky: Useful for current users where target image is generated from poky while building from sources.
# Post build binary configuration: Useful for users who want to generate custom configured images from already built package binaries.
== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata. 
:Ex: shadow package provides adduser config command. 
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.
Please refer section #Defining a new config command.
=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration (file)===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.
The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image. 
#:Ex: addpkg(shadow)
# Configuring the content
#:Ex: adduser(test, test123, life=999)
# Exporting the rootfs
#:Ex: diskimage( ext3(size=10240) )
Refer #Binary configuration file format.
== Post build binary configuration ==
Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. config-library which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to section #Binary level user configuration (file) for more details regarding Binary level user configuration.
== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.
=== Defining a new config command ===
Config commands are implemented in python in <package_name>_cfg.py file which is kept under <package>/files/configlib/pkgconf folder. Adding a new config command requires
# Deriving new config command class from ConfigCmd class.
# Add __init__( ) method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override execute() method and add logic to apply the configuration. Shell functions can also be called from here by using shellexec python library.
# Add inherit binary-config-package to shadow bb file
Sample code:
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py
from configcmd import *
class users_adduser (ConfigCmd):
 """config command to add user"""
 uname=""
 passwd=""
 life=""
 def __init__(self,uname, passwd="", life=99999):
 #any input validation can be done here
 self.uname = uname
 self.passwd = passwd
 self.life = life
 def execute(self):
 #write logic to create user
 print "User :"+self.uname+" added"
== Appendix ==
=== Binary configuration file format ===
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef	: '[' sectioninfo ']' [statements] 
# statements	: [ statement ]* 
# statement 	: operation '(' [paramlist] ')' 
# operation 	: identifier
# paramlist 	: param [, param ]* 
# param 	: directparam | fullparam | embstatement 
# directparam 	: nondelimiters | '"' anycharcters '"' 
# fullparam 	: identifier '=' (directparam | embstatement) 
# embstatement 	: statement 
# identifier 	: [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >
[general]
version( "1.2" )
[pkgs]
addlib( 32,64 )
#minimal/mandatory pkgs are default added
addpkg( libacl )
#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )
addpkgs( packagegroup-core-boot,
	openssh,
	busybox,
	update-modules,
	# Additional Package Configuration
	binutils,
	coreutils,
	cracklib,
	#Added for disk Management
	e2fsprogs-e2fsck,
	e2fsprogs-mke2fs,
	e2fsprogs-ext4,
	elfutils,
	elfutils-libebl,
	iproute2,
	iputils,
	logrotate,
	nfs-utils,
	numactl-2.0.3-r0.x86_64,
	numactl-2.0.3-r0.x86,
	libstdcxx,
	libpam,
	pam-plugin-*,
	!pam-plugin-motd,
	!pam-plugin-permit,
	pciutils,
	portmap,
	procps,
	sed,
	shadow,
	sysconfig,
	syslog-ng,
	sysvinit,
	sysvinit-pidof,
	hw-breakpoint,
	#Added for disk Management
	util-linux,
	vlan,
	perf,
	lock-task )
adddebugpkg( strace,
	 ethtool,
	 file,
	 binutils-debug,
	 oprofile,
	 iproute2-debug
	 )
[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
	 package=kernel-module-ramfs,
	 status=e,
	 param="rsize=100,caddress=0x444" )
[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 ) 
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin) 
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
	 shell=/usr/bin,
	 group="mail,ssh,rtos" ) 
[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )
[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0) 
ifconfig( eth1, ffe::200 , mode=6) 
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )
[security]
include ( security.conf )
[export]
# should be more detailed and relooked
diskimage( 
	partitions (
		partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
		partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
		)
	)
# or
diskimage(
	ext2(size=10240)
	)
# or
flashimage(
	jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
	)
# or
flashimage(
	jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
	cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
	)
# or
archiveimage( type=cpio.tar.gz )
# or
isoimage(dvd) 
[others]