bitbake: cooker,xmlrpc,servers: implement CookerFeatures

Implementing feature set selection that allows a client
to enable specific features in the server at connection time.

Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.

Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.

In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.

The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.

The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.

The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: cooker,runqueue: send the task dependency tree

Adding a CookerFeature that allows UIs to enable
receving a dependency tree once the task data has been
computed and the runQueue is ready to start.

This will allow the clients to display dependency
data in an efficient manner, and not recompute the runqueue
specifically to get the dependency data.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: build, runqueue: adds info to the *runQueue* events

This patch adds task identifying information for all
runQueue and sceneQueue events, and for bb.build.Task* events.
This will allow matching event to specific tasks in the UI
handlers processing these events.

Adds RunQueueData functions to get the task name and task
file for usage with the runQueue* events.

Adds taskfile and taskname properties to bb.build.TaskBase.

Adds taskfile and taskname properties to the *runQueue* events

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: cooker: get extra information from recipe cache

The loaded cache modules may add extra attributes to
the recipecache, that will be populated by the cache
classes required by the UI. These attributes
will be used by the UI to display relevant information.

Adds cachefields cache class field to specify
for each cache class which attributes will be set
in the recipecache.

Adds code to automatically expand depends tree with the
fields exported by the extra cache class.

Fixes a cache field name in the HOB UI.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: cooker, command: add a command to return global data

Adding the 'getAllKeysWithFlags' read-only command that will
return a dump of the global data state, together with specified
flags for each key. The flag list is passed in as the first
parameter to the command.

This will be used by UI clients to get the build configuration.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: event: adding generic event for metadata usage

Adding the generic bb.event.MetadataEvent that is
targeted specifically at metadata usage. This is
needed in order to let the metadata code send and receive
events during asynchrous execution without having
to define each event specifically in Bitbake.

Metadata code should subscribe to and fire the MetadataEvent
in order to communicate asynchronously, and identify
the object using event.type field, and parse the
data in the event.data field.

Knotty UI will ignore these event by default.

This deprecates RequestPackageInfo/PackageInfo, and that
event pair will be removed in the future.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: runqueue: add task hash to Queue events

Adding the sstate-related hash for all runqueue and
scenequeue tasks, as it's needed in the WebHob data.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
build: Add BB_TASK_NICE_LEVEL to task code

On Linux its not possible for processes to regain a previous nice level after
it has changed. Its therefore not possible to have a core low priority and
then raise the priorities of individual tasks.

This variable allows us to do something like:

BB_TASK_NICE_LEVEL = "5"
BB_TASK_NICE_LEVEL_task-testimage = "0"

to give priority to specific tasks which the BB_NICE_LEVEL functionality
doesn't give us the option of.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bin/bitbake: Improve --help text

The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.

This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.

[YOCTO #4856]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: populate "Save image" dialog when saving changes to a custom image recipe

If my build starts from a YP standard image recipe, the 'Save image
recipe' dialog should be empty.

If my build starts from a custom image recipe, the 'Save image recipe'
dialog should populate the 'Name' and 'Description' fields with the values
set for the custom image recipe.

[YOCTO #5004]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data_smart: Variable references don't contain newlines, spaces or tabs

The code is happily trying to expand variable names containing newlines,
spaces and tabs which are illegal characters in variable names. This
patch stops it doing this. This will change dependency checksums
since some rather weird dependencies were being attempted to be expanded.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
tests/data: Whitespace in key names is a really bad idea

The parser never has supported it, the datastore API did happen to work
but whitespace in key names is a really bad idea and not something I think we
should encourage or support.

Fix test case failures after excplitly ignoring it for variable expansion
purposes.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>