data_smart: Account for changes in append/prepend/remove in the config hash

bitbake wasn't reparsing when _remove items were added to its configuration
and equally, appends/prepends were also being badly tracked. This
change enrures these variables are accounted for in the configuration
hash.

[YOCTO #5172]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: xmlrpc: delete unused code BitBakeUIEventServer

BitBakeUIEventServer is an unused class that pushes UI
events over a separate thread.

The current version of XMLRPC server works just fine with
the classic UI event handlers, so this class is not needed.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Drop obsolete worker test

This call only ever happens in cooker context now so we can drop the
nasty worker check from here.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Clean up init/reset configuration code

Currently the cooker event data isn't rebuilt upon reset and the cache
configuration cannot be changed after init. These are both bad things
and this patch refactors the init/reset code so that it is possible
to reconfigure the server.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Rename confusing 'stop' state to 'forceshutdown'

The shutdown state causes the server to finish what its doing, stop was
them meant to completely stop it. It doesn't mean the server is stopped
though. Renaming the current stop event for forceshutdown gives more
meaning to what it actually does. The stopped namespace then becomes
available to indicate a completely stopped server.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker/command: Add finishcommand to reset cooker state

After running a command on the server, it needs to reset to the initial
state. This ensures that subsequent clients start from a known state
and notice any configuration changes.

Ultimately we may want to do more than this buts a good start and better
than nothing.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
command: Treat empty messages as failures, not CommandCompleted

Empty messages should trigger CommandFailed, not CommandCompleted as
otherwise the exit code will be incorrect.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty: Cleanup error/interruption handling

Only display a CommandFailed ERROR: message if there is an error to display.

Only display an errors summary if we actually displayed errors.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: updateCache should rause exceptions, not sys.exit

Exiting from the server is antisocial, instead we should raise an exception. This
will correctly fail the current command and reset the server state. We use
the handled exception since for these conditions to occur, something was
already displayed to the user.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
build: Add logfile to add TaskBase events

We add the path to the logfile for all Task events except TaskInvalid
so that we can trace back the logfile locations at some future point.

TaskInvalid doesn't ever have a logfile.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: add runQueueTaskSkipped event

Adding a runQueueTaskSkipped to notify that the tasks that are not
run either because they are set-scened or they don't need an update
(timestamp was ok).

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Avoid duplication for taskdata creation

Clean-up to avoid duplication and promote code reuse to factor
taskdata creation into a common function.

[RP: minor tweaks]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Allow profiling of the parser in profile mode

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data: Be explicit in data_db check

The if statement current causes the size of parent to be calcuated which
is like a len() operation on a datastore. Since we're only interested
whether the value is none, checking explictly for this gives a
small performance gain.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
siggen: Use lookup cache exclusively

All the values we need are already guaranteed to be in the lookupcache
so rather than fetch variables again, just use the cache. This gives a
small performance improvement and simplifies the code.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data_smart: Improve variable expansion regexp

Debugging showed the variable expansion regexp was catching python
expressions (starting with @). Since these are caught by their own
dedicated regexp, stop matching these for the plain variable expansion
for small performance improvements.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data_smart: use the expand_cache in VariableParse

When in VariableParse, use the expand_cache if possible rather than looking
up data. Ultimately it would come from the same place but this short cuts
a heavily used code block for speed improvements.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data: Use direct iteration, not keys()

Profiling shows the creation of keys() has overhead and we're better using
an iterator rather than the memory associated with the huge list of keys
when iterating the whoe datastore. We minimise the number of times
we do this to twice only per recipe.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data: Cache an list of export variables

Compute a cache of the list of potential export variables so
that we don't have to compute the list from scratch.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data_smart: Add explict None checks

Simple if xxx checks end up calling len(xxx). We're interested in the specific case
of None which means we can break out the iterator much earlier after the first
item. This adds in the specific tests for None in what is a hot path in the
data store code which gives small performance gains.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>