cooker: Split data from configuration

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lib/bb/cache.py: invalidate cache when file checksum entry no longer exists

Go through the cached list of file checksums and check if any of the
files no longer exist; if any are missing then invalidate the cached
recipe, which will force it to be reparsed and thus force the list of
files to be collected again. This prevents a warning when moving a file
to a different location that is still picked up by the recipe, e.g.
moving a file from a "files" subdirectory to one named with the recipe
name (${BPN}).

Fixes [YOCTO #4474].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
tinfoil: fix for move of data attribute to cooker

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-layers: fix for move of data object to cooker

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: Fix AUTOINC handling

AUTOINC was meant to appear once at the start of the version string.
The list of names may not be sorted meaning it could get inserted in
the middle. This patch simplifies the code and ensures it appears at
the start.

Include cache version bump to ensure the cache picks up these changes.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Split runqueue to use bitbake-worker

This is a pretty fundamental change to the way bitbake operates. It
splits out the task execution part of runqueue into a completely
separately exec'd process called bitbake-worker.

This means that the separate process has to build its own datastore and
that configuration needs to be passed from the cooker over to the
bitbake worker process.

Known issues:

* Hob is broken with this patch since it writes to the configuration
  and that configuration isn't preserved in bitbake-worker.
* We create a worker for setscene, then a new worker for the main task
  execution. This is wasteful but shouldn't be hard to fix.
* We probably send too much data over to bitbake-worker, need to
  see if we can streamline it.

These are issues which will be followed up in subsequent patches.

This patch sets the groundwork for the removal of the double bitbake
execution for psuedo which will be in a follow on patch.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Move the bitbake-worker execution to a higher level

The worker was being executed by each execution queue so would get
constructed twice for each build. This is wasteful so move execution
to the main runqueue so we only have to start the worker once.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Abstract the start and teardown worker functions

We're going to need a fakeroot/pseudo version of the worker so
abstract the code to start the worker process.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Spawn a separate worker for fakeroot tasks

This allows us to have an exec() inside bitbake after pseudo-native has been
built and hence is the final step in allowing us to drop the double execution
bitbake wrapper.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Spawn a separate worker for fakeroot tasks

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: Bump minimum python version to 2.7.3

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: Update version to 1.19.1

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>