runqueue/build: Add recideptask flag

Currently, tasks like fetchall are slightly broken since if a recipe
has specific [depends] which occur after do_fetch and add items not listed
in DEPENDS and RDEPENDS, they are not caught by recrdeptask. We've gone
around in circles on this issue (e.g
http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/runqueue.py?id=5fa6036d49ed7befe6ad50ec95c61a50aec48195
) and in many cases the behaviour of recrdepends is correct but tasks like
fetchall need the other behaviour.

To address this we add a recideptask flag which can be used in conjuction
with the recrdeptask flag to specify which task to to the inspection upon.
This means entries like do_rootfs[depends] which have do_fetch tasks are
caught and run.

I'm not 100% happy with needing another flag but I don't see any rational
way to get the correct behaviour in all cases without it.

[YOCTO #4597]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: Enhance error message for BB_NO_NETWORK

BB_NO_NETWORK can be set by bitbake internally by the use of
BB_FETCH_PREMIRRORONLY so update the error message to give users a
hint about this.

[YOCTO #3222]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: remove the rest of the old template functionality

Until now, some configuration settings was saved in a hob specific
directory. From now on, it will be saved in conf directory through bitbake.

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake & hob: modify some commands and call them from hob

This patch adds a new command to create an extra configuration
file, in order to create it from Hob.
Also, it calls the enable/disable data tracking when needed.
Modified the prefiles and postfiles variables because the cooker
expects some lists.

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: check if imageconfiguration page has the attribute 'warnings_bar'

[YOCTO #4571]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow checksums to be used for files retrieved using SFTP

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correct a typo

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: remove the function for reparse because it does nothing

The command for reparse does nothing, so I have managed to remove it,
because it can be misleading. When a reparse is needed, it can be used
the parse function.

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
build.py: Cleanup data store references

Clean up a number of old style accesses to the datastore.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Ensure configuration matches variable tracking state

When we reset, the current tracking value may change unexpectedly. This
patch ensures that the default value is preserved over a datastore reset.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: call the initConfiguration when reparsing

As it is implemented now, when parsing the configuration files is
needed, it also needs an initialization.
Also, implemented a function to create an extra configuration files
which can be used as postfile/prefiles when parsing.

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: replace the init cooker with reset in order to keep the history

When the cooker initialization is called, the variables history is lost.
It need a reset, in order to load the configuration files.
Also, concatenated the value for INHERIT variable.

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: save variables in conf/ directory

Until now the variables were saved in bitbake configuration,
now they are saved in configuration files, in order to be read by
bitbake-worker. This helps to assure the consistency for the rest
of the variables.

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data_smart: Ensure variable flags are accounted for in config data hash

Currently if the flags set against a variable in the base data store
change, it doesn't automatically trigger a reparse when it really
should. For example with the blacklist class setting:

PNBLACKLIST[qemu] = "bar"
PNBLACKLIST[bash] = "foo"

will not trigger a reparse if only one entry is changed and a
blacklisted recipe can still be built.

I did consider using BB_SIGNATURE_EXCLUDE_FLAGS in here however it
doesn't make sense, we want to trigger a reparse when any of the
flags change too (which is different to the sstate signatures which
we wouldn't want to change in those cases).

[YOCTO #4627]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data_smart: Fix bug with overrides and weak default values

(aka pay the cookie monster for weak defaults)

If you have code like:

MYVAR = "a"
MYVAR_override ??= "b"

then MYVAR will get the value "a" even when override is in OVERRIDES. The
reason is that the value of ??= is set as a flag not a value and the cookie
monster isn't paid.

The fix is to ensure appropriate payment is made for a defaultval varflag
matching the usual setVar case.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
world-image: add extra target

* similar to world target, but collects all PACKAGES
* recipes marked with EXCLUDE_FROM_WORLD are excluded from world as well
  as world-image
* recipes marked with EXCLUDE_FROM_WORLD_IMAGE are excluded completely
  from world-image, sometimes you can build 2 recipes in same sysroot,
  but cannot install them in the same image
* PACKAGES listed in EXCLUDE_PACKAGES_FROM_WORLD_IMAGE are also
  excluded. You can remove e.g. ${PN}-dbg, but keep all other PACKAGES.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>