runqueue: Fix setscene hard dependency problems

Commit c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051 added in the idea of hard dependencies
such as the case a setscene has a hard dependency on pseudo-native and that
dependency wasn't available from sstate for some reason.

Unfortunately the implementation was a bit too enthusiastic, causing rebuilds
of things when it wasn't necessary. A test case was:

bitbake quilt-native
bitbake quilt-native -c clean
bitbake <some-image>

and then you'd watch quilt-native get rebuilt for no good reason.

The clue to the problem is in the for loop where it never depends on
the item being iterated over.

The fix is to include the exact list of hard dependencies rather than
guessing. With these changes, the use case above works, the one in
the original commit also works.

This patch also adds in or cleans up various pieces of logging to
allow issues like this to be more easily debugged in future.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
user-manual-ref-variables.xml: Added BB* variables to glossary.

BB_STRICT_CHECKSUM
BB_WORKERCONTEXT
BB_SIGNATURE_EXCLUDE_FLAGS
BB_HASHBASE_WHITELIST
BB_INVALIDCONF
BB_LOGFMT
BB_RUNFMT
BB_RUNTASK
BB_HASHBASE_WHITELIST
BB_STAMP_POLICY
BB_STAMP_WHITELIST
BB_SCHEDULER
BB_SCHEDULERS
BB_SETSCENE_DEPVALID
BB_SETSCENE_VERIFY_FUNCTION
BB_SIGNATURE_HANDLER
BB_SRCREV_POLICY
BB_VERBOSE_LOGS

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Added FAKEROOT* variables to glossary.

FAKEROOT
FAKEROOTCMD
FAKEROOTBASEENV
FAKEROOTDIRS
FAKEROOTENV
FAKEROOTNOENV

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Added PREFERRED_PROVIDERS to glossary.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Edits to PREFERRED_PROVIDER.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Added 6 new variables to glossary:

MULTI_PROVIDER_WHITELIST
CACHE
PERSISTENT_DIR
FETCHCMD
FILE
FILESDIR

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Edits to FILESPATH.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Added 11 new variables to glossary.

SRCREV_FORMAT
BUILDNAME
BB_ENV_WHITELIST
BB_PRESERVE_ENV
BB_ENV_EXTRAWHITE
BB_ORIGENV
BB_TASKHASH
BITBAKE_UI
BBDUBUG
BB_DEFAULT_TASK
BBINCLUDED

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-ref-variables.xml: Review edits to several variables in glossary.

ASSUME_PROVIDED
BBCLASSEXTEND
SRC_URI
PACKAGES_DYNAMIC
BB_NUMBER_THREADS
BB_DANGLINGAPPENDS_WARNONLY

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-metadata.xml: Added "Checksums (Signatures)" section.

Added this section to the end of the Metadata chapter.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-intro.xml: General edits to Introduction chapter.

I performed a general edit to this chapter.  Some significant changes
include changing the chapter's title to "Overview" when it was titled
"BitBake User Manual", doing some consolidation of text to eliminate
a couple sections that described methods to obtain a copy of BitBake,
and various improvements as needed.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-metadata.xml: Edits through syntax section

I made some general improvements in the "Overview" and
"Basic Syntax" sections.  Additionally, I added a blank
section for "Variable Flags" that will eventually hold general
information on this concept.  Finally, come review edits to the
"Defining Pure Python Functions" section per Paul Eggleton.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-metadata.xml: Edits to "Defining Pure Python Functions"

Made a review edit here to the section per Paul Eggleton.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-metadata.xml: Edits to "Conditional Syntax (Overrides)"

Re-wrote this section to use clearer more described examples.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
user-manual-metadata.xml: Edits to the "Sharing Functionality" section.

Applied some review edits from Paul to the section.

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
runqueue: Ensure we do run 'masked' setscene tasks if specified as targets

If you specify multiple targets on bitbake's commandline and some of them are
setscene tasks which are "masked" by other tasks they may not get run.

For example <image>:do_rootfs <kernel>:do_populate_sysroot

the rootfs tasks "masks" the populate_sysroot task so bitbake would currently
decide not to run it. In this case, we do really want it to be run.

The fix is not to skip anything which has been given as an explict target.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Fix silly variable overlap

A previous commit of mine used the target variable for two different uses
resulting in a lot more sstate being installed than is needed.

Fix the variable to use two different names and unbreak the setscene
behaviour.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
world-image: add extra target

* similar to world target, but collects all PACKAGES
* recipes marked with EXCLUDE_FROM_WORLD are excluded from world as well
  as world-image
* recipes marked with EXCLUDE_FROM_WORLD_IMAGE are excluded completely
  from world-image, sometimes you can build 2 recipes in same sysroot,
  but cannot install them in the same image
* PACKAGES listed in EXCLUDE_PACKAGES_FROM_WORLD_IMAGE are also
  excluded. You can remove e.g. ${PN}-dbg, but keep all other PACKAGES.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>