cache: Don't reload the cache file since we already have this data in memory

If we're writing out merged data to disk, its safe to assume that either
we loaded the data or couldn't. Loading it again is relatively pointless
and time consuming.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cache: Optimise invalid cache file handling

If there is a corrupt/invalid cache file, we'd keep trying to reopen
it. This is pointless, simplify the code paths and delete the dead
file.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
codeparser cache improvements

It turns out the codeparser cache is the bottleneck I've been observing when running
bitbake commands, particularly as it grows. There are some things we can do about
this:

* We were processing the cache with "intern()" at save time. Its actually much
  more memory efficient to do this at creation time.

* Use hashable objects such as frozenset rather than set so that we can
  compare objects

* De-duplicate the cache objects, link duplicates to the same object saving
  memory and disk usage and improving speed

* Using custom setstate/getstate to avoid the overhead of object attribute names
  in the cache file

To make this work, a global cache was needed for the list of set objects as
this was the only way I could find to get the data in at setstate object creation
time :(.

Parsing shows a modest improvement with these changes, cache load time is
significantly better, cache save time is reduced since there is now no need
to reprocess the data and cache is much smaller.

We can drop the compress_keys() code and internSet code from the shared cache
core since its no longer used and replaced by codeparser specific pieces.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
world-image: add extra target

* similar to world target, but collects all PACKAGES
* recipes marked with EXCLUDE_FROM_WORLD are excluded from world as well
  as world-image
* recipes marked with EXCLUDE_FROM_WORLD_IMAGE are excluded completely
  from world-image, sometimes you can build 2 recipes in same sysroot,
  but cannot install them in the same image
* PACKAGES listed in EXCLUDE_PACKAGES_FROM_WORLD_IMAGE are also
  excluded. You can remove e.g. ${PN}-dbg, but keep all other PACKAGES.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>