data: Fix output inconsistencies for emit_var

VAL = ""     (not shown)
VAL = " "    (shown as "")
VAL = " x"   (shown as "x")

would all show up rather differently to what would be expected in the
bitbake -e output. This fixes things so they appear consistently.

The output for running some shell functions may also change slightly
but shouldn't change in a way that is likely to cause problems.

[YOCTO #5507]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: fetch2: Remove the unused urldata from the git fetcher.

Signed-off-by: Florin Sarbu <florin.sarbu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: Correct a few calls to latest_revision()

In 6a48474de9505a3700863f31839a7c53c5e18a8d the url parameter to a
number of functions was removed. However, not all calls to
latest_revision() were fixed...

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
codeparser: Drop unneeded variable separation

There is no good reason to separately track var_references and
references so merge them and remove the unneeded variable.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Optimise next_buildable_task()

This unlikely looking function was found to be eating a lot of CPU time
since it gets called once per trip through the idle loop if we're not
running a maximum number of processes. This was particularly true in
world builds of 13,000 tasks.

Calling the computation code is pretty pointless because until some
other task finishes nothing is going to become available to build.
We can know when things become available so this patch teaches the
scheduler this knowledge.

It also:

* skips any coputation when nothing can be built
* if there is only one available item to build, ignore the priority map
* precomputes the stamp filenames, rather than doing it every time
* saves the length of the array rather than calculating it each time
  (the extra function overhead is significant)

Timing wise, initially, 5000 iterations through here was 20s, with
the patch 200000 calls takes the same time. The end result is that
builds get up and running faster.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: Share BB_TASKDEPDATA with tasks

Currently tasks have no knowledge of which other tasks they depend
upon. This makes it impossible to do at least two things which would be
desirable/interesting:

a) Have the ability to create per recipe sysroots
b) Allow the aclocal files to be present only for the entries in
   DEPENDS (directly and indirectly)

By exporting task data through this new variable, tasks can inspect
their dependencies and then take actions based upon this.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data/codeparser: Improve handling of contains functions

One of the current frustrations with the sstate checksums is that
code like base_contains('X', 'y',...) adds a full dependency on X
and varies depend even on whitespace changes in X.

This patch adds special handling of the contains functions to expand
the first parameter and check for the flag specified by the second
parameter (assuming its a string).

The result is then appended to the value of the variable with a "Set"
or "Unset" status. If the flag is added/removed, the stored variable
value changes and hence the checksum changes. No dependency on X
is added so it is free to change with regard to other flags or
whitespace.

This code is far from ideal, ideally we'd have properly typed variables
however it fixes a major annoyance of the current checksums and
is of enough value its worth adding in a stopgap solution. It shouldn't
significantly restrict any propely typed variable implementation in
future.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hob: force notebook refresh

Redraw the entire notebook widget after
number-of-issues drawable is incremented
(because for this case, the notebook widget
doesn't refresh it's children automatically).

[YOCTO #5596]

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
crumbs/builder.py: Fix typo for indefinite article

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: Fix support for wildcards in bbappend filenames

The 'bbappend in f' incorrectly compares the current recipe with the avaliable
bbappends recipes. This comparsion causes unrequested bbappend files to be
appended, e.g. in the case of 'libgcc_4.8.bb', the bbappends for 'libgcc_4.8.bb'
and 'gcc_4.8.bb' are added to the filelist (because 'gcc_4.8.bb' is contained in
the 'libgcc_4.8.bb' string) which in turn causes the gcc_4.8.bbappend files to
be appended to the libgcc_4.8 recipe.

This should be a 'bbappend == f' to match the previous implementation of this
function, such that if no wildcard is present the recipe names must match
exactly.

This issue was introduced by commit 31bc9af9cd56e7b318924869970e850993fafc5f,
which it related to [YOCTO #5411].

Signed-off-by: Nathan Rossi <nathan.rossi@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
parse/ConfHander/BBHandler/utils: Fix cache dependency bugs

Currently bitbake only adds files to its dependency list if they exist.
If you add 'include foo.inc' to your recipe and the file doesn't exist,
then later you add the file, the cache will not be invalidated.

This leads to another bug which is that if files don't exist and then
you add them and they should be found first due to BBPATH, again the
cache won't invalidate.

This patch adds in tracking of files we check for the existence of so
that if they are added later, the cache correctly invalidates. This
necessitated a new version of bb.utils.which which returns a list of
files tested for.

The patch also adds in checks for duplicate file includes and for now
prints a warning about this. That will likely become a fatal error at
some point since its never usually desired to include a file twice.

The same issue is also fixed for class inheritance. Now when a class
is added which would be found in the usual search path, it will cause
the cache to be invalidated.

Unfortunately this is old code in bitbake and the patch isn't the
neatest since we have to work within that framework.

[YOCTO #5611]
[YOCTO #4425]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: disable layer drag and drop outside the containing widget

[YOCTO #5577]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: Add --status-only option

The --status-only option is purely to check the health of the server.
The idea is to use it from the oe environment script so as to know if
the server needs to be started or not.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake, xmlrpc.py: Implement memory resident auto port configuration/restart

This patch adds the ability to dynamically select a port for the
bitbake memory resident server when the BBSERVER port is set to -1.
This allows for running multiple instances of the bitbake memory
resident server on the same system in different build directories.

The client portion of the bitbake instance can also request that the
server automatically start when using the auto port feature.  This is
to deal with a bitbake instance that eventually times out and exits or
that has died for some unknown reason.

The new functionality allows for lazy startup of the server after
sourcing the init script for the memory resident functionality.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake.lock: Add host:port to bitbake.lock for memres server

The idea is to build on the --status-only option for bitbake and
expose a mechanism where the oe init scripts can easily switch between
memres server and the non-memres server.

In the case of the standard oe init script the following
can shut down the server:

if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
    grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
    if [ $? = 0 ] ; then
       echo "Shutting down bitbake memory resident server with bitbake -m"
       BBSERVER=`cat bitbake.lock` bitbake -m
    fi
fi

A similar function can be used to automatically detect if the server
is already running for the oe memres init script.  This new
functionality allows for the memres init script to be started in a new
shell and connect up to an alaready running server without seeing the
error of trying to start the server multiple times.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
imagedetailspage: Fix crash with more than 15 layers

If you had more than 15 layers the system would crash since one more
value is added to one array than the other. This fixes the code
so equal numbers of values are added to the arrays and hence
doesn't crash when many layers are enabled.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: fix rotating progress icon animation

I've always found it somewhat annoying that this icon's rotation
animation wasn't quite correct; this was because it was looping around
to the second position instead of the first, which made it appear to
stutter.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
codeparser/data_smart: Optimise parsing speed

The previous "contains" changes caused a ~3% parsing speed impact.
Looking at the cause of those changes was interesting:

* Use of defaultdict was slower than just checking for missing entries
  and setting them when needed.
* Even the "import collections" adversely affects parsing speed
* There was a missing intern function for the contains cache data
* Setting up a log object for each variable has noticeable overhead
  due to the changes in the code paths uses, we can avoid this.
* We can call getVarFlag on "_content" directly within VariableParse
  for a noticeable speed gain since its a seriously hot code path.

This patch therefore tweaks the code based on the above observations to
get some of the speed back.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: Use BASEDATASTORE_TRACKING feature

Might as well use this functionality now we have it available.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker/command/hob: Cleanup configuration init/reset functions and commands

initConfigurationData and loadConfigurationData are similar functions, the only
reason for them appears to be to be able to reset the pre/post configuration
files. The current code is confusing and unmaintainable.

Instead this patch creates a new Sync command which allows these to be explicitly
set. The init and load functions can then be merged into one. There is then no
need for a parseConfiguration command, we can simply reset the server to have the
settings take effect.

The reset fuction is not an instant value return and triggers an event so it should
be an Async command, not a sync one.

The number of calls for the set pre/post command is probably higher than it
need be but someone with more familiarity with the hob code base can probably
figure out the right places its needed (maybe just init_cooker?).

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>