knotty: Ensure last tasks are disaplyed correctly in the footer

There was an issue where the last tasks run by bitbake would not
correctly get displayed in the knotty footer. This was due to the
total count including active tasks. This change ensures the footer
is displayed if the are any running tasks.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uihelper: Set update flag when start event encountered

Its a minor correctness detail but the update flag should be set
when Start events are encountered.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker.py: Don't dump the environment into the console log file

Dumping the environment data into the console log files directory is
invariably not what the user wants or expects and leads to confusion
when looking at the log directory.

This change forces the logs to be disabled by default when using
the -e option.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake/server: Remove dead console log code

This code is dead and doesn't do anything so lets remove it.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: providing details about process state through porgress bar

-small changes to the text of the progress bar when parsing recipes

[YOCTO #3282]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
command: add error to return of runCommand

Currently, command.py can return an error message from runCommand, due to
being unable to run the command, yet few of our UIs (just hob) can handle it
today. This can result in seeing a TypeError with traceback in certain rare
circumstances.

To resolve this, we need a clean way to get errors back from runCommand,
without having to isinstance() the return value. This implements such a thing
by making runCommand also return an error (or None if no error occurred).

As runCommand now has a method of returning errors, we can also alter the
getCmdLineAction bits such that the returned value is just the action, not an
additional message. If a sync command wants to return an error, it raises
CommandError(message), and the message will be passed to the caller
appropriately.

Example Usage:

    result, error = server.runCommand(...)
    if error:
        log.error('Unable to run command: %s' % error)
        return 1

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: hob was freezing because it doesn't receives well the log file

-after pressing "build image" button, hob was freezing because it didn't
receive well the log file

[YOCTO #3398]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
print clear message for "bitbake -e ASSUME_PROVIDED"

"bitbake -e ASSUME_PROVIDED" should fail, but the error message wasn't
clear enough in the past:

$ bitbake -e bzip2-native
[snip]
ERROR: Command execution failed: Traceback (most recent call last):
  File "/buildarea/lyang1/poky/bitbake/lib/bb/command.py", line 94, in
runAsyncCommand
    commandmethod(self.cmds_async, self, options)
  File "/buildarea/lyang1/poky/bitbake/lib/bb/command.py", line 323, in
showEnvironmentTarget
    command.cooker.showEnvironment(None, pkg)
  File "/buildarea/lyang1/poky/bitbake/lib/bb/cooker.py", line 325, in
showEnvironment
    fnid = taskdata.build_targets[targetid][0]
KeyError: 0
[snip]

With this patch, the massage will be:
[snip]
ERROR: bzip2-native is in ASSUME_PROVIDED
ERROR: Command execution failed: Exited with 1

Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

[YOCTO #3392]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty: kill duplicated import of 'time'

The duplicated import could result in an UnboundLocalError.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
parse/cache/cooker: Preserve order in the file inclusion list

The data returned by get_file_depends() may me used in contexts like
checksums where order is important. The current usage of sets means
that some of the checksums can change in circumstances they should not.

This patch changes to use lists, thereby removing the problem.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqueue: Allow partial setscene task coverage

When the setscene code was originally written it was thought that we'd
allow "partial" coverage. For example, if we just want to build the target
"bash:do_populate_sysroot" and its available from sstate, it makes no sense
to install gcc-cross's sstate package as its simply not needed.

Due to various other issues in the codebase, this functionality was
disabled/removed to allow the setscene code and sstate to stabilise and allow
us to concentrate on other problems.

The time has now come to enable "partial" coverage. There are two major changes
in this patch:

a) Creation of an unskippable list. This lists direct dependencies of
   build targets and hence things that cannot be skipped.

b) Addition of a handler which looks at a given setscene target and what depends
   on it and then decides whether its necessary.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: warnings during the build should be displayed in the Issues tab

Any issues encountered during the build (fatal or not) is displayed
in the Issues tab, and the total number of issues is changed.

[YOCTO #3376]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty: Colorize knotty interactive console output

Add bold color output to log level name and standard color output to log msg
when bitbake is run from an iteractive console.  Color output is only
enabled
if the terminal supports color.

Used Jason Wessel's recommendation for transparency on verbose, note and
plain.

Signed-off-by: Seth Bollinger <seth.boll@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
server/process.py: Change timeout error handling

In normal usage, we never hit the timeout issue. If we do, it becomes obvious
that the current error handling is not good enough. The request may have made it
to the server and the answer will get queued. This means the next command may get
the return value from the previous command with suitably puzzling results.

Without rewriting large sections of code, its not possible to avoid this problem.
It is better to increase the timeout to several seconds giving the server a chance
to respond and if it does timeout, hard exit since recovery is not possible with the
code base today.

I'd be happy to see the structure of this code improved but this quick fix at least
stops corrupted builds from happening which has to be a good thing.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: showing when build fails because out of disk space

-to enable this in hob, first you have to enable this in bitbake using
BB_DISKMON_DIRS and/or BB_DISKMON_WARNINTERVAL vars
-created "restart the build" action on the build_fail_top_bar

[YOCTO #3357]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob/builder: Don't rerun sanity checks

Run the sanity check only once

[YOCTO #3377]

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty/msg: Avoid usage of curses initscr/endwin to avoid terminal corruption

Using curses initscr/endwin causes screen corruption if for example you
suspend bitbake and resume it. This changes the code to use a less
invasive approach to determining colour availability on the terminal.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2/local: Improve handling of wildcard matches

Currently wildcard matches end up working by FILESDIR being defined
in the metadata to a default of "." in FILESPATH which is hacky at best.

This patch adds the behaviour into the fetcher so its at least slightly
more explicit.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: Avoid using FILESDIR in unpack

Currently there is code which uses FILESDIR in unpack to ensure
parent directories are created, leading to differing behaviour depending on
which search path is used to locate the directory.

This change standardises the code and takes the data from the fetcher in
question meaning we can standardise the code and deprecate FILESDIR.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update version to 1.17.0

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>