hig.py: refactor into individual components

Since hig.py was becoming too large (which impacted maintenance and
operations like git merges) it was split into individual files for
the classes that were implemented in hig.py. hig is now a Python
package (lib/bb/ui/crumbs/hig/). The patch was tested by
building core-image-basic/qemux86 in Hob and accessing the various
UI elements implemented in hig. Note that the patch does not change
the functionality of Hob in any way, it's just a code refactoring
that should make life a bit easier for Hob developers.

[YOCTO #3200]

Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BBHandler/data: Standardise some setVar access formats

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: data_smart.py and friends: Track file inclusions for bitbake -e

This code adds inclusion history to bitbake -e output, showing
which files were included, in what order. This doesn't completely
resolve timing questions, because it doesn't show you which lines
of a file were processed before or after a given include, but it
does let you figure out what the path was by which a particular
file ended up in your build at all.

How it works: data_smart acquires a .history member, which is an
IncludeHistory; this represents the inclusion of a file and all its
inclusions, recursively. It provides methods for including files,
for finishing inclusion (done as an __exit__), and for
dumping the whole tree.

The parser is modified to run includes inside a with() to push
and pop the include filename.

RP Modifications:

a) Split Include and Variable tracking
b) Replace deepcopy usage with dedicated copy function
c) Simplify some variable and usage

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: data_smart.py and friends: Track variable history

This patch adds tracking of the history of variable assignments.
The changes are predominantly localized to data_smart.py and
parse/ast.py. cooker.py and data.py are altered to display the
recorded data, and turn tracking on for the bitbake -e case.
The data.py update_data() function warns DataSmart.finalize()
to report the caller one further back up the tree.

In general, d.setVar() does what it used to do. Optionally,
arguments describing an operation may be appended; if none
are present, the operation is implicitly ignored. If it's
not ignored, it will attempt to infer missing information
(name of variable, value assigned, file and line) by examining
the traceback. This slightly elaborate process eliminates a
category of problems in which the 'var' member of the keyword
arguments dict is set, and a positional argument corresponding
to 'var' is also set. It also makes calling much simpler for
the common cases.

The resulting output gives you a pretty good picture of what
values got set, and how they got set.

RP Modifications:

a) Split from IncludeHistory to separate VariableHistory
b) Add dedicated copy function instead of deepcopy
c) Use COW for variables dict
d) Remove 'value' loginfo value and just use 'details'
e) Desensitise code for calling order (set 'op' before/after
   infer_caller_details was error prone)
f) Fix bug where ?= "" wasn't shown correctly
g) Log more set operations as some variables mysteriously acquired
   values previously
h) Standardise infer_caller_details to be triggered from .record()
   where at all possible to reduce overhead in non-enabled cases
i) Rename variable parameter names to match inference code
j) Add VariableHistory emit() function to match IncludeHistory
k) Fix handling of appendVar, prependVar and matching flag ops
l) Use ignored=True to stop logging further events where appropriate

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
base.bbclass: Fix bitbake example class usage

Patch from Andrew Stubbs <ams@codesourcery.com> to fix listtasks and showdata
functions to output data correctly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: Sort file checksums by value, not path

Changing the path to a file could change the task hash even if the file
still has the same checksum. This occurs when the task depends on
multiple files and the sort order of their paths changes. Usually the
sorting is consistent because layers tend to have the same relative
paths, but this should take care of other configuations.

The problem arose when using a .bbappend to add files to a recipe in
another layer. If the layer is located alongside the other layers and
their parent directory is moved, the hash does not change. However,
moving the .bbappend layer outside of the common directory can change
the path sort order and the task hash.

Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hg.py: Fixed fetch failure that occurs when SRCREV is a tag.

Removed "-r REV" from hg clone invocation to fetch the entire repo,
rather than just the subset at the specified revision. This will ensure
that the specified tag exists for successive commands (e.g., the update
built on line 149.)

Signed-off-by: Jon Szymaniak <jon.szymaniak@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: Fix GTK+ and PyGtk version check error message to be more clear

Improve the error message the user sees if the versions are incorrect.

[YOCTO #3637]

Signed-off-by: Jessica Zhang <jessica.zhang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: BBHandler/ConfHandler: Improve multiline comment handling

Faced with an expression like:

 # Some comment \
 FOO = "bar"

what should bitbake do? Technically, the \ character means its multiline and
currently the code treats this as a continuation of the comment. This can
surprise some people and is not intuitive.

This patch makes bitbake simply error and asks the user to be clearer
about what they mean.

(Bitbake rev: 589d31ce41e019ee6a7cb6527d67bc76c0b6382a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BBHandler/ConfHandler: Merge fix for multiline comments

This was meant to be squashed into the previous commit for multiline comment
handling. It fixes the case the commented multiline is followed by an empty
line which was resulting in a traceback instead of a sane error message.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: progress bar changed to busy cursor when you open log file

-the first implementation for this bug used a progress bar, that is shown
during open file process; it revelead that the progress bar stops earlier
-now I have implemented using gtk.show_uri() method, that shows itself a
busy cursor when it opens a file;
-deleted the code for the first implementation

[YOCTO #2997]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: Hob should display warnings generated during parsing

-now Hob catches the warnings generated during parsing, and after the parsing
is completed, if there are any warnings, it shows a bar that contains a message
with how many warnings has encountered, and a button "View warnings"
-when "View warnings" button is clicked, Hob shows a dialog with the warnings;
if there more than 1 warning, you can use "Previous" and "Next" button to see them

[YOCTO #3215]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
parse_py: add ~ to config_regexp

- enable the '~' character in bitbake variables

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
persist_data: add get_by_pattern method to API

- one can use get_by_pattern to get a list of values associated
with keys that match the specified pattern

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: combo box updated correctly

The 'select a base image' option is no longer displayed once
an image is selected

[YOCTO #3671]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
crumbs: Add file lost from previous commit

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Merge branch 'master' into wmat
prserv: use only PRSERV_HOST

- remove PRSERV_PORT variable
- use 'hostname:port' as PRSERV_HOST format
- use 'localhost:0' for enabling the local PRServer

[YOCTO #3744]

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: hob do not use .hob dir for variables from local.conf and bblayers.conf

If .hob exists, hob takes the variable from there. Change to not override anymore
the variables which comes from bblayers.conf and local.conf

[YOCTO #2934]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cache.py: remove the duplicated self.file_depends

There are two "self.file_depends =" lines in cache.py::CoreRecipeInfo:

class CoreRecipeInfo(RecipeInfoCommon):
    __slots__ = ()

    cachefile = "bb_cache.dat"

    def __init__(self, filename, metadata):
        self.file_depends = metadata.getVar('__depends', False)
    [snip]
        self.file_depends = metadata.getVar('__depends', False)

They are duplicated, remove the last one.

[YOCTO #3795]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>