core-image-minimal-initramfs: Remove odd quote characters

The quote characters present trigger python 3 characters type warnings,
we don't need them so replace them with normal quote characters.

(From OE-Core rev: 3c104443506cb89d72944e46096a94a80838a707)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
clases/lib: Use modern exception syntax

Update older code to use modern exception handling syntax which
is the form accepted by python 3.

(From OE-Core rev: 3439cec880d2d293cf82d8ed82c75942defd8170)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fixup

(From OE-Core rev: 776e1c46674514d12d90e64eb0cc50f6e151a64a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildstats: Update to ensure files are closed

python3 doesn't like files being left open. This updates the code style
to ensure file are closed.

(From OE-Core rev: 4e47983b0237e969410567e333c7e0d6b30c73ce)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
classes: Use modern exception raising syntax

Modern expection rasing syntax is function call format, convert to this
to keep python 3 happy and model correct coding style in the core.

(From OE-Core rev: 1b14a979177e84f90e2753245af5907d8f9952ad)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fixup

(From OE-Core rev: aae441bfefb2896c4be05e519469e133fe8bba47)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
classes/recipes/lib: Fix various python whitespace issues

There are some left over tab characters in the python functions. This
removes them and resolves python 3 errors.

(From OE-Core rev: c80a11c2543a2276c6acf1da4eed35919c6c8c2a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sanity.bbclass: Use open(), not file()

file() syntax is removed in python 3, use open() instead.

(From OE-Core rev: f78e038a3525a5b4a1b7382d9ecc3bd5b4a38f57)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
metadata_scm: Replace deprecated operator

The <> operator is deprecated, use != instead which is equivalent.

(From OE-Core rev: a17a2eb18912c159bc9ec00ae7cbf2c96144ab44)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
classes/lib: Update to use print as a function call

In python 3 print is a function call. In some cases bb.note is a more
appropriate call to make.

(From OE-Core rev: c7f0561d75ac21a41b91d6b16c71b4b159849d55)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta/lib/oe: Replace StandardError with Exception

StandardError is removed in python3, replace with Exception class instead.

(From OE-Core rev: acbb4e2ff30521ad0ea1b4d5a30d6fd4204fc220)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
insane: Open file in binary mode

We open the file we're writing to in binary mode so open the input stream with the
same mode so things match. This avoids errors with python3.

(From OE-Core rev: 2475e036d4259175a91c755360016ffb3c03fc7d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sanity/patch.py: Remove commands module usage

The commands module is removed in python3. Use the subprocess module instead
and the pipes module to replace the mkargs usage.

(From OE-Core rev: 8087233eca1010f802ac1f081983dccac7c57d1e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fixup

(From OE-Core rev: 7c583a524e0fcc8bace8ac76dffbb802a7a37ae2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
image/kernel-module-split/eglibc-ld.inc: Remove has_key() usage

The has_key() attribute has been removed in python 3 since there is better
syntax available. Use the improved syntax.

(From OE-Core rev: 1963e155a781a16abd31d7ed1a570bd88ed6c9b5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fixup

(From OE-Core rev: d9b3923d1db67bab30afea83860bb51c5a3d4c24)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
base.bbclass: Drop unused/obsolete import

(From OE-Core rev: 37b26bd92f8b096dec78ec6c600586148f8c0226)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake.conf: Use casting to ensure valid comparision

python3 is stricter about type comparisions so add an explicit cast to int()
to ensure this code is portable.

(From OE-Core rev: 099d5db2ac0602cd735c33431a7686e8cc41e865)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: bin/bitbake: Ensure early errors are shown to the user

Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.

(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: utils: Improve better_exec traceback handling

The current bitbake tracebacks are hard to read/confusing and sometimes
incomplete. This patch attempts to do better by:

* Moving the note about the exact exception to the end to make things
  read in sequence
* Merged the initial stack trace to become part of the code dump
* Added handling for "/xxxx" file paths since we can load these files
  and include the data as part of the trace
* Dropped the ERROR: prefix to every line, allowing the error messages to
  be spacially accosicated in the UIs
* Moved the "From:" line to the top of each code block and ensured its present
  consistently

With the complexity now in this funciton, I've added try/except wrapping around
it to ensure we catch exceptions in the exception handler too.

Example before:

"""
ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb:
TypeError: 'filter' object is not subscriptable

ERROR: The stack trace of python calls that resulted in this exception/failure was:
ERROR:   File "do_populate_lic", line 13, in <module>
ERROR:
ERROR:   File "do_populate_lic", line 6, in do_populate_lic
ERROR:
ERROR:   File "license.bbclass", line 99, in find_license_files
ERROR:
ERROR:   File "/media/build1/poky/meta/lib/oe/license.py", line 38, in visit_string
ERROR:     if pos > 0 and license_pattern.match(elements[pos-1]):
ERROR:
ERROR: The code that was being executed was:
ERROR:      0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
ERROR:      0010:    copy_license_files(lic_files_paths, destdir)
ERROR:      0011:
ERROR:      0012:
ERROR:  *** 0013:do_populate_lic(d)
ERROR:      0014:
ERROR: [From file: 'do_populate_lic', lineno: 13, function: <module>]
ERROR:      0002:def do_populate_lic(d):
ERROR:      0003:    """
ERROR:      0004:    Populate LICENSE_DIRECTORY with licenses.
ERROR:      0005:    """
ERROR:  *** 0006:    lic_files_paths = find_license_files(d)
ERROR:      0007:
ERROR:      0008:    # The base directory we wrangle licenses to
ERROR:      0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
ERROR:      0010:    copy_license_files(lic_files_paths, destdir)
ERROR: [From file: 'do_populate_lic', lineno: 6, function: do_populate_lic]
ERROR:      0095:        lic_files_paths.append((os.path.basename(path), srclicfile))
ERROR:      0096:
ERROR:      0097:    v = FindVisitor()
ERROR:      0098:    try:
ERROR:  *** 0099:        v.visit_string(license_types)
ERROR:      0100:    except oe.license.InvalidLicense as exc:
ERROR:      0101:        bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
ERROR:      0102:    except SyntaxError:
ERROR:      0103:        bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True)))
ERROR: [From file: 'license.bbclass', lineno: 99, function: find_license_files]
ERROR: Function failed: do_populate_lic
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.17442
"""

Example after:

"""
ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'do_populate_lic', lineno: 13, function: <module>
     0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
     0010:    copy_license_files(lic_files_paths, destdir)
     0011:
     0012:
 *** 0013:do_populate_lic(d)
     0014:
File: 'do_populate_lic', lineno: 6, function: do_populate_lic
     0002:def do_populate_lic(d):
     0003:    """
     0004:    Populate LICENSE_DIRECTORY with licenses.
     0005:    """
 *** 0006:    lic_files_paths = find_license_files(d)
     0007:
     0008:    # The base directory we wrangle licenses to
     0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
     0010:    copy_license_files(lic_files_paths, destdir)
File: 'license.bbclass', lineno: 99, function: find_license_files
     0095:        lic_files_paths.append((os.path.basename(path), srclicfile))
     0096:
     0097:    v = FindVisitor()
     0098:    try:
 *** 0099:        v.visit_string(license_types)
     0100:    except oe.license.InvalidLicense as exc:
     0101:        bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
     0102:    except SyntaxError:
     0103:        bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True)))
File: '/media/build1/poky/meta/lib/oe/license.py', lineno: 38, function: visit_string
     0034:        new_elements = []
     0035:        elements = filter(lambda x: x.strip(), license_operator.split(licensestr))
     0036:        for pos, element in enumerate(elements):
     0037:            if license_pattern.match(element):
 *** 0038:                if pos > 0 and license_pattern.match(elements[pos-1]):
     0039:                    new_elements.append('&')
     0040:                element = '"' + element + '"'
     0041:            elif not license_operator.match(element):
     0042:                raise InvalidLicense(element)
Exception: TypeError: 'filter' object is not subscriptable

ERROR: Function failed: do_populate_lic
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.3275
ERROR: Task 9 (/media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb, do_populate_lic) failed with exit code '1
"""

(Bitbake rev: c5de66b870406d9bd1161a9b7e2b04fe6eb065fe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>