yocto-bsp: Add a processing pass to get rid of 'strange' filenames

Operating systems that can't handle filenames containing anything but
uppercase letters don't like to see 'strange' characters in filenames
such as the file and directory names yocto-bsp uses as part of its
templating mechanism.  To fix this, we essentially add another level
of metadata into the template files themselves rather than into their
names, and add a processing pass that internally gives us what we had
before.  Specifically:

  - strange characters in filenames are moved inside the files they
    apply to, to the first line of the file prepended with '#
    yocto-bsp-filename {{...}} filename | this'

  - strange characters in directory names are moved inside a new file
    of the same name but ending in .noinstall and with the first (and
    only) line of the file prepended with '# yocto-bsp-dirname {{...}}
    dirname'

If the new filename contained in the yocto-bsp-* tag is 'this', the
filename is the existing name of the file.  For a dirname, this would
be the filename with .noinstall removed.

"Fixes" [YOCTO #5312].

(From meta-yocto rev: 3dad2decbd682713f717950fc39fa0f63f1b8c98)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames

Give all the 'strange' yocto-bsp template filenames 'normal' names,
adding new yocto-bsp-filename and yocto-bsp-dirname tags for the new
filename processing pass where needed.

(From meta-yocto rev: c66a5b2acb2e243b857e5eb31c1427af296b4c9d)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames in x86 templates

For the x86 templates, give all the 'strange' yocto-bsp template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: 9fa30ad9cf1291a37d365515ec8d969fb4fd4aef)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames in mips templates

For the mips templates, give all the 'strange' yocto-bsp template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: afa61438aa31fbc5c87991c8b3ec88c228b9d2d1)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames in arm templates

For the arm templates, give all the 'strange' yocto-bsp template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: fb533fd98678671d319e52fc288783bbd0368218)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames in mips64 templates

For the mips64 templates, give all the 'strange' yocto-bsp template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: 8bc26cbfe2da0d6ffd08ecdce36fd50622bb1fad)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames in powerpc templates

For the powerpc templates, give all the 'strange' yocto-bsp template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: c21329385f93bd876ef675ee453627910b4d5dec)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-layer: Get rid of 'strange' filenames in templates

For the yocto-layer templates, give all the 'strange' template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: 0aa76b36ccb8262a7eec62845361c3d4b73ceb46)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-bsp: Get rid of 'strange' filenames in x86_64 templates

For the x86_64 templates, give all the 'strange' yocto-bsp template
filenames 'normal' names, adding new yocto-bsp-filename and
yocto-bsp-dirname tags for the new filename processing pass where
needed.

(From meta-yocto rev: d478b11db66795040009e9b7d7751300d4385018)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toasterconf: naming and description changes

Some minor changes to the names and descriptions
used in the Toaster configuration file:

* Change the local layer source name from Local Poky to Local
Yocto Project to match release naming, which makes no reference
to Poky

* Change Imported Layer Source to just Imported layers. It is
shorter, which helps table display

* Change the helptext of the dizzy release to remove the
"latest" reference, since might make maintenance hard

(From meta-yocto rev: ab1326597bbc505c5beade577206de78cda2dae8)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: add build artifacts table and other improvements

We add a BuildArtifacts class to store data about files
discovered during the build process and not stored anywhere
else.

Small cosmetic changes in the toasterui.

Add model methods to return file path display data relative
to the build environment instead of absolute file paths.

[YOCTO #6834]

(Bitbake rev: bbe24d912869312d561be199b2c029b0c898e049)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toasterui: add extra debug and development infos

We update and add logs throughout the code in order to help
with development. The extra logging is turned off by default,
but it can be enabled by using environment variables.

All logging happens through the Python logging facilities.

The toaster UI will save a log of all incoming events if the
TOASTER_EVENTLOG variable is set.

If TOASTER_SQLDEBUG is set all DB queries will be logged.

If TOASTER_DEVEL is set and the django-fresh module is available,
the module is enabled to allow auto-reload of pages when the
source is changed.

(Bitbake rev: 10c27450601b4d24bbb273bd0e053498807d1060)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: add option to write offline event log file

This patch adds a "-w/--write-log" option to bitbake
that writes an event log file for the current build.

The name of the file is passed as a parameter to the "-w"
argument. If the parameter is the empty string '', the file
name is generated in the form bitbake_eventlog_DATE.json,
where DATE is the current date and time, with second precision.

The "-w" option can also be supplied as the BBEVENTLOG
environment variable.

We add a script, toater-eventreplay, that reads an event
log file and loads the data into a Toaster database, creating
a build entry.

We modify the toasterui to fix minor issues with reading
events from an event log file.

Performance impact is undetectable under no-task executed builds.

(Bitbake rev: 1befb4a783bb7b7b387d4b5ee08830d9516f1ac2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toastergui: implement UI changes to allow file download

This patchset adds download links in the build analisys pages
if toaster runs in managed mode. This allows the user to access
data directly from the web interface.

[YOCTO #6837]

(Bitbake rev: 6000e1ae5c846e51932ecd0dc21e0fa02eb10357)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: add POST endpoint for uploading eventlog files

This patch adds a simple UI-less POST endpoint, where
bitbake_eventlog.json files generated by a bitbake run can be
uploaded to the running toaster instance for insertion into
the database.

(Bitbake rev: cf98a8cbc5dd7a41bbe5fdebcd81e29b1621ff35)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toaster: base Only show change project icon when > one project

In the new build button, it only makes sense to change
the selected project when there is more than one project
in the Toaster instance. If the number of projects is 1,
we hide the change project icon.

(Bitbake rev: e354a40d7dbcd85fea9d37d3983428e4470df2dd)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toaster: Initialise the 'change' icon tooltips

All 'change' icons should have a tooltip that appears
when you hover over them. The tootlip says (fittingly):
"Change". Initialise those icons in the libtoaster.js
file so that they work outside the project page.

(Bitbake rev: c635dd71670030da78c52545d2bb32b8c485c1af)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toaster: libtoaster Add a error handler to GET in makeTypehead

If the JSON data comes back from the request with an error set, have a
default handler which logs the error to the console.

(Bitbake rev: 9e3f2e2d985a575adb5520d8b517eac5ad6a15ed)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toaster: libtoaster Add editProject and getLayerDepsForProject

Add two utility functions for editing project settings and returning the
layer "dependencies" for a specified layer.

(Bitbake rev: 5f98f245bdd381a141a018a7f9953a5510fbb4df)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: toasterui: refactor project layer finding logic

This is a basic refactoring of the code computing
the layer equivalence classes for a project, in order to
bring common bits of logic in a single place.

(Bitbake rev: 9bdd0a687ba74577f2c0f0b6c696433ea662d38f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>