toasterui: refactor log saving and save out-of-build errors

We refactor log saving to go through only one code path.

All logs that happened outside the build (i.e. before build
starting) now will be logged to either toaster_ui.log if the
build command ran in interactive mode, or to the build request
errors if the command ran in managed mode.

This enables proper display of error logs in project page.
toaster: fix application discovery in settings.py

We fix application discovery, tuples don't have .append().

Other minor fixes.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: update orm models for layerindex compatibility

We add a ToasterSettings table that will keep
installation-wide settings.

We update the models for the layer-related data storage to
make them compatible with the layerindex application API.

We add a LayerSource class that can update local data from
a LayerIndex-like compatible API.

Adding a command line option to perform information update
from all upstream layer sources.

Fair warning - there is no backward migration from 0013.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: enable admin interface on select models

We enable the admin interface in Toaster.

We add admin models for Build Environments (where the
sysadmin can configure where the builds take place)
and for Layer Sources (marking the upstream provider
for layer information).

The admin interface and associated data are enabled
only for the MANAGED version.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: create project section navigation structure

We create the navigation structure for the project section.
This includes adding URLs for configuration, builds, all layers,
layer details and all targets pages.

Changes to existing pages to exemplify navigation links.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: add all layers page

We add a page where the user can browse
and import layers from all the layers known
by Toaster.

[YOCTO #6590]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: update checksettings command for auto-detection

We enhance the checksettings command to try to automatically
detect settings for running on localhost.

The checksettings will look for a nearby poky layer source,
for a nearby build directory, and will try to import settings
from "toasterconf.json" files found in the local layer.

On new configuration, it will also perform updates from the
layer source.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: update the bldcontrol to the new orm models

We update the build controller application to make proper
use of the bitbake specification in project settings.

Added heuristic to detect when the meta* layers and bitbake
are checked out from Yocto Project poky, and use a single
git checkout.

Building without a proper oe-init-build-env is not yet supported.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toatergui:  update pages to match project models

We update pages and queries to match the new project
models.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
buildinfohelper: BuildRequest project file update soft linked

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: create Build methods for calculating progress and ETA

We move the code to calculate build progress as percent
and the ETA of the build to the model, so that they can be
reused across different pages.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: add project pages for machines, targets, layers

We add new pages for the all-machines and all-targets
project-related views.

We update the existing template structure to create
a base project view, similar to a base build view, that includes
a breadcrumb.

Updating existing all layers view to use the new structure.

We update methods in the models to provide corrent
information display.

[YOCTO #6592]
[YOCTO #6593]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toastergui: added pages for project details

We add new pages for the layer importing, layer details,
showing project builds and project configuration.

The pages are  in read-only mode, but they're needed as
to be able to verify the quality of data in the system.

Write capabilities will be added in a subsequent patch.

[YOCTO #6595]
[YOCTO #6590]
[YOCTO #6591]
[YOCTO #6588]
[YOCTO #6589]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty: ignore interrupted system calls

With the improved exception handling added in an earlier commit bitbake
now stops when recieving a SIGWINCH. This happens frequently when
disconnecting and reconnecting tmux sessions and bitbake didn't survive.
Restore old behaviour of ignoring interrupted system calls but keep
proper exception handling for other errors.

Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
world-image: add extra target

* similar to world target, but collects all PACKAGES
* recipes marked with EXCLUDE_FROM_WORLD are excluded from world as well
  as world-image
* recipes marked with EXCLUDE_FROM_WORLD_IMAGE are excluded completely
  from world-image, sometimes you can build 2 recipes in same sysroot,
  but cannot install them in the same image
* PACKAGES listed in EXCLUDE_PACKAGES_FROM_WORLD_IMAGE are also
  excluded. You can remove e.g. ${PN}-dbg, but keep all other PACKAGES.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>