fetch/git: Add latest_versionstring method

Being able to generate a version string representing the most recent git commit
given git is useful, not least for the package reporting system.

This adds in a latest_versionstring method to the git fetcher
which allows users to query the latest version using ls-remote
and filtering the responses.

The patch also adds unittests for this function so that if
improvements are made, the original test urls can be used
to evaulate the those changes.

This is based on code from Irina Patru <irina.patru@intel.com>.

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch/wget: Add latest_versionstring method

Being able to query whether updated versions of a url are available
is useful, not least for the package reporting system. Since such code
is closely linked to the url type and the url itself, the fetcher
makes a locical place to contain this code.

For wget based urls this means taking upstream directory listings
and searching those for later versions, returning those that are
found.

The patch also adds unittests for this function so that if
improvements are made, the original test urls can be used
to evaulate the those changes.

This is based on code from Irina Patru <irina.patru@intel.com>.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
progressbar: use '/usr/bin/env' in shebangs with python

To support yocto on systems with python3 as default version, scripts
should use /usr/bin/env python in the shebang, as this allows the use of
a fake env to mimic python2 as default version.

This patch simply replaces occurrences of #!/usr/bin/python with
 #!/usr/bin/env python and was done with this oneliner:

     git grep -lE '^#!/usr/bin/python' | xargs \
         sed -i 's|/usr/bin/python|/usr/bin/env python|'

Signed-off-by: Martin Hundebøll <martin@hundeboll.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toaster script: webport option and other improvements

We add the "webport=" command line option as to allow starting
the web server on a custom port.

The bitbake server port is now auto-allocated. This is needed
to be able to run multiple toaster environments on a single machine.

We tackle bug 6023 (toaster refusing to start when lock file is present)
by using more specific checks, and automatically recover from
bitbake server down / webserver up error mode.

Command line parameters are now read on both interactive and managed
modes.

The localhost and ssh controllers are updated to work with the modified
toaster launcher script.

[YOCTO #6023]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toasterui: performance improvements

Improve the performance of data logging in toasterui.
We modify the data queries used to:

 * cache searching in memory
 * insert in bulk (i.e. multiple values per insert, where possible)

On development test rig (networked mysql), on no-op build,
time for data recording is reduced from 4:10 to 1:30 (minutes).

We also improve the logging, so it is easier to detect
toasterui errors.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: SQL query logging

We add a logger, enabled through the settings.py, that
logs all SQL queries, with timestamp and duration, to console.

This is helpful in profiling the SQL queries.

The facility is disabled by default.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: update Release model to pinpoint to specific Branch

We update the release model to pinpoint to specific branch,
instead of holding a branch name that needs to be matched.

This is needed because we move away from mixing branches on different
layer sources.

Various minor changes to models file.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
adding angular-animate plugin

We add a static copy of the Angular's animate plugin,
version 1.2.23

Licensed under MIT.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toastergui: Various fixes for projects, layers and targets page

This is a combined set of fixes for the project, layers and targets
pages in the project section of toaster.

The fixes correct behaviour and look in various parts of the page,
including submitting XHR commands and updating the DOM with the correct
info.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: Fix the table data so that it matches the column order

This fixes the issue where the column is under the wrong table header.

[YOCTO #6684]

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
toaster: Update filter count labels when Add/Remove layers

When the user has added/removed a layer but no page reload/requery has
happened we need to update the filter count labels client side.

[YOCTO #6861]

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
toaster: separate the load configuration file command

We separate the load configuration command as to allow
system administrators load toasterconf.json file as part of the
setup process.

[YOCTO #6895]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toastergui: changes for the Project page, round 3 of reviews

This patch implements the round 3 of reviews for the Project
page, including fixing the time display, fixing the build list
display, with fade-in and fade-out animations, and various
small layout fixes.

[YOCTO #6587]
[YOCTO #6731]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: Replace bootstrap js with the correct version

The bootstrap version we are working with is 2.3 not 3

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
toastergui: Silly UI fixes

Small fixes to the UI of the project.html and targets.html templates.

In project.html:

* Remove some inline styles and replace them with declarations
in default.css

* Make sure that the 'add' and 'build' buttons in the project
configuration areas have nice, rounded corners

* Add some space between the machine name and the change icon

* Remove the input-prepend class from the build form (we don't
need it)

* Apply the success class to target names in completed builds

* Bold machine and project name in the change notifications to
match all other notifications

* There is a bug in Twitter Boostrap in tooltips inside buttons, so
moving the tooltip in the 'build' button outside the button tag

In targets.html, just add a missing space between the semicolon
and the first layer name in the add layer notification.

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toaster: importing angular service

We import angular services ngRoute and
ngSanitize that helps performing user-input
sanitization in order to prevent XSS.

Both services are version 1.2.23.

License is MIT.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
toastergui: fix XSS injection points in projects page

We close XSS injection points in Projects page.

* modify the json filter to properly escape HTML tags in strings
* enable $sanitize to automatically sanitize dangerous HTML in
user-supplied input
* clean dangerous characters in targets field, as that field contents
will be directly passed to a shell command

Based on the vulnerability discovered and the patch provided by Michael Wood.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
cooker.py: fix loginfo op being set to an invalid value

loginfo "op" was being set to invalid values in saveConfigurationVar it was
working because set is a python global but append is not. This replaces both
the "op" with strings and removes the crash when calling appendConfigurationVar

Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>