mcf: Add --version option (v2.11.0)

:Release Notes:
Allow the different editions of mcf on each of the branches (and in each
the build-* repos) to be distinguished by a version number.

:Detailed Notes:
Unfortunately, the edition on some branches can't match that on master,
so we need a simple way to name distinguish them when documenting
behavior.

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-471] Add --version option to mcf

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I282f3357c6bdbe41e1e48131e9e37acc59e4c80a
Reviewed-on: https://g2g.palm.com/6146
Reviewed-by: DCO Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
Tested-by: Susan Montooth <susan.montooth@lge.com>
mcf: Change the parallelism settings defaults (v3.0.0)

:Release Notes:
Change the default values for the bitbake (-b) and make (-p)
parallelism options from 1 to 0 and change the meaning of 0 from 1x the
the number of processor cores to 2x the number of processor cores.

:Detailed Notes:
Also fix capitalization of ICECC.

:Testing Performed:
Performed testing with various use case scenarios. The results are
reported here: http://wiki.lgsvl.com/x/QYW0BQ and in [ES-469].

:QA Notes:

:Issues Addressed
[ES-469] mcf: Change the default -b and -p option values
[ES-806] Use number of CPU cores to configure build

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: If55e3d251e4f0a79730e9218fbb1ffd12eb43eea
Reviewed-on: https://g2g.palm.com/6147
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
mcf: Re-enable icecc by default

:Release Notes:
Re-enable icecc by default.

:Detailed Notes:
Reversed commits:
 mcf: disable ice by default
   commit ea8f9480b321bfaec20441541f0e25269933e855

  mcf: allow icecc toggle between disable/enable
    commit 8c7eb5de3f35e1e178fb032f24a0c28e846d2201

:Testing Performed:
Verified operation of the script per requirements.

:QA Notes:

:Issues Addressed:
[ES-952] Enable ICECC by default in MCF

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I6ab0b78e318e8bc233a8c04fb670430915b2c34f
Reviewed-on: https://g2g.palm.com/6148
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
mcf: Disable ICECC if it's not installed (v3.0.3)

:Release Notes:
Override definition of ICECC_DISABLED based on ICECC installation
status.

:Detailed Notes:
The implication of enabling ICECC on a build machine that does not have
icecc package installed, is that the make "j" option will assume the
value set for ICECC_PARALLEL_MAKE, potentially overloading the build
machine. This change prevents this from happening, by automatically
checking the installation of ICECC, and setting ICECC_DISABLED to the
appropriate value.

Override ICECC_DISABLED setting based on the installation status of the
"icecc" package on the build machine.  If it is not installed,
ICECC_DISABLED will be set to "1" and PARALLEL_MAKE will not be set to
ICECC_PARALLEL_MAKE.

:Testing Performed:
Verified that ICECC_DISABLED value is "1" when it's not installed.

:QA Notes:

:Issues Addressed:
[BHV-10148] Force ICECC_DISABLED to "1" if ICECC is not installed
[BHV-10505] CCC: Force ICECC_DISABLED to "1" if ICECC is not installed

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I59edb8f4ef8e0f02a841baebcf381dede7dfa6a0
Reviewed-on: https://g2g.palm.com/6149
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
local.conf: Convert variables set by mcf to use weak assignment

:Release Notes:
Convert local.conf variables set by mcf to use weak assignment. This
will allow them to use overridden in webos-local.conf and
webos-global.conf without having to resort to using the
_forcevariable override.

:Detailed Notes:
The one exception is MACHINE, as it shouldn't be overridden in
webos-*.conf . Note that the OPENWEBOS_GIT_REPO setting in local.conf
will be overridden by the assignment in webos-repositories.inc.

Also removed the COVERAGE_BUILD setting, as it appears nowhere in the
current metadata.

:Testing Performed:
Verified overrides occurred when variables are defined in
webos-local.conf and webos-global.conf .

:QA Notes:

:Issues Addressed:
[ES-978] Convert local.conf variables set by mcf to use weak assignment

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: Idf094fa48ce2fd8822189617c66e55c1b947c7bc
Reviewed-on: https://g2g.palm.com/6150
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Treat "*-engineering.MP" as "engr" (v3.3.18)

:Release Notes:
Explicitly recognize jobs named "*-engineering.MP*" as
engineering jobs.

:Detailed Notes:
Also changed previous similar logic for "*-official*" job names
explicitly recognize both "*-official-*" and "*-official.nonMP*"
as job type "official", but not other random "official*" names.

:Testing Performed:
Build tested (using parameterized build to simulate job name).
Examined artifacts.

:QA Notes:
No change to image. No effect on existing build jobs.

:Issues Addressed:
[ES-935] Need periodic production builds even for development branches

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I25760097694c271e1500b8c87d180fcf100eebe9
Reviewed-on: https://g2g.palm.com/6151
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh, mcf: Show version when running the script (v3.0.4, v3.3.19)

:Release Notes:
It's useful to see which mcf and build.sh version were used for build
e.g. in jenkins console.

:Detailed Notes:

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-629] Show version of mcf and build.sh in console log

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: Ibe49077c4c9cdd880bbcbf39b0816a7eae4f6eb7
Reviewed-on: https://g2g.palm.com/4771
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Show buildhistory branch and allow to change prefix (v3.3.20)

:Release Notes:
Changing BUILDHISTORY_BRANCH_PREFIX could be useful from [ES-390]. Show
buildhistory branch where we're pushing it, so it's easier for people
to find it when looking on jenkins console output.

:Detailed Notes:

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-390] Retain premirror tarballs for upstream components

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I8be4a2287c6245c0a9586da7fd190848852e07a0
Reviewed-on: https://g2g.palm.com/5204
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Rename M and MACHINES variables to MACHINE and BMACHINES (v3.3.21)

:Release Notes:
MACHINE variable is used by OE, use the same variable name so that we
don't need to export it, rename MACHINES to BMACHINES because it
conflicts with variable exported in oe-init-build-env (will be added
in v4.0.0).

:Detailed Notes:

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-371] Create new shell script to replace lengthy build execute shell
         script in Jenkins jobs
[ES-930] Improve build.sh to simplify jenkins-job.sh

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: Idd2a1d05e5c20713f5a377cdc94c82036633a693
Reviewed-on: https://g2g.palm.com/6152
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Add 4 new options for Jenkins-specific configuration (v3.3.22)

:Release Notes:
Add 4 new options for Jenkins-specific configuration. Move parsing of
JENKINS_URL, JOB_NAME to separate functions and call them only when
the options aren't passed to build.sh.

:Detailed Notes:
This makes it possible to have a separate script for Jenkins that will
always call build.sh with already prepared correct parameters for:
1) Build site; replaces detecting it from JENKINS_URL.
2) Jenkins server which triggered this job; replaces detecting it form
   JENKINS_URL.
3) Type of job we want to run; replaces detecting it from JOB_NAME.
4) Branch where to push buildhistory for repositories on Gerrit;
   replaces detecting it from JOB_NAME and JENKINS_URL. It should start
   with "refs/heads".
The script should be backwards compatible -- no need to change Jenkins
jobs until we convert them to use the generic script.

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-371] Create new shell script to replace lengthy build execute shell
         script in Jenkin jobs
[ES-930] Improve build.sh to simplify jenkins-job.sh

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: Ica4583a8252333b2ce30edb3426ef5f09afbda43
Reviewed-on: https://g2g.palm.com/5454
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Print all timestamps with the same function (v3.3.23)

:Release Notes:
Move printing of timestamps to function and add human readable format
to it.

:Detailed Notes:
The format is:
"TIME: ${SCRIPT_NAME}-${SCRIPT_VERSION} <description>: \
  ${BUILD_TIMESTAMP}, +${BUILD_TIMEDIFF}, +${BUILD_TIMEDIFF_START}, \
  ${BUILD_TIMESTAMPH}"
e.g.:
TIME: build.sh-4.0.4 start: 1398617883, +0, +0, 20140427T18:58:03Z
Where first +number says how many seconds passed since previous
timestamp, second +number says now many seconds passed since the
beginning of build.sh script.

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-371] Create new shell script to replace lengthy build execute shell
         script in Jenkin jobs
[ES-930] Improve build.sh to simplify jenkins-job.sh

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I61a9c291b12936752649b029713dc5cc2d702292
Reviewed-on: https://g2g.palm.com/5455
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Archive few more artifacts with vmdk builds (v3.3.24)

:Release Notes:
Preserve 5 more files for developers to be able to create different
vmdk file with the same rootfs.

:Detailed Notes:

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-371] Create new shell script to replace lengthy build execute shell
         script in Jenkin jobs
[ES-930] Improve build.sh to simplify jenkins-job.sh

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: Ib48dd98a70e794e64ccd2ec78ad1d6f5b8f7ca6c
Reviewed-on: https://g2g.palm.com/5477
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
build.sh: Remove long ignored options (v3.3.25)

:Release Notes:
These are ignored for very long time, all scripts were updated already.

:Detailed Notes:
Improve error message when unrecognized option is used, getopts should
handle it automatically, but in some cases it doesn't work correctly.

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-371] Create new shell script to replace lengthy build execute shell
         script in Jenkin jobs
[ES-930] Improve build.sh to simplify jenkins-job.sh

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I9270798eb5f2bc8ca47d9cd6426bc04f30078e73
Reviewed-on: https://g2g.palm.com/5894
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
mcf: Process remaining 2 files and get rid of autoconf (v4.0.0)

:Release Notes:
Some replacements were already done in mcf, some were handled in
configure. All options needed to be duplicated in mcf and configure.ac
both with different syntax and capabilities for parsing e.g. multiple
space separated options.

:Detailed Notes:
Simplify whole script by parsing and replacing all @tags@ from one
place.

:Testing Performed:
Only build tested (generated build directory with the same options
before and after this change and created files were identical).

:QA Notes:
No change to image.

:Issues Addressed:
[ES-628] Remove autoconf from mcf

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I69696d26c2d7d1b02cede75fb458565da04ea071
Reviewed-on: https://g2g.palm.com/4770
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
mcf: Use logger instead of calling print directly (v4.0.1)

:Release Notes:
It doesn't make much sense to combine logger and print.

:Detailed Notes:
Use %s syntax instead of explicitly calling format() like whole
bitbake/OE codebase does for consistency.

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-629] Show version of mcf and build.sh in console log

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: If1a36be9dbc7d736579712b2ecbdc53516e033ae
Reviewed-on: https://g2g.palm.com/4772
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>
mcf: Change default command to update as well as configure (v4.0.2)

:Release Notes:
In most cases the expected behavior is that one call to mcf will check
out all required layers in right versions and configure the build
directory.

Current behavior was that default configure command was ignoring
existing checkouts and update command wasn't able to checkout new
layer when it was added to weboslayers.py. So in order to use it from
jenkins we would need to call mcf twice, first to configure and possibly
checkout new layers and then again to update existing layers.

Add new default command "all" which does both and simplify the script.
Old behavior which just configures the build directory without changing
the layers is available with mcf --command configure <params>.

:Detailed Notes:
Unify downloadprojects and updatelayers functions. They do almost the
same job, only difference is that downloadprojects skips existing
checkouts and updatelayers updates it based on configuration in
weboslayers.py.

Move build directory configuration to separate function, test
options.bb_number_threads, options.parallel_make only once when they
aren't equal to 0.

Move printing update summary to update_layers().

:Testing Performed:
Only build tested.

:QA Notes:
No change to image.

:Issues Addressed:
[ES-447] mcf update should be able to checkout newly added layer

Open-webOS-DCO-1.0-Signed-off-by: Martin Jansa <martin.jansa@lge.com>
Change-Id: I1c6c24bb5397180530a1389f92e712c995fda2d9
Reviewed-on: https://g2g.palm.com/4996
Reviewed-by: DCO Verification
Build: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Martin Jansa <Martin.Jansa@lge.com>
Tested-by: Martin Jansa <Martin.Jansa@lge.com>
Reviewed-by: Build Verification
Reviewed-by: Susan Montooth <susan.montooth@lge.com>