build: Preliminary support for CM product builds

If building a cm_* product, skip crawling the filesystem for
AndroidProduct makefiles (and, of course, parsing their contents)
and aim directly for the device/cm/<product> device configuration

Change-Id: I2a5e70dda973a6fcdbba0d5e26b35b99d3f1aea2

Change how we search for CM makefiles.

build: Fix fastpath code for CM_BUILD

 * We need to adjust to the new validations

Change-Id: I9066d4f437beb9597027ee6bbb52504b5e7e84c5

build: Fix cm builds

the setup variable wasn't exported, and was being missed by part
of the scripts
CM->PLAIN
Squashed commit of breakfast/brunch commands

Add a CM-specific breakfast menu ported for Plain-Andy

We usually don't wan't all the devices in the lunch menu, so
we can now choose to pick up breakfast or brunch and get a
smaller selection of devices (i.e., those we support)

Make brunch more nutritious

* "breakfast device-name" will set up the environment, just like lunch.
* "brunch device-name" will actually do the whole setup and build :)

brunch: add a lazier mode

Since all of CM's targets have a predictable PRODUCT_NAME and BUILD_VARIANT
(i.e., "cyanogen_$(PRODUCT_DEVICE)-eng"), brunch can now be invoked passing
just the PRODUCT_DEVICE ("brunch passion" for the N1 setup+build, for
example).
The "classic" long form of $(PRODUCT_NAME)-$(BUILD_VARIANT) can still be used
if desired ("brunch cyanogen_passion-eng" still works).

breakfast in bed

breakfast in bed (aliased to "bib") is the baconless friend of lazy
brunch.
You can now use "breakfast passion", or "bib passion", to setup the
corresponding CM build without actually building it

Fix breakfast for the undecided

The menu presentation (i.e., giving no arguments to breakfast
or brunch) was broken, so if you didn't know the name of your
target you weren't given a list to choose from, just an error.

Change-Id: I85b8217daa2d511c16a1e82c700e1b0fadbb36ac
CyanogenNom: Now moar helpful!
roomservice delivers you lunch combos from the CyanogenMod github.

fix roomservice formatting

support product names with _ in them

fix roomservice to handle pagination

**Modified for use with Plain-Andy

Change-Id: I4923c2f768094dbad8e06a72d9f27d46414030ab
roomservice: Add lightweight dependencies to repositories

Modified for use with Plain-Andy

Roomservice can already fetch your cm_<device> without the need for a
manifest entry.

However, when working with common repositories, there is no way of
actually fetching them without adding to the manifest. This patch
introduces a lightweight dependency system. Each repository can have a
cm.dependencies in the following json format:

[
  {
    "repository": "repository_name_on_cm_organization"
    "target_path": "target/path"
  },
  ...
]

For instance, for cm_anzu I need android_device_semc_msm7x30-common and
android_device_semc_mogami-common. I would add both to cm.dependencies
as follows:

[
  {
    "repository": "android_device_semc_msm7x30-common",
    "target_path": "device/semc/msm7x30-common"
  },
  {
    "repository": "android_device_semc_mogami-common",
    "target_path": "device/semc/mogami-common"
  }
]

Roomservice would then fetch the anzu repository, parse the dependency
files and add/fetch/sync these additional repositories if they don't
exist already.

This also adds pretty printing to the output xml.

Change-Id: I9cc847adfc717a06439bc6094213ed6492343158

roomservice: Add branch support to cm.dependencies

Allow the cm.dependencies entries to provide an optional "branch" for
the repository dependencies. Added to fully support
http://wiki.cyanogenmod.com/wiki/Integrated_kernel_building

Change-Id: I35b51920d296afa329411af6172c7bd9aeef4af8

roomservice: Fill in dependencies for already-deployed repositories

Change-Id: I01fd408c9c4bfa78097c7f848b2556d2b2b180f3

roomservice: Extend dependency-checks to devices in main manifest

CM currently keeps devices inherited from AOSP in the main manifest,
so take that into account as well when checking device paths

Change-Id: I9663f283617f237428b4eaa0cd60b5de2b86a7b9

make compatible with github v3 api

Change-Id: Iff6f1f9099cdc5d2b49e04000b5fe3d04aa5d7e4

Fixed build for full-eng

Previously
Traceback (most recent call last):
  File "build/tools/roomservice.py", line 153, in <module>
    repo_path = get_from_manifest(device)
NameError: name 'device' is not defined

** Don't have a product spec for: 'full'
** Do you have the right repo manifest?

Now
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=4.0.4
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=IMM76L
============================================

Change-Id: Ib513705aba9a7a52a971ab64102ecbe9fddfb97a

roomservice: Bump github request per_page to 100

Change the number of repos per page from the default 30
to 100.

We seem to be hitting the rate limit on the jenkins server.

Change-Id: Ie733feaa0414cbfebb7efcfc1e24d94e1e466d1b

roomservice: Add support for netrc

Change-Id: I1f5e11e40125abd0c4e4c8d8294d4fc09bfdc30a

roomservice: Handle missing netrc file

Change-Id: If981fe79dc3e2191434301239b0cd585be8b4730

roomservice.py: Verbose error when a branch is missing.
Also add ROOMSERVICE_BRANCHES environment variable to use fallback branches.

Change-Id: I3c2b1d79fc185c1f1e1d658e5ca4f78e688780e2

roomservice.py: Fixups around fallback branches not being used by dependencies when ROOMSERVICE_BRANCHES is defined.

Change-Id: Ifb42a023cae5f62ac8f9cf7832125b91b431169c

roomservice: Allow following up tag references

This is now needed for release builds

Change-Id: I8c5f87341059b3b15ee853312b71df73790ad0d8

build: local_manifest.xml deprecated, use local_manifests

Patch Set 2:- Use roomservice.xml instead of cm.xml.

Change-Id: I3d8a6ef3907b92808662cbba912cea5ed38d0bde

Fix fallback branch search in roomservice

If you provided a fallback branch to roomservice via the
ROOMSERVICE_BRANCHES environment var the branch search would fail
if the device repo had any tags.

Fixed this by appending the tag search results to the branch
search results instead of overwriting them

Change-Id: I73a11af1500bd04e346f08ec3f83454502f3a169

roomservice: Fix wrong assumption about path of active manifest

Change-Id: Id740ff4b848e6ccbfd658be4846197b8ca519237

roomservice: When validating the presence of a repo, check main manifest too

Change-Id: If680536484074b473458723d93e783d074d7f669

roomservice: Bump devices per page to 200

Limit was reached again when attempting to
lunch various projects, such as steelhead.

Change-Id: I2f3b9705e07e1e47b86857aeb383cf7c99fcdbdc
Signed-off-by: William Roberts <bill.c.roberts@gmail.com>

roomservice: Fix assumptions about the branch naming

We can't just split from the last slash anymore, since we're using
those to distinguish the stabilization branches

Change-Id: Ia175dd317f508e99b275b56e9c83bd4729a75ddb

roomservice: Add recursive dependencies

Dependency repositories can now have dependencies themselves

Change-Id: I33a28709170da52bc98f4a62387927e3a11b2450
roomservice: python3 support

Change-Id: I7621818ba7ed997676728fe865f37a25b3a5b8b5
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
make bacon and custom releasetool (for squished zips)

Change-Id: I4f61b6a6b9b8bab8582fa5423c4134c07af17606
Add otapackage support for backuptool

Change-Id: I512554c579d444067cd3ccbb0e6946a5eb6bc964a
build: Inline kernel building as a buildtime task

Kernel image integration is now done in the build system. The
"one true way" of doing this is to download the kernel source
into kernel/TARGET_BOOTLOADER_NAME (by usage of the cm.dependencies
mechanism or otherwise), and defining the TARGET_KERNEL_CONFIG
variable in the device's BoardConfig makefile

If the kernel's location doesn't match the automagic location
(multi-device kernel source, for instance), TARGET_KERNEL_SOURCE
can be used to specify a kernel path (i.e.,
"TARGET_KERNEL_SOURCE := kernel/shared-whatever")

If the device requires out-of-kernel-tree modules to be built,
the TARGET_KERNEL_MODULES variable can be used, pointing to a
_make target_ that will build and install such modules. Definition
of such a target is the device author's responsibility, the only
restriction is that it is a normal makefile recipe (example
below)

Optionally (or as an alternative), a prebuilt binary can also be
defined at BoardConfig, by usage of the TARGET_PREBUILT_KERNEL
variable. This binary will be used if the kernel source is absent
(or undefined).

A minimal BoardConfig.mk should look something like this:

TARGET_KERNEL_CONFIG := cyanogenmod_<device>_defconfig
TARGET_PREBUILT_KERNEL := device/<vendor>/<device>/kernel

To include, for example, the TI WLAN modules, this can be used:

-----------------------------
TIWLAN_MODULES:
       make -C hardware/ti/wlan/wl1283/platforms/os/linux/ KERNEL_DIR=$(KERNEL_OUT) ARCH="arm" CROSS_COMPILE="arm-eabi-" TNETW=1273 RANDOM_MAC=n REPORT_LOG=n
       mv hardware/ti/wlan/wl1283/platforms/os/linux/tiwlan_drv.ko $(KERNEL_MODULES_OUT)
       make -C hardware/ti/wlan/wl1283_softAP/platforms/os/linux/ KERNEL_DIR=$(KERNEL_OUT) ARCH="arm" CROSS_COMPILE="arm-eabi-" TNETW=1273 REPORT_LOG=n
       mv hardware/ti/wlan/wl1283_softAP/platforms/os/linux/tiap_drv.ko $(KERNEL_MODULES_OUT)

TARGET_KERNEL_MODULES := TIWLAN_MODULES
---------------------------

Change-Id: I8634fa4c788a42dc6f62e62ca170825b66db126a

build: Fix kernel module building on Darwin/OSX

Darwin/OSX build host does not have module-init-tools which is
required to generated modules.dep file. Switch to modules.order
file marker which is always generated instead.

Change-Id: I20c0fccd905fa668202c3e7284a8778db3728a65

kernel: Change path to kernel/<vendor>/<device>

Derive the kernel path from the device's own path (just replace "device"
with "kernel")

Change-Id: Idd44a0489e1ce280adf5ec4d9cfe2385c75dd115

kernel: Improve support for non-arm architectures

Remove hardwired references to the ARM arch and toolchains, and replace
them with the respective android build variables

Change-Id: Iae3eb548ca1d58ac808b5fa430d415283a809106

kernel: Fix ARM building

The default android ARM compiler, arm-androideabi-, does NOT work
for the Linux kernel. Special-case the ARM architecture to target
the ARM_EABI_TOOLCHAIN path directly

Change-Id: Ib672c99f114cb89d5fda3343d4dc68810d042d35

kernel: allow TARGET_KERNEL_MODULES to overwrite kernel modules

This is necessary to use compat-wireless since it needs to build a newer
version of cfg80211 (and sometimes mac80211) than the kernel sources version.
There are probably other instances of this type of need.

Change-Id: Ib5bf818286bc20987d8b9f9480a43f3e7690e239

kernel.mk: make use of ccache when requested

Change-Id: I9b6e28711bd5f590a76ac2b62a50b1d2de014e3e

kernel: Fix ccache inclusion logic

Builds were broken when CCACHE was missing from the environment
Change-Id: Ie8d6048f4600f1dc9c298593a50e37af04b96438

build: show accurate information on inline kernel warnings/errors

Having the variables on AndroidBoard as suggested causes errors
with mm/mmm, whereas having them on BoardConfig doesn't. Adjust
the warnings to reflect this.

Change-Id: I554c1f1073df678d36521f73bc236a1f4b02212e

This is causing generic_armv5-userdebug builds to fail. Commenting out for now to fix recovery build servers. Will fix properly later.

Change-Id: Ibe1cda8cd2b4c1914dfa3b8a29724c9069e047a6

kernel.mk: Also search PRODUCT_COPY_FILES for the kernel copy, as that is how AOSP does it now.

Change-Id: Id2d1cf079694d1996d4a85d8435c2e4562e5d444

kernel.mk: fix compiled kernel copy

Change-Id: Ifb2a3d4968e56eed236eaa2db9258cd0b8865fda

kernel.mk: workaround to fix kernel builds on darwin until the prebuilts/gcc is checked in by upstream.

Change-Id: I6321fb7f6814b207c821d974766d945351b3f546

build: fix ccache usage when building kernels

Change-Id: Id4edd4d85d9ba3ef42575f5fdebf22ed14957a99

kernel.mk: set KERNEL_OUT properly

It was hardcoded to $(ANDROID_BUILD_TOP)/$(TARGET_OUT_INTERMEDIATES)/KERNEL_OBJ
which wouldn't exist if OUT_DIR_COMMON_BASE was set to use another dir
(e.g. ramdisk)

Fix it so that it now points to $(TARGET_OUT_INTERMEDIATES)/KERNEL_OBJ
if OUT_DIR is not out

Change-Id: I1bf767d86548e41270d9cbb8f0c00512708501c5

build: add support for uncompressed kernels

build: kernel: remove hard coded darwin toolchain.

* Use latest kernel toolchain from $(ARM_EABI_TOOLCHAIN) variable.

Change-Id: I3b43408937dd5f193fcba19c034f868272de8963

envsetup: set OUT_DIR to an absolute path always

OUT_DIR was set to $(TOPDIR)out previously,
but $(TOPDIR) was null, so it was a relative path.
This broke releasetools, inline kernel building, etc
since they require absolute paths.
Fix it so that it is set to $(shell readlink -f .)/out
if $(TOPDIR) is null.

Also remove hacks which checked if (OUT_DIR) was out
and changed it to $(ANDROID_BUILD_TOP)/out to workaround
the aforementioned problem.

Change-Id: I459a3b1325a1bbea0565cd73f6acf160d4ed9b39

build: add strip on kernel modules

kernel modules are huge for prima wlan and we need to strip them
unstriped size 40mb
striped size 2mb

Change-Id: Iefd572732cad0a6f608439618673068a3586fcd5

kernel: Ignore errors with module building

Let kernel fully disable loadable modules

Change-Id: Ia37ec927b092c041ee4c68bf9fd0f28b7339c8ca

build: Add support for extra kernel build variables

 * This allows specifying a variant defconfig, and an selinux defconfig
   which are simply configuration fragments.

Change-Id: I97882ae3b8c2e16ff6a7dce8dd3a70d70f8aa866

s/cyanogenmod.com/cyanogenmod.org/

* And fixup a wiki link while I'm at it.

Change-Id: I0355b9a47eac1becc07e81659fbb2d11b14ece36

Fixes for Xcode 5 and OSX 10.9.
kernel.mk: Put elf.h into the include path on Darwin builds.

Change-Id: I7069b956965d27caac3b2e4c3cc2e8b4c1da7a82

Make the kernel image format parametric instead of a chain of if/elses

Change-Id: I54bfcdecb8647f7bcf744e72b2de19fcf4e4e7ac

build: Add "dtbs" target when building the kernel

 * This is needed for 3.10

Change-Id: I4044ea9e67017452efc25097a3327141a6627c24
build: Run squisher for the bacon target

Change-Id: If33402a5eb1661345c7d37ec1778742096449c70

TARGET_CUSTOM_RELEASETOOL is a squisher replacement

Change-Id: I9c7126d6bbe0303c1d35bb54ec2f0c9187b131e0

Fix tabs, sorry bad commit --amend :p

Change-Id: Ia981edb520d7b5f91e0717f6b4d8180386c8a20d

Conflicts:
	core/Makefile
build in colors: Install outputs in blue/yellow [basic ics version]

"target prefix:" in yellow
"Install: file" in cyan

should be in a single command line else there are sync problems
in multithread (-j X)

These colors can be tuned in core/Makefile if you use a white terminal.
can be disabled with "export BUILD_WITH_COLORS=0"

Only the most important output is colored to reduce ics merge problems

Change-Id: I0e585079fde6900799ef209367a523d94a51cda5

Colors: add more colors to final build steps

also fix releasetool echo and visible recovery echos

Change-Id: Icf5d88468572f935610c544bf1d5d356ec9870d3

build in colors: host C/C++

Change-Id: Ic415cab53a2efa104c9d4b31ddbe8c8eb74e493d

Conflicts:
	core/Makefile
build-with-colors: moar colors

- rename CL_PFX and CL_INS to better match the
  colors they show.
- add more colors.

Output of:
- most host stuff is yellow
- most target stuff is green
- installing/copying files is cyan
- import/export includes and notice files is cyan
- bootimage/recoveryimage is cyan
- and some more colors in many places ;)

Change-Id: I5532afa4ba608e0a7c408516dc9f912f9ca389f7

Conflicts:
	core/Makefile
build: Pass OTAPACKAGE location to squisher

Pass the OTAPACKAGE location so we don't have to guess what
the package name will be. This fixes an issue when
BUILD_NUMBER is defined in an environment with automated
builds.

Change-Id: Iab18b541de37c99d237aabc3cb7b2e9ab076617b

Conflicts:
	core/Makefile
update ota_from_target_files to handle mounting/unmounting for backupscript

backupscript should not be mounting/unmounting itself as it makes other
scripts have unexpected results (such as modelid_cfg, which expects /system
to be mounted)

instead have the ota script handle the mounting/unmounting

Change-Id: I94511f4147c624d975cb3ecbeaa8b0e98f63437c
Further plain bringup
Remove vendor checks and java 7 check