python: fix _json module arbitrary process memory read vulnerability

http://bugs.python.org/issue21529

Python 2 and 3 are susceptible to arbitrary process memory reading by
a user or adversary due to a bug in the _json module caused by
insufficient bounds checking.

The sole prerequisites of this attack are that the attacker is able to
control or influence the two parameters of the default scanstring
function: the string to be decoded and the index.

The bug is caused by allowing the user to supply a negative index
value. The index value is then used directly as an index to an array
in the C code; internally the address of the array and its index are
added to each other in order to yield the address of the value that is
desired. However, by supplying a negative index value and adding this
to the address of the array, the processor's register value wraps
around and the calculated value will point to a position in memory
which isn't within the bounds of the supplied string, causing the
function to access other parts of the process memory.

(From OE-Core rev: 9ec213bf67afbdfdbe25802ec86487bb22aeb2e4)

Signed-off-by: Benjamin Peterson <benjamin@python.org>

Applied to python-native recipe in order to fix the above mentioned
vulnerability.

Upstream-Status: Submitted

Signed-off-by: Daniel BORNAZ <daniel.bornaz@enea.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
shadow-securetty: add freescale lpuart

Add Freescale lpuart tty's (ttyLPx) to securetty. Freescale Vybrid
devices running upstream kernel use this driver.

(From OE-Core rev: 1732ea461f1d0afe254e36a8bfe84a6675a42e66)

Signed-off-by: Stefan Agner <stefan@agner.ch>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wget: fix build error with gnutls

This fixes '/usr/lib/libgnutls.so: error adding symbols: File in wrong format' in do_configured.

Reported-by: Jack Mitchell <ml@communistcode.co.uk>
(From OE-Core rev: 4e1649fe2fa2d512384324efcb1f9e04efeadd81)

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
autotools: Improve configure dependency code for finding m4 files

We have an open bug about the warnings issues in builds from an sstate
cache when something like glib-2.0 gets rebuilt. The issue is that
sstate is "clever" and prunes unneeded dependencies out the tree.

For example is X depends on pkgconfig-native but we've already build X
and installed it from sstate, it will not get installed when you build Y
which depends on X.

This patch changes the logic to match the sstate behaviour and prune out
unnecessary dependencies from the scope of aclocal. This in turn removes
the warning about missing manifest files.

The issue is that this patch exposes holes in our DEPENDS in recipes,
specifically that some native tools are not listed, specifically, and
problematically, pkgconfig, gtk-doc and intltool-native in particular.

I've sent out patches against OE-Core that address the bulk of the
issues there however I'm conscious this is probably going to a bug issue
in other layers and may be too annoying to consider at this point. The
other alternative is simply to turn the warning into a debug statement.

I appreciate the code below has commented blocks, this is simply debug
I've left around for now. It will be cleaned from any final version.

(From OE-Core rev: 6d2dc279faa8b28a00895dc6a620d80e2dbac685)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cogl-1.0: Add libcogl-path package

The new library needs to be put in a specific package, or rootfs
generation fails as:

,----[ Error in rootfs generation ]
| Updating cache...               ########################### [100%]
| Computing transaction...error: Can't install
|   libclutter-1.0-0-1.18.2-r0@cortexa9hf_vfp_neon_mx6: unable to
|   install provider for libcogl-path.so.20:
|      error: libcogl-path20-1.18.2-r0@cortexa9hf_vfp_neon_mx6 is
|        conflicted by libcogl20-1.18.2-r0@cortexa9hf_vfp_neon_mx6
`----

Upstream split this library out from libcogl20.so, sothe libcogl package
no longer gets renamed to libcogl20 as it contains multiple libraries.

(From OE-Core rev: 9d63188950ecd1877adf1e40de1ca1928729d866)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
alsa-lib: remove non PN based -dev packages

All dev related items should be packaged in the core
PN-dev package not in seperate packages.

(From OE-Core rev: 9aa75f06e9e78602ad8e9a5ffa312f71057a90d7)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oeqa/rutime: Added testcase decorators for automated runtime tests. Also added LogResults decorator for oeTest class in oetest.py

(From OE-Core rev: 95b83084487d0712362ade8ac487999c3274bb96)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
insane: add checking to standardize how .bbappend files do FILESEXTRAPATHS

When adding patches or config files from bbappend files, it requires
the use of FILESEXTRAPATHS, which has been an issue and failure point
for people starting to work with bitbake and oe-core.

We add checking to standardize how to use FILESEXTRAPATHS. Only the
format of:
  FILESEXTRAPATHS_append := ":${THISDIR}/Your_Files_Path" or
  FILESEXTRAPATHS_prepend := "${THISDIR}/Your_Files_Path:"
is acceptable.

[YOCTO #5412]

(From OE-Core rev: 69e083237e632f7d84a7b218dd12d1a5ad95a229)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic: --fsoptions handling

Add handling of --fsoptions in parition definition. If no options are
specified, 'defaults' is used.

(From OE-Core rev: e91c0db15db74237606fae96c9b7d21936519a86)

Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Maciek Borzecki <maciek.borzecki@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic: squashfs partition support

It is possible to instruct wic to create a squashfs partition by setting
--fstype=squashfs in *.wks. For now this is only useable for rootfs
partitions (note that you must have squashfs support in the kernel). An
attempt to create an empty partition will produce a warning.

(From OE-Core rev: 30266a0be946bd0ce76d6920e7afe840c6c3bf80)

Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lib/oeqa: Fix accidental revert of code

(From OE-Core rev: 188545ba82119d75f80dde322a73712ce1f0f762)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
insane: Use a warning for FILEEXTRAPATHS issues for now

(From OE-Core rev: f907b3a5b414aa53ce8271763f04532d7127006f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: cache: Don't reload the cache file since we already have this data in memory

If we're writing out merged data to disk, its safe to assume that either
we loaded the data or couldn't. Loading it again is relatively pointless
and time consuming.

(Bitbake rev: 14c31e18e4a4a52ee54056aeefb09ff8c295b475)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: cache: Optimise invalid cache file handling

If there is a corrupt/invalid cache file, we'd keep trying to reopen
it. This is pointless, simplify the code paths and delete the dead
file.

(Bitbake rev: c22441f7025be012ad2e62a51ccb993c3a0e16c9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake: codeparser cache improvements

It turns out the codeparser cache is the bottleneck I've been observing when running
bitbake commands, particularly as it grows. There are some things we can do about
this:

* We were processing the cache with "intern()" at save time. Its actually much
  more memory efficient to do this at creation time.

* Use hashable objects such as frozenset rather than set so that we can
  compare objects

* De-duplicate the cache objects, link duplicates to the same object saving
  memory and disk usage and improving speed

* Using custom setstate/getstate to avoid the overhead of object attribute names
  in the cache file

To make this work, a global cache was needed for the list of set objects as
this was the only way I could find to get the data in at setstate object creation
time :(.

Parsing shows a modest improvement with these changes, cache load time is
significantly better, cache save time is reduced since there is now no need
to reprocess the data and cache is much smaller.

We can drop the compress_keys() code and internSet code from the shared cache
core since its no longer used and replaced by codeparser specific pieces.

(Bitbake rev: 4aaf56bfbad4aa626be8a2f7a5f70834c3311dd3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
util-linux: break out new package util-linux-hwclock

We'd like to include the util-linux version of hwclock in images without
having to include all of util-linux. To make this possible, break out
hwclock into its own package.

(From OE-Core rev: 7c0429827d1232c8f1b439a73af6064602ec43e7)

Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
Signed-off-by: Ben Shelton <ben.shelton@ni.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
util-linux: break out new package util-linux-findfs

We'd like to include the util-linux version of findfs in images without
having to include all of util-linux. To make this possible, break out
findfs into its own package.

(From OE-Core rev: 859bdcc1691ec3fd9e88cf60b5d96f0239d72bce)

Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
Signed-off-by: Ben Shelton <ben.shelton@ni.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic: do not overwrite autogenerated /etc/fstab with original too early

DirectImageCreator.__write_fstab() generates new /etc/fstab in sysroot
with rootfs contents. The fstab entries are generated base on the
initialn contents of /etc/fstab, plus any extra (other than / or
/boot) partitions listed in *.wks. A backup of original /etc/fstab is
done in a temp location. Subsequent call to __restore_fstab() restores
the backup copy, replacing the autogenerated one.

Calling __restore_fstab() before Wic_PartData.prepare() brings back the
original fstab before the partition image file actually is created. As
such, the autogenerated /etc/fstab will not make it to the partition.

(From OE-Core rev: a9e59d7b1fa1ed33ce9678fb77a367800d7a8f87)

Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Maciek Borzecki <maciek.borzecki@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
populate_sdk_base: Include do_rootfs depends for do_populate_sdk

"bitbake core-image-minimal -c populate_sdk; rm tmp -rf; bitbake core-image-minimal -c populate_sdk"
breaks due to missing dependencies on things like update-rc.d. Since
we're effecitvely building a rootfs, we need all the same depends
as do_rootfs has.

The easiest was to achieve this is to simply load them. This resolves
the failure when targeting the task from sstate.

(From OE-Core rev: cdb0bcef28493edd4cad2a882fb8ba402485a2f1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
apt-native: Add missing DEPENDS on db-native and curl-native

The target recipe has these DEPENDS but the native version does not
and this can lead to none deterministic builds. Fix this.

(From OE-Core rev: 9a3240acd0ac64a257bcfd48f743fc85b6c8f449)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>