sched: Make scheduler aware of cpu frequency state

Capacity of a cpu (how much performance it can deliver) is partly
determined by its frequency (P) state, both current frequency as well
as max frequency it can reach.  Knowing frequency state of cpus will
help scheduler optimize various functions such as tracking every
task's cpu demand and placing tasks on various cpus.

This patch has scheduler registering for cpufreq notifications to
become aware of cpu's frequency state. Subsequent patches will make
use of derived information for various purposes, such as task's scaled
load (cpu demand) accounting and task placement.

Change-Id: I376dffa1e7f3f47d0496cd7e6ef8b5642ab79016
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: window-based load stats for tasks

Provide a metric per task that specifies how cpu bound a task is. Task
execution is monitored over several time windows and the fraction of
the window for which task was found to be executing or wanting to run
is recorded as task's demand. Windows over which task was sleeping are
ignored. We track last 5 recent windows for every task and the maximum
demand seen in any of the previous 5 windows (where task had some
activity) drives freq demand for every task.

A per-cpu metric (rq->cumulative_runnable_avg) is also provided which
is an aggregation of cpu demand of all tasks currently enqueued on it.
rq->cumulative_runnable_avg will be useful to know if cpu frequency
will need to be changed to match task demand.

Change-Id: Ib83207b9ba8683cd3304ee8a2290695c34f08fe2
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
ion: use iounmap instead of __arm_iounmap

The API iounmap() is more generic and portable
than __arm_iounmap().

Change-Id: If9601e9266bec56442686d3f2bdb1d475cd7bead
Signed-off-by: Larry Bassel <lbassel@codeaurora.org>
ion: use PAGE_KERNEL instead of pgprot_kernel

The ion code uses PAGE_KERNEL (which is more generic than
pgprot_kernel) everywhere except in one place (so fix it).

Change-Id: I38fa1c682a3e04e9c8cb3cd25678094862fe8c9c
Signed-off-by: Larry Bassel <lbassel@codeaurora.org>
ion: remove unused function

Somehow, during [83c50939e88d0 gpu: ion: optimize zero'ing of pages] we
retained some cruft. Remove it.

Change-Id: I38f2ef5d6cd1baa3fe8a758a0bf8164695e2b657
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
ion: invalidate buffers before zero'ing

Currently, we invalidate the cache for buffers being zero'd *after* we
zero them. However, this is racey since a dirty cache line from an
earlier mapping could be written out to memory between the zero'ing and
the invalidation. Fix this by invalidating the cache *before* zero'ing
the buffer.

Before this change, I was seeing corruption with:

    $ cat <<EOF | ./memory_prof -e -i -
    simple_alloc,1,ION_SYSTEM_HEAP_ID,ION_FLAG_CACHED,0x100000,1MB
    simple_basic_sanity,1
    simple_profile,1
    simple_free,1
    EOF

After this change, no corruption.

CRs-Fixed: 585524
Change-Id: I1f2603604efc9be4ebf9112b06728e0aeca60ea0
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
ion: chunk_heap: fix leak in allocated counter

buffer->size is controlled by the outer ion layer, don't modify it
inside the heap.  Instead, compute the rounded up allocated size
on demand.

Change-Id: I288ffc1221ce96cfe2591468502ac3279065bde4
Signed-off-by: Colin Cross <ccross@android.com>
Git-commit: f526a55b400d166dafc6062e5818a2410555ef39
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
cma: Allow option to use strict memblock_reserved memory

Despite all the performance optimizations, some clients are
still unable to use CMA because of the allocation latency.
Rather than make those clients use a separate set of APIs,
extend the CMA code to allow clients to keep the memory out of
the buddy allocator. Since the pages never actually go to the
buddy allocator, allocation and freeing is only based on the bitmap
allocator to find the appropriate region.

Change-Id: Ia31bb1212fd7b19280361128453c8d25369ce592
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Remove potential deadlock situation

A report was given of a deadlock situation on the cma_mutex:

mutex_lock(cma_mutex)   (*2)
dma_release_from_contiguous
ion_secure_cma_free_chunk
ion_secure_cma_shrinker
shrink_slab
try_to_free_pages
migrate_pages
alloc_contig_range
mutex_lock(cma_mutex)  (*1)
dma_alloc_from_contiguous
ion_alloc

We may need to free CMA allocations while a current CMA allocation is in
progress if CMA is freed from a shrinker. cma_mutex currently protects
two things: the bitmap indicating which pages are allocated/free and
serialization of isolation/migration calls on allocation. There is
no need to take the mutex on free calls though as the pages are
freed back into the system via the regular __free_page call. Move
the free_contig_range call outside the cma_mutex to break the
chain dependency. We can safely free the pages back into the system
before changing the bitmap without any risk of races.

Change-Id: I4989eb2891e502b08db8117a51bd86652e902778
CRs-Fixed: 619644
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Increase the default number of regions

The total number of CMA regions that can be created is statically
decided at compile time via a config option. Incoming memory map
changes increase the number of CMA regions significantly across
almost all targets. Rather than make every target set the CONFIG,
just increase the default number. The additional overhead is very
small.

Change-Id: I2f60ee56b43c256f3bfa4d1e3ab95a8f378568ab
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
ARM: dts: msm: Move audio heaps over to use reserved CMA regions

Carveout regions are being deprecated. Convert the audio heaps over
to use a reserved CMA region instead of a carveout. At a later time,
the audio regions may be converted over to a fully functional CMA region.

Change-Id: I70d9c671ff118400ee60e669062a7b2092931536
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
ARM: dts: msm: Add reservation for framebuffer memory

The framebuffer needs contiguous memory to work. Add a
dedicated CMA region for framebuffer. This region is
specified with a default size which works for most targets
and overridden for targets that have a larger screen and
therefore require a large framebuffer.

Change-Id: If5f23884f6779ed325be15ee2e16e3bd23ba3698
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
ARM: dts: msm: Update size of modem pil heap

The modem pil heap size was incorrect. Updating the
size to reflect the correct size.

Change-Id: I1ec2a907ec1e1b74e31cbe1c23817b25057237bb
Signed-off-by: Neeti Desai <neetid@codeaurora.org>