cpu: enhance power efficiency

Here we are trying to reduce power usage through the
use of macros cpu_relaxed_read and relaxed version of
idle_cpu().

Bug 1440421

Change-Id: I0a8c5d358c154782b41570059d03d7e0de87e82c
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/426485
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: relaxed.h: defined relaxed.h

Defined a new header file relaxed.h, which contains basic
macros which will be used for improving power efficiency for
arm64.

bug 1440421

Change-Id: I5ae7503afdfbaa951827bbf466d8ddccf444f558
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398315
(cherry picked from commit dd434aeb1afea1d9ebce1099fb6ecfa7c6c762c6)
Reviewed-on: http://git-master/r/422203
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
llist: defining relaxed version of llist_empty

Defining relaxed version of llist_empty as llist_empty_relaxed,
which will be used for power-optimization.

bug 1440421

Change-Id: I1c4c34b381e49775ed08ddd606d9744a7e7e1fba
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/426483
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
seqlock: enhance power efficiency

Trying to improve the power efficiency in linux/seqlock.h,
using macros cpu_relaxed_read and cpu_read_relax.

Bug 1440421

Change-Id: I8dcc9dc9c72e5a6848c7823768c76f5d38e9f0d1
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398943
(cherry picked from commit ed77ee4d50641b1a5668545c946cda681c06fcd2)
Reviewed-on: http://git-master/r/422258
GVS: Gerrit_Virtual_Submit
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
hrtimer: enhance power efficiency

Defining relaxed version of hrtimer_callback_running(),
which will be used to improve power efficiency through
the use of macro cpu_relaxed_read_long.

Bug 1440421

Change-Id: Ie42d7ae9628a817d52f4636781e11b607327c2c5
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398789
(cherry picked from commit 0d9f5fc1d39d7d1809519b5d11bf7ac72287b7c6)
Reviewed-on: http://git-master/r/422255
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: processor.h: adding new header file

Defining a new header file and adding architecture
independent macros. Using these macros we are optimizing
power usage on ARM64.

bug 1440421

Change-Id: I7393d35703d0b0a504331653d70f109a50a197c7
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/381905
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: processor.h: remove redundant macros

Removing cpu_relaxed_read and cpu_relaxed_read_long macros from
processor.h, as these macros are defined in asm-generic/relaxed.h.

Bug 1440421

Change-Id: I5d1ba25755e1c9d33b080dfe01ba838289f306af
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/400093
(cherry picked from commit 57eb21e2d4cad3ce1f85283cfffd0eff85a6d17d)
Reviewed-on: http://git-master/r/422209
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: atomic.h: relaxed atomic_read

Defining relaxed version of atomic read as cpu_relaxed_read_atomic
which will be used for improving power efficiency for arm64.

bug 1440421

Change-Id: I6ac26653ec3d62f74d8c21f250dcdaf9dfb75b9b
Signed-off-by: Alex Van Brunt <avanbrunt@nvidia.com>
(cherry picked from commit 7360c3df73afa07361eecab730903e0697d3408f)
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/415628
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
arm: including asm-generic/processor.h

Here we are including asm-generic/processor.h
because it has defined some generic macros which
can be used by any architecture for optimaztion
of power usage.

bug 1440421

Change-Id: I39db944d348b3dad9235b76d8c7ea3867c0006ce
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/368823
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Reviewed-by: Peng Du <pdu@nvidia.com>
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sched: Trying to reduce power usage

Using macros cpu_relaxed_read, cpu_relaxed_read_long and
cpu_read_relax in core.c to improve power efficiency.

Bug 1440421

Change-Id: Ic49b88fa6fb0cb847fed49886b2865bed72203db
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/368841
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Tested-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
kernel/smp.c: use '|=' for csd_lock

csd_lock() uses assignment to data->flags rather than |=.  That is not
buggy at present because only one bit (CSD_FLAG_LOCK) is defined in
call_single_data.flags.

But it will become buggy if we later add another flag, so fix it now.

Signed-off-by: liguang <lig.fnst@cn.fujitsu.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
kernel/smp.c: cleanups

We sometimes use "struct call_single_data *data" and sometimes "struct
call_single_data *csd".  Use "csd" consistently.

We sometimes use "struct call_function_data *data" and sometimes "struct
call_function_data *cfd".  Use "cfd" consistently.

Also, avoid some 80-col layout tricks.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Shaohua Li <shli@fusionio.com>
Cc: Shaohua Li <shli@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sched: Force sleep on consecutive sched_yields

If a task sched_yields to itself continuously, force the task to
sleep in sched_yield. This will lower the CPU load of this task
thereby lowering the cpu frequency and improving power.

Added a stat variable to track how many times we sleep due these
consecutive sched_yields. Also added sysctl knobs to control the number
of consecutive sched_yields before which the sleep kicks in and the
duration fo the sleep in us.

Bug 1424617

Change-Id: Ie92412b8b900365816e17237fcbd0aac6e9c94ce
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/358455
Reviewed-by: Wen Yi <wyi@nvidia.com>
Reviewed-by: Peter Zu <pzu@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: cleanup undefined instruction entry code

We don't need to keep reloading the thread into into r10 - we can do
this once and keep the value cached in the register.  Also, schedule
some instructions better so that the pipeline doesn't stall after a
load in the neon code.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: mm: Remove unnecessary CMO in Cortex A15 startup

Cortex-A15 flush L2 cache after reset, there is no need to do this
in software, if L2 is already invalidated in bootloader and
cache is disabled. For secondary startup, there is no reason to
flush L2 as well.

This change assumes the setup code is always entered as the result
of CPU reset.

Change-Id: I6d58f8b4a638b70acfb35b97c87a09266aceef41
Signed-off-by: Bo Yan <byan@nvidia.com>
Reviewed-on: http://git-master/r/170563
Reviewed-by: Mrutyunjay Sawant <msawant@nvidia.com>
Tested-by: Mrutyunjay Sawant <msawant@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: mm: Skip I-cache invalidate for Cortex-A15 boot

This is not required since cache is invalidated by HW in the reset
sequence. Bootloader is supposed to do the same before it hands
over control to kernel.

Change-Id: I0991de3ba1015a32f2c49a0333fd0b17a51a4f31
Signed-off-by: Bo Yan <byan@nvidia.com>
Reviewed-on: http://git-master/r/197028
(cherry picked from commit 9c49ffd08d6da03caa820711db83f561d9333aee)
Reviewed-on: http://git-master/r/200854
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: remove unnecessary dcache_clean_area

Cortex-A9 has PIPT D-cache which do not require clean the cache
on creating page table.

Original-Change-Id: I42d528be83ea8def96045c7e575c7b3ed95f5980
Reviewed-on: http://git-master/r/40505
Reviewed-by: Heechul Yun <hyun@nvidia.com>
Tested-by: Heechul Yun <hyun@nvidia.com>
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
Reviewed-by: Scott Williams <scwilliams@nvidia.com>

Rebase-Id: R214ce4b138abb18e1cfe79465087c39ead248d72
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: LPAE: Invalidate the TLB for module addresses during translation fault

During the free_pgtables() call all user and modules/pkmap entries are
removed. If a translation fault for the modules/pkmap area occurs before
we switched away from the current pgd, do_translation_fault() would copy
the init_mm pud into the user pud.

There is a small window between pud_clear() and pmd_free_tlb() in
free_pmd_range() where the pud entry was cleared but the TLB has not
been invalidated yet and the CPU may have cached the original (valid)
pud entry in the TLB. A scenario like below would get stuck in
continuous prefetch abort:

1. Current process exiting. The modules pmd entries not populated
2. exit_mmap() -> ... -> pmd_free_tlb()
3. pud_clear() for the 1GB pud containing user stack and modules (no TLB
   invalidation yet)
4. Interrupt -> module interrupt routine
5. Level 2 (pmd) translation fault occurs when executing the module
   interrupt routine. The CPU previously cached (TLB) the old valid pud
   value for the modules area, so we don't get a level 1 translation
   fault
6. do_translation fault() copies the pud_k into the pud
7. Linux returns to the faulty instruction. Goes back to 5

At point 7, since the CPU still has the old pud value, it goes back to
point 5 and never gets out of this loop. With this patch, the stale pud
TLB entry is invalidated after point 6 and the subsequent prefetch abort
doesn't occur.

Reported-by: Tony Thompson <Anthony.Thompson@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit bb2c1f205618083611d044a461e019c26468a231)

Rebase-Id: R9afbe63e26001dc849f8c55d5f34d6f5e70cfc47
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: mm: cache-l2x0: Implement outer_clean_all()

There is already implemented full outer clean routine in
arch/arm/mm/cache-l2x0.c.

Make it possible to use it through outer_cache interface,
like other outer maintenance functions.

bug 983964

Change-Id: I47f1fad536c151c255e6a42d6517114c334ddfef
Reviewed-on: http://git-master/r/116074
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Kirill Artamonov <kartamonov@nvidia.com>
Tested-by: Kirill Artamonov <kartamonov@nvidia.com>
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
Reviewed-by: Justin Paver <jpaver@nvidia.com>
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>

Rebase-Id: Rf769958fea9fd10058a4805eeafe08f5f0e895dd
Signed-off-by: Dan Trifan <jimsth@gmail.com>

Conflicts:
	arch/arm/mm/cache-l2x0.c

Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: dma: Drop GFP_COMP for DMA-IOMMU memory allocations

dma_alloc_coherent wants to split pages after allocation in order to
reduce the memory footprint. This does not work well with GFP_COMP
pages, so drop this flag before allocation.

This patch is ported from arch/avr32
(commit 3611553ef985ef7c5863c8a94641738addd04cff).

Change-Id: I67eb2b15807c36c6a3ade95ee35bf5856d8e0e11
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>

Rebase-Id: Rf13a569ebe836d341cf4d9287ff4b4711dde85cc
Signed-off-by: Dan Trifan <jimsth@gmail.com>