arch: arm: lib: optimize memcpy for cortex-A15

The LDRD/STRD instruction is faster than LDM/STM on Cortex-A15.
Also optimized preload cache size for Cortex-A15.

Added USE_LDRDSTRD_OVER_LDMSTM to turn on LDRD/STRD optimization.
Added ARM_PLD_SIZE, default 32. Should set to 64 for Cortex-A15.

Bug 1185248

Change-Id: I4fa8c25bcd9b7823a11018817a4d17e3357ae681
Signed-off-by: Chandler Zhang <chazhang@nvidia.com>
Reviewed-on: http://git-master/r/211599
GVS: Gerrit_Virtual_Submit
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
Apply arch: arm: lib: optimize memcpy for cortex-A15 to memutils

Signed-off-by: Dan Trifan <jimsth@gmail.com>
defconfig: enable nVidia A15 memcpy optimizations

Signed-off-by: Dan Trifan <jimsth@gmail.com>
arm: atomic.h: defined relaxed atomic_read

Defining relaxed version of atomic read as cpu_relaxed_read_atomic.

bug 1440421

Change-Id: I39303d72350985890c7eb5a1afc768c3f8064b47
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
(cherry picked from commit 6c20e8c2aed05ad1a9d1b41cfdd875dc377db44c)
Reviewed-on: http://git-master/r/415637
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
arm: asm: relaxed.h: defined relaxed.h

Defined a new header file relaxed.h, which uses generic
definitions of some macros used by arm64 for improving
power efficiency.

bug 1440421

Change-Id: I654dcef609812e3bb54e6c892c1554f9cbb4bd3d
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398766
(cherry picked from commit a96e59b1959f3ee216503b4f9df3cb75f7093ed6)
Reviewed-on: http://git-master/r/422211
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sched: defining relaxed version of idle_cpu

Defining relaxed version of idle_cpu, which uses
macro cpu_relaxed_read_long, that will be used to
enhance power efficiency.

bug 1440421

Change-Id: I6ba2185632ad0fba766a2548b0ddac743defb1b9
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/426484
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
smp: enhance power efficiency

Here we are trying to reduce power usage through the
use of macros cpu_relaxed_read_short and cpu_relaxed_read.

Bug 1440421

Change-Id: I114d122cf58b1cf7b93b4b5f5d712360a2f1e096
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/425947
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
cpu: enhance power efficiency

Here we are trying to reduce power usage through the
use of macros cpu_relaxed_read and relaxed version of
idle_cpu().

Bug 1440421

Change-Id: I0a8c5d358c154782b41570059d03d7e0de87e82c
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/426485
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: relaxed.h: defined relaxed.h

Defined a new header file relaxed.h, which contains basic
macros which will be used for improving power efficiency for
arm64.

bug 1440421

Change-Id: I5ae7503afdfbaa951827bbf466d8ddccf444f558
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398315
(cherry picked from commit dd434aeb1afea1d9ebce1099fb6ecfa7c6c762c6)
Reviewed-on: http://git-master/r/422203
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
llist: defining relaxed version of llist_empty

Defining relaxed version of llist_empty as llist_empty_relaxed,
which will be used for power-optimization.

bug 1440421

Change-Id: I1c4c34b381e49775ed08ddd606d9744a7e7e1fba
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/426483
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
seqlock: enhance power efficiency

Trying to improve the power efficiency in linux/seqlock.h,
using macros cpu_relaxed_read and cpu_read_relax.

Bug 1440421

Change-Id: I8dcc9dc9c72e5a6848c7823768c76f5d38e9f0d1
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398943
(cherry picked from commit ed77ee4d50641b1a5668545c946cda681c06fcd2)
Reviewed-on: http://git-master/r/422258
GVS: Gerrit_Virtual_Submit
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
hrtimer: enhance power efficiency

Defining relaxed version of hrtimer_callback_running(),
which will be used to improve power efficiency through
the use of macro cpu_relaxed_read_long.

Bug 1440421

Change-Id: Ie42d7ae9628a817d52f4636781e11b607327c2c5
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/398789
(cherry picked from commit 0d9f5fc1d39d7d1809519b5d11bf7ac72287b7c6)
Reviewed-on: http://git-master/r/422255
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: processor.h: adding new header file

Defining a new header file and adding architecture
independent macros. Using these macros we are optimizing
power usage on ARM64.

bug 1440421

Change-Id: I7393d35703d0b0a504331653d70f109a50a197c7
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/381905
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: processor.h: remove redundant macros

Removing cpu_relaxed_read and cpu_relaxed_read_long macros from
processor.h, as these macros are defined in asm-generic/relaxed.h.

Bug 1440421

Change-Id: I5d1ba25755e1c9d33b080dfe01ba838289f306af
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/400093
(cherry picked from commit 57eb21e2d4cad3ce1f85283cfffd0eff85a6d17d)
Reviewed-on: http://git-master/r/422209
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
asm-generic: atomic.h: relaxed atomic_read

Defining relaxed version of atomic read as cpu_relaxed_read_atomic
which will be used for improving power efficiency for arm64.

bug 1440421

Change-Id: I6ac26653ec3d62f74d8c21f250dcdaf9dfb75b9b
Signed-off-by: Alex Van Brunt <avanbrunt@nvidia.com>
(cherry picked from commit 7360c3df73afa07361eecab730903e0697d3408f)
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/415628
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
arm: including asm-generic/processor.h

Here we are including asm-generic/processor.h
because it has defined some generic macros which
can be used by any architecture for optimaztion
of power usage.

bug 1440421

Change-Id: I39db944d348b3dad9235b76d8c7ea3867c0006ce
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/368823
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alexander Van Brunt <avanbrunt@nvidia.com>
Reviewed-by: Peng Du <pdu@nvidia.com>
Reviewed-by: Sri Krishna Chowdary <schowdary@nvidia.com>
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sched: Trying to reduce power usage

Using macros cpu_relaxed_read, cpu_relaxed_read_long and
cpu_read_relax in core.c to improve power efficiency.

Bug 1440421

Change-Id: Ic49b88fa6fb0cb847fed49886b2865bed72203db
Signed-off-by: Sumit Singh <sumsingh@nvidia.com>
Reviewed-on: http://git-master/r/368841
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Tested-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
kernel/smp.c: use '|=' for csd_lock

csd_lock() uses assignment to data->flags rather than |=.  That is not
buggy at present because only one bit (CSD_FLAG_LOCK) is defined in
call_single_data.flags.

But it will become buggy if we later add another flag, so fix it now.

Signed-off-by: liguang <lig.fnst@cn.fujitsu.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
kernel/smp.c: cleanups

We sometimes use "struct call_single_data *data" and sometimes "struct
call_single_data *csd".  Use "csd" consistently.

We sometimes use "struct call_function_data *data" and sometimes "struct
call_function_data *cfd".  Use "cfd" consistently.

Also, avoid some 80-col layout tricks.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Shaohua Li <shli@fusionio.com>
Cc: Shaohua Li <shli@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sched: Force sleep on consecutive sched_yields

If a task sched_yields to itself continuously, force the task to
sleep in sched_yield. This will lower the CPU load of this task
thereby lowering the cpu frequency and improving power.

Added a stat variable to track how many times we sleep due these
consecutive sched_yields. Also added sysctl knobs to control the number
of consecutive sched_yields before which the sleep kicks in and the
duration fo the sleep in us.

Bug 1424617

Change-Id: Ie92412b8b900365816e17237fcbd0aac6e9c94ce
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: http://git-master/r/358455
Reviewed-by: Wen Yi <wyi@nvidia.com>
Reviewed-by: Peter Zu <pzu@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Diwakar Tundlam <dtundlam@nvidia.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>