sched: Introduce primitives to account for CFS bandwidth tracking

In this patch we introduce the notion of CFS bandwidth, partitioned into
globally unassigned bandwidth, and locally claimed bandwidth.

 - The global bandwidth is per task_group, it represents a pool of unclaimed
   bandwidth that cfs_rqs can allocate from.
 - The local bandwidth is tracked per-cfs_rq, this represents allotments from
   the global pool bandwidth assigned to a specific cpu.

Bandwidth is managed via cgroupfs, adding two new interfaces to the cpu subsystem:
 - cpu.cfs_period_us : the bandwidth period in usecs
 - cpu.cfs_quota_us : the cpu bandwidth (in usecs) that this tg will be allowed
   to consume over period above.

Signed-off-by: Paul Turner <pjt@google.com>
Signed-off-by: Nikhil Rao <ncrao@google.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: Hidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20110721184756.972636699@google.com
Signed-off-by: Ingo Molnar <mingo@elte.hu>

Conflicts:
	kernel/sched.c
sched: Validate CFS quota hierarchies

Add constraints validation for CFS bandwidth hierarchies.

Validate that:
   max(child bandwidth) <= parent_bandwidth

In a quota limited hierarchy, an unconstrained entity
(e.g. bandwidth==RUNTIME_INF) inherits the bandwidth of its parent.

This constraint is chosen over sum(child_bandwidth) as notion of over-commit is
valuable within SCHED_OTHER.  Some basic code from the RT case is re-factored
for reuse.

Signed-off-by: Paul Turner <pjt@google.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20110721184757.083774572@google.com
Signed-off-by: Ingo Molnar <mingo@elte.hu>
sched: Accumulate per-cfs_rq cpu usage and charge against bandwidth

Account bandwidth usage on the cfs_rq level versus the task_groups to which
they belong.  Whether we are tracking bandwidth on a given cfs_rq is maintained
under cfs_rq->runtime_enabled.

cfs_rq's which belong to a bandwidth constrained task_group have their runtime
accounted via the update_curr() path, which withdraws bandwidth from the global
pool as desired.  Updates involving the global pool are currently protected
under cfs_bandwidth->lock, local runtime is protected by rq->lock.

This patch only assigns and tracks quota, no action is taken in the case that
cfs_rq->runtime_used exceeds cfs_rq->runtime_assigned.

Signed-off-by: Paul Turner <pjt@google.com>
Signed-off-by: Nikhil Rao <ncrao@google.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: Hidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20110721184757.179386821@google.com
Signed-off-by: Ingo Molnar <mingo@elte.hu>
sched/fair: Improve the ->group_imb logic

Group imbalance is meant to deal with situations where affinity masks
and sched domains don't align well, such as 3 cpus from one group and
6 from another. In this case the domain based balancer will want to
put an equal amount of tasks on each side even though they don't have
equal cpus.

Currently group_imb is set whenever two cpus of a group have a weight
difference of at least one avg task and the heaviest cpu has at least
two tasks. A group with imbalance set will always be picked as busiest
and a balance pass will be forced.
The problem is that even if there are no affinity masks this stuff can
trigger and cause weird balancing decisions, eg. the observed
behaviour was that of 6 cpus, 5 had 2 and 1 had 3 tasks, due to the
difference of 1 avg load (they all had the same weight) and nr_running
being >1 the group_imbalance logic triggered and did the weird thing
of pulling more load instead of trying to move the 1 excess task to
the other domain of 6 cpus that had 5 cpu with 2 tasks and 1 cpu with
1 task.
Curb the group_imbalance stuff by making the nr_running condition
weaker by also tracking the min_nr_running and using the difference in
nr_running over the set instead of the absolute max nr_running.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-9s7dedozxo8kjsb9kqlrukkf@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
modified for Linux 3.0 from Linux 3.4+ by faux123

commit b37266adbcf9c8a2b32b42580d1611b54a317fd2 1 parent d989efcbbb
faux123 authored 11 days ago
PM / Suspend: Print wall time at suspend entry and exit

Change-Id: I92f252414c013b018b9a392eae1ee039aa0e89dc
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Sync kernel/power/suspend.c with 3.4.x adapted to 3.0.x

Conflicts:
	kernel/power/suspend.c
memcg: let css_get_next() rely upon rcu_read_lock()

Remove lock and unlock around css_get_next()'s call to idr_get_next().
memcg iterators (only users of css_get_next) already did rcu_read_lock(),
and its comment demands that; but add a WARN_ON_ONCE to make sure of it.

Signed-off-by: Hugh Dickins <hughd@google.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Acked-by: Li Zefan <lizf@cn.fujitsu.com>

commit 4536a1b91dd6408d3453fcb5291457f91dd0d1a2 1 parent 8723a7a0c8
Hugh Dickins authored 5 months ago Paul Reioux committed 5 months ago
Add Early Suspend to INTERACTIVE
[PATCH 3/3] Optimize ARM memset and memzero functions

The ARM memset and memzero functions are optimized with lower overhead
for small requests, generation of more 16-bit Thumb2 instructions when
compiled in Thumb2 mode, and configurable destination alignment before
the main block-copying loop.

The new compile-time constant MEMSET_WRITE_ALIGN_BYTES is introduced
in assembler.h, to augment the CALGN macro that previously regulated
32-byte write alignment but was only used on the Feroceon platform.
MEMSET_WRITE_ALIGN_BYTES can have values of 0 (no write alignment),
8, or 32. Apart from Feroceon, memset write alignment of 32 bytes
appears to benefit the armv6 platform, while the armv7 platform
seems benefit from alignment to 8 bytes for memset/memzero.

The CALGN macro is renamed to MEMSET_CALGN for memset and memzero; the
original CALGN macro is reserved for the memcpy family of functions
(memcpy, copy_from_user, copy_to_user) currently implemented in
copy_template.S, and the associated compile time constant
WRITE_ALIGN_BYTES defines the write alignment for memcpy-related
functions that will be utilized in subsequent contributions. Because
the current CALGN implementation in copy_template.S only implements
32-byte write alignment and is broken on Thumb2, it is only enabled
when WRITE_ALIGN_BYTES is equal to 32 and Thumb2 mode is not enabled.

Finally, memset and memzero now include a directive to enable unified
ARM assembler syntax.

Signed-off-by: Harm Hanemaaijer <fgenfb@yahoo.com>
[PATCH 2/3] Optimize copy_page for modern ARM platforms

The existing implementation of copy_page for ARM appears to be
optimized for older platforms. Benchmark testing in a sandbox
environment shows suboptimal performance on modern platforms
like armv6 and armv7, with speed-ups ranging from 10% (Cortex A8)
to 80% (armv6 used in Raspberry Pi) being achievable.

This commit optimizes copy_page and introduces the new compile-time
constant PREFETCH_DISTANCE, defined in cache.h, which when
multiplied by L1_CACHE_BYTES is equal to the offset used for
prefetches performed with the PLD instruction. For platforms where
L1_CACHE_BYTES is 32 (armv5 and armv6), copy_page processes 32 bytes
at a time while doing one prefetch per iteration, while for armv7
(with L1_CACHE_BYTES equal to 64), 64 bytes are processed at at time
with one prefetch per iteration. When no preload instruction is
available (platforms earlier than armv5), no preload instructions
are generated and 32 bytes are processed at at time.

To facilitate specifying instructions for architectures with no
preload instruction, the NO_PLD macro is added to assembler.h,
augmenting the PLD macro.

Signed-off-by: Harm Hanemaaijer <fgenfb@yahoo.com>
The ARM assembler library functions use a macro called "push" that
along with a macro called "pull" is used to shift bytes around in a
word in an endian-independent way. However, the modern unified
ARM assembler syntax also defines the instruction "push" to push
data onto the stack, which has specific encodings in the Thumb2
instruction set. For prevent possible conflicts going forward, and
to allow the use of the more transparent "push" instruction along
with the modern unified assembler syntax, this patch renames all
occurrences of the "push" macro to "pushbits", as well as renaming
the macro argument, when also called "push", to "pushshift".

For consistency, the macro called "pull" with its argument name
"pull" are also renamed to "pullbits" and "pullshift", respectively.

Signed-off-by: Harm Hanemaaijer <fgenfb@yahoo.com>
A few other cache related optimizations for Cortex-A9

Creating page tables also do not need to clean cache-line because of
the same reason as above.
This patch improves lmbench3 (fork/exec/shell) performance by 10%~20%
in my test.

Signed-off-by: Heechul Yun <heechul@illinois.edu>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>