Cortex-A7 optimizations
sched: disable gentle fair sleepers
The Ultra Kernel Samepage Merging feature
Ultra KSM. Copyright (C) 2011-2012 Nai Xia

This is an improvement upon KSM. Some basic data structures and routines
are borrowed from ksm.c .

Its new features:
1. Full system scan:
     It automatically scans all user processes' anonymous VMAs. Kernel-user
     interaction to submit a memory area to KSM is no longer needed.

2. Rich area detection:
     It automatically detects rich areas containing abundant duplicated
     pages based. Rich areas are given a full scan speed. Poor areas are
     sampled at a reasonable speed with very low CPU consumption.

3. Ultra Per-page scan speed improvement:
     A new hash algorithm is proposed. As a result, on a machine with
     Core(TM)2 Quad Q9300 CPU in 32-bit mode and 800MHZ DDR2 main memory, it
     can scan memory areas that does not contain duplicated pages at speed of
     627MB/sec ~ 2445MB/sec and can merge duplicated areas at speed of
     477MB/sec ~ 923MB/sec.

4. Thrashing area avoidance:
     Thrashing area(an VMA that has frequent Ksm page break-out) can be
     filtered out. My benchmark shows it's more efficient than KSM's per-page
     hash value based volatile page detection.

5. Misc changes upon KSM:
    It has a fully x86-opitmized memcmp dedicated for 4-byte-aligned page
       comparison. It's much faster than default C version on x86.
    rmap_item now has an structpage member to loosely cache a
       address-->page mapping, which reduces too much time-costly
       follow_page().
    The VMA creation/exit procedures are hooked to let the Ultra KSM know.
    try_to_merge_two_pages() now can revert a pte if it fails. No break_
       ksm is needed for this case.

6. Full Zero Page consideration(contributed by Figo Zhang)
   Now uksmd consider full zero pages as special pages and merge them to an
   special unswappable uksm zero page.
clean
lib/memcopy: use glibc version

the kernel's memcpy and memmove is very inefficient. But the glibc version is
quite fast, in some cases it is 10 times faster than the kernel version. So I
introduce some memory copy macros and functions of the glibc to improve the
kernel version's performance.

The strategy of the memory functions is:
1. Copy bytes until the destination pointer is aligned.
2. Copy words in unrolled loops.  If the source and destination are not
   aligned in the same way, use word memory operations, but shift and merge
   two read words before writing.
3. Copy the few remaining bytes.

Signed-off-by: Miao Xie <miaox*******>
Signed-off-by: faux123 <reioux@gmail.com>
lib/string: use glibc version

the performance of memcpy and memmove of the general version is very
inefficient, this patch improved them.

Signed-off-by: Miao Xie <miaox*******>
Signed-off-by: faux123 <reioux@gmail.com>
Optimized ARM RWSEM algorithm

RWSEM implementation for ARM using atomic functions.
Heavily based on arch/sh/include/asm/rwsem.h

Signed-off-by: Ashwin Chaugule <ashwinc@codeaurora.org>
arm/crypto: Add optimized AES and SHA1 routines

Add assembler versions of AES and SHA1 for ARM platforms.  This has provided
up to a 50% improvement in IPsec/TCP throughout for tunnels using AES128/SHA1.

Platform   CPU SPeed    Endian   Before (bps)   After (bps)   Improvement

IXP425      533 MHz      big     11217042        15566294        ~38%
KS8695      166 MHz     little    3828549         5795373        ~51%

Signed-off-by: David McCullough <ucdevel@gmail.com>
Signed-off-by: faux123 <reioux@gmail.com>
initial s2w dt2w code import from @showp1984

Signed-off-by: Paul Reioux <reioux@gmail.com>
mdss:mdss_dsi_panel.c: add s2wake/dt2wake screen on/off hooks

Signed-off-by: Paul Reioux <reioux@gmail.com>
sweep2wake/doubletap2wake: adapt for falcon_umts (aka Moto G)

Signed-off-by: Paul Reioux <reioux@gmail.com>
overall_stats: add overall stats for all available cores

this is much more accurate stats tracking than just displaying core 0's
stats from most other kernels

Signed-off-by: faux123 <reioux@gmail.com>
overall_stats: make dual/quad core stats configurable via meunconfig

Signed-off-by: faux123 <reioux@gmail.com>
overall_stats: forward port to kernel 3.4+

by replacing deprecated functions with native add/sub for 64bit data types

Signed-off-by: faux123 <reioux@gmail.com>
block: cgroups, kconfig, build bits for BFQ-v6-3.4

Update Kconfig.iosched and do the related Makefile changes to include
kernel configuration options for BFQ. Also add the bfqio controller
to the cgroups subsystem.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
block: introduce the BFQ-v6 I/O sched for 3.4

Add the BFQ-v6 I/O scheduler to 3.4.
The general structure is borrowed from CFQ, as much code. A (bfq_)queue is
associated to each task doing I/O on a device, and each time a scheduling
decision has to be made a queue is selected and served until it expires.

    - Slices are given in the service domain: tasks are assigned budgets,
      measured in number of sectors. Once got the disk, a task must
      however consume its assigned budget within a configurable maximum time
      (by default, the maximum possible value of the budgets is automatically
      computed to comply with this timeout). This allows the desired latency
      vs "throughput boosting" tradeoff to be set.

    - Budgets are scheduled according to a variant of WF2Q+, implemented
      using an augmented rb-tree to take eligibility into account while
      preserving an O(log N) overall complexity.

    - A low-latency tunable is provided; if enabled, both interactive and soft
      real-time applications are guaranteed very low latency.

    - Latency guarantees are preserved also in presence of NCQ.

    - Also with flash-based devices, a high throughput is achieved while
      still preserving latency guarantees.

    - A useful feature borrowed from CFQ: static fallback queue for OOM.

    - Differently from CFQ, BFQ uses a unified mechanism (Early Queue Merge,
      EQM) to get a sequential read pattern, and hence a high throughput,
      with any set of processes performing interleaved I/O. EQM also
      preserves low latency. The code for detecting whether two queues have
      to be merged is a slightly modified version of the CFQ code for
      detecting whether two queues belong to cooperating processes and whether
      the service of a queue should be preempted to boost the throughput.

    - BFQ supports full hierarchical scheduling, exporting a cgroups
      interface.  Each node has a full scheduler, so each group can
      be assigned its own ioprio (mapped to a weight, see next point)
      and an ioprio_class.

    - If the cgroups interface is used, weights can be explictly assigned,
      otherwise ioprio values are mapped to weights using the relation
      weight = IOPRIO_BE_NR - ioprio.

    - ioprio classes are served in strict priority order, i.e., lower
      priority queues are not served as long as there are higher priority
      queues.  Among queues in the same class the bandwidth is distributed
      in proportion to the weight of each queue. A very thin extra bandwidth
      is however guaranteed to the Idle class, to prevent it from starving.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
block: fiops ioscheduler core

FIOPS (Fair IOPS) ioscheduler is IOPS based ioscheduler, so only targets
for drive without I/O seek. It's quite similar like CFQ, but the dispatch
decision is made according to IOPS instead of slice.

The algorithm is simple. Drive has a service tree, and each task lives in
the tree. The key into the tree is called vios (virtual I/O). Every request
has vios, which is calculated according to its ioprio, request size and so
on. Task's vios is the sum of vios of all requests it dispatches. FIOPS
always selects task with minimum vios in the service tree and let the task
dispatch request. The dispatched request's vios is then added to the task's
vios and the task is repositioned in the sevice tree.

Unlike CFQ, FIOPS doesn't have separate sync/async queues, because with I/O
less writeback, usually a task can only dispatch either sync or async requests.
Bias read or write request can still be done with read/write scale.

One issue is if workload iodepth is lower than drive queue_depth, IOPS
share of a task might not be strictly according to its priority, request
Bias read or write request can still be done with read/write scale.

One issue is if workload iodepth is lower than drive queue_depth, IOPS
share of a task might not be strictly according to its priority, request
size and so on. In this case, the drive is in idle actually. Solving the
problem need make drive idle, so impact performance. I believe CFQ isn't
completely fair between tasks in such case too.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
block: fiops read/write request scale

read/write speed of Flash based storage usually is different. For example,
in my SSD maxium thoughput of read is about 3 times faster than that of
write. Add a scale to differenate read and write. Also add a tunable, so
user can assign different scale for read and write.

By default, the scale is 1:1, which means the scale is a noop.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
block: fiops sync/async scale

CFQ gives 2.5 times more share to sync workload. This matches CFQ.

Note this is different with the read/write scale. We have 3 types of
requests:
1. read
2. sync write
3. write
CFQ doesn't differentitate type 1 and 2, but request cost of 1 and 2
are usually different for flash based storage. So we have both sync/async
and read/write scale here.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
block: fiops add ioprio support

Add CFQ-like ioprio support. Priority A will get 20% more share than priority
A+1, which matches CFQ.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>