block: Switch from BFQ-v5 for 3.1.0 to BFQ-v5r1 for 3.1.0.

Improvements:

- BUG FIX: Fixed stall occurring when the active queue is moved to
  a different group while idling (this caused the idling timer to be
  cancelled and hence no new queue to be selected, and no new
  request to be dispatched).
- BUG FIX: Fixed wrong assignment of too high budgets to queues during
  the first few seconds after initialization.
- BUG FIX: Added proper locking to the function handling the "weights"
  tunable.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Mauro Andreolini <mauro.andreolini@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
block: Switch from BFQ-v5r1 for 3.1.0 to BFQ-v6 for 3.1.0.

Improvements:

- Replacement of the cooperating-queue merging mechanism borrowed from
  CFQ with Early Queue Merge (EQM), a unified mechanism to get a
  sequential read pattern, and hence a high throughput with any set of
  processes performing interleaved I/O.
  The  main idea  is checking newly arrived requests against the next
  request of the active queue both in case of actual request insert and
  in case of request merge. EQM  also preserves the typical low-latency
  properties of  BFQ, by properly restoring the weight-raising state of
  a queue when it gets back to a non-merged state (see
  http://algo.ing.unimo.it/people/paolo/disk_sched/description.php for
  more details). The code for detecting whether two queues have to be
  merged is a slightly modified version of the CFQ code for detecting
  whether two queues belong to cooperating processes and whether the
  service of a queue should be preempted to boost the throughput.
- Fix a bug that caused the peak rate of a disk to be computed as zero
  in case of multiple I/O errors. Subsequent estimations of the weight
  raising duration caused a division-by-zero error.

Signed-off-by: Mauro Andreolini <mauro.andreolini@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
Reviewed-by: Paolo Valente <paolo.valente@unimore.it>
block: Adding ROW scheduling algorithm

This patch adds the implementation of a new scheduling algorithm - ROW.
The policy of this algorithm is to prioritize READ requests over WRITE
as much as possible without starving the WRITE requests.

Change-Id: I4ed52ea21d43b0e7c0769b2599779a3d3869c519
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: ROW: Correct minimum values of ROW tunable parameters

The ROW scheduling algorithm exposes several tunable parameters.
This patch updates the minimum allowed values for those parameters.

Change-Id: I5ec19d54b694e2e83ad5376bd99cc91f084967f5
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: ROW: Fix forced dispatch

This patch fixes forced dispatch in the ROW scheduling algorithm.
When the dispatch function is called with the forced flag on, we
can't delay the dispatch of the requests that are in scheduler queues.
Thus, when dispatch is called with forced turned on, we need to cancel
idling, or not to idle at all.

Change-Id: I3aa0da33ad7b59c0731c696f1392b48525b52ddc
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
row: Adding support for reinsert already dispatched req

Add support for reinserting already dispatched request back to the
schedulers internal data structures.
The request will be reinserted back to the queue (head) it was
dispatched from as if it was never dispatched.

Change-Id: I70954df300774409c25b5821465fb3aa33d8feb5
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block:row: fix idling mechanism in ROW

This patch addresses the following issues found in the ROW idling
mechanism:
1. Fix the delay passed to queue_delayed_work (pass actual delay
   and not the time when to start the work)
2. Change the idle time and the idling-trigger frequency to be
   HZ dependent (instead of using msec_to_jiffies())
3. Destroy idle_workqueue() in queue_exit

Change-Id: If86513ad6b4be44fb7a860f29bd2127197d8d5bf
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
row: Add support for urgent request handling

This patch adds support for handling urgent requests.
ROW queue can be marked as "urgent" so if it was un-served in last
dispatch cycle and a request was added to it - it will trigger
issuing an urgent-request-notification to the block device driver.
The block device driver may choose at stop the transmission of current
ongoing request to handle the urgent one. Foe example: long WRITE may
be stopped to handle an urgent READ. This decreases READ latency.

Change-Id: I84954c13f5e3b1b5caeadc9fe1f9aa21208cb35e
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: row: Add some debug information on ROW queues

1. Add a counter for number of requests on queue.
2. Add function to print queues status (number requests
   currently on queue and number of already dispatched requests
   in current dispatch cycle).

Change-Id: I1e98b9ca33853e6e6a8ddc53240f6cd6981e6024
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: row: Insert dispatch_quantum into struct row_queue

There is really no point in keeping the dispatch quantum
of a queue outside of it. By inserting it to the row_queue
structure we spare extra level in accessing it.

Change-Id: Ic77571818b643e71f9aafbb2ca93d0a92158b199
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: row: fix sysfs functions - idle_time conversion

idle_time was updated to be stored in msec instead of jiffies.
So there is no need to convert the value when reading from user or
displaying the value to him.

Change-Id: I58e074b204e90a90536d32199ac668112966e9cf
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: row: Aggregate row_queue parameters to one structure

Each ROW queues has several parameters which default values are defined
in separate arrays. This patch aggregates all default values into one
array.
The values in question are:
 - is idling enabled for the queue
 - queue quantum
 - can the queue notify on urgent request

Change-Id: I3821b0a042542295069b340406a16b1000873ec6
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: Add support for reinsert a dispatched req

Add support for reinserting a dispatched request back to the
scheduler's internal data structures.
This capability is used by the device driver when it chooses to
interrupt the current request transmission and execute another (more
urgent) pending request. For example: interrupting long write in order
to handle pending read. The device driver re-inserts the
remaining write request back to the scheduler, to be rescheduled
for transmission later on.

Add API for verifying whether the current scheduler
supports reinserting requests mechanism. If reinsert mechanism isn't
supported by the scheduler, this code path will never be activated.

Change-Id: I5c982a66b651ebf544aae60063ac8a340d79e67f
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: Add API for urgent request handling

This patch add support in block & elevator layers for handling
urgent requests. The decision if a request is urgent or not is taken
by the scheduler. Urgent request notification is passed to the underlying
block device driver (eMMC for example). Block device driver may decide to
interrupt the currently running low priority request to serve the new
urgent request. By doing so READ latency is greatly reduced in read&write
collision scenarios.

Note that if the current scheduler doesn't implement the urgent request
mechanism, this code path is never activated.

Change-Id: I8aa74b9b45c0d3a2221bd4e82ea76eb4103e7cfa
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
Add ARM cpu topology definition

The affinity between ARM processors is defined in the MPIDR register.
We can identify which processors are in the same cluster,
and which ones have performance interdependency. We can define the
cpu topology of ARM platform, that is then used by sched_mc and sched_smt.

The default state of sched_mc and sched_smt config is disable.
When enabled, the behavior of the scheduler can be modified with
sched_mc_power_savings and sched_smt_power_savings sysfs interfaces.

Changes since v4 :
*  Remove unnecessary parentheses and blank lines

Changes since v3 :
* Update the format of printk message
* Remove blank line

Changes since v2 :
* Update the commit message and some comments

Changes since v1 :
* Update the commit message
* Add read_cpuid_mpidr in arch/arm/include/asm/cputype.h
* Modify header of arch/arm/kernel/topology.c
* Modify tests and manipulation of MPIDR's bitfields
* Modify the place and dependancy of the config
* Modify Noop functions

Signed-off-by: Vincent Guittot <(address hidden)>
Reviewed-by: Amit Kucheria <(address hidden)>
arm/crypto: Add optimized AES and SHA1 routines

Add assembler versions of AES and SHA1 for ARM platforms.  This has provided
up to a 50% improvement in IPsec/TCP throughout for tunnels using AES128/SHA1.

Platform   CPU SPeed    Endian   Before (bps)   After (bps)   Improvement

IXP425      533 MHz      big     11217042        15566294        ~38%
KS8695      166 MHz     little    3828549         5795373        ~51%

Signed-off-by: David McCullough <ucdevel@gmail.com>
Optimized ARM RWSEM algorithm

RWSEM implementation for ARM using atomic functions.
Heavily based on arch/sh/include/asm/rwsem.h

Signed-off-by: Ashwin Chaugule <ashwinc@codeaurora.org>
lib/string & lib/memcpy: use GNU implementation.

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
lib/lzo: Rename lzo1x_decompress.c to lzo1x_decompress_safe.c

Rename the source file to match the function name and thereby
also make room for a possible future even slightly faster
"non-safe" decompressor version.

Signed-off-by: Markus F.X.J. Oberhumer <markus@oberhumer.com>
lib/lzo: Update LZO compression to current upstream version

This commit updates the kernel LZO code to the current upsteam version
which features a significant speed improvement - benchmarking the Calgary
and Silesia test corpora typically shows a doubled performance in
both compression and decompression on modern i386/x86_64/powerpc machines.

Signed-off-by: Markus F.X.J. Oberhumer <markus@oberhumer.com>