block: prepare I/O context code for BFQ-v5r1 for 3.0

BFQ uses struct cfq_io_context to store its per-process per-device data,
reusing the same code for cic handling of CFQ.  The code is not shared
ATM to minimize the impact of these patches.

This patch introduces a new hlist to each io_context to store all the
cic's allocated by BFQ to allow calling the right destructor on module
unload; the radix tree used for cic lookup needs to be duplicated
because it can contain dead keys inserted by a scheduler and later
retrieved by the other one.

Update the io_context exit and free paths to take care also of
the BFQ cic's.

Change the type of cfqq inside struct cfq_io_context to void *
to use it also for BFQ per-queue data.

A new bfq-specific ioprio_changed field is necessary, too, to avoid
clobbering cfq's one, so switch ioprio_changed to a bitmap, with one
element per scheduler.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>

Conflicts:

	block/Kconfig.iosched
block: cgroups, kconfig, build bits for BFQ-v5r1-3.0

Add a Kconfig option and do the related Makefile changes to compile
the BFQ I/O scheduler.  Also let the cgroups subsystem know about the
BFQ I/O controller.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>

Conflicts:

	block/Makefile
block: introduce the BFQ-v5r1 I/O sched for 3.0

Add the BFQ-v5r1 I/O scheduler to 3.0.
The general structure is borrowed from CFQ, as much code. A (bfq_)queue is
associated to each task doing I/O on a device, and each time a scheduling
decision has to be taken a queue is selected and it is served until it expires.

    - Slices are given in the service domain: tasks are assigned budgets,
      measured in number of sectors. Once got the disk, a task must
      however consume its assigned budget within a configurable maximum time
      (by default, the maximum possible value of the budgets is automatically
      computed to comply with this timeout). This allows the desired latency
      vs "throughput boosting" tradeoff to be set.

    - Budgets are scheduled according to a variant of WF2Q+, implemented
      using an augmented rb-tree to take eligibility into account while
      preserving an O(log N) overall complexity.

    - A low-latency tunable is provided; if enabled, both interactive and soft
      real-time applications are guaranteed very low latency.

    - Latency guarantees are preserved also in presence of NCQ.

    - High throughput with flash-based devices, while still preserving
      latency guarantees.

    - Useful features borrowed from CFQ: cooperating-queues merging (with
      some additional optimizations with respect to the original CFQ version),
      static fallback queue for OOM.

    - BFQ supports full hierarchical scheduling, exporting a cgroups
      interface.  Each node has a full scheduler, so each group can
      be assigned its own ioprio and an ioprio_class.

    - If the cgroups interface is used, weights can be explictly assigned,
      otherwise ioprio values are mapped to weights using the relation
      weight = IOPRIO_BE_NR - ioprio.

    - ioprio classes are served in strict priority order, i.e., lower
      priority queues are not served as long as there are higher priority
      queues.  Among queues in the same class the bandwidth is distributed
      in proportion to the weights of each queue. A very thin extra bandwidth
      is however guaranteed to the Idle class, to prevent it from starving.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
block: remove aliased requests from BFQ
Linux 3.0.57
leanKernel v5.4

Remove SIO and add BFQ
morfic's Trinity contrast
cpufreq: interactive: pin timers to associated CPU

Helps avoid waking up other CPUs to react to activity on the local CPU.

Change-Id: Ife272aaa7916894a437705d44521b1a1693fbe8e
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: use deferrable timer by default

Avoid wakeups only to handle the governor timer when the system is otherwise
idle.

For platforms where the power cost of remaining in idle at higher CPU
speed may outweigh the cost of a governor wakeup from idle to lower the speed,
set parameter cpufreq_interactive.governidle=1.

Change-Id: Id6c43eb35caecf9b0574fcdd5b769711bc7e6de6
Signed-off-by: LianWei WANG <a22439@motorola.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: kick timer on idle exit past expiry

The deferrable timer list isn't checked on all idle exits, such as when
hi-res timers expire or ISRs schedule workers.  If the idle loop is
exited and it's past time to run the governor load polling timer,
run it immediately.  This ensures we handle load spikes caused by actvity
that does not run the normal timer list.

Rename the field that timestamps the "time_in_idle" value to be more
accurate.

Change-Id: Ied590ecbefc83c9a9ec5eb9e31903557f6fa1614
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: trace actual speed in target speed decisions

Tracing adds actual speed since this is expected to be key to the
choice of target speed.

Change-Id: Iec936102d0010c4e9dfa143c38a9fd0d551189c3
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: apply above_hispeed_delay to each step above hispeed

Apply above_hispeed_delay whenever increasing speed to a new speed above
hispeed (not just the first step above hispeed).

Change-Id: Ibb7add7db47f2a4306a9458c4e1ebabb60698636
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: allow arbitrary speed / target load mappings

Accept a string of target loads and speeds at which to apply the
target loads, per the documentation update in this patch.  For example,
"85 1000000:90 1700000:99" targets CPU load 85% below speed 1GHz,  90%
at or above 1GHz, until 1.7GHz and above, at which load 99% is targeted.

Attempt to avoid oscillations by evaluating the current speed
weighted by current load against each new choice of speed, choosing a
higher speed if the current load requires a higher speed.

Change-Id: Ie3300206047c84eca5a26b0b63ea512e5207550e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: remove load since last speed change

The longer-term load since last speed change isn't terribly useful,
may delay recognition of dropping load, and would need forthcoming
changes to adjust load for changing CPU speeds.  Drop it.

Change-Id: Ic3cbb0542cc3484617031787e03ed9bdd632dec1
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: adjust load for changes in speed

Add notifier for speed transitions.  Keep a count of CPU active
microseconds times current frequency, converted to a percentage relative
to the current frequency when load is evaluated.

Change-Id: I5c27adb11081c50490219784ca57cc46e97fc28c
Signed-off-by: Todd Poynor <toddpoynor@google.com>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: specify duration of CPU speed boost pulse

Sysfs attribute boostpulse_duration specifies the duration of boosting CPU
speed in response to bootpulse events.  Duration is specified in usecs,
default 80ms.

Change-Id: Ifd41625574891a44f1787a4e85d1e7b4f2afb52b
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: add timer slack to limit idle at speed > min

Always use deferrable timer for load sampling.

Set a non-deferrable timer to an additional slack time to allow prior to
waking up from idle to drop speed when not at minimum speed.  Slack value
-1 avoids wakeups to drop speed.  Default is 80ms.

Remove the governidle module param and its timer management in idle.  For
platforms on which holding speed above mimum in idle costs power, use the
new timer slack to select how long to wait before waking up to drop speed.

Change-Id: I270b3980667e2c70a68e5bff534124b4411dbad5
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: fix boosting logic

35a84de cpufreq: interactive: apply above_hispeed_delay to each step above hispeed

caused the speed choice logic to osciallate between boosting and not boosting.
Add back code to ensure speed does not drop below boost frequency while
boosting.

Change-Id: Id420068480fcc7f5c4989ff523e2a8d22e2f4db2
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: fix racy timer stopping

When stopping the governor, del_timer_sync() can race against an
invocation of the idle notifier callback, which has the potential
to reactivate the timer.

To fix this issue, a read-write semaphore is used. Multiple readers are
allowed as long as pcpu->governor_enabled is true.  However it can be
moved to false only after taking a write semaphore which would wait for
any on-going asynchronous activities to complete and prevent any more of
those activities to be initiated.

[toddpoynor@google.com: cosmetic and commit text changes]
Change-Id: Ib51165a735d73dcf964a06754c48bdc1913e13d0
Signed-off-by: Nicolas Pitre <nicolas.pitre@linaro.org>
cpufreq: interactive: fix race on timer restart on governor start

Starting the governor, or restarting on a hotplugged-in CPU, can race
with the timer start in idle, triggering a BUG on timer already pending.
Start the timer before setting the enable flag, and use enable_sem to
protect the sequence (and ensure correct order of the update to the
enable flag).  Delete any existing timer for safety.

Change-Id: Ife77cf9fe099e8fd8543224cbf148c6722c2ffb0
Reported-by: Francisco Franco <francisco.franco@cloudcar.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>