block: cgroups, kconfig, build bits for BFQ-v6-3.4

Update Kconfig.iosched and do the related Makefile changes to include
kernel configuration options for BFQ. Also add the bfqio controller
to the cgroups subsystem.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: introduce the BFQ-v6 I/O sched for 3.4

Add the BFQ-v6 I/O scheduler to 3.4.
The general structure is borrowed from CFQ, as much code. A (bfq_)queue is
associated to each task doing I/O on a device, and each time a scheduling
decision has to be made a queue is selected and served until it expires.

    - Slices are given in the service domain: tasks are assigned budgets,
      measured in number of sectors. Once got the disk, a task must
      however consume its assigned budget within a configurable maximum time
      (by default, the maximum possible value of the budgets is automatically
      computed to comply with this timeout). This allows the desired latency
      vs "throughput boosting" tradeoff to be set.

    - Budgets are scheduled according to a variant of WF2Q+, implemented
      using an augmented rb-tree to take eligibility into account while
      preserving an O(log N) overall complexity.

    - A low-latency tunable is provided; if enabled, both interactive and soft
      real-time applications are guaranteed very low latency.

    - Latency guarantees are preserved also in presence of NCQ.

    - Also with flash-based devices, a high throughput is achieved while
      still preserving latency guarantees.

    - A useful feature borrowed from CFQ: static fallback queue for OOM.

    - Differently from CFQ, BFQ uses a unified mechanism (Early Queue Merge,
      EQM) to get a sequential read pattern, and hence a high throughput,
      with any set of processes performing interleaved I/O. EQM also
      preserves low latency. The code for detecting whether two queues have
      to be merged is a slightly modified version of the CFQ code for
      detecting whether two queues belong to cooperating processes and whether
      the service of a queue should be preempted to boost the throughput.

    - BFQ supports full hierarchical scheduling, exporting a cgroups
      interface.  Each node has a full scheduler, so each group can
      be assigned its own ioprio (mapped to a weight, see next point)
      and an ioprio_class.

    - If the cgroups interface is used, weights can be explictly assigned,
      otherwise ioprio values are mapped to weights using the relation
      weight = IOPRIO_BE_NR - ioprio.

    - ioprio classes are served in strict priority order, i.e., lower
      priority queues are not served as long as there are higher priority
      queues.  Among queues in the same class the bandwidth is distributed
      in proportion to the weight of each queue. A very thin extra bandwidth
      is however guaranteed to the Idle class, to prevent it from starving.

Signed-off-by: Paolo Valente <paolo.valente@unimore.it>
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops ioscheduler core

FIOPS (Fair IOPS) ioscheduler is IOPS based ioscheduler, so only targets
for drive without I/O seek. It's quite similar like CFQ, but the dispatch
decision is made according to IOPS instead of slice.

The algorithm is simple. Drive has a service tree, and each task lives in
the tree. The key into the tree is called vios (virtual I/O). Every request
has vios, which is calculated according to its ioprio, request size and so
on. Task's vios is the sum of vios of all requests it dispatches. FIOPS
always selects task with minimum vios in the service tree and let the task
dispatch request. The dispatched request's vios is then added to the task's
vios and the task is repositioned in the sevice tree.

Unlike CFQ, FIOPS doesn't have separate sync/async queues, because with I/O
less writeback, usually a task can only dispatch either sync or async requests.
Bias read or write request can still be done with read/write scale.

One issue is if workload iodepth is lower than drive queue_depth, IOPS
share of a task might not be strictly according to its priority, request
Bias read or write request can still be done with read/write scale.

One issue is if workload iodepth is lower than drive queue_depth, IOPS
share of a task might not be strictly according to its priority, request
size and so on. In this case, the drive is in idle actually. Solving the
problem need make drive idle, so impact performance. I believe CFQ isn't
completely fair between tasks in such case too.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops read/write request scale

read/write speed of Flash based storage usually is different. For example,
in my SSD maxium thoughput of read is about 3 times faster than that of
write. Add a scale to differenate read and write. Also add a tunable, so
user can assign different scale for read and write.

By default, the scale is 1:1, which means the scale is a noop.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops sync/async scale

CFQ gives 2.5 times more share to sync workload. This matches CFQ.

Note this is different with the read/write scale. We have 3 types of
requests:
1. read
2. sync write
3. write
CFQ doesn't differentitate type 1 and 2, but request cost of 1 and 2
are usually different for flash based storage. So we have both sync/async
and read/write scale here.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops add ioprio support

Add CFQ-like ioprio support. Priority A will get 20% more share than priority
A+1, which matches CFQ.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops preserve vios key for deep queue depth workload

If the task has running request, even it's added into service tree newly,
we preserve its vios key, so it will not lost its share. This should work
for task driving big queue depth. For single depth task, there is no approach
to preserve its vios key.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops bias sync workload

If there are async requests running, delay async workload. Otherwise
async workload (usually very deep iodepth) will use all queue iodepth
and later sync requests will get long delayed. The idea is from CFQ.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: fiops add some trace information

Add some trace information, which is helpful when I do debugging.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Add SIO I/O scheduler

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block/Kconfig.iosched: disable test I/O scheduler

Signed-off-by: Paul Reioux <reioux@gmail.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
sched: bfq: Update to version 3.4-v6-r2

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: row: add magic values.

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: Add VR and ZEN I/O schedulers

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: account iowait time when waiting for completion of IO request

Using wait_for_completion() for waiting for a IO request to be executed
results in wrong iowait time accounting. For example, a system having
the only task doing write() and fdatasync() on a block device can be
reported being idle instead of iowaiting as it should because
blkdev_issue_flush() calls wait_for_completion() which in turn calls
schedule() that does not increment the iowait proc counter and thus does
not turn on iowait time accounting.

The patch makes block layer use wait_for_completion_io() instead of
wait_for_completion() where appropriate to account iowait time
correctly.

Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block/partitions: optimize memory allocation in check_partition()

Currently, sizeof(struct parsed_partitions) may be 64KB in 32bit arch, so
it is easy to trigger page allocation failure by check_partition,
especially in hotplug block device situation(such as, USB mass storage,
MMC card, ...), and Felipe Balbi has observed the failure.

This patch does below optimizations on the allocation of struct
parsed_partitions to try to address the issue:

- make parsed_partitions.parts as pointer so that the pointed memory can
  fit in 32KB buffer, then approximate 32KB memory can be saved

- vmalloc the buffer pointed by parsed_partitions.parts because 32KB is
  still a bit big for kmalloc

- given that many devices have the partition count limit, so only
  allocate disk_max_parts() partitions instead of 256 partitions always

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Reported-by: Felipe Balbi <balbi@ti.com>
Cc: Jens Axboe <axboe@kernel.dk>
Reviewed-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
deadline: Allow 0ms deadline latency, increase the read speed

Change a timer compare from after to after-equals, thus allowing
0 timeout and making deadline schedule FIFO.

Signed-off-by: xiaobing tu <xiaobing.tu@intel.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
block: deadline: Optimize for non-rotational

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>