Added SIO based on Stratosk algorithm

Change-Id: I018541302dd1600b7f79c12f8d72ef22c733d8d1
Squash fiops commits

block: fiops ioscheduler core

FIOPS (Fair IOPS) ioscheduler is IOPS based ioscheduler, so only targets
for drive without I/O seek. It's quite similar like CFQ, but the dispatch
decision is made according to IOPS instead of slice.

The algorithm is simple. Drive has a service tree, and each task lives in
the tree. The key into the tree is called vios (virtual I/O). Every request
has vios, which is calculated according to its ioprio, request size and so
on. Task's vios is the sum of vios of all requests it dispatches. FIOPS
always selects task with minimum vios in the service tree and let the task
dispatch request. The dispatched request's vios is then added to the task's
vios and the task is repositioned in the sevice tree.

Unlike CFQ, FIOPS doesn't have separate sync/async queues, because with I/O
less writeback, usually a task can only dispatch either sync or async requests.
Bias read or write request can still be done with read/write scale.

One issue is if workload iodepth is lower than drive queue_depth, IOPS
share of a task might not be strictly according to its priority, request
size and so on. In this case, the drive is in idle actually. Solving the
problem need make drive idle, so impact performance. I believe CFQ isn't
completely fair between tasks in such case too.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

Conflicts:
	block/Kconfig.iosched
	block/Makefile

block: fiops read/write request scale

read/write speed of Flash based storage usually is different. For example,
in my SSD maxium thoughput of read is about 3 times faster than that of
write. Add a scale to differenate read and write. Also add a tunable, so
user can assign different scale for read and write.

By default, the scale is 1:1, which means the scale is a noop.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

block: fiops sync/async scale

CFQ gives 2.5 times more share to sync workload. This matches CFQ.

Note this is different with the read/write scale. We have 3 types of
requests:
1. read
2. sync write
3. write
CFQ doesn't differentitate type 1 and 2, but request cost of 1 and 2
are usually different for flash based storage. So we have both sync/async
and read/write scale here.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

block: fiops add ioprio support

Add CFQ-like ioprio support. Priority A will get 20% more share than priority
A+1, which matches CFQ.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

block: fiops preserve vios key for deep queue depth workload

If the task has running request, even it's added into service tree newly,
we preserve its vios key, so it will not lost its share. This should work
for task driving big queue depth. For single depth task, there is no approach
to preserve its vios key.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

block: fiops bias sync workload

If there are async requests running, delay async workload. Otherwise
async workload (usually very deep iodepth) will use all queue iodepth
and later sync requests will get long delayed. The idea is from CFQ.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>

block: fiops add some trace information

Add some trace information, which is helpful when I do debugging.

Change-Id: I538fee1698702a596b1d1cb797e63cf5e8f0a4be
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
cfq-iosched: Fix null pointer dereference

NULL pointer dereference can happen in cfq_choose_cfqg()
when there are no cfq groups to select other than the
current serving group. Prevent this by adding a NULL
check before dereferencing.

Unable to handle kernel NULL pointer dereference at virtual address
[<c02502cc>] (cfq_dispatch_requests+0x368/0x8c0) from
[<c0243f30>] (blk_peek_request+0x220/0x25c)
[<c0243f30>] (blk_peek_request+0x220/0x25c) from
[<c0243f74>] (blk_fetch_request+0x8/0x1c)
[<c0243f74>] (blk_fetch_request+0x8/0x1c) from
[<c041cedc>] (mmc_queue_thread+0x58/0x120)
[<c041cedc>] (mmc_queue_thread+0x58/0x120) from
[<c00ad310>] (kthread+0x84/0x90)
[<c00ad310>] (kthread+0x84/0x90) from
[<c000eeac>] (kernel_thread_exit+0x0/0x8)

CRs-Fixed: 416466
Change-Id: I1fab93a4334b53e1d7c5dcc8f93cff174bae0d5e
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
block: Add support for reinsert a dispatched req

Add support for reinserting a dispatched request back to the
scheduler's internal data structures.
This capability is used by the device driver when it chooses to
interrupt the current request transmission and execute another (more
urgent) pending request. For example: interrupting long write in order
to handle pending read. The device driver re-inserts the
remaining write request back to the scheduler, to be rescheduled
for transmission later on.

Add API for verifying whether the current scheduler
supports reinserting requests mechanism. If reinsert mechanism isn't
supported by the scheduler, this code path will never be activated.

Change-Id: I5c982a66b651ebf544aae60063ac8a340d79e67f
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: Add API for urgent request handling

This patch add support in block & elevator layers for handling
urgent requests. The decision if a request is urgent or not is taken
by the scheduler. Urgent request notification is passed to the underlying
block device driver (eMMC for example). Block device driver may decide to
interrupt the currently running low priority request to serve the new
urgent request. By doing so READ latency is greatly reduced in read&write
collision scenarios.

Note that if the current scheduler doesn't implement the urgent request
mechanism, this code path is never activated.

Change-Id: I8aa74b9b45c0d3a2221bd4e82ea76eb4103e7cfa
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: urgent request: remove unnecessary urgent marking

An urgent request is marked by the scheduler in rq->cmd_flags with the
REQ_URGENT flag. There is no need to add an additional marking by
the block layer.

Change-Id: I05d5e9539d2f6c1bfa80240b0671db197a5d3b3f
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: blk-core: add printk_ratelimit

On failure of requests the console is flooded with prints,
which result in a watchdog bark.
This patch limits the number of prints to console thus, avoiding
the bark.

CRs-Fixed: 458071
Change-Id: I614002cc1305ab7b5c6a64d278139291c1baa3c4
Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
block: urgent request: Update dispatch_urgent in case of requeue/reinsert

The block layer implements a mechanism for verifying that the device
driver won't be notified of an URGENT request if there is already an
URGENT request in flight. This is due to the fact that interrupting an
URGENT request isn't efficient.
This patch fixes the above described mechanism in case the URGENT request
was returned back to the block layer from some reason: by requeue or
reinsert.

CRs-fixed: 473376, 473036, 471736
Change-Id: Ie8b8208230a302d4526068531616984825f1050d
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: urgent: Fix dispatching of URGENT mechanism

There are cases when blk_peek_request is called not from blk_fetch_request
thus the URGENT request may be started but the flag q->dispatched_urgent is
not updated.

Change-Id: I4fb588823f1b2949160cbd3907f4729767932e12
CRs-fixed: 471736
CRs-fixed: 473036
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: Remove "requeuing urgent req" error messages

It is possible for URGENT request to be requeued/reinserted if it was
fetched during the creation of a packed list. This end case is rare and is
not handled at the moment.
This patch changes the messages notifying of the above to debug level
(instead of error) in order to clear the dmesg log.

Change-Id: Ie8bc067e61559a6f702077b95c5dbcc426404232
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: add runtime pm helpers

Add runtime pm helper functions:

void blk_pm_runtime_init(struct request_queue *q, struct device *dev)
  - Initialization function for drivers to call.

int blk_pre_runtime_suspend(struct request_queue *q)
  - If any requests are in the queue, mark last busy and return -EBUSY.
    Otherwise set q->rpm_status to RPM_SUSPENDING and return 0.

void blk_post_runtime_suspend(struct request_queue *q, int err)
  - If the suspend succeeded then set q->rpm_status to RPM_SUSPENDED.
    Otherwise set it to RPM_ACTIVE and mark last busy.

void blk_pre_runtime_resume(struct request_queue *q)
  - Set q->rpm_status to RPM_RESUMING.

void blk_post_runtime_resume(struct request_queue *q, int err)
  - If the resume succeeded then set q->rpm_status to RPM_ACTIVE
    and call __blk_run_queue, then mark last busy and autosuspend.
    Otherwise set q->rpm_status to RPM_SUSPENDED.

The idea and API is designed by Alan Stern and described here:
http://marc.info/?l=linux-scsi&m=133727953625963&w=2

Change-Id: I486612af79c7887116f70a4fde552c107f7fa360
Signed-off-by: Lin Ming <ming.m.lin@intel.com>
Signed-off-by: Aaron Lu <aaron.lu@intel.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Git-commit: 6c9546675864f51506af69eca388e5d922942c56
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
block: implement runtime pm strategy

When a request is added:
    If device is suspended or is suspending and the request is not a
    PM request, resume the device.

When the last request finishes:
    Call pm_runtime_mark_last_busy().

When pick a request:
    If device is resuming/suspending, then only PM request is allowed
    to go.

The idea and API is designed by Alan Stern and described here:
http://marc.info/?l=linux-scsi&m=133727953625963&w=2

Change-Id: If9c8332164afba6aeca984277e3398008ad9c45a
Signed-off-by: Lin Ming <ming.m.lin@intel.com>
Signed-off-by: Aaron Lu <aaron.lu@intel.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Git-commit: c8158819d506a8aedeca53c52dfb709a0aabe011
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
blkpm: avoid sleep when holding queue lock

In blk_post_runtime_resume, an autosuspend request will be initiated for
the device. Since we are holding the queue lock, we can't sleep and thus
we should use the async version to initiate an autosuspend, i.e.
pm_request_suspend instead of pm_runtime_suspend, which might sleep.

Change-Id: I28cbe7e20ffdcb8f7b3329bf3b827c942dd1b45a
Signed-off-by: Aaron Lu <aaron.lu@intel.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Git-commit: c60855cdb976c632b3bf8922eeab8a0e78edfc04
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
block: Add URGENT request notification support to CFQ scheduler

When the scheduler reports to the block layer that there is an urgent
request pending, the device driver may decide to stop the transmission
of the current request in order to handle the urgent one. This is done
in order to reduce the latency of an urgent request. For example:
long WRITE may be stopped to handle an urgent READ.

Change-Id: I3072b8a1423870fed9c04c28d93caaf9557a7b89
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: Adding ROW scheduling algorithm

This patch adds the implementation of a new scheduling algorithm - ROW.
The policy of this algorithm is to prioritize READ requests over WRITE
as much as possible without starving the WRITE requests.

Change-Id: I4ed52ea21d43b0e7c0769b2599779a3d3869c519
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: ROW: Correct minimum values of ROW tunable parameters

The ROW scheduling algorithm exposes several tunable parameters.
This patch updates the minimum allowed values for those parameters.

Change-Id: I5ec19d54b694e2e83ad5376bd99cc91f084967f5
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: ROW: Fix forced dispatch

This patch fixes forced dispatch in the ROW scheduling algorithm.
When the dispatch function is called with the forced flag on, we
can't delay the dispatch of the requests that are in scheduler queues.
Thus, when dispatch is called with forced turned on, we need to cancel
idling, or not to idle at all.

Change-Id: I3aa0da33ad7b59c0731c696f1392b48525b52ddc
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block:row: fix idling mechanism in ROW

This patch addresses the following issues found in the ROW idling
mechanism:
1. Fix the delay passed to queue_delayed_work (pass actual delay
   and not the time when to start the work)
2. Change the idle time and the idling-trigger frequency to be
   HZ dependent (instead of using msec_to_jiffies())
3. Destroy idle_workqueue() in queue_exit

Change-Id: If86513ad6b4be44fb7a860f29bd2127197d8d5bf
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
row: Adding support for reinsert already dispatched req

Add support for reinserting already dispatched request back to the
schedulers internal data structures.
The request will be reinserted back to the queue (head) it was
dispatched from as if it was never dispatched.

Change-Id: I70954df300774409c25b5821465fb3aa33d8feb5
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
row: Add support for urgent request handling

This patch adds support for handling urgent requests.
ROW queue can be marked as "urgent" so if it was un-served in last
dispatch cycle and a request was added to it - it will trigger
issuing an urgent-request-notification to the block device driver.
The block device driver may choose at stop the transmission of current
ongoing request to handle the urgent one. Foe example: long WRITE may
be stopped to handle an urgent READ. This decreases READ latency.

Change-Id: I84954c13f5e3b1b5caeadc9fe1f9aa21208cb35e
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>