blk-mq: improve support for shared tags maps

This adds support for active queue tracking, meaning that the
blk-mq tagging maintains a count of active users of a tag set.
This allows us to maintain a notion of fairness between users,
so that we can distribute the tag depth evenly without starving
some users while allowing others to try unfair deep queues.

If sharing of a tag set is detected, each hardware queue will
track the depth of its own queue. And if this exceeds the total
depth divided by the number of active queues, the user is actively
throttled down.

The active queue count is done lazily to avoid bouncing that data
between submitter and completer. Each hardware queue gets marked
active when it allocates its first tag, and gets marked inactive
when 1) the last tag is cleared, and 2) the queue timeout grace
period has passed.

Signed-off-by: Jens Axboe <axboe@fb.com>
block: move bio.c and bio-integrity.c from fs/ to block/

They really belong in block/, especially now since it's not in
drivers/block/ anymore. Additionally, the get_maintainer script
gets it wrong when in fs/.

Suggested-by: Christoph Hellwig <hch@infradead.org>
Acked-by: Al Viro <viro@ZenIV.linux.org.uk>
Signed-off-by: Jens Axboe <axboe@fb.com>
block: move ioprio.c from fs/ to block/

Like commit f9c78b2b, move this block related file outside
of fs/ and into the core block directory, block/.

Signed-off-by: Jens Axboe <axboe@fb.com>
blk-mq: move the cache friendly bitmap type of out blk-mq-tag

We will use it for the pending list in blk-mq core as well.

Signed-off-by: Jens Axboe <axboe@fb.com>
blk-mq: switch ctx pending map to the sparser blk_align_bitmap

Each hardware queue has a bitmap of software queues with pending
requests. When new IO is queued on a software queue, the bit is
set, and when IO is pruned on a hardware queue run, the bit is
cleared. This causes a lot of traffic. Switch this from the regular
BITS_PER_LONG bitmap to a sparser layout, similarly to what was
done for blk-mq tagging.

20% performance increase was observed for single threaded IO, and
about 15% performanc increase on multiple threads driving the
same device.

Signed-off-by: Jens Axboe <axboe@fb.com>
Merge branch 'for-3.16/blk-mq-tagging' into for-3.16/core

Signed-off-by: Jens Axboe <axboe@fb.com>

Conflicts:
	block/blk-mq-tag.c
block: move mm/bounce.c to block/

Continue moving some of the block files that are scattered around.
bounce.c contains only code for bouncing the contents of a bio.
It's block proper code, not mm code.

Suggested-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
htmldocs: fix bio.c location

Commit f9c78b2be2ca moved bio.c from fs/ to block/, but didn't
update the docbook location. Fix that up.

Signed-off-by: Jens Axboe <axboe@fb.com>
Revert "NVMe: CPU hot plug notification"

This reverts commit 33b1e95c90447ea73e37e837ea0268a894919f19.

Rebalancing of nvme queues can be taken care of by blk-mq.
Revert "NVMe: per-cpu io queues"

This reverts commit 42f614201e80ff4cfb8b285d7190149a8e1e6cec.

blk-mq allocated hctx on to each separate queues. Let blk-mq handle this
in the future.
Revert "NVMe: Retry failed commands with non-fatal errors"

This reverts commit edd10d33283899fb15d99a290dcc9ceb3604ca78.

This can now be tracked by nvme_timeout in the future.
Revert "NVMe: IOCTL path RCU protect queue access"

This reverts commit 4f5099af4f3d5f999d8ab7784472d93e810e3912.

Reimplement on top on blk-mq conversion.
Revert "NVMe: RCU protected access to io queues"

This reverts commit 5a92e700af2e5e0e6404988d6a7f2ed3dad3f46f.

Reimplment on top of blk-mq conversion.
NVMe: convert to blk-mq
NVMe: init scatter/gather lists

Forgot to initialize the scatter/gatter list. This result in protection
fault when the mappings are set up with 2 or more segments.
NVMe: remove jiffies from iod

Timeouts are handled by the block layer. No neep to keep track within
the iod structure.
NVMe: Enable timeout handling again

timeouts handling where disabled while debugging timeouts that occurred
after a command was completed successfully.
NVMe: no need for reserved tags in current impl.
NVMe: fix flush handling to work with requests
NVMe: remove obsolete todo