virtio_blk: allow blk-mq to merge scatter gather lists

As it's now possible to have arbitrary number of segments, mq-capable
block drivers should set BLK_MQ_F_SG_MERGE in q->queue_flags.
Especially virtio_blk must do it, __blk_segment_map_sg() may create
too large sg lists. When QUEUE_FLAG_NO_SG_MERGE turned on, such sg lists
could later trigger BUG_ON(total_sg > vq->vring.num) in virtqueue_add().

[   26.763457] ------------[ cut here ]------------
[   26.764147] kernel BUG at /home/mlin/linux/drivers/virtio/virtio_ring.c:160!
[   26.765010] invalid opcode: 0000 [#1] PREEMPT SMP
[   26.765273] Modules linked in:
[   26.765273] CPU: 3 PID: 25 Comm: kworker/u8:1 Not tainted 3.18.0-00030-g141405f #73
[   26.765273] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5.1-0-g8936dbb-20141113_115728-nilsson.home.kraxel.org 04/01/2014
[   26.765273] Workqueue: writeback bdi_writeback_workfn (flush-254:0)
[   26.765273] task: ffff88001ebf0950 ti: ffff88001ebfc000 task.ti: ffff88001ebfc000
[   26.765273] RIP: 0010:[<ffffffff81314359>]  [<ffffffff81314359>] virtqueue_add_sgs+0x70/0x2fe
[   26.765273] RSP: 0000:ffff88001ebff6a8  EFLAGS: 00010002
[   26.765273] RAX: 0000000000000000 RBX: ffff88001e1ad000 RCX: ffffea0000698540
[   26.765273] RDX: ffffea0000698500 RSI: ffff88001ebff7c8 RDI: ffff88001ebff748
[   26.765273] RBP: ffff88001ebff708 R08: ffff88001d911330 R09: 0000000000000020
[   26.765273] R10: 0000160000000000 R11: ffff88001d7f0000 R12: ffff88001ebff7c8
[   26.765273] R13: 0000000000000002 R14: 0000000000000082 R15: 0000000000000003
[   26.777232] FS:  0000000000000000(0000) GS:ffff88001fd80000(0000) knlGS:0000000000000000
[   26.777232] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   26.777232] CR2: 00007fd84f9e5000 CR3: 000000001cf6a000 CR4: 00000000000006e0
[   26.777232] Stack:
[   26.777232]  ffff88000000007f ffff880000000020 ffff88001ebff7d8 ffff88001d911330
[   26.777232]  01ffea0000000003 0000007f00000080 0000000000000418 0000000000000000
[   26.777232]  ffff88001ebff748 ffff88001d911360 0000000000000002 ffff88001d911330
[   26.777232] Call Trace:
[   26.777232]  [<ffffffff81346b3b>] __virtblk_add_req+0x137/0x149
[   26.777232]  [<ffffffff812c027e>] ? part_round_stats+0x52/0x59
[   26.777232]  [<ffffffff812c6c2f>] ? __blk_segment_map_sg+0xbd/0x190
[   26.777232]  [<ffffffff812c6e07>] ? blk_rq_map_sg+0xb6/0x1cc
[   26.777232]  [<ffffffff81346c9b>] virtio_queue_rq+0x14e/0x1ee
[   26.777232]  [<ffffffff812ca138>] __blk_mq_run_hw_queue+0x1ac/0x2ad
[   26.777232]  [<ffffffff812ca7f2>] blk_mq_run_hw_queue+0x3d/0x77
[   26.777232]  [<ffffffff812caf81>] blk_mq_insert_requests+0x103/0x156
[   26.777232]  [<ffffffff812cbdd5>] blk_mq_flush_plug_list+0xeb/0xfa
[   26.777232]  [<ffffffff812c365f>] blk_flush_plug_list+0xb8/0x1e1
[   26.777232]  [<ffffffff812c379e>] blk_finish_plug+0x16/0x38
[   26.777232]  [<ffffffff811af1f9>] ext4_writepages+0x973/0xb66
[   26.777232]  [<ffffffff810dd9ff>] do_writepages+0x1e/0x2c
[   26.777232]  [<ffffffff81145118>] __writeback_single_inode+0x84/0x298
[   26.777232]  [<ffffffff8106aa0d>] ? wake_up_bit+0x25/0x2a
[   26.777232]  [<ffffffff8114638f>] writeback_sb_inodes+0x1fc/0x33d
[   26.777232]  [<ffffffff81146544>] __writeback_inodes_wb+0x74/0xb9
[   26.777232]  [<ffffffff811466d5>] wb_writeback+0x14c/0x2eb
[   26.777232]  [<ffffffff810dc7b1>] ? global_dirty_limits+0x1b/0x117
[   26.777232]  [<ffffffff81146b92>] bdi_writeback_workfn+0x1f6/0x40c
[   26.777232]  [<ffffffff8104ded7>] process_one_work+0x1ca/0x376
[   26.777232]  [<ffffffff8104e319>] worker_thread+0x267/0x366
[   26.777232]  [<ffffffff8104e0b2>] ? process_scheduled_works+0x2f/0x2f
[   26.777232]  [<ffffffff81051e7c>] kthread+0xd2/0xda
[   26.777232]  [<ffffffff81490000>] ? ldsem_down_write+0x6/0x19f
[   26.777232]  [<ffffffff81051daa>] ? kthread_freezable_should_stop+0x48/0x48
[   26.777232]  [<ffffffff81490b6c>] ret_from_fork+0x7c/0xb0
[   26.777232]  [<ffffffff81051daa>] ? kthread_freezable_should_stop+0x48/0x48

Signed-off-by: Ming Lin <mlin@minggr.net>
[dpark: added a precise description]
Signed-off-by: Dongsu Park <dongsu.park@profitbricks.com>
block: plugging

Signed-off-by: Kent Overstreet <kmo@daterainc.com>
[dpark: fix compile errors]
md: get rid of bio_list from struct task
blk-mq: remove also remaining calls to blk_attempt_plug_merge()
block: fix panic in generic_make_request()

bio should be added to splits.
Fixed below panic.

[    1.112557] BUG: unable to handle kernel NULL pointer dereference at           (null)
[    1.112563] IP: [<ffffffff812bc2d3>] generic_make_request+0x63/0x9d
[    1.112564] PGD 0
[    1.112566] Oops: 0002 [#1] PREEMPT SMP
[    1.112568] Modules linked in:
[    1.112570] CPU: 3 PID: 530 Comm: kworker/u8:2 Not tainted 3.18.0-00033-ge76e0b3 #41
[    1.112571] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
[    1.112575] Workqueue: events_unbound async_run_entry_fn
[    1.112576] task: ffff88001d8eae90 ti: ffff88001dad0000 task.ti: ffff88001dad0000
[    1.112579] RIP: 0010:[<ffffffff812bc2d3>]  [<ffffffff812bc2d3>] generic_make_request+0x63/0x9d
[    1.112579] RSP: 0000:ffff88001dad3798  EFLAGS: 00010246
[    1.112580] RAX: 0000000000000000 RBX: ffff88001d8eae90 RCX: 0000000000000008
[    1.112581] RDX: ffff88001dad3798 RSI: 0000000000000000 RDI: ffff88001dad3798
[    1.112582] RBP: ffff88001dad37d8 R08: ffff88001e2908b8 R09: 000000000000000d
[    1.112583] R10: 000000000000000d R11: 000000000000000c R12: ffff88001dad3798
[    1.112583] R13: 0000000000000000 R14: 0000000000000000 R15: ffffea000078cb80
[    1.112585] FS:  0000000000000000(0000) GS:ffff88001fd80000(0000) knlGS:0000000000000000
[    1.112586] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    1.112588] CR2: 0000000000000000 CR3: 000000000180e000 CR4: 00000000000006e0
[    1.112593] Stack:
[    1.112595]  0000000000000000 0000000000000000 ffff88001dad37a8 ffff88001dad37a8
[    1.112597]  ffff88001dad37b8 ffff88001dad37b8 ffff88001e290840 0000000000000008
[    1.112599]  ffff88001dad3838 ffffffff812bc404 ffff88001dad3808 0000000481059f5e
[    1.112599] Call Trace:
[    1.112603]  [<ffffffff812bc404>] submit_bio+0xf7/0x136
[    1.112607]  [<ffffffff8114a1b9>] _submit_bh+0x104/0x12e
[    1.112610]  [<ffffffff8114a1f3>] submit_bh+0x10/0x12
[    1.112612]  [<ffffffff8114a4be>] block_read_full_page+0x24a/0x266
[    1.112614]  [<ffffffff8114be05>] ? I_BDEV+0xd/0xd
[    1.112618]  [<ffffffff812e8bd2>] ? debug_smp_processor_id+0x17/0x19
[    1.112621]  [<ffffffff810dd817>] ? __lru_cache_add+0x72/0x88
[    1.112623]  [<ffffffff8114c4e8>] ? blkdev_readpages+0x1f/0x1f
[    1.112625]  [<ffffffff8114c500>] blkdev_readpage+0x18/0x1a
[    1.112628]  [<ffffffff810d2f1f>] do_read_cache_page+0x97/0x16f
[    1.112630]  [<ffffffff810d3013>] read_cache_page+0x1c/0x1e
[    1.112633]  [<ffffffff812c98cf>] read_dev_sector+0x2d/0x87
[    1.112636]  [<ffffffff812cc283>] read_lba+0xb1/0x105
[    1.112638]  [<ffffffff812cc6e4>] ? find_valid_gpt+0x80/0x4d4
[    1.112640]  [<ffffffff812cc705>] find_valid_gpt+0xa1/0x4d4
[    1.112642]  [<ffffffff812dd430>] ? string.isra.6+0x3d/0xa2
[    1.112645]  [<ffffffff812ccbb4>] efi_partition+0x7c/0x364
[    1.112647]  [<ffffffff812de3b4>] ? vsnprintf+0x82/0x3a7
[    1.112649]  [<ffffffff812de77f>] ? snprintf+0x34/0x36
[    1.112651]  [<ffffffff812ca773>] check_partition+0x11e/0x1c3
[    1.112653]  [<ffffffff812c9e53>] rescan_partitions+0x98/0x271
[    1.112657]  [<ffffffff814818c8>] ? _raw_spin_lock+0x18/0x37
[    1.112659]  [<ffffffff8114d32b>] __blkdev_get+0x1c7/0x3f6
[    1.112660]  [<ffffffff814818c8>] ? _raw_spin_lock+0x18/0x37
[    1.112663]  [<ffffffff8114d5b8>] blkdev_get+0x5e/0x2a9
[    1.112666]  [<ffffffff8113513b>] ? unlock_new_inode+0x5c/0x61
[    1.112668]  [<ffffffff8114c3cb>] ? bdget+0x120/0x12f
[    1.112670]  [<ffffffff81330a9e>] ? put_device+0x17/0x19
[    1.112672]  [<ffffffff812c7e17>] add_disk+0x286/0x3f9
[    1.112676]  [<ffffffff8136ab9f>] sd_probe_async+0x106/0x19d
[    1.112678]  [<ffffffff81053a9c>] async_run_entry_fn+0x71/0x138
[    1.112681]  [<ffffffff8104d603>] process_one_work+0x1c7/0x36f
[    1.112683]  [<ffffffff8104e090>] worker_thread+0x28b/0x37f
[    1.112685]  [<ffffffff8104de05>] ? rescuer_thread+0x226/0x226
[    1.112687]  [<ffffffff810519a8>] kthread+0xdb/0xe3
[    1.112689]  [<ffffffff81480000>] ? __ww_mutex_lock_slowpath+0xcc/0x1c3
[    1.112691]  [<ffffffff810518cd>] ? kthread_create_on_node+0x17f/0x17f
[    1.112693]  [<ffffffff8148226c>] ret_from_fork+0x7c/0xb0
[    1.112695]  [<ffffffff810518cd>] ? kthread_create_on_node+0x17f/0x17f

Signed-off-by: Ming Lin <mlin@minggr.net>
direct-io: Rewrite based on immutable biovecs

Near total rewrite of fs/direct-io.c. This makes use of our new bio
splitting functionality, and the fact that generic_make_request() will
take arbitrary size bios - we allocate a bio, pin pages to it directly,
then call the getblocks() function to map it wherever the filesystem
tells us - splitting as needed.

Appears to pass xfstests with CONFIG_XFS_DEBUG=y (would appreciate testing from
someone who uses xfstests more than I).

Doesn't quite work with btrfs yet - when running xfstests it eventually get
stuck in an infinite loop somewhere. First thing in the kernel log is a warning
at fs/btrfs/ordered-data.c:288 in btrfs_add_ordered_sum() - if any of the btrfs
people want to take a look that would be a great help.

It _definitely_ needs review/auditing from someone intimately familiar with the
expected behaviour w.r.t. everything related to filesystem semantics and the
handling of the getblocks() call - dio_send_bio() is what primarily implements
that behaviour.

Signed-off-by: Kent Overstreet <kmo@daterainc.com>
dio: use submit_io() for block zeroing
block: kill bio_get_nr_vecs()

Signed-off-by: Kent Overstreet <kmo@daterainc.com>
btrfs: fix compile error on calling btrfs_dio_bio_alloc()
foo

Signed-off-by: Kent Overstreet <kmo@daterainc.com>
more dio rewriting
fs/dio: wait for completion for dio inside the loop

Fix compile errors, by waiting for completion for dio inside loop,
as well as checking dio->async instead of dio->refcount.