blk-mq: call again finish_wait(bs->wait) to avoid kernel crash

commit 2a5429fc0ab7 ("blk-mq: Micro-optimize bt_get()") introduced a
bug that occasionally causes the whole kernel to crash like following.

This occurs because of a list corruption of bs->wait.task_list[]
in the while loop of bt_get(). Previously finish_wait() was called
always at the end of while loop, so that task_list[] could be cleaned
up. Even if block IOs are submitted between prepare_wait() and
io_schedule(), task_list was not corrupted.

Since finish_wait() is not called any more, however, there's now a small
window that blk_sq_make_request() can end up with waking up bs->wait.
Then the wait queue wakes up with clearing tags.
Call chain looks like below.

blk_mq_end_request()
 -> blk_mq_free_request()
  -> __blk_mq_free_request()
   -> blk_mq_put_tag()
    -> bt_clear_tag()
     -> wake_up()

blk_sq_make_request()
 -> blk_mq_map_request()
  -> blk_mq_get_tag()
   -> __blk_mq_get_tag()
    -> bt_get()
     -> prepare_to_wait()

Hence fix it by adding back finish_wait() at the end of while loop.

This but is reproducible by running xfstests/generic/113.
Backtrace with CONFIG_DEBUG_LIST turned on is like that:

------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at lib/list_debug.c:59 __list_del_entry+0xbd/0xe0() list_del corruption.
prev->next should be ffff880036adf758, but was ffff880035cf4140
Call Trace:
 <IRQ>
 [<ffffffff817e9ee2>] dump_stack+0x4e/0x68
 [<ffffffff8108c9d1>] warn_slowpath_common+0x81/0xa0
 [<ffffffff8108ca36>] warn_slowpath_fmt+0x46/0x50
 [<ffffffff8138f0dd>] __list_del_entry+0xbd/0xe0
 [<ffffffff810c2cdb>] autoremove_wake_function+0x2b/0x40
 [<ffffffff810c2685>] __wake_up_common+0x55/0x90
 [<ffffffff810c28c7>] ? __wake_up+0x27/0x60
 [<ffffffff810c28dd>] __wake_up+0x3d/0x60
 [<ffffffff81352058>] bt_clear_tag+0xb8/0xe0
 [<ffffffff81352910>] blk_mq_put_tag+0x30/0x60
 [<ffffffff8134ef21>] __blk_mq_free_request+0x41/0xd0
 [<ffffffff8134efdd>] blk_mq_free_hctx_request+0x2d/0x30
 [<ffffffff8134f014>] blk_mq_free_request+0x34/0x40
 [<ffffffff8134f5e4>] blk_mq_end_request+0x54/0x70
 [<ffffffffa0035152>] virtblk_request_done+0x32/0x80 [virtio_blk]
 [<ffffffff8134e0d3>] __blk_mq_complete_request_remote+0x13/0x20
 [<ffffffff810f7b5a>] generic_smp_call_function_single_interrupt+0x7a/0x170
 [<ffffffff81030807>] smp_call_function_single_interrupt+0x27/0x40
 [<ffffffff817f4b6f>] call_function_single_interrupt+0x6f/0x80
 <EOI>
 [<ffffffff8100c7e6>] ? default_idle+0x26/0xe0
 [<ffffffff8100c7e4>] ? default_idle+0x24/0xe0
 [<ffffffff8100d17f>] arch_cpu_idle+0xf/0x20
 [<ffffffff810c32c4>] cpu_startup_entry+0x254/0x3f0
 [<ffffffff8103114b>] start_secondary+0x14b/0x160
---[ end trace 0c42b4a28805925e ]---
------------[ cut here ]------------
WARNING: CPU: 3 PID: 4419 at lib/list_debug.c:29 __list_add+0x70/0xd0() list_add corruption.
next->prev should be prev (ffff880036529dd0), but was ffff880036adf758. (next=ffff880036adf758).
Call Trace:
 [<ffffffff817e9ee2>] dump_stack+0x4e/0x68
 [<ffffffff810d35d2>] ? console_unlock+0x292/0x4c0
 [<ffffffff8108c9d1>] warn_slowpath_common+0x81/0xa0
 [<ffffffff8108ca36>] warn_slowpath_fmt+0x46/0x50
 [<ffffffff8138efc0>] __list_add+0x70/0xd0
 [<ffffffff810c2a39>] prepare_to_wait+0x79/0x80
 [<ffffffff81106e70>] ? __delayacct_blkio_end+0x60/0xa0
 [<ffffffff813524b0>] bt_get+0xc0/0x1c0
 [<ffffffff8134f3f4>] ? blk_mq_queue_enter+0x74/0x210
 [<ffffffff810c2cb0>] ? abort_exclusive_wait+0xb0/0xb0
 [<ffffffff813528b7>] blk_mq_get_tag+0xa7/0xd0
 [<ffffffff8134e2db>] __blk_mq_alloc_request+0x1b/0x1d0
 [<ffffffff8134fd11>] blk_mq_map_request+0x201/0x260
 [<ffffffff81351107>] blk_sq_make_request+0x87/0x3a0
 [<ffffffff811e1219>] ? do_blockdev_direct_IO+0x1079/0x2e50
 [<ffffffff81343078>] generic_make_request+0xd8/0x130
 [<ffffffff81343138>] submit_bio+0x68/0x160
 [<ffffffff810c85dd>] ? trace_hardirqs_on+0xd/0x10
 [<ffffffff811e1ecb>] do_blockdev_direct_IO+0x1d2b/0x2e50
 [<ffffffffa01d2fe0>] ? xfs_get_blocks+0x20/0x20 [xfs]
 [<ffffffff811e303c>] __blockdev_direct_IO+0x4c/0x50
 [<ffffffffa01d2fe0>] ? xfs_get_blocks+0x20/0x20 [xfs]
 [<ffffffffa01d1090>] ? xfs_finish_ioend_sync+0x20/0x20 [xfs]
 [<ffffffffa01d1270>] xfs_vm_direct_IO+0xc0/0x140 [xfs]
 [<ffffffffa01d2fe0>] ? xfs_get_blocks+0x20/0x20 [xfs]
 [<ffffffffa01d1090>] ? xfs_finish_ioend_sync+0x20/0x20 [xfs]
 [<ffffffff81143855>] generic_file_direct_write+0xb5/0x190
 [<ffffffffa01dfaf6>] xfs_file_dio_aio_write+0x146/0x360 [xfs]
 [<ffffffffa01dff80>] ? xfs_file_buffered_aio_write+0x270/0x270 [xfs]
 [<ffffffffa01e008d>] xfs_file_write_iter+0x10d/0x130 [xfs]
 [<ffffffffa01dff80>] ? xfs_file_buffered_aio_write+0x270/0x270 [xfs]
 [<ffffffff811f0b97>] aio_run_iocb+0x1f7/0x3a0
 [<ffffffff8116ba5e>] ? might_fault+0x5e/0xc0
 [<ffffffff8116ba5e>] ? might_fault+0x5e/0xc0
 [<ffffffff811f1b95>] do_io_submit+0x2b5/0x4c0
 [<ffffffff811f1ab0>] ? do_io_submit+0x1d0/0x4c0
 [<ffffffff811f1db0>] SyS_io_submit+0x10/0x20
 [<ffffffff817f3852>] system_call_fastpath+0x12/0x17
---[ end trace 0c42b4a28805925f ]---

Cc: Bart Van Assche <bvanassche@acm.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Robert Elliott <elliott@hp.com>
Cc: Ming Lei <ming.lei@canonical.com>
Cc: Alexander Gordeev <agordeev@redhat.com>
Cc: Jens Axboe <axboe@fb.com>
Signed-off-by: Dongsu Park <dongsu.park@profitbricks.com>