Create readme.md

Description of the lightnvm project
block: expose init_request_from_bio

When doing internal commands within the a device driver, we like to form
it as a bio, and use the existing infrastructure for handling bio's.
Exposing init_request_from_bio as blk_init_request_from_bio, it allows
us to initialized the request with a bio that we have initialized.
NVMe: Convert to blk-mq

This converts the NVMe driver to a blk-mq request-based driver.

The NVMe driver is currently bio-based and implements queue logic within itself.
By using blk-mq, a lot of these responsibilities can be moved and simplified.

The patch is divided into the following blocks:

 * Per-command data and cmdid have been moved into the struct request field.
   The cmdid_data can be retrieved using blk_mq_rq_to_pdu() and id maintenance
   are now handled by blk-mq through the rq->tag field.

 * The logic for splitting bio's has been moved into the blk-mq layer. The
   driver instead notifies the block layer about limited gap support in SG
   lists.

 * blk-mq handles timeouts and is reimplemented within nvme_timeout(). This both
   includes abort handling and command cancelation.

 * Assignment of nvme queues to CPUs are replaced with the blk-mq version. The
   current blk-mq strategy is to assign the number of mapped queues and CPUs to
   provide synergy, while the nvme driver assign as many nvme hw queues as
   possible. This can be implemented in blk-mq if needed.

 * NVMe queues are merged with the hardware dispatch queue (hctx) structure of
   blk-mq.

 * blk-mq takes care of setup/teardown of nvme queues and guards invalid
   accesses. Therefore, RCU-usage for nvme queues can be removed.

 * IO tracing and accounting are handled by blk-mq and therefore removed.

 * Setup and teardown of nvme queues are now handled by nvme_[init/exit]_hctx.

Contributions in this patch from:

  Sam Bradshaw <sbradshaw@micron.com>
  Jens Axboe <axboe@kernel.dk>
  Keith Busch <keith.busch@intel.com>
  Robert Nelson <rlnelson@google.com>

Acked-by: Keith Busch <keith.busch@intel.com>
Acked-by: Jens Axboe <axboe@fb.com>
lightnvm: extend rq_flag_bits

The rq_flag_bits is extended by

REQ_NVM

Used to detect request have through LightNVM on submission, and can be
detected on completion.

REQ_NVM_MAPPED

Used to detect if request have mapped appropriately through LightNVM.

The latter is added temponarily to debug the LightNVM key-value target
and will be removed later.
lightnvm: A host-side driver for Open-Channel SSDs

Open-channel SSDs are devices which exposes direct access its physical
flash storage, while keeping a subset of the internal features of SSDs.

A common SSD consists of a flash translation layer (FTL), bad block
management, and hardware units such as flash controller and host
interface controller and a large amount of flash chips.

LightNVM moves part of the FTL responsibility into the host, allowing
the host to manage data placement, garbage collection and parallelism. The
device still maintains information about bad block management, and implements a
simpler FTL, that still allows extensions such as atomic IOs, metadata
persistence and similar.

The architecture of LightNVM consists of a core and multiple targets. The core
has the parts of the driver that is shared across targets, initialization and
teardown and statistics. The targets are how physical flash are exposed to
user-land. This can be as the block device, key-value store, object-store, or
anything else.

LightNVM is currently hooked up through the null_blk and NVMe driver. The NVMe
extension allow development using the LightNVM-extended QEMU implementation,
using Keith Busch's qemu-nvme branch.
lightnvm: NVMe integration

NVMe devices are identified by the vendor specific bits:

Bit 3 in OACS (device-wide). Currently made per device, as the nvme
namespace is missing in the completion path.
Bit 1 in DSM (per-namespace).

The OACS change can be removed when the namespace is resolvable from the
completion path.

From there, the NVMe specification is extended with the following
commands:

LightNVM Identify
LightNVM Channel identify
LightNVM Synchronious/Asynchronious erase
LightNVM Get Features
LightNVM Set Responsibility
LightNVM Get Logical to Physical map
LightNVM Get Physical to Logical map

The NVMe integration can be tested using Keith Busch NVMe qemu simulator
Lwith ightNVM patches on top. This can be found at:
https://github/LightNVM/qemu-nvme
lightnvm: null_blk integration

Allows the null_blk driver to hook into LightNVM for performance
evaluation. The number of channels exposed to LightNVM can be configured
through the lightnvm_num_channels module parameter.
lightnvm: ATA/SCSI integration

LightNVM is integrated into the ATA/SCSI stack by devices identifying
themselves as LightNVM devices.

ATA devices should export bit 2 as set in its Data Set Management byte.
SCSI devices should set bit 1 at offset 0x44 in its VPD b0 page.

Currently the patch limits the incoming I/O size to 4K. This is done to allow
LightNVM to rewrite the destination address to the appropriate physical
flash address. A future approach is to extend the IO size to incorporate
appropriate mapping techniques.

For evaluation, the OpenSSD Jasmine board can be used. The source for a LightNVM
compatible firmware can be found at: https://github.com/LightNVM/OpenSSDJasmine