transfer_len for scsi_cmd, set together with resid; unified resid handling for iscsi/iser

The current code is a bit messy about handling allocation length, actual
transfer length, and residual counts in underflow and overflow conditions.
This patch attempts to tune up the exiting APIs so that their semantics follows
the relevant specs through the entire course of a command execution.

It is important to distinguish here between two layers: Transport and SCSI.
Transport is implemented as LLDs (iSCSI, iSER, FCoE etc.) in tgt architecture.
SCSI is comprised of SPC, SBC etc., jointly covered by "Device Types" and
"Backing Stores" in tgt.

Residual counts for underflow and overflow conditions are relevant only for
the Transport layer. For example, iSCSI spec (10.4. "SCSI Response") defines
them relatively to "Expected Data Transfer Length" (EDTL):
* overflow:
  "initiator's EDTL was not sufficient for the number of bytes to be transfered"
* underflow:
  "bytes that were not transferred out of expected number of bytes (EDTL)".

EDTL is communicated by the Transport (e.g. in iSCSI Command PDU) and provides
a normalized indication (which does not require CDB parsing) about the buffer
length allocated by the initiator. This is important because all CDBs also
contain similar counters, but their format varies with the opcode and does
require CDB parsing.

While EDTL reflects the expectation of the initiator, the actual number of
bytes to be transferred is produced by SCSI layer. It is regulated by specs like
SPC, SBC etc. There is a fundametal difference between the method used by
SPC (plus some commands from other groups) vs. SBC for handling buffer lengths.

SBC directly specifies TRANSFER_LENGTH, counted in logical blocks.
Any event in which the actual number of blocks to be transferred differs from
the expected value is defined as underflow or overflow, both for READ and WRITE
commands.

SPC commands convey ALLOCATION_LENGTH, counted in bytes.
For READ SPC commands SCSI layer should generate DATA-IN buffer and communicate
available length to the initiator in an appropriate field of the response
data structure, contained in the generated DATA-IN.

If the available DATA-IN length is longer than the allocation length, it is
truncated, but this does not change those length-carrying fields,
as mandated by SPC-3 4.3.5.6:
 "If the information being transferred to the Data-In Buffer includes
  fields containing counts of the number of bytes in some or all of the data,
  then the contents of these fields shall not be altered to reflect the
  truncation, if any, that results from an insufficient ALLOCATION LENGTH value,
  unless the standard that describes the Data-In Buffer format states otherwise"

Such truncation is the main reason for having non-zero residual counts in SPC.
In SPC there is usually no residual counts for WRITE commands.

Below is an algorithm for handling READ-type SPC cmds (like REPORT_LUNS,
INQUIRY etc.) with clear separation between transport and scsi layers.

1. Transport layer receives a SCSI cmd PDU, carrying CDB and EDTL
2. Transport layer passes cmd to SCSI layer:
   * scsi_execute_spc_command (cdb, edtl)
   2.1. extract alloc_len from cdb
   2.2. if (edtl < alloc_len)
           then sense(ILLEGAL_REQUEST,ASC_INVALID_FIELD_IN_CDB)
   2.3. generate Data-IN buffer, avail_len bytes long
   2.4. actual_len = MIN (alloc_len, avail_len)
   /* Note that avail_len remains internal to SCSI SPC layer, although it is
      contained in DATA-IN; Transport receives only actual_len value */
   2.5. set actual_len for IN direction
3. Transport layer is requested to transfer data_in
   * transport_send_data_in (actual_len)
   3.1. if (actual_len == edtl)
           then { resid = 0; }
   3.2. if (actual_len < edtl)
           then { resid = edtl - actual_len; underflow = 1; }
   3.3. if (actual_len > edtl)
           then { resid = actual_len - edtl; overflow = 1; }

In SPC READ cmds overflow is impossible as it would imply edtl < alloc_len,
which is responded with sense. Still it is (at least in theory) possible with
SBC commands (like READ-10).

With SBC only step 2. of the above algorithm actually differs, as there
is no alloc_len, but transfer_len (in blocks) instead, and the actual_len is
normally received from the backing store device. With emulated devices some
scenarios producing residual counts are impossible. When working with
generic SCSI devices, as in the case of SG/BSG, all scenarios are possible.

Currently, iscsid.c always truncates the case of READ overflow (resid < 0)
and sets resid = 0.

As explained above this is correct behavior with SPC cmds (no overflows),
but overflows should not have been produced by SPC module in the first place,
while with READ SBC commands overflow MUST be reported.

Current iser.c implementation behaves according to the spec, but because
the SCSI layer sometimes does not, it may occasionally produce incorrect
responses. Linux initiator disregards them, but more pedantic initiators,
like VmWare's ESX, throws errors.

This patch basically forces SCSI layer to set only the actual transfer len
as described in section 2.5. and to implement residual+overflow calculations as
described in section 3. of the above algorithm entirely in the Transpor layer.

It unifies functions setting final residual counts and over/underflow
indicators for iscsi and iser LLDs.

New bi-dir field "transfer_len" is defined for scsi cmd descriptor.
By default it is assumed that EDTL will be actually transferred.
If actual_len differs from the expected then SCSI layer must call
scsi_set_in_resid_by_actual() or scsi_set_out_resid_by_actual().
These functions always existed but now they set both the transfer len
and residual count.

This patch relies on correct behavior of SCSI layer, as described above.
Thus it is followed by a series of patches correcting this and other related
aspects.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc_memcpy: safe memcpy for SPC-type cmds tracking remaining space

To assist safe generation of data-in buffer for SPC-type read commands
a new utility function is defined:

int spc_memcpy(uint8_t *dst, uint32_t *dst_remain_len,
               uint8_t *src, uint32_t src_len)

Copies up to src_len bytes from src, making sure not to exceed *dst_remain_len
bytes available at dst. Returns actually copied length, updates remaining space
in *dst_remain_len.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for INQUIRY cmd

Generate inquiry data-in in a separate buffer and safe-copy it to the command
buffer taking into account the allocation length.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for REPORT_LUNS cmd

Generate report_luns data-in directly in the command buffer, copying LUNs
one by one while taking into account the allocation length. Set the actual
transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for REPORT_SUPPORTED_OPERATION_CODES cmd

Generate data-in in a separate buffer and safe-copy it to the command buffer
taking into account allocation length. Set the actual transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for MOD_SENSE cmd

Generate mod_sense data-in directly in the command buffer, building mode pages
one by one while taking into account the allocation length. Set the actual
transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for READ_KEYS action of PERSISTENT_RESERVE_IN

Generate read_keys data-in directly in the command buffer, building registered
keys records one by one while taking into account the allocation length.
Set the actual transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for READ_RESERVATION of PERSISTENT_RESERVE_IN

Generate reservation data-in in a separate buffer and safe-copy it to the
command buffer taking into account the allocation length. Set the actual
transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for REPORT_CAPABILITIES of PERSISTENT_RESERVE_IN

Generate PGR caps data-in in a separate buffer and safe-copy it to the
command buffer taking into account the allocation length. Set the actual
transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
spc: fix data-in buffer generation for REQUEST_SENSE cmd

Fix allocation len extraction from CDB. Calculate and set the actual
transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
sbc: fix data-in buffer generation for READ_CAPACITY(16)

READ_CAPACITY(16) is an SBC command behaving in SPC-like manner.
Generate data-in in a separate buffer and safe-copy it to the command buffer
taking into account the allocation length.
Set the actual transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
sbc: fix data-in buffer generation for GET_LBA_STATUS

GET_LBA_STATUS is an SBC command behaving in SPC-like manner.
Generate data-in directly in the command buffer, building data and hole records
one by one, while taking into account the allocation length.
Set the actual transfer len correctly.

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
bs_sg: fix residual count handling for sg and bsg, account for direction

Signed-off-by: Alexander Nezhinsky <nezhinsky@gmail.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Add support for Ceph rbd backing store

Ceph is distributed storage clustering software (http://ceph.com).
This allows tgtd to use the Ceph "RADOS block device" (rbd) as
backing store for a LUN.

Ceph storage is divided into pools, and each rbd image can have
multiple readonly snapshots, so the parameters to fully specify
an image include poolname, imagename, and snapshot name.

--bstype is 'rbd'
--backing-store path is [pool/]imagename[@snapname]
Up to 20 simultaneous images supported
All I/O is synchronous, 16 threads in worker pool by default
Based on the 'rdwr' bs driver

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Add documentation for rbd backend

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Add README.rbd to explain/document usage with Ceph rbd backing storage.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
make Ceph rbd support optional

I guess that there are lots of people who doesn't install ceph library
so (including me). So this patch makes the rbd support optional (like
iSER).

To enable Ceph rbd support, compile tgtd with:

    make CEPH_RBD=1

Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>