rbd: have __rbd_add_snap_dev() return a pointer

It's not obvious whether the snapshot pointer whose address is
provided to __rbd_add_snap_dev() will be assigned by that function.
Change it to return the snapshot, or a pointer-coded errno in the
event of a failure.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: make rbd_create_rw_ops() return a pointer

Either rbd_create_rw_ops() will succeed, or it will fail because a
memory allocation failed.  Have it just return a valid pointer or
null rather than stuffing a pointer into a provided address and
returning an errno.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: pass null version pointer in add_snap()

rbd_header_add_snap() passes the address of a version variable to
rbd_req_sync_exec(), but it ignores the result.  Just pass a null
pointer instead.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: always pass ops array to rbd_req_sync_op()

All of the callers of rbd_req_sync_op() except one pass a non-null
"ops" pointer.  The only one that does not is rbd_req_sync_read(),
which passes CEPH_OSD_OP_READ as its "opcode" and, CEPH_OSD_FLAG_READ
for "flags".

By allocating the ops array in rbd_req_sync_read() and moving the
special case code for the null ops pointer into it, it becomes
clear that much of that code is not even necessary.

In addition, the "opcode" argument to rbd_req_sync_op() is never
actually used, so get rid of that.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: fixes in rbd_header_from_disk()

This fixes a few issues in rbd_header_from_disk():
    - There is a check intended to catch overflow, but it's wrong in
      two ways.
	- First, the type we don't want to overflow is size_t, not
	  unsigned int, and there is now a SIZE_MAX we can use for
	  use with that type.
	- Second, we're allocating the snapshot ids and snapshot
	  image sizes separately (each has type u64; on disk they
          grouped together as a rbd_image_header_ondisk structure).
	  So we can use the size of u64 in this overflow check.
    - If there are no snapshots, then there should be no snapshot
      names.  Enforce this, and issue a warning if we encounter a
      header with no snapshots but a non-zero snap_names_len.
    - When saving the snapshot names into the header, be more direct
      in defining the offset in the on-disk structure from which
      they're being copied by using "snap_count" rather than "i"
      in the array index.
    - If an error occurs, the "snapc" and "snap_names" fields are
      freed at the end of the function.  Make those fields be null
      pointers after they're freed, to be explicit that they are
      no longer valid.
    - Finally, move the definition of the local variable "i" to the
      innermost scope in which it's needed.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: return obj version in __rbd_refresh_header()

Add a new parameter to __rbd_refresh_header() through which the
version of the header object is passed back to the caller.  In most
cases this isn't needed.  The main motivation is to normalize
(almost) all calls to __rbd_refresh_header() so they are all
wrapped immediately by mutex_lock()/mutex_unlock().

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: create rbd_refresh_helper()

Create a simple helper that handles the common case of calling
__rbd_refresh_header() while holding the ctl_mutex.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: kill rbd_init_watch_dev()

When an rbd image is mapped, a watch request is issued so that the
client will get notified in the event the image header object
changes.  This is done using rbd_init_watch_dev(), which calls
rbd_req_sync_watch().

rbd_init_watch_dev() is organized as a loop, arranging to re-issue
the request after refreshing the header if -ERANGE ever gets
returned.  But the only way rbd_req_sync_watch() will return -ERANGE
is if the CEPH_OSD_OP_WATCH operation returns that, which it will
not.

As a result, the whole looping structure and in fact the whole
function becomes excessive.  So get rid of rbd_init_watch_dev(),
and call rbd_req_sync_watch() directly in the one place it's used.

Signed-off-by: Alex Elder <elder@inktank.com>
rbd: __rbd_init_snaps_header() bug

There is a bug in the way __rbd_init_snaps_header() handles newly-
discovered snapshots, in the unusual case where the new snapshot id
is less than an existing rbd snapshot id.


When a new snapshot for an rbd image is created it is assigned a new
snapshot id.  The client requests the id from a monitor, and later
updates the osds with information about the snapshot.  The sequence
of snapshot ids is monotonically increasing, which guarantees each
is unique.  And typically all new snapshots will have higher ids
than any others a client has encountered.

Because of the delay between getting assigned a snapshot id and
recording the snapshot, there is a possible race between two clients
requesting new snapshots on the same rbd image.  It's possible in
this case for the client granted a higher id to be the first to
update the set of snapshots for the rbd image.  This leaves a sort
of gap in the set of snapshots until the other client's update gets
recorded.

This is an unlikely scenario, but it appears to be possible.  I'm
not sure whether we even support multiple concurrent clients for the
same rbd image.  However there is code in place to handle this case
and it's wrong, so it needs to be fixed.


The job of __rbd_init_snaps_header() is to compare an rbd's existing
list of snapshots to a new set, and remove any existing snapshots
that are not present in the new set and create any new ones not
present in the existing set.

If any new snapshots exist after the entire list of existing ones
has been examined, all the new ones that remain need to be created.
The logic for handling these last snapshots correct.

However, if a new snapshot is found whose id is less than an
existing one, the logic for handling this is wrong.  The basic
problem is that the name and current id used here are taken from
a place one position beyond where they should be.  In the event
this block of code is executed the first pass through the outer
loop, for example, these values will refer to invalid memory.

The fix is to make this block of code more closely match the
block that handles adding new snapshots at the end.  There are
three differences:
    - we use a do..while loop because we know we initially have an
      entry to process.
    - we insert the new entry at a position within the list instead
      of at the beginning
    - because we use it in the loop condition, we need to update
      our notion of "current id" each pass through.

Signed-off-by: Alex Elder <elder@inktank.com>