rgw: fix up signal handling

OMG libfcgi is annoying with shutdown and signals.  You need to close
the fd *and* resend a signal to ensure that you kick the accept loop
hard enough to make it shut down.

Document this, and switch to the async signal handlers.  Put them
tightly around the runtime loop as we do with other daemons.

Signed-off-by: Sage Weil <sage@inktank.com>
rgw: free resolver on shutdown

Signed-off-by: Sage Weil <sage@inktank.com>
rgw: fix leak of RGWDataChangesLog::renew_thread

Signed-off-by: Sage Weil <sage@inktank.com>
unit tests for sharedptr_registry

Covers 100% of the LOC and all the expected behavior, including thread
safety.

The sharedptr_registry is made friend of the test class so that it can
synthetize race conditions. The lookup and lookup_or_create methods
set the new in_method data member before calling cond.Wait() so that
the caller knows it is waiting.

http://tracker.ceph.com/issues/5527 refs #5527

Signed-off-by: Loic Dachary <loic@dachary.org>
replace in_method_t with a counter

A single counter ( waiting ) accurately reflects the number of
waiters, regardless of the method waiting. It is enough to allow
unit tests to synthetise all situations, including:

T1: x = lookup_or_create(0)
T1: release x part 1 (weak_ptrs now fail to lock)
T2: y = lookup_or_create(0)
T2: block in lookup_or_create (waiting == 1)
T1: z = lookup_or_create(1) (does not block because the key is different)
    while holding the lock it waiting++ and waiting == 2
    and before returning it waiting-- and waiting is back to == 1
T1: complete release x
T2: complete lookup_or_create(0) (waiting == 0)

The unit tests are modified to add a lookup on an unrelated key to
demonstrate that it does not reset waiting counter.

http://tracker.ceph.com/issues/5527 refs #5527

Signed-off-by: Loic Dachary <loic@dachary.org>
doc/changelog/v0.61.7

Signed-off-by: Sage Weil <sage@inktank.com>
Use dh_installinit to install upstart job files

Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
Merge pull request #425 from dachary/wip-5527

unit tests for sharedptr_registry

Reviewed-by: Samuel Just <sam.just@inktank.com>
osd: fix off-by-one in map gap logic

If we have map 250, and monitor's first is 251, but sends 260, we can
request the intervening range.

Fixes: #5784
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
osd: get initial full map after a map gap

If there is a gap in our map history, get the full range of maps that
the mon has.  Make sure the first one is a full map.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
WorkQueue: fix bracing on reset_tp_timeout

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
OSD: suspend tp timeout while taking pg lock in OpWQ

If N op_tp threads are configured, and recovery_max_active
is set to a sufficiently large number, all N op_tp threads
might grab a MOSDPGPush op off of the queue for the same PG.
The last thread to get the lock will have waited
N*time_to_handle_push before completing its item and pinging
the heartbeat timeout.  If that time exceeds the timeout
and there are enough ops waiting, each thread subsequently
will end up exceeding the timeout before completeing an
item preventing the OSD from heartbeating indefinitely.

We prevent this by suspending the timeout while we try to
get the PG lock.  Even if we do block for an excessive
period of time attempting to get the lock, hopefully,
the thread holding the lock will cause the threadpool
to time out.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
check_new_interval must compare old acting with old osdmap

When trying to establish if the old acting set is either empty or
smaller than the min_size of the osdmap,
pg_interval_t::check_new_interval compares with the min_size of the
new osdmap. Since the goal is to try to determine if the previous
interval may have been writeable, it should not enter the if when
there were not enough osds in the acting set ( i.e. < min_size ). But
it may enter it anyway if min_size was decremented in the new osdmap.

A complete set of unit tests were added to cover the logic of
check_new_interval. The parameters are prepared to describe a
situation where the function returns false (i.e. no new
interval). Each case is described in a separate bloc that introduces
the minimal changes to demonstrate the intended test case.

Because a number of cases have the same output while implementing a
different logic, the debug output is parsed to differentiate between them.

A test case demonstrating the problem ( check_new_interval must
compare old acting with old osdmap ) is added, with a link to the bug
number for future reference. The problem is fixed. The text of two
debug messages are slightly changed to make the maintenance of the
test that match them easier.

http://tracker.ceph.com/issues/5780 refs #5780

Signed-off-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
mon: fix xml element name

Should be the type, not the name.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PGMonitor: fix 'pg dump_[pools_]json'

Use the correct type for the dumpcontents arg.  Fixes the dump_pools_json
output and avoids these errors:

2013-07-29 13:09:14.089188 7fa0c5d21700 -1 0x7fa0c5d1e7a8
2013-07-29 13:09:16.306560 7fa0c5d21700 -1 bad boost::get: key dumpcontents is not type std::vector<std::string, std::allocator<std::string> >
2013-07-29 13:09:16.317104 7fa0c5d21700 -1 0x7fa0c5d1e7a8
2013-07-29 13:09:16.317136 7fa0c5d21700 -1 bad boost::get: key dumpcontents is not type std::vector<std::string, std::allocator<std::string> >

Fixes: #5786
Signed-off-by: Sage Weil <sage@inktank.com>
rgw: read / write bucket attributes from bucket instance

and not from bucket entry point.

Fixes: #5770

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
rgw: track bucket instance oid

We now keep the bucket instance oid in rgw_bucket. The reason
we need it is that the bucket might have been created before
the entrypoint / bucket instance separation.

Fixes: #5790

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
rgw: set bucket attrs are a bucket instance meta operation

Need to do the action through the bucket instance handler
and not through the bucket handler, otherwise it's wrongly
recorded (and wrongly replayed, ouch).

Fixes: #5791

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Merge pull request #463 from toabctl/upstart-handling

Use dh_installinit to install upstart job files

Reviewed-by: Gaudenz Steinlin <gaudenz@debian.org>
rgw: cors subresource needs to be in canonical auth header

Fixes: #5789

This was fixed before, however, might have been lost due to
recent refactoring + merge.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>