performance/write-behind: guarantee non-overlapping concurrent writes

Maintain a list of writes (either written behind or SYNC) which are
currently "in progress" (i.e, STACK_WIND'ed towards server) and hold
off any new STACK_WIND of write (either written behind or SYNC) which
overlaps with any of the "in progress" writes.

This is a guarantee which AFR's eager-lock depends upon (though not
strictly a write-behind requirement)

Change-Id: Icedd0b51b440366a906dc9223d62b7fd6ef2ca03
BUG: 857673
Original-author: Anand Avati <avati@redhat.com>
Signed-off-by: Anand Avati <avati@redhat.com>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4642
Tested-by: Gluster Build System <jenkins@build.gluster.com>
gfapi: dict_unref() xattr_req in fop finish instead of dict_destroy()

The current way of calling dict_destroy() at the end of an API fop assumes
that xattr_req is not stored/ref'd by any translators in the stack. However
when translators like DHT store xattr_req in dht_local_t with a dict_ref()
and perform dict_unref() in the unwind path, things get subject to a race.

The race is between the woken up thread (by syncop_wake) i.e, the gfapi
invoking thread and the thread where the FOP was unwound. As the C stack
of STACK_UNWIND unwinds back, dht_local_unref() gets invoked within the
DHT_STACK_UNWIND macro. This thread attempts dict_unref, which would
be "safe" if it wins the race against the gfapi invoking thread. However
if the gfapi invoking thread wins the race, it will perform dict_destroy()
first and therefore make dict_unref() within dht_local_unref perform
a double free.

This is the embarrassing on-screen bug which showed up in a roomful of people
during the gluster dev summit demo of qemu/libgfapi integration.

Change-Id: I284c93de87cdc128d5801f42c84aa87f753090d4
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4645
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/distribute: Fix layout overlaps due to spread-count in selfheal path

We needed to zero out the layout range, before we re-calculate the range.
When spread-count is issued, we would end up with stale ranges in the layout.

Replaced dht_selfheal_dir_xattr with dht_fix_dir_xattr, which correctly resets
the un-used (after re-cal) layouts.

Change-Id: I1a900d15df07335f59356bd23182ccec34381ab2
BUG: 884455
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4648
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
rpm: package /var/run/gluster so that statedumps can be created

Creating statedumps fail when /var/run/gluster does not exist. This
directory should be part of the 'glusterfs' package that is installed on
storage servers and native clients.

Merged Niels's change from both $HEAD and release-3.3

BUG: 917554
Change-Id: I6ffc497c0bb6bc90c97a91a72bba9118853d4c8c
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/4659
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
mgmt/glusterd: Start fs-crawl in separate thread so as not to block epoll

tests/basic/quota.t covers test case for this.

Patch is only for 3.4 branch, http://review.gluster.org/4495 fixes the issue
in master.

Change-Id: I92674f5413441cc896245d5b3d0925f44ce8b2d3
BUG: 919998
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/4680
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>