build: require /usr/bin/fusermount when not carrying our own version

The fuse.so from glusterfs-fuse will try to call /usr/bin/fusermount.
This obviously fails when the fuse package is not installed and
fusermount is not available.

In order to prevent this problem, the glusterfs-fuse package should
require /usr/bin/fusermount so that it gets automatically pulled in when
glusterfs-fuse is installed with yum.

BUG: 947830
Change-Id: I20fe836a1aaf751dbc04d9ec4ba5ea50573c71c5
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/4768
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/distribute: Start rebalance with option readdir-optimize on

With readdir-optimize set to on, we instruct the posix layer to ignore
directory entries from not first subvolume. DHT discards directories
returned from non first subvolume. By making posix itself ignore it,
we are making directory crawls faster

Change-Id: Ia1faf2dedec0c615c0632c3c063e846f5742ede6
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4613
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
core: add dispatch table for init/fini

This adds a layer of indirection so that derivative translators such as
NUFA and switch can refer to the parent's init/fini (in both cases DHT's)
without having to create stub functions.

Change-Id: I1af1fea70a9ddd2aa20485af7ae65f9660f19dd6
BUG: 924490
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4709
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
dht: make nufa/switch call dht's init/fini

These functions keep changing as new functionality is added, so copying
and pasting the code is not a good solution.  This way ensures that all
fields get initialized properly no matter how much new stuff we throw in.

Change-Id: I9e9b043d2d305d31e80cf5689465555b70312756
BUG: 924488
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4710
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
protocol/client: Print valid loc identifiers

Change-Id: I45f91105862a2484b8906a7a63b98ab4aaf80d05
BUG: 924643
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4683
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
build: enable fusermount by default

The fusermount available in gluster is customized to ensure
mounting with SELinux happens properly, i.e - to have a separate
thread for fuse_thread_proc which can process getxattr requests
and in parallel perform sys_mount() in a different thread, thereby
avoiding a deadlock.

However our build and packaging defaults to not including our
fusermount. This patch reverses the defaults.

Change-Id: I793af4c2f56aeac46efae3db30e7c64ee7c18850
BUG: 811217
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4773
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
pump: Set self-heal readdir size in pump

Problem:
In Pump entry self-heal happens for each directory during the
first opendir using conservative merge. But in entry-self-heal
readdir is issued with '0' size. So entry self-heal is not
creating any files. After pump thinks entry self-heal is complete
it proceeds to heal each of the file in the directory it just
healed. Fortunately most of the times it chooses source-brick
in pump as read-child for readdir. This happens because readchild is the
subvolume on which lookup succeeds first. In pump lookup succeeds
faster in local process than on the destination brick process most
of the times. For all the entries pump finds in readdir it does a
lookup. During this lookup the entry on the destination brick is
created and healed. This is the reason why replace-brick
succeeds whenever read-child for the directory is chosen as the
source-brick.  Which is most of the times. When read-child is chosen
as the destination brick, readdir returns no entries so replace-brick
completes without syncing the whole data.

Fix:
Set readdir-size in pump so that entry self-heal happens with
64k size. This ensures that entry self-heal triggered from
opendir actually creates the files on the destination brick.

Change-Id: I65ea45d3c2735a9578f3aa34eff771b6563241ca
BUG: 909800
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4712
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
dict: Put "goto out" in dict_unserialize to avoid process crash

Problem:
In the dictionary serialization function, if the
[(buf + vallen) > (orig_buf + size)], then memdup is getting failed.

Fix:
Put "goto out" whenever this condition is met.

Change-Id: I662628a936596dbb47825aad47d7dbab2879eb07
BUG: 947824
Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com>
Reviewed-on: http://review.gluster.org/4767
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Made afr_sh_purge_entry_common message log more clear.

FIX:
In missing entry self heal, once the source directories are determined
after the lookup and if file is not present on any of the brick which
contains the souce directory, the entry is removed from the directory.
So log message should give information of "Purging of entry".

Change-Id: I4d3deb602e0812dc1c9c8ba0a466716d81dede7e
BUG: 947312
Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com>
Reviewed-on: http://review.gluster.org/4753
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
posix: fix dangerous "sharing" of fd in readdir between two requests

posix_fill_readdir() is a multi-step function which performs many
readdir() calls, and expects the directory cursor to have not
"seeked away" elsewhere between two successive iterations. Usually
this is not a problem as each opendir() from an application has its
own backend fd, and there is nobody else to "seek away" the directory
cursor. However in case of NFS's use of anonymous fd, the same fd_t
is shared between all NFS readdir requests, and two readdir loops can
be executing in parallel on the same dir dragging away the cursor in
a chaotic manner.

The fix in this patch is to lock on the fd around the loop. Another
approach could be to reimplement posix_fill_readdir() with a single
getdents() call, but that's for another day.

Change-Id: Ia42e9c7fbcde43af4c0d08c20cc0f7419b98bd3f
BUG: 948086
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4774
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/afr: Treat all dir fop failure as success in changelog

For example:
If a new entry creation fop fails with EEXIST or a delete entry fop
fails with ENOENT, on all the subvols the fop is wound, then no
change took place to the directory. So we can treat that case as no
change happened to the directory.

Change-Id: I3b3a7931954da2166a9cba19ff9f76f37739d751
BUG: 860210
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4626
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: Remove grep process entries from pidgrep

Problem:
We were picking process with lowest pid from ps|grep
result. However, lowest pid need not be oldest process
as recycling of PIDs can take place.

Solution:
Removed grep process entries from ps entries using 
grep -v grep.

Change-Id: I2b9687a05a34cf6358f773183770d69a3fb9eb10
BUG: 858488
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/4765
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsync: Display additional information in status command

Added code to display extra information when status command
is executed.
 Information shown now are
1 Number of files synced
2 crawl time
3 total sync time
4 bytes synced

bytes synced is taken from rsync output .
--stats option of rsync gives extra infor
mation about the sync.In stats output there
is a field called Total transferred file
size which states the ammount of bytes synced .
This information is parsed from stdout output
using regular expressions.Bytes synced information
can be used to calculate throughput.

Change-Id: Id9bba9fff45ee7049bb8257c6fd918e5237e05b1
BUG: 947774
Signed-off-by: sarvotham s pai <spai@redhat.com>
Reviewed-on: http://review.gluster.org/4749
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
object-storage: Import missing sys and errno modules.

Import the missing modules and implemented unit test case for Glusterfs module.
Thanks to Paul Smith for pointing it out.

Change-Id: Ib04202aa0ae05c4da2ebbf11f87d6accc778f827
BUG: 905946
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
object-storage: use tox for unit tests; fix em too

Add the ability to use tox for unit tests, since it helps us solve the
problem of supporting multiple branches that require different
versions of dependencies, and allows us to possibly support multiple
versions of python in the future.

Also fix the code to work with pre-grizzly environments, by not
requiring the constraints backport.

Also fixed the xattr support to work with both pyxattr and xattr
modules.

And fixed the ring tests to also work without a live /etc/swift
directory.

BUG: 948657 (https://bugzilla.redhat.com/show_bug.cgi?id=948657)
Change-Id: I2be79c8ef8916bb6552ef957094f9186a963a068
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4781
Reviewed-by: Alex Wheeler <wheelear@gmail.com>
Tested-by: Alex Wheeler <wheelear@gmail.com>
Reviewed-by: Anand Avati <avati@redhat.com>
storage/posix: introduce node-uuid-pathinfo

enabling this option has an effect on pathinfo xattr
request returning <node-uuid>:<path> instead of the
default - which is <hostname>:<path>.

Change-Id: Ice1b38abf8e5df1568bab6d79ec0d53dfa520332
BUG: 765380
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/4567
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
object-storage: rebase Swift to 1.8.0 (grizzly)

Change-Id: I24265c12a45eac4cec761748096118c9647440be
BUG: 948039
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/4779
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
afr: let eager-locking do its own overlap checks

Today there is a non-obvious dependence of eager-locking on
write-behind. The reason is that eager-locking works as long
as the inheriting transaction has no overlaps with any of the
transactions already in progress. While write-behind provides
non-overlapping writes as a side-effect most of times (and only
guarantees it when strict-write-ordering option is enabled,
which is not on by default) eager-lock needs the behavior
as a guarantee. This is leading to complex and unwanted checks
for the presence of write-behind in the graph, for the simple
task of checking for overlaps.

This patch removes the interdependence between eager-locking
and write-behind by making eager-locking do its own overlap checks
with in-progress writes.

Change-Id: Iccba1185aeb5f1e7f060089c895a62840787133f
BUG: 912581
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4782
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
glusterfsd: Cleanup temporary files from /tmp

For each gluster{d,fs,fsd} start, one or more temporary
file(s) created in /tmp were not being unlinked. This
patch cleans that up.

Modified a typo in an unrelated log message as well.

Change-Id: I3dec2a2ca40c7d6828eb238ec9cd08b6072cf0dd
BUG: 949327
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4786
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cli: Address a double free with volume info.

Crash is observed when volume info is performed on a
non-exisiting volume name and the output format is xml.

Change-Id: I88aa5d9dc954b1352f5cc3b5b38742c832bc1bb8
BUG: 949298
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4785
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>