tests: Remove grep process entries from pidgrep

Problem:
We were picking process with lowest pid from ps|grep
result. However, lowest pid need not be oldest process
as recycling of PIDs can take place.

Solution:
Removed grep process entries from ps entries using 
grep -v grep.

Change-Id: I2b9687a05a34cf6358f773183770d69a3fb9eb10
BUG: 858488
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/4765
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsync: Display additional information in status command

Added code to display extra information when status command
is executed.
 Information shown now are
1 Number of files synced
2 crawl time
3 total sync time
4 bytes synced

bytes synced is taken from rsync output .
--stats option of rsync gives extra infor
mation about the sync.In stats output there
is a field called Total transferred file
size which states the ammount of bytes synced .
This information is parsed from stdout output
using regular expressions.Bytes synced information
can be used to calculate throughput.

Change-Id: Id9bba9fff45ee7049bb8257c6fd918e5237e05b1
BUG: 947774
Signed-off-by: sarvotham s pai <spai@redhat.com>
Reviewed-on: http://review.gluster.org/4749
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
object-storage: Import missing sys and errno modules.

Import the missing modules and implemented unit test case for Glusterfs module.
Thanks to Paul Smith for pointing it out.

Change-Id: Ib04202aa0ae05c4da2ebbf11f87d6accc778f827
BUG: 905946
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
object-storage: use tox for unit tests; fix em too

Add the ability to use tox for unit tests, since it helps us solve the
problem of supporting multiple branches that require different
versions of dependencies, and allows us to possibly support multiple
versions of python in the future.

Also fix the code to work with pre-grizzly environments, by not
requiring the constraints backport.

Also fixed the xattr support to work with both pyxattr and xattr
modules.

And fixed the ring tests to also work without a live /etc/swift
directory.

BUG: 948657 (https://bugzilla.redhat.com/show_bug.cgi?id=948657)
Change-Id: I2be79c8ef8916bb6552ef957094f9186a963a068
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4781
Reviewed-by: Alex Wheeler <wheelear@gmail.com>
Tested-by: Alex Wheeler <wheelear@gmail.com>
Reviewed-by: Anand Avati <avati@redhat.com>
storage/posix: introduce node-uuid-pathinfo

enabling this option has an effect on pathinfo xattr
request returning <node-uuid>:<path> instead of the
default - which is <hostname>:<path>.

Change-Id: Ice1b38abf8e5df1568bab6d79ec0d53dfa520332
BUG: 765380
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/4567
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
object-storage: rebase Swift to 1.8.0 (grizzly)

Change-Id: I24265c12a45eac4cec761748096118c9647440be
BUG: 948039
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/4779
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
afr: let eager-locking do its own overlap checks

Today there is a non-obvious dependence of eager-locking on
write-behind. The reason is that eager-locking works as long
as the inheriting transaction has no overlaps with any of the
transactions already in progress. While write-behind provides
non-overlapping writes as a side-effect most of times (and only
guarantees it when strict-write-ordering option is enabled,
which is not on by default) eager-lock needs the behavior
as a guarantee. This is leading to complex and unwanted checks
for the presence of write-behind in the graph, for the simple
task of checking for overlaps.

This patch removes the interdependence between eager-locking
and write-behind by making eager-locking do its own overlap checks
with in-progress writes.

Change-Id: Iccba1185aeb5f1e7f060089c895a62840787133f
BUG: 912581
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4782
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
glusterfsd: Cleanup temporary files from /tmp

For each gluster{d,fs,fsd} start, one or more temporary
file(s) created in /tmp were not being unlinked. This
patch cleans that up.

Modified a typo in an unrelated log message as well.

Change-Id: I3dec2a2ca40c7d6828eb238ec9cd08b6072cf0dd
BUG: 949327
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4786
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cli: Address a double free with volume info.

Crash is observed when volume info is performed on a
non-exisiting volume name and the output format is xml.

Change-Id: I88aa5d9dc954b1352f5cc3b5b38742c832bc1bb8
BUG: 949298
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4785
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: fix spurious regression test failures

Change-Id: I752aeb8e25f43281d2f5cf33d0ff5aeae49687e7
BUG: 764966
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4794
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
syncenv: be robust against spurious wake()s

In the current implementation, when the callers of synctasks perform
a spurious wake() of a sleeping synctask (i.e, an extra wake() soon
after a wake() which already woke up a yielded synctask), there is
now a possibility of two sync threacs picking up the same synctask.
This can result in a crash. The fix is to change ->slept = 0|1 and
membership of synctask in runqueue atomically.

Today we dequeue a task from the runqueue in syncenv_task(), but
reset ->slept = 0 much later in synctask_switchto() in an unlocked
manner -- which is safe, when there are no spurious wake()s.

However, this opens a race window where, if a second wake() happens
after the dequeue, but before setting ->slept = 0, it results in
queueing the same synctask in the runqueue once again, and get
picked up by a different synctask.

This is has been diagnosed to be the crashes in the regression tests
of http://review.gluster.org/4784. However that patch still has a
spurious wake() [the trigger for this bug] which is yet to be fixed.

Change-Id: I9b4b9dd5115d6e62ba45162ae90dd5e917a4f83d
BUG: 948686
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4795
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/distribute: Ignore non-participating subvols for layout checks

When subvols-per-directory is < available subvols, then there are layouts
which are not populated. This leads to incorrect identification of holes or
overlaps. We need to ignore layouts, which have err == 0, and start == stop.
In the current scenario (start == stop == 0).

Additionally, in layout-merge, treat missing xattrs as err = 0. In case of
missing layouts, anomalies will reset them.

For any other valid subvoles, err != 0 in case of layouts being zeroed out.

Also reverted back dht_selfheal_dir_xattr, which does layout calculation only
on subvols which have errors.

Change-Id: I9f57062722c9e8a26285e10675c31a78921115a1
BUG: 921408
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4668
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
mgmt/glusterd: enable valgrind usage even in non DEBUG build

* Till now running glusterfs processes were allowed to run in valgrind
  mode only when built with debug mode enabled.

Change-Id: I11e07ea2a4da4f82f70cdded6258a22d65d6db64
BUG: 922877
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/4688
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
tests: fix dependency on sleep in bug-874498.t

With the introduction of http://review.gluster.org/4784, there are
delays which breaks bug-874498.t which wrongly depends on healing
to finish within 2 seconds.

Fix this by using 'EXPECT_WITHIN 60' instead of sleep 2.

Change-Id: I2716d156c977614c719665a5e1f159dabf2878b5
BUG: 874498
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4796
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: changes in 'volume create' behaviour

This patch incorporates all the changes suggested on the behaviour of
'volume create' command in http://review.gluster.org/#change,4214
(comment #14, to be precise).

Change-Id: Iaac524a59738b177415595b18aa8a136090d3d25
BUG: 948729
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4740
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: fix further issues with bug-874498.t

The failure of bug-874498.t seems to be a "bug" in glustershd.
The situation seems to be when both subvolumes of a replica are
"local" to glustershd, and in such cases glustershd is sensitive
to the order in which the subvols come up.

The core of the issue itself is that, without the patch (#4784),
self-heal daemon completes the processing of index and no entries
are left inside the xattrop index after a few seconds of volume
start force. However with the patch, the stale "backing file"
(against which index performs link()) is left. The likely reason
is that an "INDEX" based crawl is not happening against the subvol
when this patch is applied.

Before #4784 patch, the order in which subvols came up was :

  [2013-04-09 22:55:35.117679] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-0: Connected to 10.3.129.13:49156, attached to remote volume '/d/backends/brick1'.
  ...
  [2013-04-09 22:55:35.118399] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-1: Connected to 10.3.129.13:49157, attached to remote volume '/d/backends/brick2'.

However, with the patch, the order is reversed:

  [2013-04-09 22:53:34.945370] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-1: Connected to 10.3.129.13:49153, attached to remote volume '/d/backends/brick2'.
  ...
  [2013-04-09 22:53:34.950966] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-0: Connected to 10.3.129.13:49152, attached to remote volume '/d/backends/brick1'.

The index in brick2 has the list of files/gfid to heal. It appears
to be the case that when brick1 is the first subvol to be detected
as coming up, somehow an INDEX based crawl is clearing all the
index entries in brick2, but if brick2 comes up as the first subvol,
then the backing file is left stale.

Also, doing a "gluster volume heal full" seems to leave out stale
backing files too. As the crawl is performed on the namespace and
the backing file is never encountered there to get cleared out.

So the interim (possibly permanent) fix is to have the script issue
a regular self-heal command (and not a "full" one).

The failure of the script itself is non-critical. The data files are
all healed, and it is just the backing file which is left behind. The
stale backing file too gets cleared in the next index based healing,
either triggered manually or after 10mins.

Change-Id: I5deb79652ef449b7e88684311e804a8a2aa4725d
BUG: 874498
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4798
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
build: really disable fusermount if you say so

There is no logic in configure.ac that provides a $disable_fusermount
variable. So, use the $enable_fusermount variable instead.

Follow-up-for: http://review.gluster.org/4773
Change-Id: I81cdbd0045409d0036438d542ca6dc1934f784e4
BUG: 948205
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/4803
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Sachidananda Urs <sacchi@gmail.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
build: add BuildRequires librdmacm-devel

See http://review.gluster.org/149

Installed librdmacm-devel RPM on the build server.

Change-Id: Iafb30e9cf1b15272b80329cc6141c7e92b43daee
BUG: 950083
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/4804
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
glusterd: Fixed spurious wakeups in glusterd syncops

glusterd syncops perform a barrier_wake whenever rpc_clnt_submit returned -1.
This is based on the wrong assumption that the cbkfn wasn't called.
This would result in one more wakeup than there ought to be.

Change-Id: I591e67c267f0e26d1145bf8fb5feeb2c13a751a1
BUG: 948686
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4802
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
object-storage: rebase Swift to 1.8.0 (grizzly)

Two minor tweaks found while packaging 3.4.0-0.1.alpha2 for Fedora 19

BUG: 948039
Change-Id: I97175636164702cf4042bc4a18ffead76ad386cb
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/4807
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>