replicate: don't stop checking xattrs because one was absent

The functional issue is described by the subject line.  This patch also
addresses several efficiency/structure issues, such as...

* Calling dict_set_ptr once for each txn type, instead of once overall.

* Calling afr_index_for_transaction_type once per iteration instead of
  once per call (or better yet zero since the conversion is unnecessary).

* Implementation of inner functions in a different file than their one
  caller, creating a spurious header-file dependency.

Change-Id: I29e0df906a820533b66b9ced73e015dfe77267d2
BUG: 865825
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4070
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: Replace mount -t glusterfs with glusterfs -s

Change-Id: I9aa9bd1efb9ae56caec0fee596b3f21ca496f66e
BUG: 863068
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4239
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
cli: XML output for "gluster volume geo-replication status"

Change-Id: I1a64bd5eb9b1040a2a2d9b97bfe9cc756835596e
BUG: 864506
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4227
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests/bug-865825: turn stat-prefetch off before doing any afr self-heal
                  related tests.

Change-Id: I5c45bb4f4d4aa59965c6dda34ab884071bef9c6f
BUG: 865825
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4242
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: handle GF_XATTR_LOCKINFO_KEY appropriately.

values from all children need to be aggregated into a dictionary
and serialized buffer of this aggregated dictionary has to be
the value of GF_XATTR_LOCKINFO_KEY in the dict sent as a result of
fgetxattr.

Change-Id: Ie877f7c637c07feaee4c44d7ef86aa967a17b7e7
BUG: 808400
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4121
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
mount/fuse: migrate posix locks after a graph-switch

Each posix-lock is associated with an fd and a transport.
After a graph switch, this lock-state has to be associated
with new fd and transport corresponding to new client graph.

Change-Id: Ia0855e15600c85ef902bf612738f7d96557145be
BUG: 808400
Signed-off-by: Raghavendra <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4122
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/distribute: send getxattr on LOCKINFO to only cached subvolumes.

lk is sent to only cached subvolume. Hence there is no point in
sending LOCKINFO to other children (even in case of directories).

Change-Id: Ia20fc358dfa84cee9a52d1f613564ff6f25aa0c9
BUG: 808400
Signed-off-by: Raghavendra <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4123
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
libglusterfs/mem-pool: implement gf_memdup

Change-Id: I1d7a1e7418f424fe08c041d562098b947c01580f
BUG: 808400
Signed-off-by: Raghavendra <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4124
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/stripe: handle GF_XATTR_LOCKINFO_KEY in f(get)(set)xattr

Change-Id: I4463006a7f54c05e757d877c56e1330fd91aec45
BUG: 808400
Signed-off-by: Raghavendra <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4125
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
mount/fuse: add test cases for lock migration

Change-Id: I1a48c63387c7803b36584070fd8f3684ccb696b1
BUG: 808400
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4209
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/locks: implement fgetxattr and fsetxattr

implement xattrs for GF_XATTR_LOCKINFO_KEY, which will be used
for posix-locks migration from old to new graph after a switch.

fgetxattr (fd, GF_XATTR_LOCKINFO_KEY) will return a dict.
This dict has a serialized dict stored for key
GF_XATTR_LOCKINFO_KEY. This serialized dict in turn has fdnum
value of locks acquired on this fd with modified pathinfo
(containing hostname and base directory components) as key.

fsetxattr (newfd, GF_XATTR_LOCKINFO_KEY, dict) has following
semantics. dict can be the result of a previous fgetxattr with
GF_XATTR_LOCKINFO_KEY. In that case, a dict_get on dict
constructed using serialized buffer is done on modified pathinfo
as key. If a value is got, that value is treated as fdnum and for
every lock l on newfd->inode we do,
      if (l->fdnum == fdnum) {
         l->fdnum = fd_fdnum (newfd);
         l->transport = <connection identifier of connection on which fsetxattr came>;
      }

Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Change-Id: I73a8f43aa0b6077bc19f8de52205ba748f2d8bbe
BUG: 808400
Reviewed-on: http://review.gluster.org/4120
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
glusterd: volume-sync shouldn't validate volume-id

- volume sync would overwrite volume information on local node
  from the hostname supplied. This warning is provided to the user.

- Also fixed a double free in volume-sync handler.

Change-Id: Icc68d9d563fb50ca58d5880921f063692e1e6882
BUG: 865700
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4188
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
protocol/client: add an option to filter O_DIRECT flag in open

with the option, the idea is all client-side caching will be disabled,
where as on server side process, the fd will be treated as a regular
fd, thus helping the performance better.

"gluster volume set <VOLNAME> remote-dio enable" would set
this option in client protocol volumes.

Change-Id: Id2255a167137f8fee20849513e3011274dc829b4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 845213
Reviewed-on: http://review.gluster.org/4206
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
geo-replication: catch select.error on select()

tailer() in resource.py does not correctly catch exceptions from
select(). select() can raise an instance of the select.error class and
the current expression only catches ValueError (and the instance will
have reference called selecterror).

The geo-rep log contains a call trace like this:
> E [syncdutils:190:log_raise_exception] <top>: FAIL:
> Traceback (most recent call last):
> File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 216, in twrap
> tf(*aa)
> File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 123, in tailer
> poe, _ ,_ = select([po.stderr for po in errstore], [], [], 1)
> File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 276, in select
> return eintr_wrap(oselect.select, oselect.error, *a)
> File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 269, in eintr_wrap
> return func(*a)
> error: (9, 'Bad file descriptor')

BUG: 880308
Change-Id: I2babe42918950d0e9ddb3d08fa21aa3548ccf7c5
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/4233
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Csaba Henk <csaba@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
libgfapi: "tcp" is documented as the valid transport type instead of "socket".

Change-Id: I066e9be7755a535f385c102cdd1822adeda7f319
BUG: 860203
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4226
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Protected conf->xprt_list racy access.

- epoll on RPCSVC_EVENT_ACCEPT would add corresponding xprt
  onto the xprt_list. Concurrently, synctask thread (volume op)
  would call into glusterd_fetchspec_notify which iterates on
  the xprt_list. Added a mutex to protect such a racy access of
  the list.

Change-Id: Idc51b4bdb1c814dfab7790e1c899d6977f7640f2
BUG: 878873
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4241
Reviewed-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd, cli: implement gluster system uuid reset command

A commonly faced problem among glusterfs users is: after a fresh
installation of glusterfs in a virtual machine, the VM image is
cloned to make multiple instances of the server. This breaks
glusterd because right after glusterfs installation on the first
boot glusterd would have created the node UUID and this gets
inherited into the clone. The result is wierd behavior at the time
of peer probe where glusterd does not (yet) deal with UUID
collisions in a user friendly way.

To handle it gluster peer reset command is implemented which upon
execution changes the uuid of local glusterd.

Change-Id: If207dd2ad93ab94ef1a3253f409c21c442975f87
BUG: 811493
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/3637
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
geo-rep / gsyncd,glusterd: do not hardcode socket path

... in gsyncd python code. Indeed, use the configuration
mechanism to set it suitably from glusterd.

Change-Id: I9fe2088b14d28588d1e64fe892740cc5755b8365
BUG: 868877
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.org/4143
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/dht: send ACCESS call on dir to first_up_subvol if cached is down

Change-Id: I4f518a969bbe3a11075e7c9ae10bd21bf059d5f3
BUG: 867253
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4240
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
afr: handle short writes in afr_writev_wind and self-heal to avoid corruption

The current failure to handle short writes on writev fops leaves
us open to file corruption. A short write on a user request is
ignored and leaves replicas in an inconsistent state. A short write
during a self-heal is ignored and incorrectly marks the files as
consistent if the heal completes.

Modify user writev handling to return the best case return value
from each of the replicas. Short writes that occur relative to this
value are marked as failed and will require a heal. Modify
self-heal to set an error on a short write and abort the heal.

BUG: 853690
Change-Id: I18b30f58702326249230eeebb361b29e40b535f5
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/4150
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>