cluster/dht: Ignore subvols with error in min-free-disk/inodes

Currently when selecting a alternative subvolume when hashed
subvol has exceeded min-free-disk/inodes, we do not check if
layouts have errors (including decommissioning). This leads
to data being written to those subvolumes, and in case of
decommissioning, will lead to data loss.

Change-Id: Ie0c6cf4a29d7c53d8a6d8a8c1bd595cf58a0012a
BUG: 982919
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5299
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
test-ffop: the result of (O_TRUNC|O_RDONLY) is unspecified

On RHEL-6 an open(.., O_TRUNC|O_RDONLY) fails. test-ffop expects this to
succeed, but the behaviour is unspecified according to 'man 2 open':

       O_TRUNC
              If  the  file  already exists and is a regular file and the open
              mode allows writing (i.e., is O_RDWR or  O_WRONLY)  it  will  be
              truncated to length 0.  If the file is a FIFO or terminal device
              file, the O_TRUNC flag is  ignored.   Otherwise  the  effect  of
              O_TRUNC is unspecified.

Change-Id: I76b55419871fc758cbe82532f9b75476a0c14101
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5268
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
performance/io-cache: check for non-null gfid before calling inode_path

A new non-linked inode is added to lru list. Hence it might be possible
that gfid might be NULL when inode_dump is called. To pass asserts in
inode_path, we've to check for non-null gfid before invoking that
procedure.

Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Change-Id: Iff14efc6d6e2faa33b9f7a81e0a66f6a947b77ed
BUG: 976189
Reviewed-on: http://review.gluster.org/5241
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: Don't use stripe-replicate volume in bug-905864.t

Today, stripe doesn't follow a deterministic order in sending posix
locks to its subvolume. This may lead to dead locks. To avoid
intermittent test failures, we modify bug-905864.t to run on replicate
volume.

Change-Id: I38b72f30bc31576d3ca642f0029cc8749314b1dd
BUG: 905864
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/5072
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
mount/fuse: expose 'glusterfs.gfid*' virtual xattr key

currently two keys are exposed:

'glusterfs.gfid' : output is 16byte binary gfid
'glusterfs.gfid.string' : output is 36 byte canonical format of gfid

e.g.
[root@supernova glusterfs]# getfattr -n glusterfs.gfid -e hex f0
glusterfs.gfid=0x68305acb73e541719804fcf36a4857e8

[root@supernova glusterfs]# getfattr -n glusterfs.gfid.string f0
glusterfs.gfid.string="68305acb-73e5-4171-9804-fcf36a4857e8"

early consumers for this key would be geo-replication
(as it has being designed to do namespace operations on
gfid from the mount point, thereby needing the GFID for
entry operations on the slave).

Change-Id: I10b23dbd11628566ad6924334253f5d85d01a519
BUG: 847839
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/5129
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: Correct op-version of some options

New options being introduced in the master branch should now have
op-version set to the GD_OP_VERSION_MAX (3). Some of the options have
been backported to release-3.3 branch and hence should have their
op-version reduced. Some other options had op-version incorrectly set as
1.

Change-Id: If40325b7b2da7aa36f90261024117cd18cf51ef0
BUG: 981278
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5318
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Give up biglock before brick's rpc unref

This is to prevent the possibility of a deadlock when
rpc_connection_cleanup being called in the same thread as rpc_clnt_unref

Change-Id: Ia4dcc0a8a6e6158d4ddec68b780fccbc4cd64adb
BUG: 962619
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/5321
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/dht: If linkfile unlink fails with ENOTCONN, do not fail

Currently if linkfile fails with ENOENT, we do not fail. We also
need to treat failures with ENOTCONN as success, as if cached subvol
is up, rm of a file should succeed. A stale linkfile will get removed
later

Change-Id: I71d136847933351ed9e2c939bda4a69bc96a3cfc
BUG: 983416
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5317
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/dht: Unlink dst file after cleanup during migration

If a rename happens during migration, unlink fails. This leads
to stale xattrs, and Sticky bits still being set.

By removing the xattrs and Sticky bits from dst (through fd ops),
the stale link file would be cleaned up eventually (even if unlink
fails on src).

Change-Id: Iec537d021905438327a20e1d811aa06e74034364
BUG: 983399
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5316
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli: check for null in is_server_debug_xlator()

Command: gluster volume set <volname> diagnostics.client-log-level trace
Expected output:
"volume set: failed: option log-level trace: 'trace' is not valid
(possible options are DEBUG, WARNING, ERROR, INFO, CRITICAL, NONE,
TRACE.)"
Current output: gluster cli receives a segmentation fault
Fix: check for NULL before calling strstr

Change-Id: If4c7a85a635849a388cf122543e12349c109643c
BUG: 982174
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/5298
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
mount/fuse: Provide option to use/not use kernel-readdirp

By default fuse kernel readdirp usage in fuse xlator is off.
When mount option use-readdirp=yes is provided it starts using
fuse-kernel's readdirp.

Change-Id: Id37edc53b1adc1638186d956c2f74c1e4e48aa59
BUG: 983477
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5322
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/*: get logic to calculate min() of the 'stime' xattr

* in both distribute and replicate (ignoring stripe for now),
  add logic to calculate the min() of stime values.

* What is a 'stime' ? Why is this required:
  -  stime means 'slave xtime', mainly used to keep track of slave
  node's sync status when distributed geo-replication is used.
  Logic of calculating 'min()' for this stime is very important as
  in case of crashes/reboots/shutdown, we will have to 'restart'
  with crawling from stime time value from the mount point, which
  gives the 'min()' of all the bricks, which means, we don't miss
  syncing any files in the above cases.

Change-Id: I2be8d434326572be9d4986db665570a6181db1ee
BUG: 847839
Original Author: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4893
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Adjust configure.ac to honor existing CFLAGS env variable

Change-Id: I43a8defc7a8c0c2d391e7bd2c0b7bebfcc522a2a
BUG: 961892
Signed-off-by: Justin Clift <jclift@redhat.com>
Reviewed-on: http://review.gluster.org/4979
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/dht: node-uuid for directories winds to all subvolumes

this works similar to pathinfo now except that the request is sent
to all subvolumes of dht. Underlying replica selects it's subvolume
in a round-robin fashion till one of them returns successfully.

Change-Id: Ie46c5f7090d04d8c2e487b209916ae6791e94624
BUG: 847839
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/5225
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
libxlator: implement pluggable aggregation policies

The API is described in libxlator.h.

Behavior remains the same for this commit; this
is a preparatory step for per-translator customization
of aggregation.

Change-Id: I5d42923af59b2fd78e1ff59c12763875b57c5190
BUG: 847839
Original Author: Csaba Henk <csaba@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4903
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
afr: customize client-pid=-1 xtime aggregation to tolerate a replica down

Using the new 'pluggable policies' API of libxlator.

Change-Id: Ie7528182dff8fb42c6e8287a106d3057944df775
BUG: 847839
Original Author: Csaba Henk <csaba@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4904
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests: call 'cleanup' at the end of each test

Some tests do not cleanup after themselves. That is bad behaviour and
makes it difficult to run single tests and verify the state of the
system afterwards.

Change-Id: I4ac5401d790d6bc81e6975fd1384874b21d6cf8a
BUG: 983975
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5328
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/dht: Prevent dht_access from going into a loop.

If access fails with ENOTCONN, do not wind to same subvol.
We wind to first-up-subvol if access fails with ENOTCONN.
In few cases, if dht has only 1 subvolume, and access fails with
ENOTCONN, we go into a infinite loop of winding to same subvol

The fix is to check if we previously wound to same subvol, and
fail if first-up-subvol is same.

Change-Id: Ib5d3ce7d33e8ea09147905a7df1ed280874fa549
BUG: 983431
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5319
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gfapi: fix for not able to rename folders

If oldname is being renamed to newname, we need to
check for type of newname only if newname exists.

Change-Id: I068a283f9ffe67fcd5e8754d6bf052a2339efbf2
BUG: 953694
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/5333
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
dht: fix dht_discover_cbk doing a wrong layout set.

with the sequence of operations are like below, we have issues
with current code (MP == mountpoint):
T0,MP1# mkdir /abcd (Succeeds on hash_subvol)
T1,MP2# mkdir /abcd (Gets EEXIST as dir exists in hash_subvol)
T2,MP2# mkdir /.gfid/<abcd's gfid>/xyz (lookup happens on abcd's
        gfid, calls dht_discover)
T3,MP1# (Completes mkdir(), goes to dir_selfheal to set the layouts).
T4,MP2# (dht_discover_cbk gets success for lookup as the entry
        existed, as layout is not yet written, it says normalize
        done, found holes).
T5,MP2# (as layout anomaly is not considered an issue in this patch,
        dht_layout_set happens on inode, with all xlators pointing
        to 0s)
T6,MP1# (completes mkdir call, inode has proper layouts)
T7,MP2# mkdir /.gfid/<abcd's gfid>/xyz fails with ENOENT
        (with log saying no subvol found for hash value of xyz.

Porting Amar's fix from down-stream beta branch.

Change-Id: Ibdc37ee614c96158a1330af19cad81a39bee2651
BUG: 982913
Original-author: Amar Tumballi <amarts@redhat.com>
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5302
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>