afr: pass back xdata in create

A striped, replicated volume spits an error on file creation because
stripe requires xdata to process stripe information and AFR isn't
passing it back.

This fix was suggested by Amar Tumballi.

BUG: 842373
Change-Id: Ia7063590ca5e873d4a4e155989cf067e8a07501f
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3713
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Fixes for gsyncd / geo-rep and FUSE listxattr

This patch fixes two problems with recent changes to
Geo-Replication

First:
------
Recent changes to geo-replication relies on Rsync to tranfer
extended attributes. Essentially Rsync would invoke a listxattr()
and then getxattr() the set reutrned by listxattr() and finally
transfer it to the remote slave. Xattrs like security.selinux would
create problems as they are not allowed to be set explicitly (unless
there's a rule that allows this). So, to make Rsync behave sanely we
filter out all "*.selinux*" xattrs from listxattr() (which is getxattr()
with ->name as NULL).

Second:
-------
Python's "if {..} else {..}" shortcut ".. and .. or .." was misused here.
This is a straightforward fix by interchanging last two variables (classes
in this case). Also fix a typo in sendmark_regular() definition.

Change-Id: I097b5f5d88a36c7eef5560a78d4332948a545942
BUG: 842330
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3714
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
libglusterfs: fix creation of spurious root inodes

Suppose lookup is sent on a newly created inode (gfid is root gfid),
and in the call back inode_link is done, where if there is an inode
already present in the inode table for that gfid, then we start using
the present inode and the newly created inode supposedly gets destroyed
when refcount comes down to zero through inode_unref calls.

But inodes with root gfid are not unrefed. Now since in inode_link,
the gfid has already been copied to the new inode and later the actual
inode is found from the inode table, the new inode does not gets unrefed
if its gfid is root gfid. Thus those spurious inodes with root gfid will
be present throughout the lifetime of the filesystem.

To fix this first search the inode table for an inode, with the gfid, present
in the stat structure received as an argument. If the inode is not found,
then copy the gfid to the newly created inode. Now whenever unref is called on
on the new inode it gets unrefed and thus eventually gets destroyed

Change-Id: I0f25f0a8dca3245abda1c322c216f063b52cf842
BUG: 841188
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3716
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: Fix sendmark() invocation for Normal Mixin

Change-Id: I0ae81ab01418becba83e401ec36c6db5323945e8
BUG: 842330
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3725
Tested-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/index: Fix race in this->private initialization

RCA:
In index_worker function at the time of assigning priv from
this->private, this->private may not be set in init() function
of index xlator.

Fix:
Set this->private before creating the thread.

Additional Changes:
Added code to handle error path completely.

Test cases:
Attached the process to gdb and simulated failures.
executed fini in gdb using call fini(this) after init
was successful.

Change-Id: I1874a30d009a35352173b827574cf83daf431453
BUG: 843071
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3728
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/distribute: Suppress user xattr mismatch log message

Changing the log-level to DEBUG.
Xattr mismatch can occur when parallel setxattr's race, or when
one of the bricks was down. A subsequent setxattr will fix the
condition when all the subvols are up. In this case, the 'user.swift'
xattr used by ufo was out of sync, but did not cause any other error.

Change-Id: I6fdff78869b8ff72c305bbe122033e6c1d9d3cff
BUG: 838197
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.com/3722
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/index: make pthread_attr_t local to init()

RC & FIX:
No need to carry around pthread_attr_t in index_priv.
So made it local to init()

Tests:
stepped through init() in gdb, it succeeded.

Change-Id: I0525ac0676f9a329fccb0fd064933594ec117261
BUG: 843071
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3729
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/stripe: don't fail if no fctx on a non-regular file

cluster/stripe broke directory rename. Only check for fctx on regular
files.

BUG: 842652
Change-Id: I8a1e7ff30d57c994082cb10471f610023713ee53
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3720
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Perform data self-heal for non regular files

RCA:
Data self-heal for non regular files open the files
and then proceeds using that fd. This approach
does not work for symlinks because open on symlink opens
the file resolved by it.

Fix:
If the file is not a regular file then perform self-heal using
loc. It needs to get 'big' lock and then perform lookup to get
changelog then erase data part of chagelog, then unlock.

Test cases:
Automated at
https://github.com/pranithk/gluster-tests/blob/master/afr/special-file-self-heal-test.sh

Change-Id: I924a922f5135872efe2cccf2e712ada082c5689f
BUG: 811317
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3724
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster: fix crash on link of named pipe in stripe/replicate vol

A crash occurs when attempting to link a named pipe on a striped,
replicated volume. The cause for this crash is attempting to deref
a NULL inode pointer in stripe_link_cbk(). The RCA for this bug
uncovered a couple of problems:

- AFR ignores the inode pointer it receives on failure (returning
  NULL).
- stripe assumes the inode pointer is valid on failure.

Either one of these changes addresses the crash, but this patch
includes both changes. AFR is modified to pass along the inode
pointer it receives (which could still be NULL). stripe is
modified to not assume the inode pointer is valid on fop failure.

BUG: 842825
Change-Id: I9cb2cc918552620929c3ecbd69bc66d4635eafdc
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3727
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
performance/write-behind: preserve lk-owner while syncing writes.

  - This patch also makes syncing of non-overlapping but consecutive
    writes parallel. Till now only contiguous writes were synced
    parallely.

Change-Id: Icf0d5ea373f30c79fcdc90ba44b7e7a1bc5f0111
BUG: 765141
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.com/269
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
performance/write-behind: detect short writes and pend an EIO error

Write-behind returns write requests immediately and queues the request
in memory for merging, etc. If a write is incomplete, pend an EIO
error for the next fop. This ensures that write failures are not
silent and potentially ignored.

BUG: 809975
Change-Id: I0e0e6c8e710efab58ccfaf746501d00e459eb7ef
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3712
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Filter O_TRUNC in afr-fix-open

RCA:
When open was done while a brick is down, afr opens the file after
the brick comes backup. If this happens after the self-heal on the file
is completed by self-heald etc, the file will end up in truncated state.

Fix:
Filter O_TRUNC while afr-fix-open because afr_open turns O_TRUNC
into truncate transaction, so there will be pending changelog for
the subvolume on which open fails.

Testing:
Had to simulate the race by stopping fix-open until self-heald completes
self-heal on the file after brick online.

Change-Id: I32759cc37f4bb34f206d01606a279f17b246dba4
BUG: 841840
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3705
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Modified split-brain handling

RCA
The bug is observed because the decision to mark
a file in split-brain is taken outside appropriate locks.
Lookup gathers xattrs outside any lock. The xattrs being
in split-brain in lookup should only be taken as a hint.
Appropriate inodelks should be taken before confirming
a split-brain. Self-heal confirms this at the moment.
If data/metadata self-heal is turned off, inspecting of
xattrs could not be performed so split-brain behavior
does not work correctly if the self-heal options are turned off.

Fix
Self-heals are launched to inspect xattrs even when the
data/metadata self-heal options are turned off. The decision
to heal data/metadata after the xattrs are inspected is based
on whether the options are turned on/off. So decision to set/reset
split-brain flag is taken inside appropriate locks.

Testcases:
tests 33-36 in
https://github.com/pranithk/gluster-tests/blob/master/afr/self-heal.sh

Change-Id: Ia8aeab08208b50c06609ad35a9d72f3d553ee343
BUG: 833727
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3626
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gfapi: unglobalize oldvol{file,len}

Multiple initialization of volumes works fine now.

Change-Id: Ibc16957b39325817bc6f9ab90ebac9cefdafabf8
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.com/3734
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
acl: enable handling of FMODE_EXEC flag

on linux systems, with open(), we can get below flag as per
'linux/fs.h'.

/* File is opened for execution with sys_execve / sys_uselib */

Instead of adding '#include <linux/fs.h>, its better to copy this
absolute number into other variable because then we have to deal
with declaring fmode_t etc etc..

With the fix, we can handle the file with '0711' permissions in
the same way as backend linux filesystems.

Change-Id: Ib1097fc0d2502af89c92d561eb4123cba15713f5
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.com/3739
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Warnings: Fix set, unused variable warnings

Change-Id: If7ee85aa0d93f9018a76e7f60fcb7dcaf849dbdc
BUG: 839950
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3742
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
glusterd: Ensured 'store' data reaches disk.

- Opened temporary file(s) with O_SYNC flag to avoid explicit fsync'ing.
- Sync'd directory entry after creation and rename of 'store' files.
- Thanks to Jeff Moyer's article on http://lwn.net/Articles/457667/

Change-Id: I68a8672dc6a0b24d128de53f3b60c74dd08d8ab8
BUG: 765434
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.com/3726
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli: Prevent creation of volumes with tokens as volnames

RCA:
yyparse confuses volnames with tokens when any of the
tokens 'volume', 'type', 'subvolumes', 'option', 'end-volume'
are used as volnames. This happens because io-stats xlator
name is same as volname in fuse volfile. Both nfs, fuse volfiles
are affected by this problem.

Fix:
We could fix this also by changing io-stats xlator name to
something other than 'volname'. But I am worried of the
backward compatibility issues it may introduce. Disallowing
creation of volumes with tokens as volname seems like a safer
fix.

Tests:
All volume creation operations with tokens as volname give
invalid volume name error.

Change-Id: Ifc63a5c31375e92541b954ec133aa3c8e6a56a02
BUG: 844030
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3745
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
features/locks: Fix statedump code

RCA:
Taking blocking mutex/spin locks lead to dead locks
because of the locking order in statedumps. Also we
were asked to remove gf_logs if possible to avoid extra
cost in signal handlers.

Fix:
changed blocking mutes/spin locks to their non-blocking variants.
Removed gf_logs in locks xlator statedump code-path.

Tests:
State-dump success cases are working fine.
Triggered try-lock failures by putting statedumps in a while loop.
In parallel did chown of the same file in a while loop.

Change-Id: I81539a62f8216f267f57bb703ef132c85bfd557d
BUG: 843781
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3747
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>