cluster/afr: Club missing entry, missing gfid self-heals

Problem:
gfid-self-heal always assigns the gfid(GFID-1) it gets from lookup.
Between the time of lookup to triggering the gfid-self-heal the
entry could have changed. Now lets say there is a case where
one of the files of the replica subolumes already has a gfid
(GFID-2) and the other does not. In that case healing should
happen with GFID-2 instead of GFID-1.

Fix:
Missing-entry-self-heal already handles all these cases. So removed
separate handling of gfid-self-heal.

Change-Id: Ie96261e9036c8f3cb4cad89347f9bf7b681cdc1a
BUG: 767585
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/2670
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
extras: /etc/init.d/glusterd should create a lockfile under /var/lock/sybsys

Without a lockfile under /var/lock/subsys, the glusterd service is not
stopped on shutdown or reboot.

Change-Id: I9dc9b4204c5c39879c22191b2e8a2b81a4f3a58a
BUG: 960476
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/4960
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
nfs3-server: call truncate fop only if necessary

* nfs3svc_setattr_cbk: only truncate if requested size != current size

Change-Id: I3d89e4d2b0710118f90cf5bf9cfdea61d8877491
BUG: 960725
Signed-off-by: Michael Brown <michael@netdirect.ca>
Reviewed-on: http://review.gluster.org/4917
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: Give up big lock before performing any RPC

Change-Id: Ib0a772dc1cb9afc8adccd8f7092f480d2b525ea0
BUG: 960580
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4964
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gluster/CLI: crash upon executing "peer status" command

Problem:
While doing "gluster peer status", cli_cmd_peer_status_cbk() creates
the frame and passes as arg to gf_cli_list_friends() which sets
frame->local to GF_CLI_LIST_PEERS flag (value: 0x1). It expects
gf_cli_list_friends_cbk() [invoked through cli_cmd_submit()] to
reset frame->local to NULL. But if cli_cmd_submit() fails some
where before gf_cli_list_friends_cbk() gets invoked, then the
flag value remains in frame->local and causes a SEGV while
destroying the stack i.e. [CLI_STACK_DESTROY => cli_local_wipe()].

Fix:
In gf_cli_list_friends(), if cli_cmd_submit() fails, then
reset the value of frame->local to NULL.

Change-Id: Ied19f07eaf67e3bd42c75cdc2ff3729b0789e632
BUG: 961691
Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>
Reviewed-on: http://review.gluster.org/4976
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
object-storage: final removal of ufo code

See https://git.gluster.org/gluster-swift.git for the new location of
the Gluster-Swift code.

With this patch, no OpenStack Swift related RPMs are constructed.

This patch also removes the unused code that references the
user.ufo-test xattr key in the DHT translator.

Change-Id: I2da32642cbd777737a41c5f9f6d33f059c85a2c1
BUG: 961902 (https://bugzilla.redhat.com/show_bug.cgi?id=961902)
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4970
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glupy: Importing Jeff's glupy project into glusterfs

Change-Id: I3891ef6eaf6ede7c8cbedc3298ce2501a69b2b05
BUG: 961856
Original-author: Jeff Darcy <jdarcy@redhat.com>
Signed-off-by: Ram Raja <rraja@redhat.com>
Reviewed-on: http://review.gluster.org/4906
Reviewed-by: Justin Clift <jclift@redhat.com>
Tested-by: Justin Clift <jclift@redhat.com>
glusterd: Set op-version on startup based on install status

If the current installation of glusterfs doesn't have a stored
op-version and is,
a. a fresh install, then set op-version to maximum
b. an upgrade from release which didn't have op-version support, set it
to minimum.

The install status is detected using the peer-uuid.
If both peer-uuid and op-version are not present in the store, the
installation is fresh.
If peer-uuid is present, but op-version is absent in the store, the
installation has been upgraded from a version which didn't support
op-versions.

By setting the initial op-version as above, we can ensure that
a. features are not enabled accidentally during upgrades
b. a fresh install starts with all possible features enabled.

Change-Id: I52aed9ebeb5d3823c81e65f739a78a924ecf7e12
BUG: 954256
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/4867
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Fix uuid to hostname conversion for 'volume status'

Change-Id: I46c41c29c2d11652f6d8ccd5637be0ac9774fc1d
BUG: 927648
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/4873
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd: remove-brick: prevent removal from a replicate volume.

Prevent the removal of brick(s) from a plain replicate volume and
display the error message at the CLI.

Change-Id: I8e182404564147329d8cd364b7c7931d19f14570
BUG: 961669
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/4975
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
rpcsvc: fix dangerous setting of pointer on free'd structure

The current code is setting @req->hdr_iobuf = NULL _after_ calling
actor_fn() on @req. Calling actor_fn() takes away all guarantees
of whether @req is still a valid object or destroyed.

Unfortunately most of the times the object is allocated from mem-pool
an a mem_put() still keeps the arena allocated (no crash). However
once the mem-pool is full and allocation falls back to malloc()/free()
the code actually becomes dangerous. This resulted in random crashes
when the system load is high (when there were sufficient outstanding
calls that @rpc pool got full)

Change-Id: I4398c717aa0e2c5f06733212b64dd79e7b2a4136
BUG: 884452
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4990
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Perform NULL check on rsp.op_errstr before using it

Change-Id: Id18b215a91cf016964ea98d2f414293b82167d24
BUG: 962362
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4992
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glupy patch by Ram, Justin: Add/Modify fops, structure types, utility fns

Extend the following fops with Python:
  * open
  * readv
  * writev
  * opendir
  * readdir
  * readdirp
  * stat
  * fstat
  * statfs
  * setxattr
  * getxattr
  * fsetxattr
  * fgetxattr
  * removexattr
  * fremovexattr
  * link
  * unlink
  * readlink
  * symlink
  * mkdir
  * rmdir

Add fd_t, inode_t and iatt_t structure types.

Modify loc_t structure type; Alter the data types of the following
attributes - inode, parent, gfid, pargfid.

Modify uuid2str function, which returns a string equivalent for a ctype
object representing a gfid, to make use of python's 'uuid' module for
accurate representation of uuids.

by Justin Clift:
           Adjust debug-trace.py to work with Python 2.6

           Work around 'zero length field name in format' bug in
           negative.py's uuid2str function

           Fix indentation errors in negative.py, glupy.h,
           glupy.c, gluster.py

Change-Id: If0fcfb2866e21c0380a973f8ffab9ea7b6a4cd5d
BUG: 961856
Signed-off-by: Ram Raja <rraja@redhat.com>
Reviewed-on: http://review.gluster.org/4907
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Justin Clift <jclift@redhat.com>
Tested-by: Justin Clift <jclift@redhat.com>
mount/fuse: enable fuse real async dio when available

fuse has support for optimized async. direct I/O handling via the
FUSE_ASYNC_DIO init flag. Enable FUSE_ASYNC_DIO when advertised
by fuse.

performance/write-behind: fix dio hang

Also fix a hang observed during aio-stress testing due to conflicting
request handling in write-behind. Overlapping requests are skipped
in pick_winds and may never continue when the conflicting write in
progress returns. Add a wb_process_queue() call after a non-wb request
completes to keep the queue moving.

BUG: 963258
Change-Id: Ifba6e8aba7a7790b288a32067706b75f263105d4
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/5014
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
tests: basic/rpm.t leaves /var/tmp/rpm-tests.* directories

When the test is terminated early because nothing changed that requires
this test, the /var/tmp/rpm-tests.xxxxxx is not removed.

BUG: 963333
Change-Id: I98d44ca34f14d62da9d836c41a9015cfb520ff96
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/5015
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: Ensure portmap registration before nfs mount.

Change-Id: I12a660a7dfbe4a2d0428910d762434043395fe02
BUG: 927616
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/5012
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
syncop: Remove task from synclock's waitq before 'wake'

Removing task from synclock's waitq after wake could result in
a subsequent unlock, wake'ing up the already running task.
This fix makes the removal from waitq and wake 'atomic'.

Change-Id: Ie51ccf9d38f2cee84471097644aab95496f488b1
BUG: 948686
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4987
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Start bricks on glusterd startup, only once

The restarting of bricks has been deffered until the cluster 'stabilizes'
itself volumes' view. Since glusterd_spawn_daemons is executed everytime
a peer 'joins' the cluster, it may inadvertently restart bricks that
were taken offline for say, maintenance purposes. This fix avoids that.

Change-Id: Ic2a0a9657eb95c82d03cf5eb893322cf55c44eba
BUG: 960190
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4973
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd: Fix misleading log messages of the kind "Node <n> responded to <n>"

PROBLEM:

glusterd logs coming from glusterd_xfer_friend_add_resp() (wrongly)
indicate that a node responded to itself, although it actually
responded to one of its peers.

FIX:

Make glusterd_xfer_friend_add_resp() distinguish between remote host
and self and print the appropriate hostname.

Change-Id: I2a504eeb058c08a0d378443888eb6f1dc7edc76f
BUG: 963537
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/5017
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
rpc-transport: Moved unix socket options function to rpc-transport

This change removes the asymmetry in the 'layer' (read rpc, transport
etc) in which transport options were being filled for inet and unix
sockets.

Change-Id: Iaa080691fd5e4c3baedffa97e9c3f16642c1fc12
BUG: 955919
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4850
Reviewed-by: Raghavendra G <raghavendra@gluster.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>