distribute: use global synctask 'env' instead of local

creating a local synctask_env can lead to creating of many more
syncop threads than required. The current syncop logic can handle
the scale-up/scale-down of threads depending on the load. Hence,
its neater to use global synctask env.

Change-Id: Id46f963a0190c0154513317ae03323db155ac15a
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 823774
Reviewed-on: http://review.gluster.com/3412
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
performance/quick-read: fix race-conditions in qr_unlink.

The list of fds on which open needs to be done as part of unlink,
was being modified at different places using different locks.
This resulted in a race-condition where open was marked as in-transit,
but fdctx was removed from the list of fds on which open was being
sent even before open was done. Because of this, open_in_transit would
be set forever (as an open was never actually sent, there would be no
open_cbk called and hence we could not reset the variable), blocking
all the future fd based fops on this fd.

Change-Id: Ie84a55bee578869a9a060a094ba28480e7643ae8
BUG: 819490
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.com/3371
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
Revert "fuse-resolve: consider cases where an entry should be resolved even when parent belongs to active itable."

This reverts commit 595bc33c1d0a3a6fe3d54d3eb876ac0b336d9dea

Change-Id: I6c67404d4ff0bc83cfac83d786bc6e66d221a84a
Reviewed-on: http://review.gluster.com/3422
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Fix build error from recent FUSE changes.

Change-Id: Ia878949d00b1ef188b18c853763918b83521ee34
BUG: 762389
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.com/3423
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
protocol/client: use correct xlator object while creating frames
during reopening files and reacquiring locks.

Change-Id: I29f42fcfa9e782ce9e323e53024e5034029914a7
BUG: 822337
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.com/3420
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Release inodelk on erase changelog failures

Change-Id: I58271e1ac5a116b5bc717d7cad9f03eb7dc8a1a4
BUG: 811551
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3417
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: Disconnected bricks of 'stale' volume before deleting it.

If glusterd_delete_bricks is called before glusterd got the DISCONNECT event
from the brick that was stopped, then glusterd_brick_rpc_notify would
dereference a free'd brickinfo. This can happen if the brick had not been
disconnected before.

Change-Id: I6c07ec50f6739422a14478a549edd06c4c0ce913
BUG: 802015
Signed-off-by: Krishnan Parthasarathi <kp@gluster.com>
Reviewed-on: http://review.gluster.com/3183
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
mount.glusterfs : multiple fixes

* made log-level and other string comparisons be case insensitive
* fixed wrong spelling, wrong command usage in case of brick inode
  computation
* used 'cut' instead of 'tr' as piping the result to read was not
  working fine in few cases.

Change-Id: I9caab481cfd80000b8ef9de7a44006729c88cc1b
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 823403
Reviewed-on: http://review.gluster.com/3413
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd/rebalance: Display run-time of rebalance process

Added run-time value field to cli output of rebalance/remove-brick.

A new cluster/distribute boolean option rebalance-stats when set to
ON, time taken for migration of each file is logged.

With rebalance-stats OFF (default), rebalance logs will only have
entries showing time spent in each directory.

Change-Id: I02a8918621120068cd71ffaf2999d30b3a2d10a2
BUG: 821987
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3303
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
rpc: fix the vecsizer issue

* currently, we needed 'base-address' of the header for glusterfs
  writev vecsizer and 'current-address' of the buffer which is
  reading the data on socket, for nfs write vecsizer.

* nfs write issues started coming after http://review.gluster.com/3182
  was pushed into repo, now fixed.

* fixed by sending both as argument for vecsizer from the transport

Change-Id: I6db360ce265ce5f083f1794ebdb3867f8cfad9ec
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 824472
Reviewed-on: http://review.gluster.com/3431
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: decrement changelog when fdctx is absent

Change-Id: I9d76ddbd2cf8e4e8e4ad70529ba3a70178489a68
BUG: 765194
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3435
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
geo-rep / gsyncd: add "--super" to rsync invocation

This forces rsync to perform supposedly privileged operations on
unprivileged slaves (like chown(2)).

For consistent behavior (with gsyncd's "chown" RPC call that's
being used for symlinks and directories), we also pass
"--numeric-ids" to rsync.

Also took the chance to retire gsyncd's "--rsync-extra" option
which was there for debugging purposes (related to a resolved
issue).

Change-Id: I4ee4d0d3a8c4e0f6746d34d7722c8a567a67491c
BUG: 822121
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3426
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd/remove-brick: Reset decommission stats on all nodes

Decommissioning is started only on nodes where the bricks which
are being decommissioned are present. The stats were reset only
when decommission was started. Hence stale stats were being
shown on nodes where the bricks were not present.

BUG: 822778

Change-Id: I2d839f877d4e040b463bebde5ba753b7265ab633
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3425
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
nfs/nlm: avoid duplicate replies for nlm procedures

The way NLM handles errors and corresponding response
messages has been simplified to avoid duplicate replies
in case of failures.

Also, unlock_cbk and unlock_fd_resume functions are moved
in with other unlock functions.

Change-Id: I94100aa3c8de95dabebed4598651bbcd49d95782
BUG: 824316
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.com/3414
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishna Srinivas <krishna@gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd-portmap: adhere to IANA standards while assigning brick ports

RFC 6335

Change-Id: Iafbba70d24db95807ae73d27c96288d18d7232e5
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 824233
Reviewed-on: http://review.gluster.com/3339
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
geo-rep / gsyncd: further cleanup refinements

- Regarding issue of leftover ssh control dirs:

  If master side worker is stuck in connection establishment
  phase, have the monitor kill it softly (ie. first by SIGTERM,
  to let it cleanup). This is trickier than sounds on first hearing,
  because if worker is stuck in waiting for a RePCe answer
  (in threading.Condition().wait()), then SIGTERM is ignored
  (more precisely, Python holds it back for the wait and resends it to
  itself when wait is over).

  So instead of signalling the worker only, we send TERM to the
  whole process group -- that brings down the ssh connection, which
  wakes up the waiting worker, which then can cleanup. Only problem
  is that monitor is also in the process group and it should not coomit
  a suicide. That is taken care by setting up a one-time SIGTERM
  handler in the monitor.

- Regarding slave gsyncd stuck in chdir:

  Slave gsyncd is usually well behaved: if master does not send
  keepalives, it takes care to exit. However, if a hang occurs
  in early phase, when slave is to change to the gluster mountpoint,
  no timeout is set up for that (and unlike on master side, neither
  is there an external actor like the monitor to do that).

  So, to manage this scenario, we do the chdir in a (supposedly)
  short lived thread, and in the main thread we wait for the termination
  of this thread. If that does not happen within the time limit, main
  thread calls for cleanup and exit. (This logic explicitely takes the
  appropriate action in the cases when chdir succeeds or when hangs;
  but what about the remaining case, when chdir fails? Well in that case
  the chdir thread's exception handler will put the process to
  cleanup and exit route.)

Change-Id: I6ad6faa9c7b1c37084d171d1e1a756abaff9eba8
BUG: 786291
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3376
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Switch to GNU basename_r() and dirname_r()

BSD basename() and dirname() return static storage, hence they are not
thread safe, while GNU flavors are. glusterfs assumes the GNU behavior.
As a fix, we added a BSD-derived, thread-safe version of basename_r()
and dirname_r() in contrib/libgen. BSD systems can use it instead of
their built in versions.

Unfortunately, there are more reliance on GNU behavior in glusterfs.
There are place where free() is called on static storage returned by
basename(), place where a second call to dirname() overwrite the
string obtained once, and so forth.

Each of these problems can be fixed, but it is likely to be a pandora
box. We need to accept that glusterfs heavily rely on GNU behavior,
and switch contrib/libgen's basename_r() and dirname_r() to the GNU
version. This change does just that.

BUG: 764655
Change-Id: Id874b9c7aacd9aa3a7a4bd6a92a9633f5b2d6ac0
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.com/3430
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd / geo-rep : fix rotating slave log file

Attempt to rotate log files for slave that are local.
Remote slaves (identified by the 'ssh' scheme) are
ignored.

Change-Id: I89959278c0eefffabc6a6702d4ef661cf52644a6
BUG: 821443
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3444
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
glusterd/geo-rep: cleanup and fixes

- fix the hilarious fd leak of "geo-rep status"
- instead of "corrupt", which can trip up users to think their
  data is in danger, use the term "defunct" to describe the
  condition when gsyncd is dead/unresponsive
- don't use buffered I/O when unnecessary
- stop using PATH_MAX for sizing buffers that don't hold paths
- some cleanups wrt. memory management

Change-Id: I396aacc45dc06a002318b19c60c44041fa9fa18d
BUG: 764268
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3454
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
glusterd/geo-rep: do away with unneded function exports

Change-Id: I0c0b500bcb0b183ae445800fd334cd838b8af0d3
BUG: 764890
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3455
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>