cluster/distribute: Add filter to support file patterns to be migrated

'gluster volume rebalance' command will be enhanced to support passing of these
options/pattern.

<pattern> is comma separated list as show below. The Precedence is from right
to left.

e.g- "*avi,*pdf:10MB,*:1KB"
     The precedence is as follows:
     migrate all files with size equal or greater than 1KB "*:1KB"
     migrate all pdf files with size equal or greater than 10MB "*pdf:10MB"
     migrate all avi files "*avi"

With this option, it is possible to choose which files to migrate.

Change-Id: I6d6d6a015bcbacf1debae2f278a2d92306fb055d
BUG: 896456
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4366
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/dht: print hash function munging logs in DEBUG mode

Change-Id: Ia2e6bce80710d103da9d78afdb389ea162b00686
BUG: 912564
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4590
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Added validation function for stripe-block-size.

Change-Id: I050d01b01eac46550aa435da7d96a972e0393d35
BUG: 770655
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4561
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Do not call xdr_string() with a NULL error message

It is illegal to call xdr_string() with a NULL string. Linux
just retruns false, NetBSD gets a SIGSEGV when xdr_string()
calls strlen(NULL)

BUG: 916439
Change-Id: Ia958470ada6e8e55a86d439922ec942d038f5f13
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/4589
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/distribute: Prevent spurious multiple defrag crawls

In dht_notify, we used to create a thread to start defrag
crawls after we had heard from all child subvols.
This was in-correct, as a later event, could also trigger the
crawl again(due to the fact that all subvols had responded).

The fix is to make sure, the thread is started only once after
all subvols have responded the first time

Change-Id: Ifc2978b9dc866af2395b79911eca50ab38ff9457
BUG: 916449
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4587
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
glusterd: Fix some options in vme table

Some of the options had invalid '.flags' members. In the original table
these table these were supposed to be the op-versions, but, the entries
for the below options were missing the flags field the op-version was
entered in that place.

Change-Id: I408f5a972743eb37d9a58a809e8be8cb385bced8
BUG: 903478
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/4593
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Added the validation function for subvols-per-directory

Change-Id: Ie2259023b9001311a2032792639c3093054f6750
BUG: 896431
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4552
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
performance/write-behind: Add test case for fd being marked bad
                          after write failures.

Change-Id: I5a3c27be0ad77b7f2660f45977f9f1f8f34ae31c
BUG: 765473
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4560
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
libglusterfs: avoid the logging which says the port is invalid

If the reserved ports file in proc contains just a newline, then
do not proceed with ports checking and reserving.

Change-Id: If610b0e49a97e5c22e7e2b1bb4155c9616a4612a
BUG: 762989
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/4583
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
cluster/afr: Turn on eager-lock for fd DATA transactions

Problem:
With the present implementation, eager-lock is issued for
any fd fop. eager-lock is being transferred to metadata
transactions. But the lk-owner is set to local->fd address
only for DATA transactions, but for METADATA transactions
it is frame->root. Because of this unlock on the eager-lock fails
and rebalance hangs.

Fix:
Enable eager-lock for fd DATA transactions

Change-Id: If30df7486a0b2f5e4150d3259d1261f81473ce8a
BUG: 916226
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4588
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Modified validation parameters for owner-uid and owner-gid.

owner-uid and owner-gid will not receive negative values anymore.

Change-Id: I82741d3d01b29e448294b2ec093fb70d22a5c77e
BUG: 912297
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4581
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: Added description for nfs.transport-type option in volume set help.

Change-Id: I9fe81dc1c3172158e8dd86c4fa2a04af18cb9dde
BUG: 782285
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4582
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster xlators: s/-1/GF_CLIENT_PID_GSYNCD/

Change-Id: I03be3cb23684de4ab36cf2953002708466edd580
BUG: 765433
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.org/4601
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
rpm: package /var/run/gluster so that statedumps can be created

Creating statedumps fail when /var/run/gluster does not exist. This
directory should be part of the 'glusterfs' package that is installed on
storage servers and native clients.

BUG: 917561
Change-Id: Ibbc1e4016469fab3db0423337fabc4ba5a8302ab
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/4616
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
nlm: use appropriate open flags while locking

In case of a shared/read lock, open the file in O_RDONLY
mode, and in the case of an exclusive lock, open the file
in O_WRONLY mode to emulate the behaviour posix fcntl
implementation as given in the man pages.

Change-Id: Ib9eab6570c3bc65f8bd48a14a9d801616213b295
BUG: 916930
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.org/4603
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
storage/posix: Remove a redundant lstat in posix_handle_hard.

Change-Id: I9129b71d5568eff3513c17e3607256783fdc42ec
BUG: 903396
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4641
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
object-storage: Restoring multi volume support in UFO.

* Currently, the users of UFO are restricted to use just one volume
  at any given point of time. This patch removes this limitation.

* The usage of gluster-swift-gen-builders has also changed. With this
  commit the users should mention the list of volumes that they want
  to expose through UFO. So, only the volumes mentioned during the
  ring file generation can be accessed.

  Usage: gluster-swift-gen-builders <vol-name1> [<vol-name2>]...

This is an intermediate fix until we remove the account, container and
object server processes. Once we have this frame work running, it will
completely eliminate the ring files.

Change-Id: I9ad3808519fec9c7c60ad846c4f8b653117a8337
BUG: 909053
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4485
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
object-storage: Fixing the errors and warnings in unittest.

Change-Id: Id22c968aefd82c4b62445b3ecc93cbabc2b35ffc
BUG: 887301
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4394
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
gfapi: dict_unref() xattr_req in fop finish instead of dict_destroy()

The current way of calling dict_destroy() at the end of an API fop assumes
that xattr_req is not stored/ref'd by any translators in the stack. However
when translators like DHT store xattr_req in dht_local_t with a dict_ref()
and perform dict_unref() in the unwind path, things get subject to a race.

The race is between the woken up thread (by syncop_wake) i.e, the gfapi
invoking thread and the thread where the FOP was unwound. As the C stack
of STACK_UNWIND unwinds back, dht_local_unref() gets invoked within the
DHT_STACK_UNWIND macro. This thread attempts dict_unref, which would
be "safe" if it wins the race against the gfapi invoking thread. However
if the gfapi invoking thread wins the race, it will perform dict_destroy()
first and therefore make dict_unref() within dht_local_unref perform
a double free.

This is the embarrassing on-screen bug which showed up in a roomful of people
during the gluster dev summit demo of qemu/libgfapi integration.

Change-Id: I284c93de87cdc128d5801f42c84aa87f753090d4
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4644
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/distribute: Fix layout overlaps due to spread-count in selfheal path

We needed to zero out the layout range, before we re-calculate the range.
When spread-count is issued, we would end up with stale ranges in the layout.

Replaced dht_selfheal_dir_xattr with dht_fix_dir_xattr, which correctly resets
the un-used (after re-cal) layouts.

Change-Id: I1a900d15df07335f59356bd23182ccec34381ab2
BUG: 884455
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4647
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>