glusterd: Backport of vme table changes from master

This patch backports the following changes from the master branch

99fe09f glusterd: Moved the volume entry table to a separate file.
e306d08 glusterd: Changing the volume entry table's representation.
eac54f6 glusterd: Added option description, and validation function fields.
bcb4235 glusterd: Added validation function for performance cache max and min size.
8897d08 glusterd: Added validation function for quota-timeout.
4579609 glusterd: Added validation function for stripe-block-size.
6788bad glusterd: Fix some options in vme table
549231d glusterd: Added the validation function for subvols-per-directory
9636e63 glusterd: Added description for nfs.transport-type option in volume set help.

Change-Id: I4a64ad94f17df4b45a3a32262a83e2c35fb5f7da
BUG: 907311
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/4956
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd: Introduce volume op-versions

This is a backport of change
        9153855 glusterd: Introduce volume op-versions
from master.

Each volume is now associated with two op-versions,
 * op_version - the op-version of the highest op-versioned feature enabled
 * client_op_version - the op-version of the highest op-versioned feature
   enabled which affects the clients only.

These two op-versions are generated dynamically and kept updated during
runtime. Glusterd now uses the respective volumes' client-op-version during
getspec requests.

To achieve the above a new field in the vme table is introduced,
client_option, this boolean field tells if the option is a client side
option.

BUG: 907311
Change-Id: I59af02644a714e1c54fc89f1ead5aa551bba7ee7
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/4957
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd: Set op-version on startup based on install status

This is a backport of change
        fa227c0 glusterd: Set op-version on startup based on install status
from master.

If the current installation of glusterfs doesn't have a stored
op-version and is,
a. a fresh install, then set op-version to maximum
b. an upgrade from release which didn't have op-version support, set it
to minimum.

The install status is detected using the peer-uuid.
If both peer-uuid and op-version are not present in the store, the
installation is fresh.
If peer-uuid is present, but op-version is absent in the store, the
installation has been upgraded from a version which didn't support
op-versions.

By setting the initial op-version as above, we can ensure that
a. features are not enabled accidentally during upgrades
b. a fresh install starts with all possible features enabled.

BUG: 954256
Change-Id: I5cdd0c63fd16ecfa2fede99684da6fd6167823a8
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5001
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd-volgen: Enable open-behind based on op-version

Backport of patch on master branch, under review at
        http://review.gluster.org/4866

This patch enables the open-behind by default only when the op-version
allows it. Also the volume op-version calculations take account of this
enablement.

BUG: 954256
Change-Id: Ie739bc23ba90ec2f009feecef28187912a37487c
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5095
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd-volgen: Improve volume op-versions calculation

Backport of patch on master branch, under review at
        http://review.gluster.org/4952

Volume op-versions calculations now take into account if an option,
 a. enables/disables an xlator, or
 b. is a boolean option.

This prevents op-versions from being updated when a feature is disabled.

BUG: 954256
Change-Id: Ic68032b9e55a3f0191f8fc3ecd6b5ced385ad943
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5094
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Avoid order mismatch in blocking entrylks

Problem:
When taking blocking entrylks, afr orders the entrylks based on
uuid_compare of gfids of parent dirs, if they are equal then it orders
them based on the basenames. While this approach works fine, the
implementation assumes loc->gfids to be populated at the time of
the comparison, but loc may have gfid in loc->inode->gfid instead
of loc->gfid which was leading to order mismatches and dead-locks.

Fix:
Implemented loc_gfid which gives gfid by checking both loc->gfid,
loc->inode->gfid. Used this for ordering the blocking entrylks.

Change-Id: I2743fcaff3d670fbeb6b8e0a496f106a3585dde1
BUG: 965987
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5063
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: support for chdir() and family

Add support for chdir, fchdir, getcwd, realpath equivalents in
GFAPI. These are required for the Samba VFS plugin.

Change-Id: I91d2db9146994403c98961c489c7640c51d5add2
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5151
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: handle root (/) resolution gracefully for realpath()

Resolving path of / is crucial for realpath() to work properly.

Change-Id: I8a1b482beba37f01afbd20446d9e6f6125f2ec34
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5152
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
syncop: set credentials of running process in @frame

Inherit the pid/euid/egid/groups of the running process in the
frame. Do this only in cases where a loaded frame was not
presented to the synctask.

This behavior is required for Samba VFS.

Change-Id: Ib181c90f47c6741197b9ce9f67a19e2914b647d2
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5153
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: implement readdirplus_r support

This call is used by Samba VFS.

Change-Id: Ib371502ad5a71b3b0e993f6b30e3dfb1f839c020
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5154
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: POSIX locking support

Change-Id: I37d9e1fb4a715094876be6af3856c1b4cf398021
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5155
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
posix-acl: fetch ACLs in readdirplus

Not fetching ACLs in readdirplus can potentially result in spurious
wrong ACL decisions (which magically go away on a lookup() which
populates the ACLs)

Change-Id: Ided38b4d868fab482b477ce51b4878289ef9eed0
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5156
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: add new API glfs_set_xlator_option()

This is the equivalent of specifying --xlator-option command line
argument to glusterfsd.

Change-Id: I1171dd518a85db4bd456fa0790f450e889a956cd
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5157
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
quick-read: prune cache on write/[f]truncate

Cache needs to be pruned on write and [f]truncate. The lack of this
is causing Samba ping-pong test to return wierd 'data increment' values
during startup.

Change-Id: I9cd6a839bcd02de738d78638211b78f382f58e0a
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5158
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: optimistic resolution with ESTALE detection

Enhance the path resolver to be optimistic. Instead of performing
a wasteful ->lookup() fop for every component in every API call,
build in speculation logic to consider the previous resolution
result temporarily, and if it results in ESTALE either later in
the path resolution or in the FOP, then retry resolution with
->lookup() at every level and retry the FOP.

This is the same optimistic resolution/retry logic Linux kernel
resolver is proposing to use.

Change-Id: Iecbc25248754caf0915cd7205910563ff88ce5e7
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5159
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
md-cache: support negative xattr entries

Add support for negative xattr caching. For this, we need
to fetch xattrs in every opportunity (including readdirplus)
in order to treat missing key in cached dict as negative entry.

This is crucial to detect missing ACL xattrs in Samba workload.

Change-Id: I918a2ef4ab804724256f7546b15e808332ed518d
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5160
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
syncop: copy inode pointer in readdirplus

Change-Id: I9ab2b8ac2da9fe13f56b8b08f715a0b603ece0cb
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5161
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: link inodes in relevant entry FOPs

Do not let inode linking to happen only in lookup(). While
that works, it is inefficient.

Change-Id: I51bbfb6255ec4324ab17ff00566375f49d120c06
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5162
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
dht,posix: support for case discovery

This is support for discovering a filename in a given directory
which has a case insensitive match of a given name. It is implemented
as a virtual extended attribute on the directory where the required
filename is specified in the key.

E.g:

  sh# getfattr -e "text" -n user.glusterfs.get_real_filename:FiLe-B /mnt/samba/patchy
  getfattr: Removing leading '/' from absolute path names
  # file: mnt/samba/patchy
  user.glusterfs.get_real_filename:FiLe-B="file-b"

In reality, there can be multiple "answers" as the backend filesystem is
case sensitive and there can be multiple files which can strcasecamp()
successfully. In this case we pick the first matched file from the first
responding server.

If a matching file does not exist, we return ENOENT (and NOT ENODATA).
This way the caller can differentiate between "unsupported" glusterfs
API and file not existing.

This API is used by Samba VFS to perform efficient discovery of the real
filename without doing a full scan at the Samba level.

Change-Id: I53054c4067cba69e585fd0bbce004495bc6e39e8
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5163
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: provide stable st_dev in stat structure

Samba depends on st_dev being unique and stable for different
filesystems (it uses st_dev+st_ino as key to store metadata of
a file).

Change-Id: Ia022fabadfb5ef3fc9724c1b1ca86f1a20f98af9
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5164
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>