porting: various fixes regression tests OSX/FreeBSD

- `wc -l` on OSX/FreeBSD adds spurious spaces, this clobbers
  up TAP output parsers - fix it.
- `umount -l` doesn't exist on OSX/FreeBSD use 'umount -f' if
   available.
- Add check for 'file' version, to handle mime type variations
  across versions
- Converge 'glusterfs --attribute-timeout=0 --entry-timeout=0'
  into '$GFS'
- Modify remaining 'mount -t nfs' to use 'mount_nfs'
- Update sha1sum for OSX to use 'openssl sha1'.

Change-Id: Id1012faa5d67a921513d220e7fa9cebafe830d34
BUG: 1131713
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/8501
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
porting: Remove unnecessary code from mount_darwin.c

- Cleanup mount_darwin.c to make it cleaner
- Restructure the code to be more readable
- Avoid unnecessary delays invoking `mount_osxfusefs`

Change-Id: I7f28875b0ec872a08bf8e77dfc8ebe5eca750d0e
BUG: 1135348
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/8564
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
build: fixes for compiling with -Werror=format-security

Nightly builds on Fedora Rawhide (x86_64 and i686) started to fail with the
following error:

  cli-rpc-ops.c: In function 'gf_cli_get_vol_opt_cbk':
  cli-rpc-ops.c:9339:25: error: format not a string literal and no format arguments [-Werror=format-security]
                           cli_err (msg);
                           ^

The cli_err() should be called with a format-string like "%s" to prevent
unexpected errors.

Change-Id: I6a287795fb8b7a305c8f14323456eeabf68d100c
BUG: 1134773
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/8560
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
statedump: Print curr_stdalloc in mempool statedump ...

 ...for, it is curr_stdalloc that is incremented for every mem_get()
and decremented on every call to mem_put() and can be used to detect
leaks, when cold_count is 0.

Change-Id: I418a132c3ea4c0b99ea5c6840ff3024d8d19ddf4
BUG: 1134221
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/8557
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Hook scripts should exit with correct exit status.

When volume set options other than nfs-ganesha.host and nfs-ganesha.enable
are executed, the hook script S31ganesha-set.sh should exit with
exit status 0. The present behaviour results in an error message in
one of the logs that could be  misleading.

Similarly, hook script S30samba-set.sh should exit with
exit status 0 if samba is not installed. Fixing the issue
with a minor change in the script.

Change-Id: I35d1dc8400bb7043385800ed97adc5321829437f
BUG: 1122978
Signed-off-by: Meghana Madhusudhan <mmadhusu@redhat.com>
Reviewed-on: http://review.gluster.org/8385
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Regression test portability: LVM

Skip any test involving LVM on NetBSD as LVM is not supported

BUG: 1129939
Change-Id: I2237bae1128d1a81047c9ff7f905431156daf8b7
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/8556
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Regression test portability: EXPECT

Make sure test_expect_not_footer() and test_expect_footer() work on
non empty strings, otherwise it may produce errors such as in pgfid_feat.t
on NetBSD:
not ok 12 Got "" instead of ""

BUG: 1129939
Change-Id: I9cb76ba863897126534c3808fb0c9e564659835f
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/8568
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Regression test portability: mknod

Linux mknod(2) is able to create fifo and named sockets. NetBSD and
FreeBSD use mkfifo(2) and socket(2)/bind(2) for that.

BUG: 764655
Change-Id: I1d3969e3fcb6afdbd184c28bd268ffa2da7ae202
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/8433
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
storage/posix: Prefer gfid links for inode-handle

Problem:
File path could change by other entry operations in-flight so if renames are in
progress at the time of other operations like open, it may lead to failures.
We observed that this issue can also happen while renames and readdirps/lookups
are in progress because dentry-table is going stale sometimes.

Fix:
Prefer gfid-handles over paths for files. For directory handles prefering
gfid-handles hits performance issues because it needs to resolve paths
traversing up the symlinks.
Tests which test if files are opened should check on gfid path after this change.
So changed couple of tests to reflect the same.

Note:
This patch doesn't fix the issue for directories. I think a complete fix is to
come up with an entry operation serialization xlator. Until then lets live with
this.

Change-Id: I10bda1083036d013f3a12588db7a71039d9da6c3
BUG: 1136159
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/8575
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Propagate EIO on inode's type mismatch

Original author of the test script:
            Pranith Kumar K <pkarampu@redhat.com>

Change-Id: If515ecefd3c17f85f175b6a8cb4b78ce8c916de2
BUG: 1132469
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/8574
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
USS: initialize a list before using it.

Change-Id: Iba35e4397646a67384e878fbe7bdfb7586fc07f9
BUG: 1130888
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: http://review.gluster.org/8569
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
core: update INSTALL file.

Change-Id: Ie6f041810da647e39d4bbb5384f51dadd6f69b30
Signed-off-by: hchiramm <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/8573
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: Treat linkto file rename failure as non-critial error

It is a critical failure iff we fail to rename the cached file
if the rename of the linkto failed, it is not a critical failure,
and we do not want to lose the created hard link for the new
name as that could have been read by other clients.

NOTE: If another client is attempting the same oldname -> newname
rename, and finds both file names as existing, and are hard links
to each other, then FUSE would send in an unlink for oldname. In
this time duration if we treat the linkto as a critical error and
unlink the newname we created, we would have effectively lost the
file to rename operations.

Repercussions of treating this as a non-critical error is that
we could leave behind a stale linkto file and/or not create the new
linkto file, the second case would be rectified by a subsequent
lookup, the first case by a rebalance, like for all stale linkto
files

Change-Id: Ia53ad8b43c3cf8f48ef5b43fd1fec4274e807556
BUG: 1130888
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/8563
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: Rename should not fail post hardlink creation

In the rename path, we wind the creation of newname hardlink and
linkto file in dst hashed a the same time. If the linkto creation
fails, but the link creation succeeds, we enter the failure code
and cleanup the created newname hardlink.

In the interim if another client looks up newname and finds it as
a hardlink from FUSE, it could send an unlink for oldname instead
of a rename. This combined with the above cleanup code could end
up losing all the files copies, and thereby losing data.

This fix separates these steps into 2 parts, creating the linkto
first and then the link file, so that post link file creation no
failures would cleanup the newname file. If linkto fails then link
is not attempted, thereby not polluting the name space with
newname.

Change-Id: I61da8e906060da16a31ea1076eec2f01fd617f44
BUG: 1130888
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/8570
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: Modified test case to note rename failures as errors

The bug referenced in this change, had an race condition that is now
fixed by the following commits that are posted for review.

    http://review.gluster.org/#/c/8563/
    http://review.gluster.org/#/c/8570/

These changes would now make the winning client not fail a rename,
in case it failed to rename the linkto file. Hence when one client
wins the link race, and the other still deletes the linkto file,
the rename failure by the winning client is not a critical failure,
hence it resolves the issue posted in the bug.

As a result modifying the test case to treat the rename failures
as errors, to catch any future issues.

Change-Id: Ibe9caac7ee87dcbc4f581cfbd36173b734859ccb
BUG: 1123950
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/8579
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/dht: remove specifying cached-subvol as part of name in
unlink.

commit 667b2496c3f29e24ed359a05b0f44df0d1894969 introduced a
functionality where we can specify the subvol where file is stored. As
part of same commit, dht_unlink was also changed to accept
cached-subvol as part of name. While it makes sense to specify subvol
while creating file, there is no necessity for specifying the subvol
during unlink, since the default unlink logic works fine with this
functionality too. Also, this code in unlink doesn't work well when
files get migrated by rebalance process. Hence removing it.

Change-Id: Ic3fc32ad657e2dcd677a4c80b04a618029eddd89
BUG: 1130888
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: http://review.gluster.org/8548
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Cluster/DHT: Changing rename log severity

Changing log level for a rename message from debug
to info to improve debuggability

Change-Id: I53031fcf97fffd62095692477330ecde0cf47dcd
BUG: 1130888
Signed-off-by: Nithya Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/8582
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
CLI: Adding warning message in case remove-brick commit executed

Change-Id: Ia2f1b2cd2687ca8e739e7a1e245e668a7424ffac
BUG: 1136702
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/8577
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd: Set the rlimit for Open FDs to higher value

Default 'open FD limit' is 1024. As the number of volumes/bricks
increases, brick-to-glusterd socket FDs also increases in glusterd and
runs out of the limit.

Solution is to set the 'Open FD' limit to higher value in glusterd

Change-Id: Iaa60b2155df2fa5a0759e054bdebffbc09f63ec1
BUG: 1136352
Signed-off-by: Vijaikumar M <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/8578
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
glusterd: Prevent rebalance starting with old clients

Glusterd will prevent rebalance from starting when clients older than
glusterfs-v3.6.0 are connected to a volume. This is needed as running
rebalance with old clients connected could lead to data loss in some
cases. The DHT xlator on newer clients (>= 3.6.0) has been fixed to
prevent the data loss issues.

Change-Id: If58640236382a2fc13f73f6b43777f01713859f7
BUG: 1136201
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/8583
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>