cli: remove-brick no longer defaults to commit-force

Problem : When gluster volume remove-brick is executed with out any option, it
defaults to force commit which results in data loss.

Fix : remove-brick can not be executed with out explicit option, user needs to
provide the option in the command line else the command will throw back an usage
error.

Earlier usage : volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
                [start|stop|status|commit|force]

Current usage : volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
                <start|stop|status|commit|force>

Change-Id: I2a49131f782a6c0dcd03b4dc8ebe5907999b0b49
BUG: 1077682
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7292
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <sam.somari@gmail.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/gfid-access: Fix possible inode memory corruption.

        During lookup, the inode is not ref'd. Added code
        to ref the inode in call path and unref in cbk path.
        Also fixed a case where we should always be putting
        linked inode into context as it is not guaranteed
        that we get same inode that we passed in a call to
        inode_link.

Change-Id: Iaec083a9258658bef3047e83956729d3dbcd9a59
BUG: 1080295
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7329
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
features/glupy: Add mem accounting support

When glusterfs is built from source using -DDEBUG flag and glupy
xlator is added to vol file, the brick process used to crash when
mounting the volume.

This fix is largely derived from the fix submitted for BZ #1035751.
Thanks to Justin Clift for helping in tracking this down.

BUG: 1035751
Change-Id: Id64f92eecc9335e34dd08812fe176774e7723c2c
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/7332
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
tests: Adjust test 767095 to cope with long hostnames

BUG: 1084147
Change-Id: Ie1ff8852a501690e681072c54620d305b5e20d6a
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7395
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests/afr: select correct read-child for entry OPs.

Change-Id: If375c937579a18d603ed70232130a4664060e9d6
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7344
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli : Removal of dead code

dead code reported by covscan is removed from cli-cmd-parser.c

Fix for coverity CID: 1195423

Change-Id: Ice1771dc8b3ef47fd2e63b380b12e850dc1d5d95
BUG: 789278
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7389
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
afr: Simple 1-liner fix for crash in Rackspace

BUG: 1084485
Change-Id: I89ddf10add041638ef70baebbce0ec2807ef4b6d
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7402
Reviewed-by: Anand Avati <avati@redhat.com>
cli: Trivial wording improvement of a comment

Change-Id: Ib569b39bdf0357c30c94c7a1b8d3ff87b811841c
Reviewed-on: http://review.gluster.org/7403
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
Reviewed-by: Anand Avati <avati@redhat.com>
build: move argp-standalone into contrib/ directory

Change-Id: Iedcddf95c3577da644c0aebbb297b04c93f1b6fe
BUG: 1081274
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7352
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
rpm: fix broken glusterfs-server install of hooks

the %ghost hook directories are no longer ghosts now that files
are installed in them

Change-Id: I6133b891c73d87e4e35dc9c6c7ba7febbc9e2e7f
BUG: 1080970
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/7391
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
log: Add missing log message from glusterfsd.c to glusterfsd-messages.h

... by retaining GLFS_NUM_MESSAGES as 33 which is its correct value.

Also replace all occurrences of gf_log with gf_msg/gf_msg_debug.

Change-Id: Ibfbe1d645de521e8d59ca406f78b1a8eb08aa7e0
BUG: 1075611
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/7371
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/barrier: add barrier translator

gluster feature page:
http://www.gluster.org/community/documentation/index.php/Features/Server-side_Barrier_feature

Change-Id: Ia9f8802a54d1ffbd1cf789b80f5d30819bf65f64
BUG: 1060002
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/6928
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
feature/barrier: Add statedump support

This patch adds statedump support for barrier. This currently dumps
barrier xlators private information and the queue of barriered fops.

Change-Id: I273eb6e676db02c40c363feeff58a79737dc041e
BUG: 1060002
Reviewed-on: http://review.gluster.org/7136
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests/afr: self-heal

Basic functional tests related to self-heal.

arequal-checksum.c is taken from
https://github.com/raghavendrabhat/arequal after consent from all
authors.

Change-Id: I43facc31c61375f4dbe58bbb46238e15df5c9011
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7357
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
geo-rep: code pep8/flake8 fixes

pep8 is a style guide for python.
http://legacy.python.org/dev/peps/pep-0008/

pep8 can be installed using, `pip install pep8`

Usage: `pep8 <python file>`, For example, `pep8 master.py`
will display all the coding standard errors.

flake8 is used to identify unused imports and other issues
in code.

pip install flake8

cd $GLUSTER_REPO/geo-replication/
flake8 syncdaemon

Updated license headers to each source file.

Change-Id: I01c7d0a6091d21bfa48720e9fb5624b77fa3db4a
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7311
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
tests: Increase bug-865825.t wait time for self-heal daemon

BUG: 1084653
Change-Id: I057bbd2e50803344552314b32d2d0e6240bf9604
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7404
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: In glfs_set_volfile_server() remove the port number check.

The documentation for glfs_set_volfile_server() api says that the
port zero is a valid value. Specifying 0 uses the default port
number GF_DEFAULT_BASE_PORT. Hence removing the check for port being
zero.

Change-Id: I942a7fe54a418231a438ab67756537df8f28d2de
BUG: 1084964
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/7409
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
geo-rep: fix the code bug introduced due to flake8 refactoring

Sorry for the bug, which got introduced due to code refactoring.
http://review.gluster.org/#/c/7311/

Change-Id: Ide519ca114aa8a7d7624d7af99945c857f069069
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7417
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
rpc: warn and truncate grouplist if RPC/AUTH can not hold everything

The GlusterFS protocol currently uses AUTH_GLUSTERFS_V2 in the RPC/AUTH
header. This header contains the uid, gid and auxiliary groups of the
user/process that accesses the Gluster Volume.

The AUTH_GLUSTERFS_V2 structure allows up to 65535 auxiliary groups to
be passed on. Unfortunately, the RPC/AUTH header is limited to 400 bytes
by the RPC specification: http://tools.ietf.org/html/rfc5531#section-8.2

In order to not cause complete failures on the client-side when trying
to encode a AUTH_GLUSTERFS_V2 that would result in more than 400 bytes,
we can calculate the expected size of the other elements:

    1 | pid
    1 | uid
    1 | gid
    1 | groups_len
   XX | groups_val (GF_MAX_AUX_GROUPS=65535)
    1 | lk_owner_len
   YY | lk_owner_val (GF_MAX_LOCK_OWNER_LEN=1024)
  ----+-------------------------------------------
    5 | total xdr-units

  one XDR-unit is defined as BYTES_PER_XDR_UNIT = 4 bytes
  MAX_AUTH_BYTES = 400 is the maximum, this is 100 xdr-units.
  XX + YY can be 95 to fill the 100 xdr-units.

  Note that the on-wire protocol has tighter requirements than the
  internal structures. It is possible for xlators to use more groups and
  a bigger lk_owner than that can be sent by a GlusterFS-client.

This change prevents overflows when allocating the RPC/AUTH header. Two
new macros are introduced to calculate the number of groups that fit in
the RPC/AUTH header, when taking the size of the lk_owner in account. In
case the list of groups exceeds the maximum possible, only the first
groups are passed over the RPC/GlusterFS protocol to the bricks.
A warning is added to the logs, so that most system administrators will
get informed.

The reducing of the number of groups is not a new inventions. The
RPC/AUTH header (AUTH_SYS or AUTH_UNIX) that NFS uses has a limit of 16
groups. Most, if not all, NFS-clients will reduce any bigger number of
groups to 16. (nfs.server-aux-gids can be used to workaround the limit
of 16 groups, but the Gluster NFS-server will be limited to a maximum of
93 groups, or fewer in case the lk_owner structure contains more items.)

Change-Id: I8410e59d0fd246d601b54b961d3ae9cb5a858c10
BUG: 1053579
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7202
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Init local on txn-frame for zerofill

Change-Id: I516f4fb0237dd0b3e512117bf987cea69f8678b8
BUG: 1084485
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7407
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>