geo-rep: fix the code bug introduced due to flake8 refactoring

Sorry for the bug, which got introduced due to code refactoring.
http://review.gluster.org/#/c/7311/

Change-Id: Ide519ca114aa8a7d7624d7af99945c857f069069
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/7417
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
rpc: warn and truncate grouplist if RPC/AUTH can not hold everything

The GlusterFS protocol currently uses AUTH_GLUSTERFS_V2 in the RPC/AUTH
header. This header contains the uid, gid and auxiliary groups of the
user/process that accesses the Gluster Volume.

The AUTH_GLUSTERFS_V2 structure allows up to 65535 auxiliary groups to
be passed on. Unfortunately, the RPC/AUTH header is limited to 400 bytes
by the RPC specification: http://tools.ietf.org/html/rfc5531#section-8.2

In order to not cause complete failures on the client-side when trying
to encode a AUTH_GLUSTERFS_V2 that would result in more than 400 bytes,
we can calculate the expected size of the other elements:

    1 | pid
    1 | uid
    1 | gid
    1 | groups_len
   XX | groups_val (GF_MAX_AUX_GROUPS=65535)
    1 | lk_owner_len
   YY | lk_owner_val (GF_MAX_LOCK_OWNER_LEN=1024)
  ----+-------------------------------------------
    5 | total xdr-units

  one XDR-unit is defined as BYTES_PER_XDR_UNIT = 4 bytes
  MAX_AUTH_BYTES = 400 is the maximum, this is 100 xdr-units.
  XX + YY can be 95 to fill the 100 xdr-units.

  Note that the on-wire protocol has tighter requirements than the
  internal structures. It is possible for xlators to use more groups and
  a bigger lk_owner than that can be sent by a GlusterFS-client.

This change prevents overflows when allocating the RPC/AUTH header. Two
new macros are introduced to calculate the number of groups that fit in
the RPC/AUTH header, when taking the size of the lk_owner in account. In
case the list of groups exceeds the maximum possible, only the first
groups are passed over the RPC/GlusterFS protocol to the bricks.
A warning is added to the logs, so that most system administrators will
get informed.

The reducing of the number of groups is not a new inventions. The
RPC/AUTH header (AUTH_SYS or AUTH_UNIX) that NFS uses has a limit of 16
groups. Most, if not all, NFS-clients will reduce any bigger number of
groups to 16. (nfs.server-aux-gids can be used to workaround the limit
of 16 groups, but the Gluster NFS-server will be limited to a maximum of
93 groups, or fewer in case the lk_owner structure contains more items.)

Change-Id: I8410e59d0fd246d601b54b961d3ae9cb5a858c10
BUG: 1053579
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7202
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Init local on txn-frame for zerofill

Change-Id: I516f4fb0237dd0b3e512117bf987cea69f8678b8
BUG: 1084485
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7407
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
features/locks: Fix a missing assignment in new_entrylk_lock()

Change-Id: If5c03456d61ec930d588b57781fb545eed18e4a2
BUG: 1085220
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/7413
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
cluster/afr: Mem leak fixes found in valgrind for iozone

Change-Id: I869d191dc3470b2208c17343bbf772f01ef744cb
BUG: 1085511
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7424
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: call runner_end even if runner_start fails

Change-Id: I5eca01a131307ba3be2aed4922eea73025ff284c
BUG: 1081013
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/7360
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Set right argument order for STACK_WIND_COOKIE

Change-Id: Ia26e17a7147ed825319c7c29880b9cf4ae80a48c
BUG: 1085259
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/7416
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
build: set version based on git tag or specific

This patch brings version and release number generated at build time
using git tags or fixed content from VERSION file.

With git tag, version/release number are got from output of
'git describe --tags --match "v[0-9]*"' command.

This behavior can be overriden by having VERSION file with fixed
version/release.  The VERSION file should have text describing version
and release for example something like 'v3.4.0-1'

For testing this patch, its required to remove autom4te.cache
directory to avoid seeing previously set version.

BUG: 1074919
Change-Id: I8f68172e8b389b0ba0846e9adb4b597e67a909aa
Signed-off-by: Bala.FA <barumuga@redhat.com>
Reviewed-on: http://review.gluster.org/7164
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
build: rearrange spec file elements

This is majorly cleanup wrt spec file elements/definitions.  This
patch fixes to ease adding downstream specific conditions.

BUG: 1074939
Change-Id: I311abf8457417591a0d08569d3f83d73fa63cdc0
Signed-off-by: Bala.FA <barumuga@redhat.com>
Reviewed-on: http://review.gluster.org/7165
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
gluster: GlusterFS Volume Snapshot Feature

This is the initial patch for the Snapshot feature. Current patch
includes following features:
* Snapshot create
* Snapshot delete
* Snapshot restore
* Snapshot list
* Snapshot info
* Snapshot status
* Snapshot config

Change-Id: I2f46920c0d61c515f6a60e0f8b46fff886d9f6a9
BUG: 1061685
Signed-off-by: shishir gowda <sgowda@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Signed-off-by: Vijaikumar M <vmallika@redhat.com>
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7128
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli,glusterd: Improve detach check validation

This patch improves the validation for the 'peer detach' command.

A check for if volumes exist with some bricks on the peer being detached
validation is added in peer detach code flow (even force would have this
validation).

This patch also gurantees that peer detach doesn't fail for a volume with all
its brick on the peer which is getting detached and there are no other bricks on
this peer.

The following steps need to be followed for removing a downed and unrecoverable
peer.
* If a replacement system is available
  - add it to the cluster
  - use replace-brick to migrate bricks of the downed peer to the new
    peer (since data cannot be recovered anyway use the 'replace-brick
    commit force' command)
  or,
  If no replacement system is available,
  - remove bricks of the downed peer using 'remove-brick'

Change-Id: Ie85ac5b66e87bec365fdedd8352b645bb25e1c33
BUG: 983590
Signed-off-by: Kaushal M <kaushal@redhat.com>
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/5325
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
doc: add documentation for the "Brick Failure Detection" feature

The documentation from the feature page should be included in the
sources.

Change-Id: I4fd67ce1c56afc5236c00de8be9110dfa6bbe91f
BUG: 1086700
Feature-page: http://www.gluster.org/community/documentation/index.php/Features/Brick_Failure_Detection
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7449
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Adding admin-guide for distributed-geo-rep

Change-Id: Icbbc9d3fc14fc1a5ae9550f096eb69dda41e92a4
BUG: 1086796
Signed-off-by: M S Vishwanath Bhat <vbhat@redhat.com>
Reviewed-on: http://review.gluster.org/7437
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
mount.glusterfs: return an error when mounting failed

When mounting fails, mount.glusterfs incorrectly returns 0 for some
error cases, it should return 1 instead. Also make sure that error
messages are redirected to /dev/stderr and not printed to stdout.

Unfortunately it is not possible with the current test-scripts to test
commands like 'mount -t glusterfs ...'. Any mounting of Gluster volumes
is done directly with the 'glusterfs' command instead.

Change-Id: Ica9d45b6d5ae537de869a1fa0f6c3edab47225d1
BUG: 1031973
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7441
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
logging: Make logger and log format configurable through cli

Change-Id: Ic4b701a6621578848ff67ae4ecb5a10b5f32f93b
BUG: 1075611
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/7372
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
pkg-version: Use /bin/bash instead of /bin/sh

`function` is reserved word only available in /bin/bash,
Using `/bin/sh` with `function` keyword fails on Ubuntu.

Change-Id: Ia0fff8f13ace2a0eb1dac50093ff42294d960814
BUG: 1074919
Reviewed-on: http://review.gluster.org/7439
Reviewed-by: Bala FA <barumuga@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
build: set run levels for glusterd init script properly

'chkconfig --add glusterd' does not set run levels for start/stop.
This patch fixes it.

Change-Id: I8962f5207d034b6875353e15eb60c4d08c5a4fe4
BUG: 1086141
Signed-off-by: Bala.FA <barumuga@redhat.com>
Reviewed-on: http://review.gluster.org/7432
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
configure: argp-standalone is always built, even when not needed

This patch still configures argp, i.e. creates Makefile in
/contrib/argp-standalone, for the `make dist`, but there's no reason
to compile it when it's not going to be linked into anything.

Change-Id: Id0fcb717b5821d8a4c176e6274339f46b3a4a249
BUG: 1086773
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/7440
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
dict: add dict_set_dynstr_with_alloc

There is an overwhelming no. of instances of the following pattern in
glusterd module.

    ...

    char *dynstr = gf_strdup (str);
    if (!dynstr)
       goto err;
    ret = dict_set_dynstr (dict, key, dynstr);
    if (ret)
       goto err;

    ...

With this changes it would look as below,

   ret = dict_set_dynstr_with_alloc (dict, key, str);
   if (ret)
       goto err;

Change-Id: I6a47b1cbab4834badadc48c56d0b5c8c06c6dd4d
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/7379
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
doc: Add doc for network compression xlator

Change-Id: Iaed94f1bfe7c97eade95923759980a1d10805663
BUG: 1086752
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/7479
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>