mgmt/glusterd: Update sub_count on remove brick

Change-Id: I7c17de39da03c6b2764790581e097936da406695
BUG: 1002556
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/5902
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: Use volume op-versions during volgen

  Backport of '3af61d6 glusterd: Use volume op-versions during volgen'
  from master

Instead of using the cluster op-version, volume op-version is used to
enable open-behind during volgen. For doing this, the volume op-versions
are updated before regenerating the volfiles.

BUG: 990830
Change-Id: I07e4a34004816c803fcbb3ee1ddd4b1e4c3a8006
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5831
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: Calculate volume op-versions only on set/reset

  Backport of http://review.gluster.org/5568

The volume op-versions are calculated during a volume set/reset, reading a
volume from disk and importing a volume during probe or volume sync. The
calculation of the volume op-version depends on the clusters op-version as some
features are enabled automatically depending on the clusters op-version. We
also don't store the volume op-versions persistently and don't export the
volume op-versions during sync. Due to this, there can occur cases which will
lead to inconsistencies in volumes in different peers. One such case is below,
Consider, a cluster made up 3 peers P1, P2 and P3, operating at op-version N.
The cluster has two volumes V1 and V2, which have volume op-versions N (since
volume op-version cannot be greater than cluster op-version). We have,
 Cluster-op-version = N
 V1 op-version = N
 V2 op-version = N
A set operation on V1 causes the clusters op-version to be bumped up to N+1.
Assume that there exist some features that are automatically enabled on
op-version N+1. The op-version of V2 remains at N as no operation has been
performed on it. So,
 Cluster op-version = N+1
 V1 op-version = N+1
 V2 op-version = N
Now, we probe a new peer P4. On the new peer we will have the following
op-versions,
 Cluster op-version = N+1
 V1 op-version = N+1
 V2 op-version = N+1
This happens because we don't send volume op-versions during the sync after
probe. P4 will freshly calculate the op-version of V2 (assuming features have
been auto enabled due to the cluster op-version being N+1) as N+1.
Another case is when glusterd on a peer restarts. Assume P3 was restarted,
glusterd will recalculate the volume op-versions during the restore state.
Again, op-version of V2 will be calculated as N+1 assuming auto enabled
features. This will lead to inconsistency in the volume representation in
memory and on disk, as glusterd will assume the volume contains auto enabled
features, but the volfiles don't contain them as they were not regenrated.
These kind of issues can be solved by calculating the volume op-version only
when features are enabled and disabled (ie. during volume set/reset),
persisting the volume-op-versions and exporting/importing them.

BUG: 1005043
Change-Id: Id8bb05ba2a77e510739b3b1833f98b4d6d1fa4d7
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5832
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests: Remove grep process entries from pidgrep

Problem:
We were picking process with lowest pid from ps|grep
result. However, lowest pid need not be oldest process
as recycling of PIDs can take place.

Solution:
Removed grep process entries from ps entries using
grep -v grep.

Change-Id: I2b9687a05a34cf6358f773183770d69a3fb9eb10
BUG: 858488
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/5930
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli: Add statusStr xml tag to task list and rebalance/remove brick status

 Backport of cd7951aa31ae4bbdf35fc6d1f2466636844b889d from master

New xml tag statusStr added to following gluster cli commands
gluster volume status all --xml (For Task status)
gluster volume rebalance <VOLNAME> status --xml
gluster volume remove-brick <VOLNAME> <BRICK1..> status --xml

Example(volume status all):
<task>
    <type>Rebalance</type>
    <id>82d8d122-8738-4144-8507-d93fc98b61df</id>
    <status>3</status>
    <statusStr>completed</statusStr>
</task>

Example(volume rebalance <VOL> status)
<node>
    <nodeName>localhost</nodeName>
    <files>0</files>
    <size>0</size>
    <lookups>0</lookups>
    <failures>0</failures>
    <status>3</status>
    <statusStr>completed</statusStr>
</node>

Also modified task status as string instead of showing number
in gluster volume status all

Example:
Status of volume: gv1
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick sumne.sumne:/gfs/b1                               49154   Y       15489
Brick sumne.sumne:/gfs/b2                               49155   Y       15493
NFS Server on localhost                                 N/A     N       15913

           Task                                      ID         Status
           ----                                      --         ------
      Rebalance    82d8d122-8738-4144-8507-d93fc98b61df      completed

BUG: 1003521
Change-Id: I2f6845b621cbd5a74aeb3a3195f944536745e0ec
Original-author: Aravinda VK <avishwan@redhat.com>
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5969
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli: Fix 'status all' xml output when volumes are not started

 Backport of 7d9bc0d21408c31651a65a6ec0e67c3b8acd0fde from master

CLI now only outputs one XML document for 'status all' only containing
those volumes which are started.

BUG: 1004218
Change-Id: I119ac40282380886b46a09fd9a19d35115fd869d
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5970
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
mount/fuse: Implement forget in cbks for fuse.

With the introduction of inode_ctx_set in fuse as part of  2991503d014,
forget cbk gets called for fuse xlator. Though nothing needs to be done
inf forget_cbk, excessive log messages of the following kind are observed:

[2013-09-16 06:09:50.758063] W [defaults.c:1331:default_forget]
(-->/usr/local/lib/glusterfs/3git/xlator/mount/fuse.so(+0xa1f2) [0x7f51432781f2]
(-->/usr/local/lib/libglusterfs.so.0(inode_unref+0x3c) [0x7f5144e5
816c] (-->/usr/local/lib/libglusterfs.so.0(+0x2d061) [0x7f5144e58061]))) 0-fuse:
xlator does not implement forget_cbk

This patch prevents such log messages from being seen.
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
BUG: 979910
Change-Id: Ie5874138f46822b10ff4213bd1134d78330ec460
Reviewed-on: http://review.gluster.org/5932
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5975
Tested-by: Gluster Build System <jenkins@build.gluster.com>
rpcsvc: allocate large auxgid list on demand

For rpc requests having large aux group list, allocate large list
on demand. Else use small static array by default.

Without this patch, glusterfsd allocates 140+MB of resident memory
just to get started and initialized.

Change-Id: I3a07212b0076079cff67cdde18926e8f3b196258
Signed-off-by: Anand Avati <avati@redhat.com>
BUG: 953694
Reviewed-on: http://review.gluster.org/5927
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5928
glusterd: Don't reset rebalance status on add-brick

 Backport of 67c28b19355c47e96d1420405cc38753a3e5f9be from master

The rebalance status was being reset to 'Not started' when add-brick was
performed. This would lead to odd cases where a 'rebalance status' on a
volume would show status as 'not started' but would also include the
rebalance statistics. This also affected the showing of asynchronus task
status in 'volume status' command.

By not resetting the status prevent the above issues from happening.
Since we use the running/not-running of the rebalance process as the
check when performing other operations we can safely leave the rebalance
stats collected on an add-brick.

BUG: 1006247
Change-Id: Idade88d9e5a6f27659490b3e6d85495d426ef0a3
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5971
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli: add aggregate status for rebalance and remove-brick status xml output

 Backport of 91e4b7aa1361087317238b9bf6427ef274737c8c from master

Add aggregate status information in <aggregate> section of gluster volume
'rebalance status' and 'remove-brick status cli xml output.

The aggregate status determined based on the most critical level
and the aggregate status will have 'Complete' only when all
individual status are 'Complete'.

BUG: 1006813
Change-Id: I803d4bfb31dbda53b5665332263576c910562805
Original-author: Timothy Asir <tjeyasin@redhat.com>
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5972
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli,glusterd: Task parameters in xml output

 Backport of 91cd0eae2cc1d96cbafa6457835f146503355238 from master

This patch introduces task parameters for the asynchronus task shown in
volume status. The parameters are only given for xml output. The
parameters shown currently are,
- source and destination bricks for replace-brick tasks
  ......
        <tasks>
          <task>
            <type>Replace brick</type>
            <id>3d1a1005-9d2e-4ae0-bd62-577bc1d333a3</id>
            <status>1</status>
            <params>
              <srcBrick>archm:/export/test4</srcBrick>
              <dstBrick>archm:/export/test-replace1</dstBrick>
            </params>
          </task>
        </tasks>
  ......
- list of bricks being removed for remove-brick tasks
  ......
        <tasks>
          <task>
            <type>Remove brick</type>
            <id>901c20ca-0da2-41de-8669-5f0caca6b846</id>
            <status>1</status>
            <params>
              <brick>archm:/export/test2</brick>
              <brick>archm:/export/test3</brick>
            </params>
          </task>
        </tasks>
  ......

The changes for non-xml output will be done in a subsequent patch.

BUG: 916577
Change-Id: Iade8a4974aefc5ffb080553496ae5a3169055090
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5973
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gfapi: implement a minimial glfs_fini()

At the very least, we should PARENT_DOWN on the currently active
graph and disconnect ourselves from glusterd.

Further cleanups underway.

Change-Id: I9276686a84b0975b5ce272b4cbec1b80920d5c5c
BUG: 1004519
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5903
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/dht: Treat migration failures due to space constraints as skipped

Currently rebalance/remove-brick op's display migration failed count even
for files which failed due to space issues (not enough space for file, or
migration leading to cluster imbalance)

These will now be counted as skipped, and rebalance/remove-brick status
will display the additional counter

BUG: 989846
Change-Id: I4efa7ce69dd43680ff47181afed0c561954c5080
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5977
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gNFS: avoid double fd unref in opendir

Noticed that the fd_unref was called on the fd regardless of the return
value at nfs3svc_opendir_readdir_cbk(), hence removing an extra unref in
the negative case in nfs_inode_opendir_cbk, which fixes the spurious
fd_unref().

Back port of: http://review.gluster.org/4943 (Rajesh Amaravathi)

Change-Id: Ibddf487c7890407d01befedd65eefb10cb9c989f
BUG: 1011761
Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>
Reviewed-on: http://review.gluster.org/5996
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
posix-acl: fixup extended ACL entries properly

Typically when updating cached ACL from backend, we get both iatt and
ACL xattrs (like lookup, readdirplus etc.) However in calls like
setattr(), the mode would have updated but we receive only iatt and not
the ACL xattrs. In such case we need to "spread" the effects of the
changed mode properly into the cached ACL xattr ourselves.

Change-Id: I23a7bc9c14722ff6848e175ed4bbe863a21ce2c9
BUG: 998967
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6002
Tested-by: Gluster Build System <jenkins@build.gluster.com>
doc: Release notes for 3.4.1

Change-Id: Id4bb25c3fbda52700b1d4d624cd622db3b3c0d8a
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
BUG: 811311
Reviewed-on: http://review.gluster.org/6011
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
build: sync glusterfs.spec.in with Fedora glusterfs.spec

(this is the release-3.4 branch)

sync with Fedora glusterfs.spec 3.4.1-2+

Change-Id: I13e7161dca969804415045a47e37705819fe173d
BUG: 819130
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/6058
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
nfs/mount3: fix crash in subdir resolution

* __mnt3_resolve_export_subdir_comp: if nfs_entry_loc_fill fails,
  mres->resolveloc does not contain valid data
* gf_log should use 'gfid' instead of mres->resolveloc.inode->gfid
* fixes a crash if program flow gets to this line

Backport FIX link: http://review.gluster.org/4948

Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>

Change-Id: I6dea17ae3f07210430a0b71bd08be6f59f739c9a
BUG: 1018547
Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>
Reviewed-on: http://review.gluster.org/6083
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
cli: runtime in xml output of rebalance/remove-brick status

"runtime in secs" is available in the CLI output of
rebalance status and remove-brick status, but not available
in xml output when --xml is passed.

runtime in aggregate section will be max of all nodes runtimes.

Example output:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
  <volRebalance>
    <op>3</op>
    <nodeCount>1</nodeCount>
    <node>
      <nodeName>localhost</nodeName>
      <files>0</files>
      <size>0</size>
      <lookups>0</lookups>
      <failures>0</failures>
      <skipped>0</skipped>
      <runtime>1.00</runtime>
      <status>3</status>
      <statusStr>completed</statusStr>
    </node>
    <aggregate>
      <files>0</files>
      <size>0</size>
      <lookups>0</lookups>
      <failures>0</failures>
      <skipped>0</skipped>
      <runtime>1.00</runtime>
      <status>3</status>
      <statusStr>completed</statusStr>
    </aggregate>
  </volRebalance>
</cliOutput>

BUG: 1012773
Change-Id: I6de59d4ed03983b6ffc014d6a331251cf635a690
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/5997
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/6118
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli: skipped tag in xml output of rebalance/remove-brick status

Skipped files count is available in CLI output of rebalance status
and remove-brick status, but not available in xml output.

Example output:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
  <volRebalance>
    <op>3</op>
    <nodeCount>1</nodeCount>
    <node>
      <nodeName>localhost</nodeName>
      <files>0</files>
      <size>0</size>
      <lookups>0</lookups>
      <failures>0</failures>
      <skipped>0</skipped>
      <status>0</status>
      <statusStr>completed</statusStr>
    </node>
    <aggregate>
      <files>0</files>
      <size>0</size>
      <lookups>0</lookups>
      <failures>0</failures>
      <skipped>0</skipped>
      <status>0</status>
      <statusStr>completed</statusStr>
    </aggregate>
  </volRebalance>
</cliOutput>

BUG: 1012772
Change-Id: I4f5128c3fa3b2b791bcbaf41ac7f229f80bfefe9
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/6000
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/6119
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>