build: sync glusterfs.spec.in with Fedora glusterfs.spec

 * add glusterfs-api and glusterfs-api-devel sub-packages
 * add extras/systemd/glusterd.service. Use it (or extras/init.d/glusterd)
   instead of the Fedora versions.
 * restores ability to build glusterfs RPMs from the dist tarball with
   'rpmbuild {-ts,-tb,-ta} glusterfs-XXX.tar.gz'
 * other minor cleanups mostly to sync with fedora .spec and to build
   from the dist tarball. Any differences will be resolved in the fedora
   .spec in the next release (i.e. beta4 or GA).

(still considering whether to add fedora glusterfsd.{init,service} files
to glusterfs or remove them from fedora.)

BUG: 950083
Change-Id: Ibda3cd57b24ef5c2a27446703e718e5044ec546c
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/5230
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Bala FA <barumuga@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
rpm: move the mount/api xlator to the glusterfs-api sub-package

The main glusterfs package included the xlator/api.so file which caused
a dependency on the glusterfs-api package:

  # rpm -i glusterfs-3.4.0-0.6.beta3.el6.x86_64.rpm
  error: Failed dependencies:
  libgfapi.so.0()(64bit) is needed by glusterfs-3.4.0-0.6.beta3.el6.x86_64

Moving the mount/api xlator to the glusterfs-api package will prevent
the automatic dependency generator to add the dependency to the main
package.

Change-Id: I4650c40cdfe610b9b69bab07d262093854aa6583
BUG: 950083
Reported-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5255
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
nfs/auth: reject mounts if getaddrinfo fails

When nfs.addr-namelookup is turned on, if the
getaddrinfo call fails while authenticating client's
ip/hostname, the mount request is denied

Change-Id: I744f1c6b9c7aae91b9363bba6c6987b42f7f0cc9
BUG: 947055
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.org/5143
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
build: declare lvm_lv_from_name() if it is missing from lvm2app.h

The bd-xlator can not be built successfully on certain Debian
distributions due to a missing declaration of lvm_lv_from_name(). This
function is available for linking, but it does not exist in the header
file.

This change adds a detection for lvm_lv_from_name() in both the library
for linking, and the declaration in the header file. If the 1st is
missing, the bd-xlator can not be built, and if only the 2nd one is
missing, we'll declare lvm_lv_from_name() ourselves. This makes it
possible to build the bd-xlator on the affected Debian distributions
too.

Change-Id: I0c823a7861b02bb5d9c1abb76ebfff92f272f9eb
BUG: 976946
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5250
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
protocol/server: Initialize connection variable

Change-Id: I46badd812e9b936911ddd2793cef7ce30ec220a6
BUG: 979237
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5266
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/afr: Handle NULL fdctx in fsync

Problem:
If fdctx is NULL in afr_fsync, process crashes because
of NULL dereference.

Fix:
if fdctx is NULL, always say witnessed unstable write so
that fsyncs are done properly. Handled fdctx being null
in afr_delayed_changelog_post_op otherwise fsync stub is
never resumed and the mount was hanging.

Change-Id: Icacc900e9be63c29db3325cb0e19cc250adebaac
BUG: 978794
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5258
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
nfs: exit when all volumes are disabled

Instead of triggering 4-5 error logs, when nfs is
disabled for all volumes, exit the process.

Change-Id: Ib286f143c4f74ba22f502aca0e7dcd0907db6563
BUG: 976750
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.org/5245
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
build: sync glusterfs.spec.in with Fedora glusterfs.spec

* fix the hardening fix for shlibs
* use %{__sed} macro
* better short ChangeLog
* fix hard-coded path in extras/systemd/glusterd.service in, use @prefix@

Change-Id: I267244b68a97cbcd6f6ab62fc322c78db45023bb
BUG: 950083
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/5264
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
extras/hook-scripts: Set of changes for new samba share method.

New method of creating a share of gluster volume
through samba eliminates the requirement of fuse
mount and changes in fstab.

glusterfs vfs plugin has to be specified as required
vfs object.
Path now is relative to the root of gluster volume.
/ signifies complete volume.

Change-Id: I3d80452cd58513ef59d6a816c1ab62ebc3b76a62
BUG: 953694
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/5247
Reviewed-by: José Rivera <jrivera@redhat.com>
Tested-by: José Rivera <jrivera@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
libglusterfs: Fix valid_host_name for consecutive dots

The valid_host_name() function must reject addresses with consecutive
dots in them.

Change-Id: I1749de80c66e8fbad63b2e014f79e0203906030e
BUG: 977246
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5249
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Allow data/entry self heal for metadata split-brain

Problem:
Currently whenever there is metadata split-brain, a variable
sh->op_failed is set to 1 to denote that self heal got failed.
But if we proceed for data self heal, even code-path of data
self heal also relies on the sh->op_failed variable. So if will
check for sh->op_failed variable and will eventually fails to
do data self heal. So needed a mechanism to allow data self heal
even if metadata is in split brain.

Fix:
Some data structure revamp is done in
http://review.gluster.com/#/c/5106/ fix and this patch is
based on the above fix. Now we can store which particular self-heal
got failed i.e GFID_OR_MISSING_ENTRY_SELF_HEAL, METADATA, DATA,
ENTRY. And we can do two types of self heal failure check.
1. Individual type check: We can check which among all four
   (Metadata, Data, Gfid or missing entry, entry self heal)
   got failed.

2. In afr_self_heal_completion_cbk, we need to make check
   based on the fact that if any specific self heal got failed treat
   the complete self heal as failure so that it will populate
   corresponding circular buffer of event history accordingly.

Change-Id: Icb91e513bcc752386fc8a78812405cfabe5cac2d
BUG: 977797
Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com>
Reviewed-on: http://review.gluster.org/5253
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
store: Fix resource leaks in gf_store_iter_* functions

Also, removed (redundant) member fd from gf_store_iter_t

Change-Id: I40f0469997f77fa2f578a5495ca4ce285f1a59f2
BUG: 904065
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/5243
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
glusterd: More checks before starting rebalance/remove-brick

Check if a previous remove-brick operation has been committed before
starting a new rebalance/remove-brick task.

Change-Id: I553e5ba64a6a352ca91032ab1a17997051a4494e
BUG: 963541
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5019
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/afr: post-op should complete before starting flush

Problem:
At the moment afr-flush makes sure that a delayed post-op
is woken up but it does not wait for it to complete the
post-op before flush unwinds.
These are the steps that are happening:
1) flush fop comes on an fd which wakes up a delayed post-op
and continues with the flush fop.
2) post-op sends fsync on the wire.
3) flush completes and unwinds to fuse.
4) graph switch happens on the fuse mount disconnecting the
old graph's client connections to bricks.
5) xattrop after fsync fails with ENOTCONN because the
connections from old graph are taken down now.

Fix:
Wait for post-op to complete before starting to flush.
We could make flush act similar to fsync (i.e.) wind
flush as is but wait for post-op to complete before unwinding
flush, but it is better to send flush as the final fop. So
wind of flush will start after post-op is complete. Had to
change fsync to accommodate this change.

Change-Id: I93aa642647751969511718b0e137afbd067b388a
BUG: 980548
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5274
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Provide an option to disable afr durability

Change-Id: I40eec20ca6b3f857245a2438883822e251077ee9
BUG: 979365
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5269
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Refactor inodelk to handle multiple domains

- afr_local_copy should not be memduping locked nodes, that would
  mean that lock is taken in self-heal on those nodes even before
  it actually takes the lock. So removed memdup code. Even entry
  lock related copying (lockee info) is also not necessary for
  self-heal functionality, so removing that as well. Since it is
  not local_copy anymore changed its name.

- My editor changed tabs to spaces.

Change-Id: I8dfb92cb8338e9a967c06907a8e29a8404782d61
BUG: 967717
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5099
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Let two data-self-heals compete in new domain

Problem:
At the moment data-self-heal acquires locks in following
pattern. It takes full file lock then gets xattrs on files on both
replicas. Decides sources/sinks based on the xattrs. Now it acquires
lock from 0-128k then unlocks the full file lock. Syncs 0-128k range
from source to sink now acquires lock 128k+1 till 256k then unlocks
0-128k, syncs 128k+1 till 256k block... so on finally it takes full file
lock again then unlocks the final small range block.
It decrements pending counts and then unlocks the full file lock.

     This pattern of locks is chosen to avoid more than 1 self-heal
to be in progress. BUT if another self-heal tries to take a full
file lock while a self-heal is already in progress it will be put in
blocked queue, further inodelks from writes by the application will
also be put in blocked queue because of the way locks xlator grants
inodelks. So until the self-heal is complete writes are blocked.

Here is the code:
xlators/features/locks/src/inodelk.c - line 225
if (__blocked_lock_conflict (dom, lock) && !(__owner_has_lock (dom, lock))) {
         ret = -EAGAIN;
         if (can_block == 0)
                 goto out;

         gettimeofday (&lock->blkd_time, NULL);
         list_add_tail (&lock->blocked_locks, &dom->blocked_inodelks);
}

This leads to hangs in applications.

Fix:
Since we want to prevent two parallel self-heals. We let them compete
in a separate "domain". Lets call the domain on which the locks have
been taken on in previous approach as "data-domain".

In the new approach When a self-heal is triggered,
it acquires a full lock in the new domain "self-heal-domain".
    After this it performs data-self-heal using the locks in
    "data-domain" as before.
unlock the full file lock in "self-heal-domain"

With this approach, application's writevs don't have to wait
in pending queue when more than 1 self-heal is triggered.

Change-Id: Id79aef3dfa888945977fb9758374ac41c320d0d5
BUG: 967717
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5100
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli: Fix in letter case in volume heal output

Change-Id: I25d13444c2cbff9b26642e91677ad1e09e77aa1e
BUG: 978936
Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com>
Reviewed-on: http://review.gluster.org/5259
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
fuse: pass the mountflags to older (< 2.6.21) kernels

The change for Bug 853895 fixed mounting volumes read-only for recent
kernels. Older kernels fail the first mount() syscall, and the second
mount() did not add the 'mountflags'.

Full analysis and a description for reproducing is in the bugreport. The
test included in http://review.gluster.org/4163 would have caught this
problem when the tests are executed on RHEL-5 or similar systems.

Change-Id: I440591344a6a5af7b2018e37a2a1fda9de8b5ab2
Bug: 980770
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5278
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
posix: add a simple health-checker

Goal of this health-checker is to detect fatal issues of the underlying
storage that is used for exporting a brick. The current implementation
requires the filesystem to detect the storage error, after which it will
notify the parent xlators and exit the glusterfsd (brick) process to
prevent further troubles.

The interval the health-check runs can be configured per volume with the
storage.health-check-interval option. The default interval is 30
seconds.

It is not trivial to write an automated test-case with the current
prove-framework. These are the manual steps that can be done to verify
the functionality:

- setup a Logical Volume (/dev/bz970960/xfs) and format is as XFS for
  brick usage

- create a volume with the one brick

    # gluster volume create failing_xfs glufs1:/bricks/failing_xfs/data
    # gluster volume start failing_xfs

- mount the volume and verify the functionality

- make the storage fail (use device-mapper, or pull disks)

    # dmsetup table
    ..
    bz970960-xfs: 0 196608 linear 7:0 2048

    # echo 0  196608 error > dmsetup-error-target
    # dmsetup load bz970960-xfs dmsetup-error-target
    # dmsetup resume bz970960-xfs

    # dmsetup table
    ...
    bz970960-xfs: 0 196608 error

- notice the errors caught by syslog:

    Jun 24 11:31:49 vm130-32 kernel: XFS (dm-2): metadata I/O error: block 0x0 ("xfs_buf_iodone_callbacks") error 5 buf count 512
    Jun 24 11:31:49 vm130-32 kernel: XFS (dm-2): I/O Error Detected. Shutting down filesystem
    Jun 24 11:31:49 vm130-32 kernel: XFS (dm-2): Please umount the filesystem and rectify the problem(s)
    Jun 24 11:31:49 vm130-32 kernel: VFS:Filesystem freeze failed
    Jun 24 11:31:50 vm130-32 GlusterFS[1969]: [2013-06-24 10:31:50.500674] M [posix-helpers.c:1114:posix_health_check_thread_proc] 0-failing_xfs-posix: health-check failed, going down
    Jun 24 11:32:09 vm130-32 kernel: XFS (dm-2): xfs_log_force: error 5 returned.
    Jun 24 11:32:20 vm130-32 GlusterFS[1969]: [2013-06-24 10:32:20.508690] M [posix-helpers.c:1119:posix_health_check_thread_proc] 0-failing_xfs-posix: still alive! -> SIGTERM

- these errors are in the log of the brick as well:

    [2013-06-24 10:31:50.500607] W [posix-helpers.c:1102:posix_health_check_thread_proc] 0-failing_xfs-posix: stat() on /bricks/failing_xfs/data returned: Input/output error
    [2013-06-24 10:31:50.500674] M [posix-helpers.c:1114:posix_health_check_thread_proc] 0-failing_xfs-posix: health-check failed, going down
    [2013-06-24 10:32:20.508690] M [posix-helpers.c:1119:posix_health_check_thread_proc] 0-failing_xfs-posix: still alive! -> SIGTERM

- the glusterfsd process has exited correctly:

    # gluster volume status
    Status of volume: failing_xfs
    Gluster process						Port	Online	Pid
    ------------------------------------------------------------------------------
    Brick glufs1:/bricks/failing_xfs/data			N/A	N	N/A
    NFS Server on localhost					2049	Y	1897

Change-Id: Ic247fbefb97f7e861307a5998a9a7a3ecc80aa07
BUG: 971774
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5176
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>