gNFS: NFS segfaults with nfstest_posix tool

Problem:
nfs3_stat_to_fattr3() missed a NULL check.

FIX:
(1) Added a NULL check.
(2) In all fop cbk path, if the op_ret is -1 and op_errno is 0,
    then handle it as a special case. Set the NFS3 status as
    NFS3ERR_SERVERFAULT instead of NFS3_OK.
(3) The other component of FIX would be in DHT module and
    is on the way.

Change-Id: I6f03c9a02d794f8b807574f2755094dab1b90c92
BUG: 1010241
Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>
Reviewed-on: http://review.gluster.org/6026
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
posix: Fix excessive logging resulting from get_real_filename failure from samba

Change-Id: I641d028165da7b8501bd372c62d2df89a9d4db1f
BUG: 1027174
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/6237
Reviewed-by: poornima g <pgurusid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
glusterd: modify remove-brick CLI message.

In the current context "replica_cnt" is used just to know whether the
specific key exists or not by calling "dict_get_int32", which we can
replace by "dict_get ()". And changing the log message as it is more
appropriate to say "migration of data" rather than "rebalance".

This patch refactors commit 51c6fa7a354826744de98 against BZ 961669

reviewed on : http://review.gluster.org/5566

Change-Id: I48eae206a28d4083975e64407ed8fe4539f9c24b
BUG: 1027270
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Original patch: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/6001
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: susant palai <spalai@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gfapi: remove unnecessary call to glfs_resolve_base()

Calling glfs_resolve_base() on the root inode for every resolver
invocation is unnecessary and wasteful.

Here are the results from running a test program which performs
path based operations (creates and deletes 1000 files):

Without patch:
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.314s
user    0m1.923s
sys     0m1.144s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.383s
user    0m1.940s
sys     0m1.177s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.339s
user    0m1.863s
sys     0m1.129s

With patch:

[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m3.005s
user    0m1.162s
sys     0m0.816s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m3.188s
user    0m1.222s
sys     0m0.867s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m2.999s
user    0m1.131s
sys     0m0.832s

Change-Id: Id160a24f44b4dccfcfce99a6f69ddb8938523cd5
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6131
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
libglusterfs: fix bug in timespec adjustment

The argument to the timespec_adjust_delta() function introudced in
commit 6836118b21 needs to be passed by reference rather than by value
for the function to do it's job.

BUG: 1028663
Change-Id: I62a3636906e67ed35b7786e9553f6819b48f3626
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/6243
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterfs: zerofill support

Add support for a new ZEROFILL fop. Zerofill writes zeroes to a file in
the specified range. This fop will be useful when a whole file needs to
be initialized with zero (could be useful for zero filled VM disk image
provisioning or  during scrubbing of VM disk images).

Client/application can issue this FOP for zeroing out. Gluster server
will zero out required range of bytes ie server offloaded zeroing. In
the absence of this fop,  client/application has to repetitively issue
write (zero) fop to the server, which is very inefficient method because
of the overheads involved in RPC calls  and acknowledgements.

WRITESAME is a  SCSI T10 command that takes a block of data as input and
writes the same data to other blocks and this write is handled
completely within the storage and hence is known as offload . Linux ,now
has support for SCSI WRITESAME command which is exposed to the user in
the form of BLKZEROOUT ioctl.  BD Xlator can exploit BLKZEROOUT ioctl to
implement this fop. Thus zeroing out operations can be completely
offloaded to the storage device , making it highly efficient.

The fop takes two arguments offset and size. It zeroes out 'size' number
of bytes in an opened file starting from 'offset' position.

This patch adds zerofill support to the following areas:
	- libglusterfs
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

Client applications can exloit this fop by using glfs_zerofill introduced in
libgfapi.FUSE support to this fop has not been added as there is no system call
for this fop.

Changes from previous version 3:
* Removed redundant memory failure log messages

Changes from previous version 2:
* Rebased and fixed build error

Changes from previous version 1:
* Rebased for latest master

TODO :
     * Add zerofill support to trace xlator
     * Expose zerofill capability as part of gluster volume info

Here is a performance comparison of server offloaded zeofill vs zeroing
out using repeated writes.

[root@llmvm02 remote]# time ./offloaded aakash-test log 20

real	3m34.155s
user	0m0.018s
sys	0m0.040s
[root@llmvm02 remote]# time ./manually aakash-test log 20

real	4m23.043s
user	0m2.197s
sys	0m14.457s
[root@llmvm02 remote]# time ./offloaded aakash-test log 25;

real	4m28.363s
user	0m0.021s
sys	0m0.025s
[root@llmvm02 remote]# time ./manually aakash-test log 25

real	5m34.278s
user	0m2.957s
sys	0m18.808s

The argument log is a file which we want to set for logging purpose and
the third argument is size in GB .

As we can see there is a performance improvement of around 20% with this
fop.

Change-Id: I081159f5f7edde0ddb78169fb4c21c776ec91a18
BUG: 1028673
Signed-off-by: Aakash Lal Das <aakash@linux.vnet.ibm.com>
Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com>
Reviewed-on: http://review.gluster.org/5327
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/qemu-block: add qemu backing image support (clone)

Add basic backing image support to the block-format mechanism. This
is a functionality checkpoint that enables the raw mechanism
required to support client driven "snapshot" and "clone" requests.

This change enhances the block-format setxattr command to support
an additional and optional backing image reference. For example:

setxattr -n trusted.glusterfs.block-format -v "qcow2:10GB:<bimg>" ./newimage

... where <bimg> refers to the backing image for unallocated blocks
in newimage. <bimg> can be provided in one of two formats:

- a gfid string in the following format (assuming a valid gfid):

<gfid:00000000-0000-0000-0000-000000000000>

- or a filename that must be resident in the same directory as the
new clone file being formatted. E.g.,

setxattr -n trusted.glusterfs.block-format -v "qcow2:10GB:baseimg" ./newimage

This latter format is more restrictive, simply provided for
convenience or until something more refined is available.

This change makes no assumptions about the backing image file and
affords no additional protection. It is up to the user/client to
recognize the relationship between the files and manage them
appropriately (i.e., no writes to the backing image, etc.).

BUG: 986775
Change-Id: I7aff7bdc59b85a6459001a6bfeae4db6bf74f703
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/5967
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
features/qemu-block: simplify coroutine model to use single synctask, ucontext

The current coroutine model, mapping synctasks 1-1 with qemu internal
Coroutines, has some unresolved raciness issues. This problem usually
manifests as lifecycle mismatches between top-level (gluster created)
synctasks and the subsequently created internal coroutines from that
context. Qemu's internal queueing (and locking) can cause situations
where the top-level synctask is destroyed before the internal scheduler
has released references to memory, leading to use after free crashes
and asserts.

Simplify the coroutine model to use a single synctask as a coroutine
processor and rely on the existing native ucontext coroutine
implementation. The syncenv thread is donated to qemu and ensures a
single top-level coroutine is processed at a time. Qemu now has
complete control over coroutine scheduling.

BUG: 986775
Change-Id: I38223479a608d80353128e390f243933fc946fd6
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/6110
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli: write 'volume rebalance' error message in xml format when
--xml is specified

When 'volume rebalance' encounters an error the cli prints the
error message in plain text independent of whether --xml is
specified. This throws off client application that expect xml
output (as mentioned in bz1026143).

Now, if the --xml flag is supplied, the cli print 'volume
rebalance' error messages in xml format.

Change-Id: I16c6a7a4cdd2819eb73422ab849125986dc299a6
BUG: 1026143
Signed-off-by: Dawit Alemu <dalemu@redhat.com>
Reviewed-on: http://review.gluster.org/6242
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
server/rpc: Numerous entries of error - "No such file or directory" in bricks log

Problem:
Messages were getting logged very frequently at log level INFO.

One of the log file snippet -

[2013-10-27 00:05:01.501355] I [server3_1-fops.c:1707:server_stat_cbk] 0-vol-server: 24846575: STAT (null) (--) ==> -1 (No such file or directory)
[2013-10-27 00:05:01.505101] I [server3_1-fops.c:1707:server_stat_cbk] 0-vol-server: 24846577: STAT (null) (--) ==> -1 (No such file or directory)
[2013-10-27 00:05:01.507299] I [server3_1-fops.c:1707:server_stat_cbk] 0-vol-server: 24846578: STAT (null) (--) ==> -1 (No such file or directory)
[2013-10-20 19:50:35.554563] I [server3_1-fops.c:1538:server_open_cbk] 0-vol-server: 18714687: OPEN <gfid:01c70ca0-1952-4e82-abee-a07205757d8e>
(01c70ca0-1952-4e82-abee-a07205757d8e) ==> -1 (No such file or directory)
[2013-10-20 19:50:35.555520] I [server3_1-fops.c:1538:server_open_cbk] 0-vol-server: 18714697: OPEN <gfid:01c70ca0-1952-4e82-abee-a07205757d8e>
(01c70ca0-1952-4e82-abee-a07205757d8e) ==> -1 (No such file or directory)
[2013-10-20 19:50:35.558292] I [server3_1-fops.c:1538:server_open_cbk] 0-vol-server: 18714712: OPEN (null) (--) ==> -1 (No such file or directory)

Solution:
Moved them to DEBUG log level if error number equlas to ENOENT else to ERROR log level.
It will help in decreasing the size of log files at INFO log level.

Change-Id: I23d74320c9c21bbce4805c20295556cc2cc0a8d8
BUG: 808073
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Reviewed-on: http://review.gluster.org/6241
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli: Set the o/p width of hostname to 8 characters

Change-Id: I91dcb19ba4d31c17e6041155c0e59af457b87f1b
BUG: 1028871
Signed-off-by: Vijaykumar M <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/6245
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>