performance/io-threads: Provide option to turn off least-priority

RCA:
In cases when self-heal is in progress, self-heal fops are starved
because of least-priority. This affects other fops with conflicting
inode, entry locks with self-heal.

Fix:
This patch provides configuring enable/disable of least-priority.

Additional changes:
Moved RCHECKSUM fop to low instead of least because it will still
affect the performance of other fops if RCHECKSUM is in LEAST
priority.

Tests:
Tested that the enabling/disabling of fops is working fine.
Tested that RCHECKSUM fop priority is assigned LOW when
least-priority is disabled.

Change-Id: I8d72872b33e2ac7e1ad3afb27582682b0eb98a80
BUG: 843704
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3743
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
libglusterfs/statedump: add start and end markers in the statedump file

Also do not overwrite the statedump file if statedump is taken on the same
process again. Instead add the timestamp to the filename

Change-Id: I451a0decf2dad5c398631cab1fc2a69811c69af8
BUG: 843408
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3738
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
geo-rep / gsycnd: restore the generic signature for sendmark_regular

Earlier fixes to 842330 changed the generic (*a, **kw) signature,
although that was not related to the issue.

We restore the generic signature as it was used for a reason
(proxy methods that do none or only algebraic transformations
on passed arguments idiomatically have generic signature, both
to serve as visual cue and agnosticism wrt. the inner API).

Change-Id: Ib609a3a58be53d78b7f1221a3c162c6aec8fd488
BUG: 842330
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
performance/io-threads: dump the io-threads specific information in statedump

Change-Id: Ib9b9eab2d2e90fce21ef32f0f884101938978859
BUG: 843377
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3736
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/stripe: handle short writes and errors in writev callback

cluster/stripe write callback handling is broken in the event of
server side errors and short writes due to crudely summing up the
return values from each node. This can produce incorrect results
or cause an application to rewrite the wrong portions of a buffer
in an attempt to handle this condition.

Modify cluster/stripe writev handling to record the requested size
of each write and use this data to return the number of consecutive
bytes written from the original request. This allows an application
to retry a write at the point of error (and potentially consume
said error).

BUG: 809975
Change-Id: Ic35cb1e092c29545205aa32e352485c507534ce0
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.com/3700
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shishir Gowda <sgowda@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: Modifed replace-brick mount cleanup to use lazy umount

Change-Id: I83a985967cbaee4927f1b8ca4bc01c9fb0171522
BUG: 806890
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.com/3143
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
rpc/socket: finish initialization in own thread

...if we have one, that is.  This addresses Avati's review comment on
ea7759f1240b1e97684273b9369472695173a66.

Change-Id: I71a272f7eeff8279981a75241d60620ca4975c99
BUG: 764731
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.com/3701
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: update loc inode after inode_link

RCA:
inode passed to inode_link is not assigned any gfid if the
inode with that gfid is already linked, so loc for opendir
does not have a valid inode

Fix:
Use the linked_inode returned by inode_link in the loc to
perform further operations on the entry.

Tests:
Checked that opendir comes with an loc with valid inode.
Checked that re-opendir happens successfully. Tested index,
full self-heal work fine with the fix.

Change-Id: Idf4ced4cc2320133744962059d363e373af0e5ec
BUG: 826580
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3748
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
 Fix issues in make dist

Change-Id: I2fb3b0b35e2b5fdb08ee1857bf31ec8f65889306
BUG: 844669
Signed-off-by: M S Vishwanath Bhat <msvbhat@gmail.com>
Reviewed-on: http://review.gluster.com/3759
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Handle failures in fop_cbk gracefully

RCA:
Afr crashes when a last fop response fails and
'fop output' arguments are NULL. Afr does not handle
these gracefully.

Fix:
Changed the fops to not access the 'fop output' arguments
in case of failures.

Tests:
Changed afr wind_cbk code to fail the last response by setting
op_ret as -1 and op_errno as ENOMEM and setting all other output
variables as NULL to test the change. Removed the code to verify
success cases. No crashes or errors seen.

Change-Id: Iad9bc54db093a162f85bfb8dbeeda5b95acd21d8
BUG: 844689
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3760
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
protocol/client: Fix negative return in client_setvolume

PROBLEM:
The function dict_serialized_length could, owing to an error,
return a negative integer (-EINVAL) that gets assigned to an
unsigned int member 'dict_len' of gf_setvolume_req structure.

FIX:
Hold the value returned by dict_serialized_length in local
variable ret (which is a signed int). Test if ret is negative,
in which case the control would anyway branch to the label fail
where the function returns. Otherwise dict_len is assigned with
ret, in turn giving a more meaningful value to the attribute
length.

TEST:
Attached gdb to glusterfs mount process, set breakpoint at
client_setvolume, forced dict_serialized_length to return
-EINVAL (indirectly by forcing _dict_serialized_length to return
-EINVAL after setting count to -1 within its body) and checked
the value of ret (which is now sure to contain a negative value)
whose value will be appropriately tested to decide the next
course of action within client_setvolume: whether to simply
exit due to an error or execute the subsequent statements.

Change-Id: Ib22ad8f30d8ae04acaf2ff5bfee9c348a2c47148
BUG: 789278
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.com/3755
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
mount/fuse: initialize entry and negative timeouts correctly.

There was a typo which resulted in updation of attribute_timeout
for both options.

Change-Id: I6e212eaf3570cadde88d2a2c3b44121c893c9427
BUG: 804592
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.com/3763
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
event-history: implement gf_log_eh for saving events

Change-Id: I2ddf4e7849bda91bb99423e27082c2bae6800a70
BUG: 795419
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3532
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pranithk@gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
performance/read-ahead: use pthread_mutex_trylock to hold mutex in statedumps

Change-Id: I4491c96fc7d9879ba9040dcda3700c81d99ba184
BUG: 843796
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3766
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pranithk@gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
core: remove libglusterfs/src/scheduler.[ch]

* as its not part of the build
* confuses with some TAGS and 'git greps'

Change-Id: I1f97307a8ce486ba56a4a3ad0151276b0a72e096
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 764890
Reviewed-on: http://review.gluster.com/3769
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
supportability: print all missing frames

earlier, the statedump and backtrace were printing frames only if
it matched type GF_FOPs or GF_MGMTs (GF_MGMT is not in use now).
This made all the frames from glusterd, or any other frames which
got created outside the scope of 'fops' not getting printed.

now, made two classifications, one as FOPs (which gets logged
with the proper name), and another set is !FOPs (ie, else part),
which gets logged with only the op number.

Change-Id: Ifa18cdb75f7afefe6bb489d319b589cda8e64c1d
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 845114
Reviewed-on: http://review.gluster.com/3768
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cli: more clear separation of error messages

the intention is to make CLI output more script friendly.
use ':' as field separator per command and each field should
be meaningful.

this is phase-1 of the change, and hence RFC, once I get review
comments, will be doing more changes, if required

Change-Id: I11e70084f78a197ac50f9401dd7af312e63647ef
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 815194
Reviewed-on: http://review.gluster.com/3753
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
mem-pool: remove 'if' by asserting condition directly.

before:

if (!condition) {
    GF_ASSERT (0);
}

after:

GF_ASSERT (condition);

BUG: 764890
Change-Id: I9894fcc609579f5efd3f98dbff41ca1487ce6dd2
Signed-off-by: Jules Wang <lancelotds@163.com>
Reviewed-on: http://review.gluster.com/3762
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Handle child_up & fd not opened case in xaction

RCA:
When an fd is opened while a brick is down, after the brick
comes back up afr issues open on the other brick. It can
fail for a number of reasons (enoent etc). While the system
is in that state, inode/entrylks pre-op happen only on the
brick that is up and fd is opened for fd-fops. post-op should
consider only the bricks where both pre-op and fop succeeded
as success, rest of them as failures. Code now marks only the
children that are down as failures as opposed to child_down &
fd-not-opened. This makes change-log appear as success on the
subvolume where we did not do any fop leading to no change-log
but differences in data/metadata for reg-files.

Fix:
Mark non-participants of fop as failure. This is tracked in
transaction.pre_op[].

Tests:
Simulated the scenario using err-gen on top of one of the client
xlator which fails all fops always. Performed fops and the changelog
represented pending fops on the brick with err-gen loaded. Tested
the case of brick down and perform entry/metadata/data operations
to confirm they still work as expected.

Change-Id: I41905936126b19abba56ca581c0301a894507e1a
BUG: 844987
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3765
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
set_volume commit changes for op-version

Change-Id: Id79b19292f9b105e4fd438526e4a98a205ff537c