geo-rep / gsyncd: gsyncd binary wrapper context fixes

Changes to __gf_calloc (commit ed4b76ba) triggers a recursive
sequence of _gf_log() and __glusterfs_this_location() calls. This
is due to __gf_calloc() trying to access THIS which is not valid
in the context of geo replication binary wrapper. This patch fixes
the problem by introducing the relevant context in the wrapper.

Change-Id: I32b0b6927074692dbf66bad2f3e93434201aaa82
BUG: 846569
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/3859
Reviewed-by: Csaba Henk <csaba@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
cluster/afr: Wake up post-op on non-co-operative transaction

Problem:
The problem is observed when kernel untar is done. One file untar
happens every second. The reason for this is, setattr lock is blocked
on the prev fd data-transaction full-lock (because of eager-lock).
Because of post-op-delay the post-op (xattrop + unlock) of the prev
data-transaction happens after 1 sec.
Until this the setattr is blocked resulting in performance problems
in untar.

Fix:
Whenever an loc data, meta-data transaction comes, it should wakeup
the prev-post-op on the same process' fd.

Tests:
The performance problem in untar went away. I put a breakpoint in
client_finodelk for a 2G file dd and the inodelk is hit only 4 times.
This confirms that the change does not affect post-op-delay in a
-ve way.

Change-Id: Ice3c2a1211f4dca6520a19bc4ba6cb9efb2902ad
BUG: 845754
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.org/3975
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gfapi: fix the failure in glfs_open()

* the issue is happening because the resolve_symlink() logic was
  reversed

Change-Id: I10eca14d82e3ebf5a0195d5db1330f9e1a3e2dc4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 839950
Reviewed-on: http://review.gluster.org/3978
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
protocol/server: put the connection back if setvolume fails

In server_setvolume, put the connection back (thus removing it
from a the list of connections that protocol/server is maintaining
if the connection object is a newly created one) if some operations
such as getting the bound_xl etc fails. Otherwise the process might
segfault upon statedump, when it tries to access NULL bound_xl of a
connection.

Change-Id: Ifa340f534cd88d5601a9219da622b323fbe98b80
BUG: 858191
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/3953
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Trigger heal on local subvols on any child_up

Problem:
The index in the child that comes online is generally empty
because the changes would have happened on the other child which
has been up.  So the sync begins when the other child's poll
time-out happens (i.e. 10 minutes). The expectation is that the
sync must be triggered as soon as the connection with any brick
is established.

Fix:
Whenever any child_up happens trigger the index self-heal on all
local children in the replicate subvolume.

Tests:
1) Checked that the self-heal is triggered on all local children
whenever any child comes online.
2) Checked that the volume heal commands are working fine.

Change-Id: I4f64737866470a2f989349a889ea52782930e11d
BUG: 852741
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.org/3972
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
protocol/client: quick-reconnect after portmap query

Currently the disconnect after a portmap query is treated like an
ordinary disconnect and the reconnection attempt (in this case, to
the brick) is attempted only after 3 secs. This results in a delay
which is unnecessary.

Mark the disconnection happening because of a successful portmap
query as needing a 'quick reconnect' to avoid the delay for this
special case.

Change-Id: I43c8292ff0c30858d883ff3569a3761acbf2f5eb
BUG: 860220
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/3994
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
build: Install glusterfs-logrotate as config file

Change-Id: I8255eb4249503eac0add87444da934256faffc01
BUG: 860037
Reviewed-on: http://review.gluster.org/3973
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Fix parallel build issues

Prevent creating symbolic links in not yet created sbindir.
Since lex creates two files, make them depend on each other to
prevent race condition.

Change-Id: I72b633b24de33407006d70571b0a40adaa99dc2b
BUG: 859843
Signed-off-by: Kacper Kowalik (Xarthisius) <xarthisius.kk@gmail.com>
Reviewed-on: http://review.gluster.org/3968
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
mgmt/glusterd: Prevent extra unref in vol start

Problem:
Volume start cli handler does dict_unref of the dict
unconditionally. It needs to be unrefed only on failure.

Fix:
Unref the dict only on failure.

Tests:
Volume start works fine after the fix.

Change-Id: I64e5ad3d48f69545dd5e1132672182732b98c466
BUG: 860833
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.org/4001
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Fixed some general typing errors.

Eg: changed recieved to received

Change-Id: I360fcb99c97c8a0222e373fee20ea2fccfb938db
BUG: 860543
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/3998
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli: removed extra dict unrefs and memory leaks

PROBLEMS ADDRESSED:
a. The following change
      http://review.gluster.com/#change,3948
   introduces extra dict unrefs in cli.
b. There are instances of memory leak in gf_cli_*_cbk functions.

FIX:
Problem (a) is fixed by ensuring the dict is ref'd (indirectly at
the time of creation) and unref'd (in CLI_STACK_DESTROY) ONLY
once. The following is the rationale behind this approach:

   The number of refs and unrefs on dict varies across the different
   commands that use it. The cli thread MUST wait for the
   gf_cli_*_cbks to complete before the frame is destroyed. This
   rules out the need to do extra refs and unrefs in the code path.

Problem (b) is fixed by doing unref on dicts that are created for
the purpose of unserializing the response but never freed in
gf_cli_*_cbk functions.

Change-Id: I5a7431543fc8e3cac4d256f5c87d1e3c62a331be
BUG: 823081
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/3966
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cluster/afr: Provide option to set readdir-size in entry-self-heal

Problem:
Entry self-heal does lookups on all the entries that are read
in readdir. More the size of readdir more number of lookups happen
in parallel. It is observed that it leads to HUGE cpu spikes
rendering everything else on the system unusable.

Fix:
Provided the option self-heal-readdir-size to configure the size.
Default value is at 1KB.

Tests:
Checked that the readdirs are happening with the configured value
in entry-self-heal.

Change-Id: Icaa937ad88857e6f9a12375b1e7f6a49192bc8b1
BUG: 860895
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.org/4002
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
write-behind: implement causal ordering and other cleanup

Rules of causal ordering implemented:

- If request A arrives after the acknowledgement (to the app,
  i.e, STACK_UNWIND) of another request B, then request B is
  said to have 'caused' request A.

- (corollary) Two requests, which at any point of time, are
  unacknowledged simultaneously in the system can never 'cause'
  each other (wb_inode->gen is based on this)

- If request A is caused by request B, AND request A's region
  has an overlap with request B's region, then then the fulfillment
  of request A is guaranteed to happen after the fulfillment of B.

- FD of origin is not considered for the determination of causal
  ordering.

- Append operation's region is considered the whole file.

Other cleanup:

- wb_file_t not required any more.

- wb_local_t not required any more.

- O_RDONLY fd's operations now go through the queue to make sure
  writes in the requested region get fulfilled before getting
  processed.

- O_SYNC fd's operations now go through the queue to make sure
  previously acknowledged writes on the file (via other fds) are
  fulfilled before getting processed.

- Option to not honor O_SYNC is now removed.

- Option to ignore O_DIRECT is added (useful when running a VM and the
  drive appears with NCQ/TCQ or WCE=1 for the guest.)

- Option to disable_first_nbytes is removed (as the cause of the
  bug which required this was diagnosed to be missing TCP_NODELAY.)

- General cleanup and better conformance to coding style and convention.

Change-Id: Ib44fb72da3727246b4a85174cb568c2f0231f6de
BUG: 857673
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/3947
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
 Made test-ffop.c exhaustive by making it to test all types of fops.

Change-Id: I26052d0cbf72f81a6aa0c52498e99b635d1273e9
BUG: 861380
Signed-off-by: M S Vishwanath Bhat <vbhat@redhat.com>
Reviewed-on: http://review.gluster.org/3931
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
dict: no need to memdup() the XDR buffer for dict_unserialize

* current memdup() of full buffer is not required as
  'dict_unserialize()' does its own memdup() of the value.

Change-Id: Iba866b68b753d060ee35bb300ee687e7d293ac82
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 850917
Reviewed-on: http://review.gluster.org/3845
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Fix problems of mismatch type.

Checking if an uint32_t is less than zero is unnecessary, so we change
the parameter and return value to int.

Change-Id: I5c08eaf8fd4c33768a366b22f0f5db9575056a1e
BUG: 789278
Signed-off-by: linbaiye <linbaiye@gmail.com>
Reviewed-on: http://review.gluster.org/3965
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
linux-aio: fixes while setting O_DIRECT flag

Linux AIO needs O_DIRECT to be set for effective operation. O_DIRECT
in turn has constraints on when it can work (offset, size alignment)

So use O_DIRECT (unless instructed by application) only when offset
and size alignments match. Else, io_submit() will happen over
non-O_DIRECT fd, effectively blocking till the completion of the IO.

Also fix a multithreading bug where detection/setting of O_DIRECT
for a request was not atomic with io_submit() of that request.

Change-Id: I190017e8bc78217429aff0714dca224cbe6f251d
BUG: 837495
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4006
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
build: add missing GF_CFLAGS in api/src/

GF_CFLAGS seems missing in two spots.

In one of those, adding it to libgfapi_la_CFLAGS would make it equal
to AM_CFLAGS, so remove libgfapi_la_CFLAGS there.

Change-Id: I50c22f16210ecd40371d4abb0e418c7a4f94c12f
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4009
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
build: consolidate common compilation flags into one variable

Some -D flags are present in all files, so collect them.
This adds -D${GF_HOST_OS} to some compiler command lines,
but this should not be a problem.

Change-Id: I1aeb346143d4984c9cc4f2750c465ce09af1e6ca
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4013
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
init.d: use proper dependencies in SUSE init script

"/usr", the place where the binaries usually reside, requires
$remote_fs.

Change-Id: I830c45b09b78b6c3b36c1475ac3f13cf6c2df588
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4011
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>