gsyncd / geo-rep: fix regular file creation on rename

On creation of a regular file on the slave (this happens
when an unlink()/rmdir() operation is performed on master
after a rename() or a link().

This patch fixes the regular file creation on the slave when
the above mentioned series of operation occurs. entry_ops()
on the slave for a rename() was missing the stat() information.

Change-Id: Idb80c6b3e27c1797dade1113ac8ce920db9c740c
BUG: 994353
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5509
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: remove deprecated code!

This includes "xtime" base crawling and syncing,
"Blind" and "Wrapup" mixins for failover/failback.

Change-Id: I9bc8695dc1c19b1261332f64621305dff2929b32
BUG: 994461
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5514
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: introduce "recovery" mode

This mode introduces and optimized recovery of master,
used during failover-failback cases. Files are only
transferred if they are identified as changed on the slave.

A normal failback mechanism would be just to reverse the
direction of geo-replication syncing. Doing this is the
correct way but could be time consuming. "Recovery" mode
optimizes this by identifying files (as skipping them) duing
the one-shot crawl.

NOTE: Use this mode when you are _absolutely_ sure that the
namespace on the master is intact, ie. there is number of files
in master is equal to the number of files in the slave, but the
slave was written to (after a failover) when master was brought
back up.

- gluster volume geo <MV> <SLAVE>::<SV> config special_sync_mode recover
- gluster volume geo <MV> <SLAVE>::<SV> start

Change-Id: Idb698949e4e86698b15d04a35d33095562f51714
BUG: 994462
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5515
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
features/marker: force xtime updates (configurable) for client-pid = -1

This is required by Geo-Replication that does auxillary mount
with client-pid as -1 (which has special treatment at specific
places in GlusterFS), to trigger xtime updates on the intermediate
master in a cascading setup.

Marker too had a check to "not" mark updates for geo-replication's
auxillary mounts. With the new geo-replication design, xtimes are
not set by the master on the slave for all entities. Due to this
cascading setups were broken.

This patch introduces "geo-replication.ignore-pid-check" option
as a "override" for the client-pid check for gsyncd's client-pid.
When this options is enabled, marker start "marking" even if the
updates are from the special client.

Geo-Replication on the detection of itself being an intermediate
master, enables this option.

Change-Id: I9f7140edd12fef5480595ee0f93f35b94cdb8345
BUG: 996371
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5591
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: "disjoint" cascading geo-replication sessions

Slave's xtime is now stored on the master itself (and that too only on
the root), which implies it cannot be propogated to the cascaded slave.
Thus the intermediate master now makes use of it's own volume information
to propogate volume-mark and xtime.

On starting Geo-Replication "geo-replication.ignore-pid-check" marker
option is enabled, which is an override for the client-pid check in
marker. This options triggers marker update only for geo-replication
auxillary mount (client-pid == -1). Since gsyncd not does setxattr()
directly on the bricks, this option won't trigger a chain of spurious
metadata updates that would need to be processed by gsyncd.

Change-Id: If50c5ef275dfb6b4ff4fd35be2565587e2fdf3e1
BUG: 996371
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5592
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: fix "regular file" overloading rename

entry operation on the slave was using source parent gfid
and basename when renames are overloaded to use regular file
creation. This patch fixes the issue by using the destination
parent gfid and basename for these cases.

Change-Id: I1a4e8df7f07905224ce44ef5abd6f180234285ab
BUG: 1003800
Tested-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: overload rename with an unlink

This is certainly a "hack" for cases when after a rename
the gfid is unlinked (which is now pointed by a different
pargfid/basename). This was osbserved when a file edited
(uisng vi/vim) on the glusterfs mount point which has a
geo-replication session running.

Change-Id: I0a5d9f0f549aa4403671fa1f11e95510142e5720
BUG: 1003800
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5755
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: handle ENOENT during readlink()

Change-Id: I8611122fda63ea496ad4491473ecabe96bf19cfa
BUG: 1003803
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5756
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: fix hardlink creation on slave

Change-Id: I20fbd518bf519cbf2362b97aeb8be7c3b105087a
BUG: 1003805
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5757
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: maximum number of retries per changelog

With entry retries MAX_RETRIES is the threashold for the number
of times we retry a changelog. After hitting this limit, the
changelog is marked as processed after warning for incompleteness
in the log.

Change-Id: I777b71b4cdaa63335d9c936f2d64e5c0264a0f26
BUG: 1003803
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
geo-replication: fix the logic of choosing the remote node to sync

Change-Id: Ie15636357d89e94b6bfad0e168b1fcad53508c47
BUG: 1003807
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5759
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: Fix "regular file" overloading link()

... missing entry2pb() call before going ahead with create.

Change-Id: I48de4df3f0ea6c789c9eeb10cd332ab461f3c868
BUG: 1003805
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5760
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
gsyncd / geo-rep: distributify slave

commit fbb8fd92 introduced slave distributification but had
some problems (monitor would crash upon gsyncd start). This
patch fixes the issue and makes code more pythonic ;)

Change-Id: I2cbf5669d81966046a4aeeb4a6ad11a947aa8f09
BUG: 1003807
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5761
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>