cluster/afr: Sparse file self-heal canges

- Fix boundary condition for offset
- Honour data-self-heal-algorithm option
- Added tests for sparse file self-healing

Change-Id: I14bb1c9d04118a3df4072f962fc8f2f197391d95
BUG: 1080707
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7339
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
glusterd: suppress spurious error message during startup

From glusterd log:
----------
E [glusterd-store.c:1981:glusterd_store_retrieve_volume] 0-: Unknown key:
brick-0
----------

The message is emitted from glusterd_store_retrieve_volume() when it reads the
volinfo file because it doesn't do anything with the key-value pair. Suppress the
error. The key is needed by glusterd_store_retrieve_bricks() which anyway
re-reads it.

Also change the log level to WARNING since we do not error out if an unknown key
is got while parsing the volinfo file.

Change-Id: Icd7962d9e16e0f90e6a37ee053dcafe97d2cab94
BUG: 1079279
Reviewed-on: http://review.gluster.org/7314
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
rpm: Include the hook scripts of add-brick, volume start, stop and set.

Change-Id: I9485da3d9a54a52cc524daff8f4f4caca3a6065d
BUG: 1080970
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/7346
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
log: enhance gluster log format with message ID and standardize errno reporting

	Currently there are quite a slew of logs in Gluster that do not
	lend themselves to trivial analysis by various tools that help
	collect	and monitor logs, due to the textual nature of the logs.

	This FEAT is to make this better by giving logs message IDs so
	that the tools do not have to do complex log parsing to break
	it down to problem areas and suggest troubleshooting options.

	With this patch, a new set of logging APIs are introduced that
	take additionally a message ID and an error number, so as to
	print the message ID and the descriptive string for the error.

	New APIs:
		- gf_msg, gf_msg_debug/trace, gf_msg_nomem, gf_msg_callingfn

	These APIs follow the functionality of the previous gf_log*
	counterparts, and hence are 1:1 replacements, with the delta
	that, gf_msg, gf_msg_callingfn take additional parameters as
	specified above.

	Defining the log messages:
	Each invocation of gf_msg/gf_msg_callingfn, should provide an ID
	and an errnum (if available). Towards this, a common message id
	file is provided, which contains defines to various messages and
	their respective strings. As other messages are changed to the
	new infrastructure APIs, it is intended that this file is edited
	to add these messages as well.

	Framework enhanced:
	The logging framework is also enhanced to be able to support
	different logging backends in the future. Hence new configuration
	options for logging framework and logging formats are introduced.

	Backward compatibility:
	Currently the framework supports logging in the traditional
	format, with the inclusion of an error string based on the errnum
	passed in. Hence the shift to these new APIs would retain the log
	file names, locations, and format with the exception of an
	additional error string where applicable.

	Testing done:
	Tested the new APIs with different messages in normal code paths
	Tested with configurations set to gluster logs (syslog pending)
	Tested nomem variants, inducing the message in normal code paths
	Tested ident generation for normal code paths (other paths
		pending)
	Tested with sample gfapi program for gfapi messages
	Test code is stripped from the commit

	Pending work (not to be addressed in this patch (future)):
	- Logging framework should be configurable
	- Logging format should be configurable
	- Once all messages move to the new APIs deprecate/delete older
	  APIs to prevent misuse/abuse using the same
	- Repeated log messages should be suppressed (as a configurable
	  option)
	- Logging framework assumes that only one init is possible, but
	  there is no protection around the same (in existing code)
	- gf_log_fini is not invoked anywhere and does very little
	  cleanup (in existing code)
	- DOxygen comments to message id headers for each message

Change-Id: Ia043fda99a1c6cf7817517ef9e279bfcf35dcc24
BUG: 1075611
Signed-off-by: ShyamsundarR <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/6547
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
doc: Update manpages

Change-Id: Id14c7c3229ed266cd15915a2136e3290ce2c5ed2
BUG: 1031328
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7338
Reviewed-by: Lalatendu Mohanty <lmohanty@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
tests: Stale file lookup test

Change-Id: I6edfc5b7ee42677e92d9cff6a7180692d20e9310
BUG: 1080759
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7341
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
DHT/Rebalance : Hard link Migration Failure

Probelm : __is_file_migratable used to return ENOTSUP
for all the cases. Hence, it will add to the failure
count. And the remove-brick status will show failure
for all the files.

Solution : Added 'ret = -2' to gf_defrag_handle_hardlink to
be deemed as success. Otherwise dht_migrate_file will try to
migrate each of the hard link, which not intended.

Change-Id: Iff74f6634fb64e4b91fc5d016e87ff1290b7a0d6
BUG: 1066798
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/7124
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests/afr: select correct read-child for data OPs.

Change-Id: If84bc489b6c45bde3bdb858da5f1600cea78c8a5
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7345
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests/afr: gfid mismatch test

Change-Id: I12bae9c4035d5b28292e8085a5b600a3e22abaf4
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7342
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
gfapi: glfs_set_volfile_server() now entertains multiple calls

Previous API:

glfs_set_volfile_server (..., const char *host, ...) - single call

New API's:

glfs_set_volfile_server (..., const char *host1, ...)
glfs_set_volfile_server (..., const char *host2, ...)

Multiple calls to this function with different volfile servers,
port or transport-type would create a list of volfile servers
which would be polled during `volfile_fetch_attempts()`

glfs_unset_volfile_server (..., const char *host, ...) to remove
a server from the list (this is provided for future usage)

Change-Id: I313efbd3efbd0214e2a71465f33195788df406cc
BUG: 986429
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7317
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
tests/quota: Wait till the rebalance is complete

Change-Id: Ia6f0c81fb1542ce1de965a69a61535691df056c3
BUG: 1077159
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/7380
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Remove eager-lock stub on finodelk failure

Problem:
For write fops afr's transaction eager-lock init adds transactions
that can share eager-lock to fdctx list. But if eager-lock finodelk
fop fails the stub remains in the list. This could later lead to
corruption of the list and lead to infinite loop on the list
leading to a mount hang.

Fix:
Remove the stub when finodelk fails.

Change-Id: I0ed4bc6b62f26c5e891c1181a6871ee6e4f4f5fd
BUG: 1063190
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/6944
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
cli: remove-brick no longer defaults to commit-force

Problem : When gluster volume remove-brick is executed with out any option, it
defaults to force commit which results in data loss.

Fix : remove-brick can not be executed with out explicit option, user needs to
provide the option in the command line else the command will throw back an usage
error.

Earlier usage : volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
                [start|stop|status|commit|force]

Current usage : volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
                <start|stop|status|commit|force>

Change-Id: I2a49131f782a6c0dcd03b4dc8ebe5907999b0b49
BUG: 1077682
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7292
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <sam.somari@gmail.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
features/gfid-access: Fix possible inode memory corruption.

        During lookup, the inode is not ref'd. Added code
        to ref the inode in call path and unref in cbk path.
        Also fixed a case where we should always be putting
        linked inode into context as it is not guaranteed
        that we get same inode that we passed in a call to
        inode_link.

Change-Id: Iaec083a9258658bef3047e83956729d3dbcd9a59
BUG: 1080295
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/7329
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
features/glupy: Add mem accounting support

When glusterfs is built from source using -DDEBUG flag and glupy
xlator is added to vol file, the brick process used to crash when
mounting the volume.

This fix is largely derived from the fix submitted for BZ #1035751.
Thanks to Justin Clift for helping in tracking this down.

BUG: 1035751
Change-Id: Id64f92eecc9335e34dd08812fe176774e7723c2c
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/7332
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
tests: Adjust test 767095 to cope with long hostnames

BUG: 1084147
Change-Id: Ie1ff8852a501690e681072c54620d305b5e20d6a
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7395
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
tests/afr: select correct read-child for entry OPs.

Change-Id: If375c937579a18d603ed70232130a4664060e9d6
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7344
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cli : Removal of dead code

dead code reported by covscan is removed from cli-cmd-parser.c

Fix for coverity CID: 1195423

Change-Id: Ice1771dc8b3ef47fd2e63b380b12e850dc1d5d95
BUG: 789278
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7389
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
afr: Simple 1-liner fix for crash in Rackspace

BUG: 1084485
Change-Id: I89ddf10add041638ef70baebbce0ec2807ef4b6d
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7402
Reviewed-by: Anand Avati <avati@redhat.com>
cli: Trivial wording improvement of a comment

Change-Id: Ib569b39bdf0357c30c94c7a1b8d3ff87b811841c
Reviewed-on: http://review.gluster.org/7403
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
Reviewed-by: Anand Avati <avati@redhat.com>