[Master] update to new version of hadoop.

Summary: Update to the latest dingo

Test Plan: mvn clean package -DskipTests

Reviewers: daviddeng, manukranthk

Reviewed By: manukranthk

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1339592

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42403 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Coprocessor uploader

Summary:
Create a class CoprocessorAdmin as a console tool for manipulating coprocessors.
Currently only implementing upload function.

Test Plan: `TestCoprocessorAdmin`

Reviewers: manukranthk, gauravm, adela

Reviewed By: adela

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1338310

Tasks: 4321287

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42406 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-11228] HLog Benchmarks on HDFS

Summary: Adding HLog benchmark to run against Dingo servers to compare performance of Cheetah vs Dingo.

Test Plan: Tested on HBASEDEV134-ASH3's HDFS from my devserver. Worked fine. The sync time was almost 99.8 % of the total time(leaving the log creation at the start and closing at the end) in my experiment.

Reviewers: rshroff, aaiyer, pritam, elliott

Reviewed By: elliott

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1341160

Tasks: 4303055

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42420 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] fix TestSimpleOperations

Summary: Fix TestSimpleOperations

Test Plan: mvn clean test -Dtest=TestSimpleOperations

Reviewers: manukranthk, daviddeng

Reviewed By: daviddeng

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1341928

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42433 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Make load shedding changable

Summary:
Add a method to TableServers so that maxOutStandingRequests can be
changed on the fly.

Test Plan: ./bin/test/loop_tests.sh -t TestLoadShedding

Reviewers: aaiyer

Reviewed By: aaiyer

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1337579

Tasks: 4362701

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42434 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-8500] Add an option to move the region to a random region server

Summary:
The patch tries to find an alive random region server, in case there
are no preferred region servers alive for this region.

Test Plan: unit tests, shadow testing

Reviewers: manukranthk, liyintang, aaiyer

Reviewed By: aaiyer

Subscribers: hbase-prodeng@, hbase-eng@

Differential Revision: https://phabricator.fb.com/D1339433

Conflicts:
	VENDOR.hbase/hbase-trunk/src/main/java/org/apache/hadoop/hbase/util/RollingRestart.java

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42439 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-9704] Fixing HTable not being AutoCloseable on Bellatrix

Summary: This diff fixes a backport breaking Bellatrix because HTableInterface in Bellatrix did not extend AutoCloseable.

Test Plan: mvn clean; mvn -DskipTests package

Reviewers: rshroff

Reviewed By: rshroff

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1342959

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42440 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-11106] Change assignment when moving regions

Summary:
In many testcases, `HBaseAdmin.moveRegion` is called and then wait for the change to happen. If AssigmentPlan is used, we should change the assignment plan as well, otherwise that movement could be ignored.

Define `HBaseTestingUtility.moveRegionAndAssignment` to change assignment before submit region move command.

In `HBaseTestingUtility.waitForTableConsistent` check whether region has been moved to most favored server.`

Test Plan: `TestGetRegionLocation`

Reviewers: gauravm, adela, elliott, manukranthk

Reviewed By: manukranthk

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1306567

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42452 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-9704] Fix TableServers.isNetworkException()

Summary:
Found that the client had a bug in identifying Network Exceptions in some cases where the network exception is wrapped inside an IOException.
Hence we should always check for the all the internal expections of an Exception while checking if it was a Network Exception. This will help in bringing down the amplification that was seen during the rack switch/single machine tests.

Test Plan: Verified the logs of TestFastFail that PFFE was being thrown. Also, this makes TestFastFail more stable.

Reviewers: aaiyer, rshroff, elliott

Reviewed By: elliott

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1343510

Tasks: 4377615

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42454 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Make TestServerSideException more stable

Summary: Broke TestServerSideException into two classes to make this more stable.

Test Plan: ./bin/test/loop_tests.sh -t TestServerSideExceptionDestructive

Reviewers: manukranthk, daviddeng, fan

Reviewed By: fan

Subscribers: fan, hbase-prodeng@, hbase-eng@

Differential Revision: https://phabricator.fb.com/D1341890

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42455 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Make TestSplitLogManager stable

Summary: Move dfs cluster creation out of tests.

Test Plan: bin/test/loop passed

Reviewers: daviddeng

Reviewed By: daviddeng

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1344164

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42458 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-9704] Deleting all connections while closing the test utility

Summary: We should be closing all connection pools instead of clearing them. Also calling deleteAllConnections() during the MiniCluter.shutDown() will clear any pending connections.

Test Plan: Verified that RejectedExcecutionException(https://phabricator.fb.com/P10774040) which was being caused because the ConnectionPool was not closed before closing the ThriftClientManager.

Reviewers: rshroff, gauravm, aaiyer, daviddeng, elliott

Reviewed By: daviddeng, elliott

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1344252

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42459 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-2000] Loading coprocessors per table

Summary: added a possibility to load coprocessors per table

Test Plan:
enhanced existing unit test. Ran:
- TestClassLoading
- TestHRegionObserverBypassCoprocessor

Reviewers: gauravm, manukranthk, daviddeng

Reviewed By: daviddeng

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1340566

Tasks: 4247961

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42473 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-11257][0.89-fb] Remove the timestamp from the annotation of Put

Summary:
David and I, recently found out, when discussing the C++ client with @csliu from the Search Team, that we have an extraneous timestamp field in the Put. Actually, that field is used when we create a Put object like:

  Put p = new Put(row, ts);

And then, if you do:

  p.add(cf, qualifier, value);

it would use `ts` as a timestamp for the KeyValue for the cf, qualifier. If you did not specify it, it will use `HConstants.LATEST_TIMESTAMP`.

One can also do this, where you explicitly state the timestamp to be used:

  p.add(cf, qualifier, ts, value)

In either case, when the add() method is called, the KeyValue is constructed, and it has the proper timestamp. Therefore, once you have created the family map with all these KeyValues, you don't need to send the `ts` field provided during the construction. All the KVs will have the correct timestamp embedded by the time it will be sent across.

This diff removes the timestamp field from the Put object. This will save us some network bandwidth, hopefully :)

Test Plan: Ran the standard thrift/swift unit tests.

Reviewers: manukranthk, fan, daviddeng, rshroff, adela

Reviewed By: adela

Subscribers: csliu, hbase-eng@

Differential Revision: https://phabricator.fb.com/D1343435

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42496 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Add hadoop-hdfs-highavailability and hadoop-lzo as an explicit dependency.

Summary: Add hadoop-hdfs-highavailability and hadoop-lzo as a runtime dependency in the pom.

Test Plan: deploy to dev cluster and see if the master comes up.

Reviewers: manukranthk

Reviewed By: manukranthk

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1349198

Tasks: 4385893

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42498 e7acf4d4-3532-417f-9e73-7a9ae25a1f51

Conflicts:
	pom.xml
[master] Fix TestHeaderSendReceive

Summary:
Test got broken because serverside profiling was enabled on second test case and not disabled - so profiling was enabled on the first test case when the order of execution changed. Fixed by disabling profiling in ##setUp()## method.

Also cleared warnings from the class

Test Plan: TestHeaderSendReceive was failing - now is passing

Reviewers: gauravm, elliott, manukranthk

Reviewed By: manukranthk

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1349454

Tasks: 4405009

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42510 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-11258] Pull in Integration Tests from open source 0.96/trunk

Summary:
Opensource trunk has a set of tests that run on a real cluster.  These
tests can also run a chaos monkey while writing and reading data.  In
the open source world this has exposed lots of bugs and is generally used
now as a way to verify a release canidate.

This pulls in the most important of these tests.

Test Plan: Test this on dev cluster with trunk and dingo.

Reviewers: liyintang, manukranthk

Reviewed By: manukranthk

Subscribers: manukranthk, hbase-eng@

Differential Revision: https://phabricator.fb.com/D1287532

Tasks: 4185690

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42517 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Load endpoint from HDFS, use CoprocessorClassLoader to load class.

Summary: Change testcase in `TestEndpointReload` to use endpoint in HDFS

Test Plan: `TestEndpointReload`

Reviewers: manukranthk, gauravm, adela

Reviewed By: adela

Subscribers: ssl-diffs@, hbase-eng@

Differential Revision: https://phabricator.fb.com/D1343242

Tasks: 4379795

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42526 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[master] Assembly shouldn't duplicate hbase jars in /lib

Summary: mvn assembly:assembly includes hbase jars in two places this excludes it from /lib.

Test Plan: mvn assembly:assembly and then check tar.gz to make sure hbase jar is only in the root dir

Reviewers: pwhite, manukranthk

Reviewed By: pwhite, manukranthk

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1349899

Tasks: 4185690

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42528 e7acf4d4-3532-417f-9e73-7a9ae25a1f51
[HBASE-11257][0.89-fb] Fix Put.equals() method to unbreak TestMultiput

Summary: My diff to remove the timestamp from the Put annotation broke TestMultiput because the Put.equals() method was still testing for the defaul ts, which was no longer being serialized. This diff fixes that.

Test Plan: The test doesn't break now :)

Reviewers: elliott, adela

Reviewed By: elliott

Subscribers: hbase-eng@

Differential Revision: https://phabricator.fb.com/D1352169

git-svn-id: svn+ssh://tubbs/svnhive/hadoop/branches/titan/VENDOR.hbase/hbase-trunk@42534 e7acf4d4-3532-417f-9e73-7a9ae25a1f51