[HBASE-7366] Make old logs directory have a subdirectory structure

Author: amirshim

Summary:
To speed up replication, it is useful to have each regionserver's old logs in a separate directory, so we don't have to do LS on a 300,000 file directory (namenode doesn't like that).  So we're changing the oldlogs directory structure to contain subdirectories of the format: "10.159.11.45%3A60020", for example:
/TITAN029-ASH3-HBASE/.oldlogs/10.159.11.45%3A60020/10.159.11.45%3A60020.1355452542203

Test Plan:
Unit tests - passed.

Run on cluster, and make sure that files are being moved to the correct subdirectories, and being deleted after the retention time has elapsed.

Reviewers: liyintang, nspiegelberg, mycnyc

Reviewed By: liyintang

CC: hbase-eng@, davejwatson

Differential Revision: https://phabricator.fb.com/D659154

git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/0.89-fb@1423101 13f79535-47bb-0310-9956-ffa450edef68
[master] Add support in hbase-daemon for regionserver stop-for-restart

Author: mbm

Summary:
Since hadoopctl wants (and kind of needs) to own the stopping and starting
of its daemons for restarts, this exposes an interface to the the jruby used
in doing the localised restart of the regionserver. hadoopctl can pick this
up and use this during a restart.

Test Plan:
This has been tested successfully on hbase430.frc1 (though it gets reverted
by fbdeploy - or something similar - periodically). Basically - the tests
for the positive situation and negative situations with hadoopctl (other
types of stop, hbase-daemon.sh that doesn't support stop-for-restart etc)
all now pass.

Reviewers: arjen, aleksandr, paultuckfield, rthiessen, kannan, kranganathan, liyintang

Reviewed By: arjen

CC: hbase-prodeng@, hbase-eng@

Differential Revision: https://phabricator.fb.com/D645234

Task ID: 906907

git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/0.89-fb@1424044 13f79535-47bb-0310-9956-ffa450edef68
[HBASE-7408] Removes the pread/seek+read option and defaults it to pread

Author: rshroff

Summary:
The change removes the pread/seek+read option and
defaults the option to always do pread.

During the compaction phase, HBase was using sequential reads to load the HDFS blocks via DFSCLient. Sequential reads causes HDFS to perform prefetching of the data which might not be used necessarily used by HBase. The reasong being that the HFile Block might be already present in the cache or it might need to skip a few blocks in the HFile(Bloom filter blocks), etc. This leads to un necessary iops on disks as Hbase might have to seek to a new location to fetch more data which might not have been prefetched.
Doing preads will prevent this prefetching of the data and hence reduce the iops on disks.

The actual performance numbers will be obtained by deploying this change on one of the shadow clusters.

Test Plan:
Execute the unit tests

http://hbasectrl056.ash3.facebook.com:50030/jobdetails.jsp?jobid=job_201209211223_31395

TestHFileWriterV2 is failing, looking into it.
After the review, will deploy it on shadow cluster to measure the
performance.

Reviewers: liyintang

Reviewed By: liyintang

CC: hbase-eng@, kannan

Differential Revision: https://phabricator.fb.com/D661385

Task ID: 1964373

git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/0.89-fb@1425100 13f79535-47bb-0310-9956-ffa450edef68
[HBASE-7416] augments tier compaction with fixed boundary option

Author: liyintang

Summary: Currently, in tier compaction the aged-based algorithm considers HFile's age in disk relative to the current time, thus the tiers are actually shifting along the time. In order to best use our prior information about how applications consume the data, it needs another feature to perceive the tiers relative to a fixed time point. For more details, please refer to https://our.intern.facebook.com/intern/wiki/index.php/HBase/Boundary_Based_Tier_Compaction

Test Plan: Modify the TestTierCompaction.java to accommodate the new feature.

Reviewers: aaiyer, kannan, cjin

Reviewed By: kannan

CC: hbase-eng@

Differential Revision: https://phabricator.fb.com/D619488

git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/0.89-fb@1425101 13f79535-47bb-0310-9956-ffa450edef68
[HBASE-7276] Fix TestRegionServerMetrics.testNumReadsAndWrites to work with the recent changes in HRegion metrics.

Author: adela

Summary:
-removed read counter increase in close();
-changed increment of read operation during getScanner from HRegion to HRegionServer

Test Plan: unit test is passing now

Reviewers: liyintang, andrewl

Reviewed By: liyintang

CC: hbase-eng@, kannan

Differential Revision: https://phabricator.fb.com/D666035

Task ID: 1981461

git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/0.89-fb@1426297 13f79535-47bb-0310-9956-ffa450edef68
just a test
Merge remote-tracking branch 'upstream/0.89-fb' into 0.89-fb