HDFS-4403. DFSClient can infer checksum type when not provided by reading first byte. Contributed by Todd Lipcon.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1436731 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4949. Enable multiple pi jobs to run in parallel. (sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437030 13f79535-47bb-0310-9956-ffa450edef68
merge MAPREDUCE-4946 from trunk. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437104 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-9231. Merge 1437140 and 1437143 from trunk

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437144 13f79535-47bb-0310-9956-ffa450edef68
merge -c 1437156 from trunk to branch-2 to fix YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contriubed by Bikas Saha

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437159 13f79535-47bb-0310-9956-ffa450edef68
merge -c 1437245 from trunk to branch-2 to fix YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437246 13f79535-47bb-0310-9956-ffa450edef68
HDFS-4426. Merge change 1437627 from trunk.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437628 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1437775 FIXES: YARN-354. WebAppProxyServer exits immediately after startup. Contributed by Lian Xie


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437776 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438604 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4049. Experimental api to allow for alternate shuffle plugins. Contributed by Anver BenHanoch.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438793 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4809. Change visibility of classes for pluggable sort changes. (masokan via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438794 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4807. Allow MapOutputBuffer to be pluggable. (masokan via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438795 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4808. Refactor MapOutput and MergeManager to facilitate reuse by Shuffle implementations. (masokan via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438796 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-2264. Job status exceeds 100% in some cases. (devaraj.k and sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438800 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1438808 from trunk for HDFS-4443. Remove a trailing '`' character from the HTML code generated by NamenodeJspHelper.generateNodeData(..).


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438809 13f79535-47bb-0310-9956-ffa450edef68
HDFS-4259. Improve pipeline DN replacement failure message. Contributed by Harsh J. (harsh)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1439127 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-9241. DU refresh interval is not configurable. Contributed by Harsh J. (harsh)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1439130 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-9247. Merge r1438698 from trunk

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1439531 13f79535-47bb-0310-9956-ffa450edef68
Merge pull request #6 from apache/branch-2

Merge branch-2 upstream changes