HDFS-4393. Merge 1434844 from trunk.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1434852 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1434868 FIXES: HADOOP-8849. FileUtil#fullyDelete should grant the target directories +rwx permissions (Ivan A. Veselovsky via bobby)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1434870 13f79535-47bb-0310-9956-ffa450edef68
HDFS-4415. HostnameFilter should handle hostname resolution failures and continue processing. Contributed by Robert Kanter.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1434952 13f79535-47bb-0310-9956-ffa450edef68
YARN-331. Fill in missing fair scheduler documentation. (sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1434991 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4923. Add toString method to TaggedInputSplit. (sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1434994 13f79535-47bb-0310-9956-ffa450edef68
YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1434997 13f79535-47bb-0310-9956-ffa450edef68
HDFS-4359. Slow RPC responses from NN can prevent metrics collection on DNs. Contributed by liang xie.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1435300 13f79535-47bb-0310-9956-ffa450edef68
merge MAPREDUCE-4948 from trunk. Fix a failing unit test - TestYARNRunner.testHistoryServerToken. Contributed by Junping Du


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1435313 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1435386 FIXES: MAPREDUCE-4458. Warn if java.library.path is used for AM or Task (Robert Parker via jeagles)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1435394 13f79535-47bb-0310-9956-ffa450edef68
HDFS-4403. DFSClient can infer checksum type when not provided by reading first byte. Contributed by Todd Lipcon.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1436731 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4949. Enable multiple pi jobs to run in parallel. (sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437030 13f79535-47bb-0310-9956-ffa450edef68
merge MAPREDUCE-4946 from trunk. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437104 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-9231. Merge 1437140 and 1437143 from trunk

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437144 13f79535-47bb-0310-9956-ffa450edef68
merge -c 1437156 from trunk to branch-2 to fix YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contriubed by Bikas Saha

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437159 13f79535-47bb-0310-9956-ffa450edef68
merge -c 1437245 from trunk to branch-2 to fix YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437246 13f79535-47bb-0310-9956-ffa450edef68
HDFS-4426. Merge change 1437627 from trunk.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437628 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1437775 FIXES: YARN-354. WebAppProxyServer exits immediately after startup. Contributed by Lian Xie


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1437776 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-8857. hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated. (tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438604 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4049. Experimental api to allow for alternate shuffle plugins. Contributed by Anver BenHanoch.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438793 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-4809. Change visibility of classes for pluggable sort changes. (masokan via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1438794 13f79535-47bb-0310-9956-ffa450edef68