merge YARN-720 and MAPREDUCE-5291 from trunk. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488830 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1347867 from trunk for HADOOP-8469. Make NetworkTopology class pluggable.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488838 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1351445 from trunk for HADOOP-8470. Add NetworkTopologyWithNodeGroup, a 4-layer implementation of NetworkTopology.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488839 13f79535-47bb-0310-9956-ffa450edef68
Merge -c 1488840 from trunk to branch-2 to fix YARN-730. Fix NMClientAsync to remove completed containers. Contributed by Zhijie Shen.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488841 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1353807 from trunk for HDFS-3498. Support replica removal in BlockPlacementPolicy and make BlockPlacementPolicyDefault extensible for reusing code in subclasses.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488844 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1357442 from trunk for HDFS-3601. Add BlockPlacementPolicyWithNodeGroup to support block placement with 4-layer network topology.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488845 13f79535-47bb-0310-9956-ffa450edef68
Merge r1414874 and r1414878 from trunk for HDFS-3495. Update Balancer to support new NetworkTopology with NodeGroup.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488848 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1417130 from trunk for HDFS-4234. Use generic code for choosing datanode in Balancer.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488850 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1416691 from trunk for HDFS-4240. For nodegroup-aware block placement, when a node is excluded, he nodes in the same nodegroup should also be excluded.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488851 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1430917 from trunk for HDFS-4261. Fix bugs in Balaner causing infinite loop and TestBalancerWithNodeGroup timeing out.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488855 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1431726 from trunk for HDFS-4382. Fix typo MAX_NOT_CHANGED_INTERATIONS.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488856 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1489012 FIXES: MAPREDUCE-5268. Improve history server startup performance. Contributed by Karthik Kambatla


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489015 13f79535-47bb-0310-9956-ffa450edef68
Merge -r 1489059:1489060 from trunk to branch-2. Fixes: MAPREDUCE-5036. Default shuffle handler port should not be 8080. Contributed by Sandy Ryza.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489061 13f79535-47bb-0310-9956-ffa450edef68
YARN-326. Add multi-resource scheduling to the fair scheduler. (sandyr via tucu)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489072 13f79535-47bb-0310-9956-ffa450edef68
Merge -c 1489087 from trunk to branch-2 to fix YARN-398. Make it possible to specify hard locality constraints in resource requests for CapacityScheduler. Contributed by Arun C. Murthy.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489088 13f79535-47bb-0310-9956-ffa450edef68
HADOOP-9614. smart-test-patch.sh hangs for new version of patch (2.7.1) (Ravi Prakash via jeagles)

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489153 13f79535-47bb-0310-9956-ffa450edef68
Merge -c 1489187 from trunk to branch-2 to fix MAPREDUCE-5220. Fix backward incompatibility for TaskCompletionEvent by adding back setTaskID. Contributed by Zhijie Shen.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489188 13f79535-47bb-0310-9956-ffa450edef68
Merge -c 1489219 from trunk to branch-2 to fix MAPREDUCE-5296. Fix backward incompatibility for JobControl by adding the omitted addJob. Contributed by Zhijie Shen.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489220 13f79535-47bb-0310-9956-ffa450edef68
Merge -c 1489227 from trunk to branch-2 to fix MAPREDUCE-5300. Fix backward incompatibility for o.a.h.mapreduce.filecache.DistributedCache. Contributed by Zhijie Shen.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489228 13f79535-47bb-0310-9956-ffa450edef68
MAPREDUCE-5299. Changed jira number for the commit in CHANGES.txt.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489230 13f79535-47bb-0310-9956-ffa450edef68