svn merge -c 1347867 from trunk for HADOOP-8469. Make NetworkTopology class pluggable.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488838 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1351445 from trunk for HADOOP-8470. Add NetworkTopologyWithNodeGroup, a 4-layer implementation of NetworkTopology.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488839 13f79535-47bb-0310-9956-ffa450edef68
Merge -c 1488840 from trunk to branch-2 to fix YARN-730. Fix NMClientAsync to remove completed containers. Contributed by Zhijie Shen.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488841 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1353807 from trunk for HDFS-3498. Support replica removal in BlockPlacementPolicy and make BlockPlacementPolicyDefault extensible for reusing code in subclasses.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488844 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1357442 from trunk for HDFS-3601. Add BlockPlacementPolicyWithNodeGroup to support block placement with 4-layer network topology.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488845 13f79535-47bb-0310-9956-ffa450edef68
Merge r1414874 and r1414878 from trunk for HDFS-3495. Update Balancer to support new NetworkTopology with NodeGroup.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488848 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1417130 from trunk for HDFS-4234. Use generic code for choosing datanode in Balancer.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488850 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1416691 from trunk for HDFS-4240. For nodegroup-aware block placement, when a node is excluded, he nodes in the same nodegroup should also be excluded.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488851 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1430917 from trunk for HDFS-4261. Fix bugs in Balaner causing infinite loop and TestBalancerWithNodeGroup timeing out.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488855 13f79535-47bb-0310-9956-ffa450edef68
svn merge -c 1431726 from trunk for HDFS-4382. Fix typo MAX_NOT_CHANGED_INTERATIONS.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1488856 13f79535-47bb-0310-9956-ffa450edef68