Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.0.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add support to turn off launching privileged containers in the container-executor | Blocker | nodemanager | Varun Vasudev | Varun Vasudev | | | Update Dockerfile to use Xenial | Major | build, test | Allen Wittenauer | Allen Wittenauer | | | ReconfigurationTaskStatus is exposing guava Optional in its public api | Major | common | Haibo Chen | Xiao Chen | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-core | Major | . | Jinjiang Ling | Jinjiang Ling | | | ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED | Blocker | erasure-coding | Xiao Chen | Xiao Chen | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | truly shared cache for jars (jobjar/libjar) | Major | . | Sangjin Lee | Chris Trezzo | | | Router-based HDFS federation | Major | fs | igo Goiri | igo Goiri | | | OrgQueue for easy CapacityScheduler queue configuration management | Major | . | Min Shen | Min Shen | | | Add support for the YARN Shared Cache | Major | . | Chris Trezzo | Chris Trezzo | | | Add additional deSelects params in RMWebServices#getAppReport | Major | resourcemanager, router | Giovanni Matteo Fumarola | Tanuj Nayak | | | Tool to estimate resource requirements of an application pipeline based on prior executions | Major | tools | Subru Krishnan | Rui Li | | | Support Application timeout feature in YARN. | Major | scheduler | nijel | Rohith Sharma K S | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add peak memory usage counter for each task | Major | mrv2 | Yufei Gu | Miklos Szegedi | | | Remove FSLeafQueue#addAppSchedulable | Major | fairscheduler | Yufei Gu | Sen Zhao | | | Add more states and transitions to stabilize the NM Container state machine | Major | . | Arun Suresh | kartheek muthyala | | | Fix the word of "erasure encoding" in the top page | Trivial | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Document caveats about the default JavaKeyStoreProvider in KMS | Major | documentation, kms | Xiao Chen | Xiao Chen | | | Update site release notes for 3.0.0 GA | Major | site | Andrew Wang | Andrew Wang | | | Add an option to disallow 'namenode format -force' | Major | . | Ajay Kumar | Ajay Kumar | | | KMS client needs retry logic | Major | . | Rushabh S Shah | Rushabh S Shah | | | add ability in Fair Scheduler to optionally configure maxResources in terms of percentage | Major | fairscheduler, scheduler | Ashwin Shankar | Yufei Gu | | | Cache the RM proxy server address | Major | RM | Yufei Gu | Yufei Gu | | | Update project release notes with HDFS-10467 for 3.0.0 | Major |

| igo Goiri | igo Goiri | | | Divide the total block metrics into replica and ec | Major | erasure-coding, metrics, namenode | Takanobu Asanuma | Takanobu Asanuma | | | Add nameServiceId to QJournalProtocol | Major | journal-node | Bharat Viswanadham | Bharat Viswanadham | | | Enable async edit logging by default | Major | namenode | Andrew Wang | Andrew Wang | | | Log block and datanode details in BlockRecoveryWorker | Major | datanode | Xiao Chen | Xiao Chen | | | Configuration.updatingResource map should be initialized lazily | Major | . | Misha Dmitriev | Misha Dmitriev | | | Native EC coder should implement release() as idempotent function. | Major | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Enable try-with-resources for RecordReader | Major | . | Zoltan Haindrich | Zoltan Haindrich | | | [KMS] Document&test missing KMS client side configs | Minor | . | Wei-Chiu Chuang | Gabor Bota | | | Do not catch and throw unchecked exceptions if IBRs fail to process | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | hadoop-client does not include hadoop-yarn-client | Critical | common | Haibo Chen | Ajay Kumar | | | TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic | Minor | . | Haibo Chen | Haibo Chen | | | Make sure user defined erasure coding policy ID will not overflow | Major | erasure-coding | Sammi Chen | Huafeng Wang | | | Add JvmMetrics to KMS | Major | kms | Xiao Chen | Xiao Chen | | | Add debug message for better download latency monitoring | Major | nodemanager | Yufei Gu | Yufei Gu | | | Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations | Trivial | resourcemanager | Daniel Templeton | Sen Zhao | | | Audit getQueueInfo and getApplications calls | Major | . | Chang Li | Chang Li | | | NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports | Minor | net | Steve Loughran | Varun Saxena | | | Make TestResourceManager Scheduler agnostic | Major | test | Robert Kanter | Robert Kanter | | | SnapshotDiff - support diff generation on any snapshot root descendant directory | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler | Minor | resourcemanager | Haibo Chen | Haibo Chen | | | Duplicate LiteralByteStrings in SystemCredentialsForAppsProto.credentialsForApp\_ | Major | . | Misha Dmitriev | Misha Dmitriev | | | Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow | Major | . | Robert Kanter | Robert Kanter | | | Reduce lock contention in FairScheduler#getAppWeight() | Major | fairscheduler | Daniel Templeton | Daniel Templeton | | | Upgrade Avro patch version | Major | build | Chris Douglas | Bharat Viswanadham | | | FileSystem based Yarn Registry implementation | Major | amrmproxy, api, resourcemanager | Ellen Hui | Ellen Hui | | | Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery | Minor | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | More logs when short-circuit read is failed and disabled | Major | datanode | Weiwei Yang | Weiwei Yang | | | Add genstamp and block size to metasave Corrupt blocks list | Minor | . | Kuhu Shukla | Kuhu Shukla | | | Improve KMSClientProvider log around delegation token checking | Major |

| Xiaoyu Yao | Xiaoyu Yao | | | Logging task counters | Major | task | Prabhu Joseph | Prabhu Joseph | | | Reduce lock contention in ClusterNodeTracker#getClusterCapacity() | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | Avoid taking locks when sending heartbeats from the DataNode | Major | . | Haohui Mai | Jiandan Yang | | | Support resource type in SLS | Major | scheduler-load-simulator | Yufei Gu | Yufei Gu | | | Duplicate Strings in various places in Yarn memory | Major | . | Misha Dmitriev | Misha Dmitriev | | | Add GC time percentage monitor/alerter | Major | . | Misha Dmitriev | Misha Dmitriev | | | Add site release notes for OrgQueue and resource types | Major | . | Andrew Wang | Andrew Wang | | | Create downstream developer docs from the compatibility guidelines | Critical | documentation | Daniel Templeton | Daniel Templeton | | | ber-jira adl:// Azure Data Lake Phase I: Stabilization | Major | fs/adl | Steve Loughran | John Zhuge | | | AliyunOSS: change the default value of max error retry | Major | fs/oss | wujinhu | wujinhu | | | ber-jira: S3a phase III: scale and tuning | Major | fs/s3 | Steve Loughran | Steve Loughran | | | ber-jira: WASB client phase II: performance and testing | Major | fs/azure | Steve Loughran | Thomas Marquardt | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | ResourceCalculator.fitsIn() should not take a cluster resource parameter | Major | scheduler | Daniel Templeton | Sen Zhao | | | ReuseObjectMapper in Hadoop Common | Minor | . | Hanisha Koneru | Hanisha Koneru | | | NM returns new SCHEDULED container status to older clients | Blocker | nodemanager | Jason Lowe | Arun Suresh | | | LoadGenerator#genFile write close timing is incorrectly calculated | Major | fs | Jason Lowe | Hanisha Koneru | | | TestSchedulingMonitor#testRMStarts fails sporadically | Major | . | Jason Lowe | Jason Lowe | | | TestDataNodeHotSwapVolumes fails in trunk Jenkins runs | Critical | test | Arpit Agarwal | Lei (Eddy) Xu | | | method name is incorrect in ConfServlet | Minor | . | Bharat Viswanadham | Bharat Viswanadham | | | TestCopyFromLocal#testCopyFromLocalWithThreads is fleaky | Major | test | Andras Bokor | Andras Bokor | | | Unset EC policy logs empty payload in edit log | Blocker | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Update Yarn to YARN in documentation | Minor | documentation | Miklos Szegedi | Chetna Chaudhari | | | AMSimulator in SLS does't work due to refactor of responseId | Blocker | scheduler-load-simulator | Yufei Gu | Botong Huang | | | SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined | Minor | . | Nandor Kollar | Nandor Kollar | | | TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails | Major | capacity scheduler, test | Wangda Tan | Akira Ajisaka | | | Whitelisted variables do not support delayed variable expansion | Major | nodemanager | Jason Lowe | Jason Lowe | | | Client fails to read a block with erasure code (XOR, native) when one of the data block is lost | Blocker | . | Ayappan | Huafeng Wang | | | Fix typo in helper message of ContainerLauncher | Trivial |

| Elek, Marton | Elek, Marton | | | Add Node and Rack Hints to Opportunistic Scheduler | Major | . | Arun Suresh | kartheek muthyala | | | TestNMClient.testNMClientNoCleanupOnStop is flaky by design | Major | . | Miklos Szegedi | Miklos Szegedi | | | BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes | Major | erasure-coding | Andrew Wang | Andrew Wang | | | libhdfs SIGSEGV in setTLSExceptionStrings | Major | libhdfs | John Zhuge | John Zhuge | | | Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | When using native decoder, DFSStripedStream#close crashes JVM after being called multiple times. | Critical | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Remove Mockito dependency from DataNodeTestUtils | Minor | test | Ted Yu | Ted Yu | | | TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | FairCallQueue may defer servicing calls | Major | ipc | Daryn Sharp | Daryn Sharp | | | Unnecessary exception declaration of the CellBuffers constructor | Minor | . | Huafeng Wang | Huafeng Wang | | | Fix enumerate in HDFSErasureCoding.md | Minor | documentation | Akira Ajisaka | Yiqun Lin | | | TestContainerManagerSecurity failing in trunk | Major | . | Varun Saxena | Akira Ajisaka | | | Change Configuration.getPropsWithPrefix to use getProps instead of iterator | Major | . | Larry McCay | Larry McCay | | | Update GroupsMapping documentation to reflect the new changes | Major | documentation | Anu Engineer | Esther Kundin | | | Fix unsafe casting from long to int for class Resource and its sub-classes | Major | resourcemanager | Yufei Gu | Yufei Gu | | | TestAMRMProxy#testAMRMProxyE2E fails with FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications | Critical | . | Sumana Sathish | Tan, Wangda | | | Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval | Minor | . | Ajay Kumar | Ajay Kumar | | | expunge may fail to remove trash from encryption zone | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestKMS#testACLs fails intermittently | Major | kms, test | Xiao Chen | Xiao Chen | | | LogAggregationTFileController deletes/renames while file is open | Critical | nodemanager | Daryn Sharp | Jason Lowe | | | container-executor fails to remove entries from a directory that is not writable or executable | Critical | . | Jason Lowe | Jason Lowe | | | TestApplicationACLs fails with FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | Document missing config key hadoop.treat.subject.external | Minor | security | Wei-Chiu Chuang | Ajay Kumar | | | FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | TestRouterWebServiceUtil#testMergeMetrics is flakey | Major | federation | Robert Kanter | Robert Kanter | | | DFSStripedOutputStream#close will throw if called a second time with a failed streamer | Major | erasure-coding | Andrew Wang | Lei (Eddy) Xu | | | CLONE - Fix source-level compatibility after HADOOP-11252 | Blocker | . | Junping Du | Junping Du | |

TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair scheduler | Major | . | Miklos Szegedi | Miklos Szegedi | | | Improve bower dependencies for YARN UI v2 | Critical | webapp | Sunil Govindan | Sunil Govindan | | | TestDistributedShell should be scheduler agnostic | Major | . | Haibo Chen | Haibo Chen | | | Re-enable TestDFSStripedOutputStreamWithFailure tests | Major | erasure-coding | Andrew Wang | Huafeng Wang | | | DistCp#cleanup() should check whether jobFS is null | Minor | . | Ted Yu | Andras Bokor | | | Fix shell check warnings of SLS. | Major | . | Wangda Tan | Gergely Novk | | | Re-encryption should handle task cancellation and progress better | Major | encryption | Xiao Chen | Xiao Chen | | | Handle JDK-8071638 for hadoop-common | Blocker | . | Bibin A Chundatt | Bibin A Chundatt | | | dfsadmin -metaSave to output maintenance mode blocks | Minor | namenode | Wei-Chiu Chuang | Wellington Chevreuil | | | Add a link to HDFS router federation document in site.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | NoSuchElementException in FairScheduler after failover causes RM crash | Blocker | fairscheduler | Robert Kanter | Robert Kanter | | | TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE | Major | test | Robert Kanter | Yufei Gu | | | PreCommit TestKDiag failure | Major | security | John Zhuge | Wei-Chiu Chuang | | | Upgrade maven-dependency-plugin to 3.0.2 | Major | build | liyunzhang | liyunzhang | | | Xenial dockerfile needs ant in main build for findbugs | Trivial | build, test | Allen Wittenauer | Akira Ajisaka | | | LocalityMulticastAMRMProxyPolicy should handle cancel request properly | Minor | . | Botong Huang | Botong Huang | | | Fix configurable WebhDFS connect/read timeout | Blocker | hdfs-client, webhdfs | Xiaoyu Yao | Xiaoyu Yao | | | Possible NPE in RMWebapp when HA is enabled and the active RM fails | Major | . | Chandni Singh | Chandni Singh | | | Replace HdfsFileStatus constructor with a builder pattern. | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Enforce JDK limitations | Major | build | Chris Douglas | Chris Douglas | | | Add missing figures to Resource Estimator tool | Major | . | Subru Krishnan | Rui Li | | | Fix TestDistributedScheduler | Major | . | Jason Lowe | Arun Suresh | | | TestFSAppStarvation.testPreemptionEnable fails intermittently | Major | . | Sunil Govindan | Miklos Szegedi | | | Unsafe cast from long to int Resource.hashCode() method | Critical | resourcemanager | Daniel Templeton | Miklos Szegedi | | | ShuffleHandler is not aware of disks that are added | Major | . | Kuhu Shukla | Kuhu Shukla | | | Clean up jdiff xml files added for 2.8.2 release | Blocker | . | Subru Krishnan | Junping Du | | | BZip2 drops records when reading data in splits | Critical | . | Aki Tanaka | Jason Lowe | | | TestMountTable fails with Java 7 | Major | . | igo Goiri | igo Goiri | | | Hadoop 3 missing fix for HDFS-5169 | Major | native | Joe McDonnell | Joe McDonnell | | | Javadoc for FSNamesystem#getMaxObjects is incorrect | Trivial | . | Erik Krogen | Erik Krogen | | |

is failing | Critical | nodemanager | Haibo Chen | Eric Badger | | | Fix typo "onf" in DynamicInputChunk.java | Trivial | . | fang zhenyi | fang zhenyi | | | Queue dash board in new YARN UI has incorrect values | Critical | webapp | Sunil Govindan | Sunil Govindan | | | Preemption properties should be refreshable | Major | capacity scheduler, scheduler preemption | Eric Payne | Gergely Novk | | | incorrect log preview displayed in jobhistory server ui | Major | yarn | Santhosh B Gowda | Xuan Gong | | | Fix ResourceEstimator findbugs issues | Blocker | . | Allen Wittenauer | Arun Suresh | | | Fix DominantResourceFairnessPolicy serializable findbugs issues | Blocker | . | Allen Wittenauer | Daniel Templeton | | | Router getApps REST invocation fails with multiple RMs | Critical | . | Subru Krishnan | igo Goiri | | | BlockPlacementPolicyRackFaultTolerant fails with very uneven racks | Major | erasure-coding | Xiao Chen | Xiao Chen | | | Add support for change of container ExecutionType | Major | . | Arun Suresh | Arun Suresh | | | Add containerId to Localizer failed logs | Minor | nodemanager | Prabhu Joseph | Prabhu Joseph | | | Cleanup FixedValueResource to avoid dependency to ResourceUtils | Major | resourcemanager | Sunil Govindan | Wangda Tan | | | TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic | Major | test | Haibo Chen | Haibo Chen | | | Update JAVA\_HOME in create-release for Xenial Dockerfile | Blocker | build | Andrew Wang | Andrew Wang | | | Reset the upload button when file upload fails | Critical | ui, webhdfs | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix issue where RM fails to switch to active after first successful start | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | TestContainerManagerSecurity is still flakey | Major | test | Robert Kanter | Robert Kanter | | | Ensure singleton for ResourceEstimatorService | Major | . | Subru Krishnan | Rui Li | | | Correct spellings of ramdomly to randomly in log. | Trivial | . | hu xiaodong | hu xiaodong | | | RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup | Minor | resourcemanager | Jason Lowe | Jason Lowe | | | TestAMRestart should be scheduler agnostic | Major | . | Haibo Chen | Haibo Chen | | | start-yarn.sh fails to start ResourceManager unless running as root | Blocker | . | Sean Mackrory | | | | ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used | Major | io | Mike Percy | Mike Percy | | | Fix Container log link in new YARN UI | Major | . | Sunil Govindan | Sunil Govindan | | | Update LICENSE.txt for HADOOP-14840 | Major | . | Arun Suresh | Arun Suresh | | | Containers stuck in Localizing state | Major | . | Jagadish | Billie Rinaldi | | | Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | GC time percentage reported in JvmMetrics should be a gauge, not counter | Major |

| Misha Dmitriev | Misha Dmitriev | | | Method canContainerBePreempted can return true when it shouldn't | Major | fairscheduler | Steven Rand | Steven Rand | | | Lease renewal can hit a deadlock | Major | . | Kuhu Shukla | Kuhu Shukla | | | RM can crash during transitionToStandby due to InterruptedException | Critical | resourcemanager | Jason Lowe | Bibin A Chundatt | | | Make FsServerDefaults cache configurable. | Minor | . | Rushabh S Shah | Mikhail Erofeev | | | AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby | Critical | . | Tao Yang | Tao Yang | | | TestPBImplRecords fails with NullPointerException | Major | . | Jason Lowe | Daniel Templeton | | | create-release site build outputs dummy shaded jars due to skipShade | Blocker | . | Andrew Wang | Andrew Wang | | | Router UI is missing robots.txt file | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | EC Checksum broken when BlockAccessToken is enabled | Critical | erasure-coding | Xiao Chen | Xiao Chen | | | Block recovery will fail indefinitely if recovery time \> heartbeat interval | Critical | datanode, hdfs, namenode | Lukas Majercak | Lukas Majercak | | | Remove the trailing duplicated timestamp in container diagnostics message | Minor | nodemanager | Weiwei Yang | Weiwei Yang | | | Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog | Blocker | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x "json-lib" | Blocker | fs/oss | Chris Douglas | Sammi Chen | | | 3.0 deployment cannot work with old version MR tar ball which breaks rolling upgrade | Blocker | security | Junping Du | Jason Lowe | | | NPE in async-scheduling mode of CapacityScheduler | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled by default in yarn-default.xml | Critical | . | Xuan Gong | Xuan Gong | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add support for multiple resource types in the Resource class | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Extend DominantResourceCalculator to account for all resources | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support to read resource types from a config file | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support for binary units | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support for resource types in the nodemanager | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | Update DominantResourceCalculator to consider all resource types in calculations | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Update the Resources class to consider all resource types | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM | Major | distributed-scheduling | Arun Suresh | Hitesh Sharma | | | Changes to allow CapacityScheduler to use configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Create YarnConfigurationStore interface and InMemoryConfigurationStore class | Major |

| Jonathan Hung | Jonathan Hung | | | Update resource usage and preempted resource calculations to take into account all resource types | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Implement MutableConfigurationManager for handling storage into configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Create REST API for changing YARN scheduler configurations | Major | . | Jonathan Hung | Jonathan Hung | | | Federation State Store Interface API | Major | fs | Jason Kace | Jason Kace | | | [YARN-3926] Performance improvements in resource profile branch with respect to SLS | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Federation State Store FS Implementation | Major | hdfs | igo Goiri | Jason Kace | | | Add pluggable configuration ACL policy interface and implementation | Major | . | Jonathan Hung | Jonathan Hung | | | Support global configuration mutation in MutableConfProvider | Major | . | Jonathan Hung | Jonathan Hung | | | Create CLI for changing YARN configurations | Major | . | Jonathan Hung | Jonathan Hung | | | Fix build for YARN-3926 branch | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | ResourcePBImpl imports cleanup | Trivial | resourcemanager | Daniel Templeton | Yeliang Cang | | | NM Container lifecycle and state transitions to support for PAUSED container state. | Major | . | Hitesh Sharma | Hitesh Sharma | | | Rebasing HDFS-10467 | Major | fs | igo Goiri | igo Goiri | | | Federation Membership State Store internal API | Major | hdfs | igo Goiri | Jason Kace | | | Create LeveldbConfigurationStore class using Leveldb as backing store | Major | . | Jonathan Hung | Jonathan Hung | | | Disable queue refresh when configuration mutation is enabled | Major | . | Jonathan Hung | Jonathan Hung | | | Federation Namenode Heartbeat | Major | fs | igo Goiri | igo Goiri | | | Improve performance of resource profile branch | Blocker | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | Federation Mount Table State Store internal API | Major | fs | Jason Kace | igo Goiri | | | Federation admin tool | Major | fs | igo Goiri | igo Goiri | | | Remove last uses of Long from resource types code | Minor | resourcemanager | Daniel Templeton | Daniel Templeton | | | Improve API implementation in Resources and DominantResourceCalculator class | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | [Documentation] Router-based federation documentation | Minor | fs | igo Goiri | igo Goiri | | | Rebasing HDFS-10467 (2) | Major | . | igo Goiri | igo Goiri | | | DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | Performance optimizations in Resource and ResourceUtils class | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Clean up unit tests after YARN-6610 | Major | test | Daniel Templeton | Daniel Templeton | | | Add Client API to get all supported resource types from RM | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | |

ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter | Minor | resourcemanager | Daniel Templeton | Yu-Tang Lin | | | Federation State Store ZooKeeper implementation | Major | fs | igo Goiri | Jason Kace | | | Optimize ResourceType information display in UI | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Fix javac and javadoc errors in YARN-3926 branch | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | Fix issues on recovery in LevelDB store | Major | . | Jonathan Hung | Jonathan Hung | | | Improve log message in ResourceUtils | Trivial | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | ResourceUtils.DISALLOWED\_NAMES check is duplicated | Major | resourcemanager | Daniel Templeton | Manikandan R | | | Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem | Major | resourcemanager | Subru Krishnan | Subru Krishnan | | | Fixing compilation issue with BanDuplicateClasses | Major | fs | igo Goiri | igo Goiri | | | Federation Metrics | Major | fs | Giovanni Matteo Fumarola | igo Goiri | | | SharingPolicy enhancements required to support recurring reservations in ReservationSystem | Major | resourcemanager | Subru Krishnan | Carlo Curino | | | Add a new log aggregation file format controller | Major | . | Xuan Gong | Xuan Gong | | | Additional Performance Improvement for Resource Profile Feature | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Move newly added APIs to unstable in YARN-3926 branch | Blocker | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Update paused container state in the NM state store | Blocker | . | Hitesh Sharma | Hitesh Sharma | | | Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218 | Major | fs | igo Goiri | igo Goiri | | | Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat | Major | . | Xuan Gong | Xuan Gong | | | Update fair scheduler policies to be aware of resource types | Major | fairscheduler | Daniel Templeton | Daniel Templeton | | | Fixing TestNamenodeHeartbeat and support non-HA | Major | fs | igo Goiri | igo Goiri | | | Implement zookeeper based store for scheduler configuration updates | Major | . | Wangda Tan | Jonathan Hung | | | Add closing logic to configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | [Documentation] Adding configuration keys for the Router | Minor | fs | igo Goiri | igo Goiri | | | Documentation for API based scheduler configuration management | Major | . | Jonathan Hung | Jonathan Hung | | | Removing queue then failing over results in exception | Critical | . | Jonathan Hung | Jonathan Hung | | | Misc changes to YARN-5734 | Major | . | Jonathan Hung | Jonathan Hung | | | Update Shared cache client api to use URLs | Minor | . | Chris Trezzo | Chris Trezzo | | | Add a size threshold beyond which yarn logs will require a force option | Major | . | Siddharth Seth | Xuan Gong | | | Add size-based rolling policy to LogAggregationIndexedFileController | Major | . | Xuan Gong | Xuan Gong | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-app | Major | . | Jinjiang Ling | Jinjiang Ling | | | Capture launch\_container.sh logs to a separate log file | Major |

| Wangda Tan | Suma Shivaprasad | | | Rebasing HDFS-10467 after HDFS-12447 | Major | fs | igo Goiri | igo Goiri | | | Moving logging APIs over to slf4j in hadoop-yarn-server-common | Major | . | Akira Ajisaka | Akira Ajisaka | | | Make AllocateRequestProto compatible with branch-2/branch-2.8 | Critical | yarn | Ray Chiang | Ray Chiang | | | Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router | Major | . | Yeliang Cang | Yeliang Cang | | | Add work preserving restart support for Unmanaged AMs | Major | resourcemanager | Karthik Kambatla | Botong Huang | | | Placement Agent enhancements required to support recurring reservations in ReservationSystem | Blocker | resourcemanager | Subru Krishnan | Carlo Curino | | | Fix alignment issues and missing information in new YARN UI's Queue page | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | Add documentation for the YARN shared cache | Major | . | Chris Trezzo | Chris Trezzo | | | Federation UI | Major | fs | igo Goiri | igo Goiri | | | Rename Router tooling | Major | fs | igo Goiri | igo Goiri | | | Extend TestSafeModeWithStripedFile with a random EC policy | Major | erasure-coding, test | Takanobu Asanuma | Takanobu Asanuma | | | Merge YARN-5734 branch to branch-3.0 | Major | . | Xuan Gong | Xuan Gong | | | Extend TestQuotaWithStripedBlocks with a random EC policy | Major | erasure-coding, test | Takanobu Asanuma | Takanobu Asanuma | | | Log improvements for the ResourceUtils | Major | nodemanager, resourcemanager | Jian He | Sunil Govindan | | | Extend TestDistributedFileSystemWithECFile with a random EC policy | Major | erasure-coding, test | Takanobu Asanuma | Takanobu Asanuma | | | NM Statestore cleanup for Container updates | Blocker | . | Arun Suresh | kartheek muthyala | | | Fix TestRMWebServicesReservation parametrization for fair scheduler | Blocker | fairscheduler, reservation system | Yufei Gu | Yufei Gu | | | SLS is slow while loading 10k queues | Major | scheduler-load-simulator | Yufei Gu | Yufei Gu | | | GPU Isolation: Incorrect minor device numbers written to devices.deny file | Major | . | Jonathan Hung | Jonathan Hung | | | Support same origin policy for cross site scripting prevention. | Major | yarn-ui-v2 | Vrushali C | Sunil Govindan | | | Backporting HDFS-10467 to branch-2 | Major | . | igo Goiri | igo Goiri | | | Make Collections.sort() more efficient by caching resource usage | Major | fairscheduler | Xianyin Xin | Yufei Gu | | | Docker permitted volumes don't properly check for directories | Major | yarn | Eric Badger | Eric Badger | | | DominantResourceCalculator.isAnyMajorResourceZero() should test all resources | Major | scheduler | Daniel Templeton | Sunil Govindan | | | Basic Federation UI | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | igo Goiri | | | Document configuration of ReservationSystem for FairScheduler | Blocker | capacity scheduler | Subru Krishnan | Yufei Gu | | | Add REST API for supporting recurring reservations | Major | resourcemanager | Sangeetha Abdu Jyothi | Sean Po | | | Define the strings used in SLS JSON input file format | Major | scheduler-load-simulator | Yufei Gu | Gergely Novk | | | Add documentation for Container Update API | Blocker |

| Arun Suresh | Arun Suresh | | | Improve performance of DRF comparisons for resource types in fair scheduler | Critical | fairscheduler | Daniel Templeton | Daniel Templeton | | | Add support for individual resource types requests in MapReduce | Major | resourcemanager | Daniel Templeton | Gergo Repas | | | Moving logging APIs over to slf4j in hadoop-yarn-applications | Major | . | Yeliang Cang | Yeliang Cang | | | Moving logging APIs over to slf4j in hadoop-yarn-client | Major | . | Yeliang Cang | Yeliang Cang | | | Federation Router Web Service fixes | Major | . | igo Goiri | igo Goiri | | | Add hadoop-aliyun as dependency of hadoop-cloud-storage | Minor | fs/oss | Genmao Yu | Genmao Yu | | | Application lifetime does not work with FairScheduler | Major | resourcemanager | Miklos Szegedi | Miklos Szegedi | | | Add support for docker to have no capabilities | Major | yarn | Eric Badger | Eric Badger | | | Support ErasureCoding section in OIV XML/ReverseXML | Blocker | tools | Wei-Chiu Chuang | Huafeng Wang | | | Allow client/AM update supported resource types via YARN APIs | Blocker | nodemanager, resourcemanager | Wangda Tan | Sunil Govindan | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs | Major | . | Akira Ajisaka | Gergely Novk | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle | Trivial | . | Jinjiang Ling | Jinjiang Ling | | | Container REST endpoints should report resource types | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | FileNotFound handling in ResourceUtils is inconsistent | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | Rename PlacementSet and SchedulingPlacementSet | Major | . | Wangda Tan | Wangda Tan | | | Use LightWeightedResource when number of resource types more than two | Critical | resourcemanager | Daniel Templeton | Sunil Govindan | | | Moving logging APIs over to slf4j in hadoop-yarn-api | Major | . | Yeliang Cang | Yeliang Cang | | | [YARN-7069] Limit format of resource type name | Blocker | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Improve the resource types docs | Major | docs | Daniel Templeton | Daniel Templeton | | | AliyunOSS: Override listFiles and listLocatedStatus | Major | fs/oss | Genmao Yu | Genmao Yu | | | Enable user re-mapping for Docker containers by default | Blocker | security, yarn | Eric Yang | Eric Yang | | | AliyunOSS: support user agent configuration and include that & Hadoop version information to oss server | Major | fs, fs/oss | Sammi Chen | Sammi Chen | | | Node updates don't update the maximum cluster capability for resources other than CPU and memory | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy | Minor | . | Botong Huang | Botong Huang | | | Regenerate editsStored and editsStored.xml in HDFS tests | Major | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Compatibility Docs should clarify the policy for what takes precedence when a conflict is found | Critical | documentation | Robert Kanter | Daniel Templeton |