Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.1.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Metrics sinks may emit too frequently if multiple sink periods are configured | Minor | metrics | Erik Krogen | Erik Krogen | | | Fsck report shows config key name for min replication issues | Minor | hdfs | Harshakiran Reddy | Gabor Bota | | | RBF: Document Router and State Store metrics | Major | documentation | Yiqun Lin | Yiqun Lin | | | RBF: Add ACL support for mount table | Major | . | Yiqun Lin | Yiqun Lin | | | Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath | Major | timelineclient, timelinereader, timelineserver | Vrushali C | Varun Saxena | | | S3 blob etags to be made visible in S3A status/getFileChecksum() calls | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Use the ZooKeeper as the default State Store | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Docker image cannot set HADOOP\CONF\DIR | Major | . | Eric Badger | Jim Brennan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: Fix doc error setting up client | Major | federation | tartarus | tartarus | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Support meta tag element in Hadoop XML configurations | Major | . | Ajay Kumar | Ajay Kumar | | | [Umbrella] Extend the YARN resource model for easier resource-type management and profiles | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | [Umbrella] Support maintenance state for datanodes | Major | datanode, namenode | Ming Ma | Ming Ma | | | Implement linkMergeSlash and linkFallback for ViewFileSystem | Major | fs, viewfs | Zhe Zhang | Manoj Govindassamy | | | Add additional deSelects params in RMWebServices#getAppReport | Major | resourcemanager, router | Giovanni Matteo Fumarola | Tanuj Nayak | | | Tool to estimate resource requirements of an application pipeline based on prior executions | Major | tools | Subru Krishnan | Rui Li | | | Support for head in FSShell | Minor | . | Olga Natkovich | Gabor Bota | | | [Umbrella] Native YARN framework layer for services and beyond | Major | . | Vinod Kumar Vavilapalli | | | | [Umbrella] Simplified discovery of services via DNS mechanisms | Major | . | Vinod Kumar Vavilapalli | | | | Add S3A committers for zero-rename commits to S3 endpoints | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Allow HDFS block replicas to be provided by an external storage system | Major | . | Chris Douglas | | | | [Umbrella] Rich placement constraints in YARN | Major |

| Konstantinos Karanasos | | | | SnapshotDiff - Provide an iterator-based listing API for calculating snapshotDiff | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | NUMA awareness support for launching containers | Major | nodemanager, yarn | Olasoji | Devaraj K | | | [Umbrella] Support for FPGA as a Resource in YARN | Major | yarn | Zhankun Tang | Zhankun Tang | | | [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN | Major | . | Wangda Tan | Wangda Tan | | | Create official Docker images for development and testing features | Major | . | Elek, Marton | Elek, Marton | | | RBF: Support global quota | Major | . | igo Goiri | Yiqun Lin | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve click interaction in queue topology in new YARN UI | Major | yarn-ui-v2 | Abdullah Yousufi | Abdullah Yousufi | | | Add support for NM Recovery of assigned resources (e.g. GPU's, NUMA, FPGA's) to container | Major | nodemanager | Devaraj K | Devaraj K | | | Read HttpServer2 resources directly from the source tree (if exists) | Major | . | Elek, Marton | Elek, Marton | | | some wrong spelling words update | Trivial | . | Chen Hongfei | Chen Hongfei | | | Remove requirement to specify TenantGuid for MSI Token Provider | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | correct wrong parameters format order in core-default.xml | Trivial | . | Chen Hongfei | Chen Hongfei | | | FSDataInputStream#unbuffer UOE should include stream class name | Minor | fs | John Zhuge | Bharat Viswanadham | | | Suppress UnresolvedPathException in namenode log | Minor | . | Kihwal Lee | Kihwal Lee | | | Tighten up our compatibility guidelines for Hadoop 3 | Blocker | documentation | Karthik Kambatla | Daniel Templeton | | | Remove unused TaskLogAppender configurations from log4j.properties | Major | conf | Todd Lipcon | Todd Lipcon | | | Remove FSLeafQueue#addAppSchedulable | Major | fairscheduler | Yufei Gu | Sen Zhao | | | GetConf to get journalnodeslist | Major | journal-node, shell | Bharat Viswanadham | Bharat Viswanadham | | | Add quantiles for transactions batched in Journal sync | Major | metrics, namenode | Hanisha Koneru | Hanisha Koneru | | | Suppress the fsnamesystem lock warning on nn startup | Major | . | Ajay Kumar | Ajay Kumar | | | Remove unused parameter from FsDatasetImpl#addVolume | Minor | . | Chen Liang | Chen Liang | | | Reduce RM app memory footprint once app has completed | Major | resourcemanager | Jason Lowe | Manikandan R | | | Audit log for admin commands/ logging output of all DFS admin commands | Major | namenode | Raghu C Doppalapudi | Kuhu Shukla | | | Remove the extra word "it" in HdfsUserGuide.md | Trivial | . | fang zhenyi | fang zhenyi | | | Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold | Major | fairscheduler | Yufei Gu | Chetna Chaudhari | | | Use slf4j instead of log4j in FSNamesystem | Major | . | Ajay Kumar | Ajay Kumar | | | CrossOriginFilter should trigger regex on more input | Major | common, security | Allen Wittenauer | Johannes Alberti | | | WebHDFS - Adding "snapshot enabled" status to ListStatus query result. | Major | snapshots, webhdfs | Ajay Kumar | Ajay Kumar | | | Add an option to disallow 'namenode format -force' | Major |

| Ajay Kumar | Ajay Kumar | | | add ability in Fair Scheduler to optionally configure maxResources in terms of percentage | Major | fairscheduler, scheduler | Ashwin Shankar | Yufei Gu | | | Cache the RM proxy server address | Major | RM | Yufei Gu | Yufei Gu | | | KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. | Major | kms | Xiaoyu Yao | Xiaoyu Yao | | | Remove service loader config entry for ftp fs | Minor | fs | John Zhuge | Sen Zhao | | | Update javadoc and documentation for listStatus | Major | documentation | Ajay Kumar | Ajay Kumar | | | TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic | Minor | . | Haibo Chen | Haibo Chen | | | Add debug message for better download latency monitoring | Major | nodemanager | Yufei Gu | Yufei Gu | | | Use slf4j instead of log4j in LeaseManager | Major | . | Ajay Kumar | Ajay Kumar | | | Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations | Trivial | resourcemanager | Daniel Templeton | Sen Zhao | | | Audit getQueueInfo and getApplications calls | Major | . | Chang Li | Chang Li | | | NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports | Minor | net | Steve Loughran | Varun Saxena | | | Reduce lock contention in FairScheduler#getAppWeight() | Major | fairscheduler | Daniel Templeton | Daniel Templeton | | | API - expose a unique file identifier | Major | . | Sergey Shelukhin | Chris Douglas | | | FileSystem based Yarn Registry implementation | Major | amrmproxy, api, resourcemanager | Ellen Hui | Ellen Hui | | | Add genstamp and block size to metasave Corrupt blocks list | Minor | . | Kuhu Shukla | Kuhu Shukla | | | Add logging to successful standby checkpointing | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | | | Reduce lock contention in ClusterNodeTracker#getClusterCapacity() | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | Avoid taking locks when sending heartbeats from the DataNode | Major | . | Haohui Mai | Jiandan Yang | | | CryptoInputStream should implement unbuffer | Major | fs, security | John Zhuge | John Zhuge | | | Support resource type in SLS | Major | scheduler-load-simulator | Yufei Gu | Yufei Gu | | | Create downstream developer docs from the compatibility guidelines | Critical | documentation | Daniel Templeton | Daniel Templeton | | | FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight() | Minor | fairscheduler | Daniel Templeton | Soumabrata Chakraborty | | | Add blockId when warning slow mirror/disk in BlockReceiver | Trivial | hdfs | Jiandan Yang | Jiandan Yang | | | Upgrade maven surefire plugin to 2.20.1 | Major | build | Ewan Higgs | Akira Ajisaka | | | Remove unused FairSchedulerEventLog | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat | Minor | . | Wei Yan | Wei Yan | | | Improve robustness of the AggregatedLogDeletionService | Major | log-aggregation | Jonathan Eagles | Jonathan Eagles | | | snapshotDiff fails if the report exceeds the RPC response limit | Major | hdfs | Shashikant Banerjee | Shashikant Banerjee | | | Add open(PathHandle) with default buffersize | Trivial |

| Chris Douglas | Chris Douglas | | | Set HADOOP\SHELL\EXECNAME explicitly in scripts | Major | . | Arpit Agarwal | Arpit Agarwal | | | Move SemaphoredDelegatingExecutor to hadoop-common | Minor | fs, fs/oss, fs/s3 | Genmao Yu | Genmao Yu | | | Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM | Major | resourcemanager | Juan Rodrguez Hortal | Juan Rodrguez Hortal | | | Fix TestUnbuffer#testUnbufferException failure | Minor | test | Jack Bearden | Jack Bearden | | | Add readahead, dropbehind, and unbuffer to StreamCapabilities | Major | fs | John Zhuge | John Zhuge | | | AliyunOSS: change the default value of max error retry | Major | fs/oss | wujinhu | wujinhu | | | Ability to disable elasticity at leaf queue level | Major | capacityscheduler | Scott Brokaw | Zian Chen | | | Support full open(PathHandle) contract in HDFS | Major | hdfs-client | Chris Douglas | Chris Douglas | | | Expose NM node/containers resource utilization in JVM metrics | Major | nodemanager | Weiwei Yang | Weiwei Yang | | | Change to a safely casting long to int. | Major | . | Yufei Gu | Ajay Kumar | | | Secure Datanode Starter should log the port when it fails to bind | Minor | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Add test case to verify context update after container promotion or demotion with or without auto update | Minor | nodemanager | Weiwei Yang | Weiwei Yang | | | When partial log aggregation is enabled, display the list of aggregated files on the container log page | Major | . | Siddharth Seth | Xuan Gong | | | FileSystem::open(PathHandle) should throw a specific exception on validation failure | Minor | . | Chris Douglas | Chris Douglas | | | Support multiple storages in DataNodeCluster / SimulatedFSDataset | Minor | datanode, test | Erik Krogen | Erik Krogen | | | Fix confusing LOG message for block replication | Minor | hdfs | Chao Sun | Chao Sun | | | When NN is not able to identify DN for replication, reason behind it can be logged | Critical | hdfs-client, namenode | Surendra Singh Lilhore | Xiao Chen | | | ContainersMonitorImpl logged message lacks detail when exceeding memory limits | Major | nodemanager | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Explicitly describe the minimal number of DataNodes required to support an EC policy in EC document. | Minor | documentation, erasure-coding | Lei (Eddy) Xu | Hanisha Koneru | | | NameNode UI should report total blocks count by type - replicated and erasure coded | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | ContainerLogAppender Improvements | Trivial | . | BELUGA BEHR | | | | Miscellaneous Improvements To ProcfsBasedProcessTree | Minor | nodemanager | BELUGA BEHR | | | | Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | Ability to enable logging of container memory stats | Major | nodemanager | Jim Brennan | Jim Brennan | | | Enhance dfsadmin listOpenFiles command to list files under a given path | Major | . | Manoj Govindassamy | Yiqun Lin | | | Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods | Minor | . | Wei Yan | Wei Yan | | | Add LOG.isDebugEnabled() guard for LOG.debug("...") | Minor |

| Mehran Hassani | Bharat Viswanadham | | | Rename variables in MockNM, MockRM for better clarity | Trivial | . | lovekesh bansal | lovekesh bansal | | | Allow fair-scheduler configuration on HDFS | Minor | fairscheduler, resourcemanager | Greg Phillips | Greg Phillips | | | Use java.util.zip.CRC32C for Java 9 and above | Major | performance, util | Dmitry Chuyko | Dmitry Chuyko | | | Improve container-executor validation check | Major | security, yarn | Eric Yang | Eric Yang | | | Zookeeper authentication related properties to support CredentialProviders | Minor | security | Gergo Repas | Gergo Repas | | | FileOutputCommitter is slow on filesystems lacking recursive delete | Minor | . | Karthik Palaniappan | Karthik Palaniappan | | | Add closeStreams(...) to IOUtils | Major | . | Ajay Kumar | Ajay Kumar | | | MR AM to clean up temporary files from previous attempt in case of no recovery | Major | applicationmaster | Gergo Repas | Gergo Repas | | | Reusing the volume storage ID obtained by replicaInfo | Major | datanode | liaoyuxiangqin | liaoyuxiangqin | | | Clean up deprecation messages for allocation increments in FS config | Minor | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fast fail rogue jobs based on task scratch dir size | Major | task | Johan Gustavsson | Johan Gustavsson | | | Use pipes when localizing archives | Major | nodemanager | Jason Lowe | Miklos Szegedi | | | Reduce verbosity for ThrottledAsyncChecker.java:schedule | Minor | datanode | Mukul Kumar Singh | Mukul Kumar Singh | | | Provide support for JN to use separate journal disk per namespace | Major | federation, journal-node | Bharat Viswanadham | Bharat Viswanadham | | | Add symlink support to FileUtil#unTarUsingJava | Minor | util | Jason Lowe | Ajay Kumar | | | Add kdiag tool to hadoop command | Minor | . | Bharat Viswanadham | Bharat Viswanadham | | | Cleanup code in InterQJournalProtocol.proto | Minor | journal-node | Bharat Viswanadham | Bharat Viswanadham | | | Add independent secret manager method for logging expired tokens | Major | security | Daryn Sharp | Daryn Sharp | | | Cleanup AllocationFileLoaderService's reloadAllocations method | Minor | yarn | Szilard Nemeth | Szilard Nemeth | | | Limit the number of Snapshots allowed to be created for a Snapshottable Directory | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Improve logging when DFSStripedOutputStream failed to write some blocks | Minor | erasure-coding | Xiao Chen | chencan | | | Expose container preemptions related information in Capacity Scheduler queue metrics | Major | . | Eric Payne | Eric Payne | | | Avoid AM preemption caused by RRs for specific nodes or racks | Major | fairscheduler | Steven Rand | Steven Rand | | | Remove ADL mock test dependency on REST call invoked from Java SDK | Major | fs/adl | Vishwajeet Dusane | Vishwajeet Dusane | | | Uber AM can crash due to unknown task in statusUpdate | Major | mr-am | Peter Bacsko | Peter Bacsko | | | With SELinux enabled, directories mounted with start-build-env.sh may not be

| Major | build | Grigori Rybkine | Grigori Rybkine | | | [Umbrella] Improve S3A error handling & reporting | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Add Configuration API for parsing storage sizes | Minor | conf | Anu Engineer | Anu Engineer | | | Define and Implement a DiifList Interface to store and manage SnapshotDiffs | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | ADLS to support per-store configuration | Major | fs/adl | John Zhuge | Sharad Sonker | | | Enable HDFS diskbalancer by default | Major | diskbalancer | Ajay Kumar | Ajay Kumar | | | Create end user documentation from the compatibility guidelines | Critical | documentation | Daniel Templeton | Daniel Templeton | | | add test to verify FileSystem and paths differentiate on user info | Minor | fs, test | Steve Loughran | Steve Loughran | | | Capacity Scheduler Intra-queue Preemption should be configurable for each queue | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | XmlImageVisitor - Prefer Array over LinkedList | Minor | hdfs | BELUGA BEHR | BELUGA BEHR | | | DatanodeAdminManager Improvements | Trivial | hdfs | BELUGA BEHR | BELUGA BEHR | | | Authentication Tokens should use HMAC instead of MAC | Major | security | Robert Kanter | Robert Kanter | | | KerberosAuthenticator.authenticate to include URL on IO failures | Minor | security | Steve Loughran | Ajay Kumar | | | Add more information for checking argument in DiskBalancerVolume | Minor | diskbalancer | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Optimize disk access for last partial chunk checksum of Finalized replica | Major | datanode | Wei-Chiu Chuang | Gabor Bota | | | Upper/Lower case conversion support for group names in LdapGroupsMapping | Major | . | Nanda kumar | Nanda kumar | | | Add the L&N verification script | Major | . | Xiao Chen | Allen Wittenauer | | | Generalize NetUtils#wrapException to handle other subclasses with String Constructor | Major | . | Ajay Kumar | Ajay Kumar | | | Various Improvements for BlockTokenSecretManager | Trivial | hdfs | BELUGA BEHR | BELUGA BEHR | | | DelegationTokenAuthenticator.authenticate() to wrap network exceptions | Minor | net, security | Steve Loughran | Ajay Kumar | | | Make Job History File Permissions configurable | Major | . | Andras Bokor | Gergely Novk | | | Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment | Minor | encryption | LiXin Ge | LiXin Ge | | | SingleCluster setup document needs to be updated | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake | Minor | build | Steve Loughran | Steve Loughran | | | Stabilize and document Configuration \<tag\> element | Blocker | conf | Steve Loughran | Ajay Kumar | | | Implement SnapshotSkipList class to store Multi level DirectoryDiffs | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Fix the outdated javadocs in HAUtil | Trivial | . | Chao Sun | Chao Sun | | | RMStateStore should trim down app state for completed applications | Major | resourcemanager | Karthik Kambatla | Gergo Repas | | | increase maven heap size recommendations | Minor | build, documentation, test | Allen Wittenauer | Allen Wittenauer | | | Handle Deletion of nodes in SnasphotSkipList | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Checkstyle version is not compatible with IDEA's checkstyle plugin | Major |

| Andras Bokor | Andras Bokor | | | Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | HADOOP-15235 broke TestHttpFSServerWebServer | Major | test | Robert Kanter | Robert Kanter | | | Port webhdfs unmaskedpermission parameter to HTTPFS | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Reduce DiffListBySkipList memory usage | Major | snapshots | Tsz Wo Nicholas Sze | Shashikant Banerjee | | | Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList | Minor | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Update getBlocks method to take minBlockSize in RPC calls | Major | balancer & mover | Bharat Viswanadham | Bharat Viswanadham | | | StripeReader#checkMissingBlocks() 's IOException info is incomplete | Major | erasure-coding, hdfs-client | lufei | lufei | | | Support for getting erasure coding policy through WebHDFS#FileStatus | Major | erasure-coding, namenode | Kai Sasaki | Kai Sasaki | | | Code refactoring: Remove Diff.ListType | Major | snapshots | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Fix spelling mistake in DistCpUtils.java | Trivial | . | Jianfei Jiang | Jianfei Jiang | | | HttpServer2 needs a way to configure the acceptor/selector count | Major | common | Erik Krogen | Erik Krogen | | | DiskBalancer: Update Documentation to add newly added options | Major | diskbalancer, documentation | Bharat Viswanadham | Bharat Viswanadham | | | dfsadmin -report should report number of blocks from datanode | Minor | . | Lohit Vijayarenu | Bharat Viswanadham | | | Refactor TestDFSStripedOutputStreamWithFailure test classes | Minor | erasure-coding, test | Andrew Wang | Sammi Chen | | | Code cleanup: INode never throws QuotaExceededException | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Adding log for BlockPoolManager#refreshNamenodes failures | Minor | datanode | Xiaoyu Yao | Ajay Kumar | | | FileInputStream redundant closes in readReplicasFromCache | Minor | datanode | liaoyuxiangqin | liaoyuxiangqin | | | DistCp to eliminate needless deletion of files under already-deleted directories | Major | tools/distcp | Steve Loughran | Steve Loughran | | | Make HAR tool support IndexedLogAggregtionController | Major | . | Xuan Gong | Xuan Gong | | | BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo | Major | namenode | Konstantin Shvachko | chencan | | | Use cgroup to get container resource utilization | Major | nodemanager | Miklos Szegedi | Miklos Szegedi | | | Upgrade Maven surefire plugin | Major | build | Arpit Agarwal | Arpit Agarwal | | | ber-JIRA: S3Guard Phase II: Hadoop 3.1 features | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Undocumented KeyProvider configuration keys | Major | . | Wei-Chiu Chuang | LiXin Ge | | | Fix the CapacityScheduler Queue configuration documentation | Major | . | Arun Suresh | Jonathan Hung | | | NameNode should optionally exit if it detects FsImage corruption | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Support to specify application tags in distributed shell | Major | distributed-shell | Weiwei Yang | Weiwei Yang | | | ber-jira: S3a phase IV: Hadoop 3.1 features | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | [Umbrella] Enable configuration of queue capacity in terms of absolute resources | Major |

| Sean Po | Sunil Govindan | | | Kms client should disconnect if unable to get output stream from connection. | Major | kms | Xiao Chen | Rushabh S Shah | | | Reduce the HttpServer2 thread count on DataNodes | Major | datanode | Erik Krogen | Erik Krogen | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Incorrect ReservationId.compareTo() implementation | Minor | reservation system | Oleg Danilov | Oleg Danilov | | | [ATSv2] Registering timeline client before AMRMClient service init throw exception. | Major | timelineclient | Rohith Sharma K S | Rohith Sharma K S | | | Kill application button is visible even if the application is FINISHED in RM UI | Major | . | Sumana Sathish | Suma Shivaprasad | | | CollectorInfo should have Public visibility | Minor | . | Varun Saxena | Varun Saxena | | | ATSv2 documentation changes post merge | Major | timelineserver | Varun Saxena | Varun Saxena | | | dfsadmin command prints "Exception encountered" even if there is no exception, when debug is enabled | Minor | hdfs-client | Nanda kumar | Nanda kumar | | | Unable to override the $HADOOP\CONF\DIR env variable for container | Major | nodemanager | Terence Yim | Jason Lowe | | | RMContext need not to be injected to webapp and other Always Running services. | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Datatable sorting on the Datanode Information page in the Namenode UI is broken | Major | . | Shawna Martell | Shawna Martell | | | NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout | Major | namenode, qjm | Erik Krogen | Erik Krogen | | | Cross-queue preemption sometimes starves an underserved queue | Major | capacity scheduler | Eric Payne | Eric Payne | | | ResourceCalculator.fitsIn() should not take a cluster resource parameter | Major | scheduler | Daniel Templeton | Sen Zhao | | | Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests | Blocker | . | Botong Huang | Botong Huang | | | Shuffle Handler prints disk error stack traces for every read failure. | Major | . | Kuhu Shukla | Kuhu Shukla | | | TestNameNodeMetrics#testTransactionAndCheckpointMetrics Fails in trunk | Blocker | test | Brahma Reddy Battula | Hanisha Koneru | | | Introduce a config to allow setting up write pipeline with fewer nodes than replication factor | Major | . | Yongjun Zhang | Brahma Reddy Battula | | | Fix finicky TestContainerManager tests | Major | . | Arun Suresh | Arun Suresh | | | Use classloader inside configuration class to make new classes | Major | . | Jongyoul Lee | Jongyoul Lee | | | FSDirectory should use Time.monotonicNow for durations | Minor | . | Chetna Chaudhari | Bharat Viswanadham | | | "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX | Major | metrics | Sai Nukavarapu | Hanisha Koneru | | | DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor | Blocker | nodemanager | Miklos Szegedi | Miklos Szegedi | | | StripedBlockUtil.java:694: warning - Tag @link: reference not found: StripingCell | Minor | documentation | Tsz Wo Nicholas Sze | Mukul Kumar Singh | | | DistSum should use Time.monotonicNow for measuring durations | Minor |

| Chetna Chaudhari | Chetna Chaudhari | | | TestCapacityScheduler.testDefaultNodeLabelExpressionQueueConfig() has the args to assertEqual() in the wrong order | Trivial | capacity scheduler, test | Daniel Templeton | Sen Zhao | | | Reuse object mapper in HDFS | Minor | . | Mingliang Liu | Hanisha Koneru | | | Change the Scope of the Class DFSUtilClient to Private | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Add documentation for getconf command with -journalnodes option | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Fix javadocs issues in Hadoop HDFS-NFS | Major | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | Fix javadocs issues in Hadoop HDFS | Minor | . | Mukul Kumar Singh | Mukul Kumar Singh | | | NFS Gateway on Shutdown Gives Unregistration Failure. Does Not Unregister with rpcbind Portmapper | Major | nfs | Sailesh Patel | Mukul Kumar Singh | | | Fail to start/stop journalnodes using start-dfs.sh/stop-dfs.sh. | Major | federation, journal-node, scripts | Wenxin He | Bharat Viswanadham | | | Remove duplicated code in AMRMClientAsyncImpl.java | Minor | client | Sen Zhao | Sen Zhao | | | Loosen compatibility guidelines for native dependencies | Blocker | documentation, native | Chris Douglas | Daniel Templeton | | | Get source for config tags from file name | Major | . | Ajay Kumar | Ajay Kumar | | | AHS REST API can return NullPointerException | Major | . | Prabhu Joseph | Billie Rinaldi | | | TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently | Major | . | Eric Badger | Eric Badger | | | hadoop-project/pom.xml is executable | Minor | . | Akira Ajisaka | Ajay Kumar | | | Add admin configuration to filter per-user's apps in secure cluster | Major | webapp | Sunil Govindan | Sunil Govindan | | | AggregatedLogsBlock reports a bad 'end' value as a bad 'start' value | Major | log-aggregation | Jason Lowe | Jason Lowe | | | TestSchedulingMonitor#testRMStarts fails sporadically | Major | . | Jason Lowe | Jason Lowe | | | Incorrect statement in Downgrade section of HDFS Rolling Upgrade document | Minor | documentation | Nanda kumar | Nanda kumar | | | JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured | Major | journal-node | Bharat Viswanadham | Bharat Viswanadham | | | ViewFS: StoragePolicies commands fail with HDFS federation | Major | hdfs | Mukul Kumar Singh | Mukul Kumar Singh | | | Update Yarn to YARN in documentation | Minor | documentation | Miklos Szegedi | Chetna Chaudhari | | | AMSimulator in SLS does't work due to refactor of responseId | Blocker | scheduler-load-simulator | Yufei Gu | Botong Huang | | | SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined | Minor | . | Nandor Kollar | Nandor Kollar | | | Fix typo in helper message of ContainerLauncher | Trivial | . | Elek, Marton | Elek, Marton | | | Add Node and Rack Hints to Opportunistic Scheduler | Major |

| Arun Suresh | kartheek muthyala | | | ContainerExecutor always launches with priorities due to yarn-default property | Minor | nodemanager | Jason Lowe | Jason Lowe | | | libhdfs SIGSEGV in setTLSExceptionStrings | Major | libhdfs | John Zhuge | John Zhuge | | | Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | Supporting HDFS NFS gateway with Federated HDFS | Major | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | Upgrade netty-all jar to latest 4.0.x.Final | Critical | . | Vinayakumar B | Vinayakumar B | | | Improve exception message when mapreduce.jobhistory.webapp.address is in wrong format | Major | applicationmaster | Prabhu Joseph | Prabhu Joseph | | | Fix typo in DFSAdmin command output | Trivial | . | Ajay Kumar | Ajay Kumar | | | Update GroupsMapping documentation to reflect the new changes | Major | documentation | Anu Engineer | Esther Kundin | | | Fix unsafe casting from long to int for class Resource and its sub-classes | Major | resourcemanager | Yufei Gu | Yufei Gu | | | LogAggregationTFileController deletes/renames while file is open | Critical | nodemanager | Daryn Sharp | Jason Lowe | | | TestRouterWebServiceUtil#testMergeMetrics is flakey | Major | federation | Robert Kanter | Robert Kanter | | | CLONE - Fix source-level compatibility after HADOOP-11252 | Blocker | . | Junping Du | Junping Du | | | TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair scheduler | Major | . | Miklos Szegedi | Miklos Szegedi | | | TestDistributedShell should be scheduler agnostic | Major | . | Haibo Chen | Haibo Chen | | | DFSZKFailOverController re-order logic for logging Exception | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Handle JDK-8071638 for hadoop-common | Blocker | . | Bibin A Chundatt | Bibin A Chundatt | | | Add a link to HDFS router federation document in site.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE | Major | test | Robert Kanter | Yufei Gu | | | Xenial dockerfile needs ant in main build for findbugs | Trivial | build, test | Allen Wittenauer | Akira Ajisaka | | | JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to construct servlet Url | Major | . | Hanisha Koneru | Hanisha Koneru | | | Possible NPE in RMWebapp when HA is enabled and the active RM fails | Major | . | Chandni Singh | Chandni Singh | | | TestFSAppStarvation.testPreemptionEnable fails intermittently | Major | . | Sunil Govindan | Miklos Szegedi | | | Unsafe cast from long to int Resource.hashCode() method | Critical | resourcemanager | Daniel Templeton | Miklos Szegedi | | | Clean up jdiff xml files added for 2.8.2 release | Blocker | . | Subru Krishnan | Junping Du | | | [JDK9] Upgrade maven-javadoc-plugin to 3.0.0-M1 | Minor | build | ligongyi | ligongyi | | | Hadoop 3 missing fix for HDFS-5169 | Major | native | Joe McDonnell | Joe McDonnell | | | Many RM unit tests failing with FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | NPE when accessing container logs due to null dirsHandler | Major | . | Jonathan Hung | Jonathan Hung | | | Preemption properties should be refreshable | Major | capacity scheduler, scheduler preemption | Eric Payne | Gergely Novk | | | incorrect log preview displayed in jobhistory server ui | Major | yarn | Santhosh B Gowda | Xuan Gong | | | Fix ResourceEstimator findbugs issues | Blocker |

| Allen Wittenauer | Arun Suresh | | | Fix DominantResourceFairnessPolicy serializable findbugs issues | Blocker | . | Allen Wittenauer | Daniel Templeton | | | Router getApps REST invocation fails with multiple RMs | Critical | . | Subru Krishnan | igo Goiri | | | TestConfigurationFieldsBase to use SLF4J for logging | Trivial | conf, test | Steve Loughran | Steve Loughran | | | Add containerId to Localizer failed logs | Minor | nodemanager | Prabhu Joseph | Prabhu Joseph | | | [Umbrella] Simplified API layer for services and beyond | Major | . | Vinod Kumar Vavilapalli | | | | Update JAVA\_HOME in create-release for Xenial Dockerfile | Blocker | build | Andrew Wang | Andrew Wang | | | Reset the upload button when file upload fails | Critical | ui, webhdfs | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix issue where RM fails to switch to active after first successful start | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | TestContainerManagerSecurity is still flakey | Major | test | Robert Kanter | Robert Kanter | | | RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup | Minor | resourcemanager | Jason Lowe | Jason Lowe | | | start-yarn.sh fails to start ResourceManager unless running as root | Blocker | . | Sean Mackrory | | | | NameNode Fsck http Connection can timeout for directories with multiple levels | Major | tools | Mukul Kumar Singh | Mukul Kumar Singh | | | Add Test for NFS mount of not supported filesystems like (file:///) | Minor | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | Cleanup usage of decodecomponent and use QueryStringDecoder from netty | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Journal Syncer is not started in Federated + HA cluster | Major | federation, journal-node | Bharat Viswanadham | Bharat Viswanadham | | | Decommissioning node default value to be zero in new YARN UI | Trivial | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Render Applications and Services page with filters in new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Fix javadoc issues in Hadoop Common | Minor | common | Mukul Kumar Singh | Mukul Kumar Singh | | | WebHdfsFileSystem exceptions should retain the caused by exception | Major | hdfs | Daryn Sharp | Hanisha Koneru | | | Render outstanding resource requests on application page of new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Introduce filters in Nodes page of new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Improve the docker container runtime documentation | Major | . | Shane Kumpf | Shane Kumpf | | | Set up SASS for new YARN UI styling | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env | Major | common | Peter Bacsko | Peter Bacsko | | | ConcurrentModificationException

RMAppImpl#getRMAppMetrics | Major | capacityscheduler | Tao Yang | Tao Yang | | | Incorrect query parameters in cluster nodes REST API document | Minor | documentation | Tao Yang | Tao Yang | | | RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1 | Major | ha | Mukul Kumar Singh | Mukul Kumar Singh | | | Document Apache Hadoop does not support Java 9 in BUILDING.txt | Major | documentation | Akira Ajisaka | Hanisha Koneru | | | Remove scheduler lock in FSAppAttempt.getWeight() | Minor | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. | Major | fairscheduler, reservation system | Yufei Gu | Yufei Gu | | | Method canContainerBePreempted can return true when it shouldn't | Major | fairscheduler | Steven Rand | Steven Rand | | | Fix java doc errors in jdk1.8 | Major | . | Rohith Sharma K S | Steve Loughran | | | ContainerLocalizer doesn't have a valid log4j config when using LinuxContainerExecutor | Major | nodemanager | Yufei Gu | Yufei Gu | | | Lease renewal can hit a deadlock | Major | . | Kuhu Shukla | Kuhu Shukla | | | Layout changes to Application details page in new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | StoragePolicyAdmin should support schema based path | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit | Critical | namenode | DENG FEI | Konstantin Shvachko | | | upgrade hadoop dependency on commons-codec to 1.11 | Major | . | PJ Fanning | Bharat Viswanadham | | | Make FsServerDefaults cache configurable. | Minor | . | Rushabh S Shah | Mikhail Erofeev | | | Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0 | Minor | fs/azure | Rajesh Balamohan | Rajesh Balamohan | | | AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby | Critical | . | Tao Yang | Tao Yang | | | Make HdfsLocatedFileStatus a subtype of LocatedFileStatus | Major | . | Chris Douglas | Chris Douglas | | | "yarn logs" command fails to get logs for running containers if UI authentication is enabled. | Critical | . | Namit Maheshwari | Xuan Gong | | | Delete copy-on-truncate block along with the original block, when deleting a file being truncated | Blocker | hdfs | Jiandan Yang | Konstantin Shvachko | | | Layout changes in Queue UI to show queue details on right pane | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | startTxId could be greater than endTxId when tailing in-progress edit log | Major | hdfs | Chao Sun | Chao Sun | | | TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently | Major | resourcemanager | Daniel Templeton | Gergo Repas | | | AM lacks flow control for task events | Major | mr-am | Jason Lowe | Peter Bacsko | | | TestPBImplRecords fails with NullPointerException | Major | . | Jason Lowe | Daniel Templeton | | | quote\and\append\_arg can overflow buffer | Major | nodemanager | Jason Lowe | Jim Brennan | | | LocatedFileStatus constructor forces RawLocalFS to exec a process to get the permissions | Major | fs | Steve Loughran | Ping Liu | |

TestNMWebServices#testGetNMResourceInfo fails on trunk | Major | nodemanager, webapp | Gergely Novk | Gergely Novk | | | Handle old RMDelegationToken format when recovering RM | Major | resourcemanager | Tatyana But | Robert Kanter | | | create-release site build outputs dummy shaded jars due to skipShade | Blocker | . | Andrew Wang | Andrew Wang | | | Remove subversion related code from VersionInfoMojo.java | Minor | build | Akira Ajisaka | Ajay Kumar | | | Application Placement should be done before ACL checks in ResourceManager | Blocker | . | Suma Shivaprasad | Suma Shivaprasad | | | DFSZKFailoverController daemon exits with wrong status code | Major | auto-failover | Doris Gu | Bharat Viswanadham | | | Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x "json-lib" | Blocker | fs/oss | Chris Douglas | Sammi Chen | | | TestClusterTopology#testChooseRandom fails intermittently | Major | test | Zsolt Venczel | Zsolt Venczel | | | Incorrect sTarget column causing DataTable warning on RM application and scheduler web page | Major | resourcemanager, webapp | Weiwei Yang | Gergely Novk | | | Do not invalidate blocks if toInvalidate is empty | Major | . | Zsolt Venczel | Zsolt Venczel | | | TestRMWebServicesSchedulerActivities fails in trunk | Major | test | Sunil Govindan | Sunil Govindan | | | Distcp : Update the usage of delete option for dependency with update and overwrite option | Minor | distcp, hdfs | Harshakiran Reddy | usharani | | | NM print inappropriate error log when node-labels is enabled | Minor | . | Yang Wang | Yang Wang | | | em-table improvement for better filtering in new YARN UI | Minor | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Allow read-only access to reserved raw for non-superusers | Major | namenode | Daryn Sharp | Rushabh S Shah | | | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Lowe | Ajay Kumar | | | Container launching code suppresses close exceptions after writes | Major | nodemanager | Jason Lowe | Jim Brennan | | | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Lowe | Jim Brennan | | | TestContainerLaunch# fails after YARN-7381 | Major | . | Jason Lowe | Jason Lowe | | | Several javadoc errors | Blocker | . | Sean Mackrory | Sean Mackrory | | | KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5\_CONFIG | Minor | security | Vipin Rathor | Vipin Rathor | | | TestDFSIO -read -random doesn't work on file sized 4GB | Minor | fs, test | zhoutai.zt | Ajay Kumar | | | NodeManager metrics return wrong value after update node resource | Major | . | Yang Wang | Yang Wang | | | Remove the extra space in HdfsImageViewer.md | Trivial | documentation | Yiqun Lin | Rahul Pathak | | | [Atsv2] Define new set of configurations for reader and collectors to bind. | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | ResourceRequest has a different default for allocationRequestId than Container | Major | . | Chandni Singh | Chandni Singh | | | Update Timeline Reader web app address in UI2 | Major | . | Rohith Sharma K S | Sunil Govindan | | | Unit Fail: TestAMRestart#testPreemptedAMRestartOnRMRestart | Major | . | Miklos Szegedi | Miklos Szegedi | | | Fix findbugs warning in ImageWriter.java | Major |

| Akira Ajisaka | Akira Ajisaka | | | TestErasureCodigCLI testAll failing consistently. | Major | erasure-coding, hdfs | Rushabh S Shah | Ajay Kumar | | | Incorrect javadoc in SaslDataTransferServer.java#receive | Major | encryption | Mukul Kumar Singh | Mukul Kumar Singh | | | Fix TestOpenFilesWithSnapshot redundant configurations | Minor | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED | Major | . | Arun Suresh | Sampada Dehankar | | | Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy | Major | namenode | Wei-Chiu Chuang | Chris Douglas | | | Support multiple resource types in YARN native services | Critical | yarn-native-services | Wangda Tan | Wangda Tan | | | Lock down version of doxia-module-markdown plugin | Blocker | . | Elek, Marton | Elek, Marton | | | NPE due to Invalid KerberosTicket in UGI | Major | . | Jitendra Nath Pandey | Mukul Kumar Singh | | | Typo in javadoc of ReconfigurableBase#reconfigurePropertyImpl | Trivial | common | Nanda kumar | Nanda kumar | | | Error in javadoc of ReconfigurableBase#reconfigureProperty | Minor | . | Ajay Kumar | Ajay Kumar | | | NodeManager should go unhealthy when state store throws DBException | Major | nodemanager | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | RM Apps API returns only active apps when query parameter queue used | Minor | resourcemanager, restapi | Grant Sohn | Gergely Novk | | | Skip validating priority acls while recovering applications | Blocker | resourcemanager | Charan Hebri | Sunil Govindan | | | Concurrent task progress updates causing NPE in Application Master | Blocker | mr-am | Gergo Repas | Gergo Repas | | | NM should reference the singleton JvmMetrics instance | Major | nodemanager | Haibo Chen | Haibo Chen | | | Deprecation of yarn.resourcemanager.zk-address is undocumented | Major | documentation | Namit Maheshwari | Ajay Kumar | | | Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum | Major | encryption | Mukul Kumar Singh | Mukul Kumar Singh | | | DiskBalancer report command top option should only take positive numeric values | Minor | diskbalancer | Namit Maheshwari | Shashikant Banerjee | | | TestDNFencingWithReplication.testFencingStress fix mini cluster not yet active issue | Major | . | Zsolt Venczel | Zsolt Venczel | | | Document - Disabling the Lazy persist file scrubber. | Trivial | documentation, hdfs | Karthik Palanisamy | Karthik Palanisamy | | | StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB | Major | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers is flakey with FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | queueUsagePercentage is coming as INF for getApp REST api call | Major | webapp | Sunil Govindan | Sunil Govindan | | | NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | Ignore expired containers from removed nodes in FairScheduler | Critical | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | DistributedShell failed to specify resource other than memory/vcores from container\_resources | Critical |

| Wangda Tan | Wangda Tan | | | NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode | Major | capacityscheduler | Tao Yang | Tao Yang | | | RMAppImpl:Invalid event: START at KILLED | Major | resourcemanager | lujie | lujie | | | Invalid event: ATTEMPT\ADDED at FINAL\SAVING | Major | yarn | lujie | lujie | | | TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout | Major | erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Allow FS scheduler state dump to be turned on/off separately from FS debug log | Major | . | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | TestRMContainerAllocator fails after YARN-6124 | Major | scheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fix S3ACommitter documentation | Minor | documentation, fs/s3 | Alessandro Andrioni | Alessandro Andrioni | | | TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky | Major | . | Miklos Szegedi | Miklos Szegedi | | | Fix typo in YARN documentation | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector | Minor | yarn | Prabhu Joseph | Prabhu Joseph | | | BlockPoolSlice can leak in a mini dfs cluster | Major | . | Robert Joseph Evans | Ajay Kumar | | | Sync rbw dir on the first hsync() to avoid file lost on power failure | Critical | . | Kanaka Kumar Avvaru | Vinayakumar B | | | RegistryDNS should handle upstream DNS returning CNAME | Major | . | Billie Rinaldi | Eric Yang | | | Improve Diagonstic message for stop yarn native service | Major | . | Yesha Vora | Chandni Singh | | | Create the container log directory with correct sticky bit in C code | Major | nodemanager | Yufei Gu | Yufei Gu | | | globStatus javadoc refers to glob pattern as "regular expression" | Trivial | documentation, hdfs | Ryanne Dolan | Mukul Kumar Singh | | | Fix the javadoc warning in WriteOperationHelper.java | Minor | documentation, fs/s3 | Mukul Kumar Singh | Mukul Kumar Singh | | | TestContainerManagerSecurity.testContainerManager[Simple] flaky in trunk | Major | test | Botong Huang | Akira Ajisaka | | | TestLeaseRecoveryStriped#testLeaseRecovery is failing when safeLength is 0MB or larger than the test file | Major | hdfs | Zsolt Venczel | Zsolt Venczel | | | Make Datanode Netty reverse proxy port to be configurable | Major | datanode | Vinayakumar B | Vinayakumar B | | | Add an additional check to the validity of container and application ids passed to container-executor | Major | nodemanager | Miklos Szegedi | Yufei Gu | | | Add configuration consistency for module.enabled and docker.privileged-containers.enabled | Major | . | Yesha Vora | Eric Badger | | | in FsShell, UGI params should be overidden through env vars(-D arg) | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | [UI2] Render time related fields in all pages to the browser timezone | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Fix logging for destroy yarn service cli when app does not exist and some minor bugs | Major | yarn-native-services | Yesha Vora | Jian He | | | FairScheduler: finished applications are always restored to default queue | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | [UI2] Meta information about Application logs has to be pulled from ATS

instead of ATS2 | Major | yarn-ui-v2 | Sunil Govindan | Sunil Govindan | | | Credentials readTokenStorageFile to stop wrapping IOEs in IOEs | Minor | security | Steve Loughran | Ajay Kumar | | | StripedBlockReader#createBlockReader leaks socket on IOException | Critical | datanode, erasure-coding | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Typo in SecureMode.md | Trivial | documentation | Masahiro Tanaka | Masahiro Tanaka | | | CapacityScheduler: Support refresh maximum allocation for multiple resource types | Blocker | . | Sumana Sathish | Wangda Tan | | | Introduce a new config property for YARN Service dependency tarball location | Major | applications, client, yarn-native-services | Gour Saha | Gour Saha | | | Error log level in ShortCircuitRegistry#removeShm | Minor | . | hu xiaodong | hu xiaodong | | | Container-executor fails with segfault on certain OS configurations | Major | nodemanager | Gergo Repas | Gergo Repas | | | [UI2] GPU information tab in left hand side disappears when we click other tabs below | Major | . | Sumana Sathish | Vasudevan Skm | | | Distributed Shell should use timeline async api's | Major | distributed-shell | Sumana Sathish | Rohith Sharma K S | | | Journal Sync does not work on a secure cluster | Major | journal-node | Namit Maheshwari | Bharat Viswanadham | | | Encounter NullPointerException when using DecayRpcScheduler | Major | . | Tao Jie | Tao Jie | | | Possible race condition in JHS if the job is not loaded | Major | jobhistoryserver | Peter Bacsko | Peter Bacsko | | | prelaunch.err file not found exception on container failure | Major | . | Jonathan Hung | Keqiu Hu | | | Fix user name format in YARN Registry DNS name | Major | . | Jian He | Jian He | | | [Documentation] Documenting the ability to disable elasticity at leaf queue | Major | capacity scheduler | Zian Chen | Zian Chen | | | Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md | Trivial | documentation | Jianfei Jiang | Jianfei Jiang | | | NM heartbeat stuck when responseId overflows MAX\_INT | Critical | . | Botong Huang | Botong Huang | | | MR should not try to clean up at first job attempt | Major | . | Takanobu Asanuma | Gergo Repas | | | [UI2] Duplicated containers are rendered per attempt | Major | . | Rohith Sharma K S | Vasudevan Skm | | | [UI2] Clicking 'Master Node' or link next to 'AM Node Web UI' under application's appAttempt page goes to OLD RM UI | Major | . | Sumana Sathish | Vasudevan Skm | | | Task timeout in uber mode can crash AM | Major | mr-am | Akira Ajisaka | Peter Bacsko | | | [Atsv2] GSSException: No valid credentials provided - Failed to find any Kerberos tgt thrown by Timelinev2Client & HBaseClient in NM | Blocker | . | Sumana Sathish | Rohith Sharma K S | | | TestErasureCodingMultipleRacks#testSkewedRack3 is failing | Major | hdfs | Gabor Bota | Gabor Bota | | | Exception message is not printed when creating an encryption zone fails with AuthorizationException | Minor | encryption | fang zhenyi | fang zhenyi | | | A misleading variable's name in ApplicationAttemptEventDispatcher | Minor | resourcemanager | Jinjiang Ling | Jinjiang Ling | | | Improve Capacity Scheduler Async Scheduling to better handle node failures | Critical |

| Sumana Sathish | Wangda Tan | | | Add an option to not disable short-circuit reads on failures | Major | hdfs-client, performance | Andre Araujo | Xiao Chen | | | [UI2] Logs page shows duplicated containers with ATS | Major | yarn-ui-v2 | Sunil Govindan | Sunil Govindan | | | Clicking on yarn service should take to component tab | Major | yarn-ui-v2 | Yesha Vora | Sunil Govindan | | | SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel | Major | . | Xiaoyu Yao | Ajay Kumar | | | Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver | Major | datanode, security | Xiaoyu Yao | Ajay Kumar | | | getErasureCodingPolicy should handle .snapshot dir better | Major | erasure-coding, hdfs, snapshots | Harshakiran Reddy | LiXin Ge | | | Map outputs implicitly rely on permissive umask for shuffle | Critical | mrv2 | Jason Lowe | Jason Lowe | | | LowRedundancyReplicatedBlocks metric can be negative | Major | metrics | Akira Ajisaka | Akira Ajisaka | | | Correct the spelling in CopyFilter.java | Major | tools/distcp | Mukul Kumar Singh | Mukul Kumar Singh | | | YARN Service CLI should use hadoop.http.authentication.type to determine authentication method | Major | . | Eric Yang | Eric Yang | | | NM user is unable to access the application filecache due to permissions | Critical | . | Shane Kumpf | Jason Lowe | | | Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. | Critical | hdfs, webhdfs | Yongjun Zhang | Yongjun Zhang | | | Localized jars that are expanded after localization are not fully copied | Blocker | . | Miklos Szegedi | Miklos Szegedi | | | TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error | Major | test | Jason Lowe | Botong Huang | | | AmFilterInitializer should addFilter after fill all parameters | Critical | . | Sumana Sathish | Wangda Tan | | | Missing kerberos token when check for RM REST API availability | Major | yarn-native-services | Eric Yang | Eric Yang | | | [UI2] Log Aggregation status to be displayed in Application Page | Major | yarn-ui-v2 | Yesha Vora | Gergely Novk | | | [UI2] Error to be displayed correctly while accessing kerberized cluster without kinit | Major | yarn-ui-v2 | Sumana Sathish | Sunil Govindan | | | NPE during container relaunch | Major | . | Billie Rinaldi | Jason Lowe | | | NPE from Unresolved Host causes permanent DFSInputStream failures | Major | hdfs-client | James Moore | Lokesh Jain | | | In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted | Major | . | Yongjun Zhang | Yongjun Zhang | | | Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up | Major | tools | Jianfei Jiang | Jianfei Jiang | | | Stop and Delete Yarn Service from RM UI fails with HTTP ERROR 404 | Critical | yarn-ui-v2 | Yesha Vora | Sunil Govindan | | | Snapshot diff could be corrupted after concat | Major | namenode, snapshots | Xiaoyu Yao | Xiaoyu Yao | | | YARN service REST API returns charset=null when kerberos enabled | Major | yarn-native-services | Eric Yang | Eric Yang | | | Log object instance get incorrectly in SlowDiskTracker | Minor |

| Jianfei Jiang | Jianfei Jiang | | | Fix mvn site fails with error: Multiple sources of package comments found for package "o.a.h.y.client.api.impl" | Blocker | build, documentation | Akira Ajisaka | Akira Ajisaka | | | Remove unnecessary public/crossdomain.xml from YARN UIv2 sub project | Blocker | yarn-ui-v2 | Allen Wittenauer | Sunil Govindan | | | NM goes down with OOM due to leak in log-aggregation | Blocker | . | Santhosh B Gowda | Xuan Gong | | | DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation | Blocker | . | Wangda Tan | Wangda Tan | | | ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster. | Major | namenode | He Xiaoqiao | He Xiaoqiao | | | refreshNamenodes does not support adding a new standby to a running DN | Critical | datanode, ha | Jian Fang | Ajith S | | | TestFixedLengthInputFormat#testFormatCompressedIn is flaky | Major | client, test | Peter Bacsko | Peter Bacsko | | | Token expiration edits may cause log corruption or deadlock | Critical | namenode | Daryn Sharp | Daryn Sharp | | | Fix the javadoc error in ReplicaInfo | Minor | . | Bharat Viswanadham | Bharat Viswanadham | | | Timed out tasks can fail to produce thread dump | Major | . | Jason Lowe | Jason Lowe | | | Fix dfs.namenode.shared.edits.dir in TestJournalNode | Major | journal-node, test | Bharat Viswanadham | Bharat Viswanadham | | | BZip2 drops and duplicates records when input split size is small | Major | . | Aki Tanaka | Aki Tanaka | | | Fix http method name in Cluster Application Timeout Update API example request | Minor | docs, documentation | Charan Hebri | Charan Hebri | | | Replace Collections.EMPTY\ with empty\ when available | Minor | . | Akira Ajisaka | fang zhenyi | | | TestTruncateQuotaUpdate fails in trunk | Major | test | Arpit Agarwal | Nanda kumar | | | Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | Use Log.\*(Object, Throwable) overload to log exceptions | Major | . | Arpit Agarwal | Andras Bokor | | | apparent bug in concatenated-bzip2 support (decoding) | Major | io | Greg Roelofs | Zsolt Venczel | | | Yarn ServiceClient does not not delete znode from secure ZooKeeper | Blocker | yarn-native-services | Eric Yang | Billie Rinaldi | | | Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider | Trivial | documentation | Akira Ajisaka | Gabor Bota | | | [UI2] Support loading pre-2.8 version /scheduler REST response for queue page | Major | yarn-ui-v2 | Gergely Novk | Gergely Novk | | | [UI2] ArtifactsId should not be a compulsory field for new service | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | ContainerExecutor does not order environment map | Trivial | nodemanager | Remi Catherinot | Remi Catherinot | | | HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails | Critical | mrv2 | Gergely Novk | Gergely Novk | | | RequestHedgingProxyProvider should handle case when none of the proxies are available | Major | ha | Mukul Kumar Singh | Mukul Kumar Singh | | | Correct the wrong word spelling 'intialize' | Minor | . | fang zhenyi | fang zhenyi | | | After Datanode down, In Namenode UI Datanode tab is throwing warning

| Major | datanode | Harshakiran Reddy | Brahma Reddy Battula | | | Failed block recovery leaves files open indefinitely and at risk for data loss | Major | . | Daryn Sharp | Kihwal Lee | | | Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively | Major | . | Nishant Bangarwa | Nishant Bangarwa | | | TestServiceAM and TestServiceMonitor test cases are hanging | Major | yarn-native-services | Eric Yang | Chandni Singh | | | SBN crash when transition to ANN with in-progress edit tailing enabled | Major | ha, namenode | Chao Sun | Chao Sun | | | DiskBalancer: Add an configuration for valid plan hours | Major | diskbalancer | Bharat Viswanadham | Bharat Viswanadham | | | SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | CachePool permissions incorrectly checked | Major | . | Yiqun Lin | Jianfei Jiang | | | CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path | Major | encryption, hdfs | Hanisha Koneru | Hanisha Koneru | | | Datanode#checkSecureConfig should allow SASL and privileged HTTP | Major | datanode, security | Xiaoyu Yao | Ajay Kumar | | | Service name is validated twice in ServiceClient when a service is created | Trivial | yarn-native-services | Chandni Singh | Chandni Singh | | | Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster | Critical | job submission | Jiandan Yang | Jiandan Yang | | | [Atsv2] Race condition in NM while publishing events if second attempt is launched on the same node | Critical | . | Rohith Sharma K S | Rohith Sharma K S | | | Incorrect javadoc for return type of RetryPolicy#shouldRetry | Minor | documentation | Nanda kumar | Nanda kumar | | | ServiceMaster should only wait for recovery of containers with id that match the current application id | Critical | yarn | Chandni Singh | Chandni Singh | | | Fix a bug in DirectoryDiffList.getMinListForRange | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Fix the typo in MiniDFSCluster class | Trivial | test | Yiqun Lin | fang zhenyi | | | NPE in ContainerLocalizer when localization failed for running container | Major | nodemanager | Tao Yang | Tao Yang | | | Upgrade commons-io from 2.4 to 2.5 | Major | minikdc | PandaMonkey | PandaMonkey | | | TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun | Minor | test | Gergely Novk | Gergely Novk | | | Disk Balancer: Add skipDateCheck option to DiskBalancer Execute command | Major | diskbalancer | Bharat Viswanadham | Bharat Viswanadham | | | Remove unused imports from TestKMSWithZK.java | Minor | test | Akira Ajisaka | Ajay Kumar | | | Remove unnecessary boxings and unboxings from PlacementConstraintParser.java | Minor |

| Akira Ajisaka | Sen Zhao | | | Kerberized inotify client fails despite kinit properly | Major | namenode | Wei-Chiu Chuang | Xiao Chen | | | TestSwiftFileSystemBlockLocation doesn't compile | Critical | build, fs/swift | Steve Loughran | Steve Loughran | | | Fix itemization in YARN federation document | Minor | documentation | Akira Ajisaka | Sen Zhao | | | File not closed if streamer fail with DSQuotaExceededException | Major | hdfs-client | Xiao Chen | Xiao Chen | | | FileStatus.readFields() assertion incorrect | Critical | . | Steve Loughran | Steve Loughran | | | Disk Balancer: Support multiple block pools during block move | Major | diskbalancer | Bharat Viswanadham | Bharat Viswanadham | | | Support fully qualified hdfs path in EZ commands | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Fix a wrong link for RBF in the top page | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma | | | TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails sometimes in trunk | Minor | . | Tao Yang | Tao Yang | | | Distcp's use of pread is slowing it down. | Minor | tools/distcp | Virajith Jalaparti | Virajith Jalaparti | | | distcp can't handle remote stores with different checksum algorithms | Critical | tools/distcp | Steve Loughran | Steve Loughran | | | TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple fail in trunk | Major | . | Ray Chiang | Bharat Viswanadham | | | [UI2] Remove master node link from headers of application pages | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml | Major | mrv2 | Daniel Templeton | Sen Zhao | | | Yarn Service: component instance name shows up as component name in container record | Major | . | Chandni Singh | Chandni Singh | | | Document WebHDFS support for snapshot diff | Major | documentation, webhdfs | Xiaoyu Yao | Lokesh Jain | | | Add stack, conf, metrics links to utilities dropdown in NN webUI | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition | Major | . | Eric Badger | Eric Badger | | | LOG in class MaxRunningAppsEnforcer is initialized with a faulty class FairScheduler | Major | fairscheduler | Yufei Gu | Sen Zhao | | | TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available | Major | . | Yiqun Lin | Yiqun Lin | | | TestFSImage fails without -Pnative | Major | test | Akira Ajisaka | Akira Ajisaka | | | WebHDFS: Add constructor in SnapshottableDirectoryStatus with HdfsFileStatus as argument | Major | webhdfs | Lokesh Jain | Lokesh Jain | | | Fix non-empty dir warning message when setting default EC policy | Minor | . | Hanisha Koneru | Bharat Viswanadham | | | ResourceManager UI cluster/app/\<app-id\> page fails to render | Blocker | webapp | Tarun Parimi | Tarun Parimi | | | Document webhdfs support for getting snapshottable directory list | Major | documentation, webhdfs | Lokesh Jain | Lokesh Jain | | | Flaky test TestTaskAttempt#testReducerCustomResourceTypes | Major | client, test | Peter Bacsko | Peter Bacsko | | | Fix incorrect null value check | Minor | hdfs | Jianfei Jiang | Jianfei Jiang | | | Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings | Minor | . | Akira Ajisaka | fang zhenyi | | | TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently. | Major | . | Rushabh S Shah | Ajay Kumar | | | Avoid using hard coded datanode data dirs in unit tests | Major | test | Xiaoyu Yao | Ajay Kumar | | | WebHDFS: Fix NPE in get snasphottable directory list call | Major | webhdfs | Lokesh Jain | Lokesh Jain | | | RM should be able to recover log aggregation status after restart/fail-over | Major |

| Xuan Gong | Xuan Gong | | | Throw meaningful message on null when initializing KMSWebApp | Major | kms | Xiao Chen | fang zhenyi | | | Re-reservation count may overflow when cluster resource exhausted for a long time | Major | capacityscheduler | Tao Yang | Tao Yang | | | Ignore minReplication for block recovery | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer | Minor | . | Miklos Szegedi | Miklos Szegedi | | | GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases | Major | . | Xiao Liang | Xiao Liang | | | TestWebHdfsFileContextMainOperations fails on Windows | Major | . | igo Goiri | Xiao Liang | | | Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread | Major | . | Jonathan Eagles | Jonathan Eagles | | | Revert YARN-6078 | Blocker | . | Billie Rinaldi | Billie Rinaldi | | | DataNode conf page cannot display the current value after reconfig | Minor | datanode | maobaolong | maobaolong | | | VersionInfo should load version-info.properties from its own classloader | Major | common | Thejas M Nair | Thejas M Nair | | | DistributedShellTimelinePlugin wrongly check for entityId instead of entityType | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | yarn rmadmin -getGroups returns group from which the user has been removed | Critical | . | Sumana Sathish | Sunil Govindan | | | Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ | Blocker | yarn | Sunil Govindan | Sunil Govindan | | | DShell does not fail when we ask more GPUs than available even though AM throws 'InvalidResourceRequestException' | Major | . | Sumana Sathish | Wangda Tan | | | Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake | Major | fs/adl, fs/azure | shanyu zhao | shanyu zhao | | | ResourceProfilesManager should be set in RMActiveServiceContext | Blocker | capacityscheduler | Tao Yang | Tao Yang | | | ManagedParentQueue with no leaf queues cause JS error in new UI | Blocker | . | Suma Shivaprasad | Suma Shivaprasad | | | NPE occurred when container allocation proposal is applied but its resource requests are removed before | Critical | . | Tao Yang | Tao Yang | | | ASF License warning in hadoop-mapreduce-client | Minor | test | Takanobu Asanuma | Takanobu Asanuma | | | Over-allocate node resource in async-scheduling mode of CapacityScheduler | Major | capacityscheduler | Tao Yang | Tao Yang | | | Document how to use classpath isolation for aux-services in YARN | Major | . | Xuan Gong | Xuan Gong | | | Avoid taking FSN lock while doing group member lookup for FSD permission check | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | | | Avoid fsync storm triggered by DiskChecker and handle disk full situation | Blocker | . | Kihwal Lee | Arpit Agarwal | | | Upgrading to 3.1 kills running containers with error "Opportunistic container queue is full" | Blocker |

| Rohith Sharma K S | Jason Lowe | | | Reduce unnecessary UGI synchronization | Critical | security | Daryn Sharp | Daryn Sharp | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Skip the testcase testJobWithChangePriority if FairScheduler is used | Major | client | Peter Bacsko | Peter Bacsko | | | Verify open files captured in the snapshots across config disable and enable | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | open(PathHandle) contract test should be exhaustive for default options | Major | . | Chris Douglas | Chris Douglas | | | Need to exercise all HDFS APIs for EC | Major | hdfs | Haibo Yan | Haibo Yan | | | Add Mover Cli Unit Tests for Federated cluster | Major | balancer & mover, test | Bharat Viswanadham | Bharat Viswanadham | | | TestDebugAdmin#testComputeMetaCommand fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add support for multiple resource types in the Resource class | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Extend DominantResourceCalculator to account for all resources | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support to read resource types from a config file | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support for binary units | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support for resource types in the nodemanager | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | Update DominantResourceCalculator to consider all resource types in calculations | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Update the Resources class to consider all resource types | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add manager class for resource profiles | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Implement APIs to get resource profiles from the RM | Major | client | Varun Vasudev | Varun Vasudev | | | Add support for resource profiles | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Changes to allow CapacityScheduler to use configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Create YarnConfigurationStore interface and InMemoryConfigurationStore class | Major | . | Jonathan Hung | Jonathan Hung | | | Add support for resource profiles in distributed shell | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Update resource usage and preempted resource calculations to take into account all resource types | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Implement MutableConfigurationManager for handling storage into configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Create REST API for changing YARN scheduler configurations | Major | . | Jonathan Hung | Jonathan Hung | | | [READ] Add tool generating FSImage from external store | Major | namenode, tools | Chris Douglas | Chris Douglas | | | [YARN-3926] Performance improvements in resource profile branch with respect to SLS | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | [READ] ProvidedReplica should return an InputStream that is bounded by its length | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Fix NullPointerException in ProvidedBlocksBuilder | Major |

| Virajith Jalaparti | Virajith Jalaparti | | | [READ] Tests for ProvidedStorageMap | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Add pluggable configuration ACL policy interface and implementation | Major | . | Jonathan Hung | Jonathan Hung | | | [READ] Test for increasing replication of provided files. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Test cases for ProvidedVolumeDF and ProviderBlockIteratorImpl | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Handle failures of Datanode with PROVIDED storage | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Support global configuration mutation in MutableConfProvider | Major | . | Jonathan Hung | Jonathan Hung | | | Create CLI for changing YARN configurations | Major | . | Jonathan Hung | Jonathan Hung | | | Fix build for YARN-3926 branch | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | ResourcePBImpl imports cleanup | Trivial | resourcemanager | Daniel Templeton | Yeliang Cang | | | Create LeveldbConfigurationStore class using Leveldb as backing store | Major | . | Jonathan Hung | Jonathan Hung | | | Disable queue refresh when configuration mutation is enabled | Major | . | Jonathan Hung | Jonathan Hung | | | [API] Introduce Placement Constraint object | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Improve performance of resource profile branch | Blocker | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | [READ] Check that the replicas served from a {{ProvidedVolumeImpl}} belong to the correct external storage | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Share remoteFS between ProvidedReplica instances. | Major | . | Ewan Higgs | Virajith Jalaparti | | | Support to add min/max resource configuration for a queue | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | ResourceProfilesManagerImpl.parseResource() has no need of the key parameter | Major | resourcemanager | Daniel Templeton | Manikandan R | | | [READ] HDFS-12091 breaks the tests for provided block reads | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Remove last uses of Long from resource types code | Minor | resourcemanager | Daniel Templeton | Daniel Templeton | | | Improve API implementation in Resources and DominantResourceCalculator class | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | ResourceProfilesManagerImpl is missing @Overrides on methods | Minor | resourcemanager | Daniel Templeton | Sunil Govindan | | | DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | Performance optimizations in Resource and ResourceUtils class | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Clean up unit tests after YARN-6610 | Major | test | Daniel Templeton | Daniel Templeton | | | Add Client API to get all supported resource types from RM | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter | Minor | resourcemanager | Daniel Templeton | Yu-Tang Lin | | | Cleanup ResourceProfileManager | Critical |

| Wangda Tan | Wangda Tan | | | Optimize ResourceType information display in UI | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Fix javac and javadoc errors in YARN-3926 branch | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | Fix issues on recovery in LevelDB store | Major | . | Jonathan Hung | Jonathan Hung | | | Improve log message in ResourceUtils | Trivial | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | Better styling for donut charts in new YARN UI | Major | . | Da Ding | Da Ding | | | Sort out hadoop-aws contract-test-options.xml | Minor | fs/s3, test | Steve Loughran | John Zhuge | | | ResourceUtils.DISALLOWED\_NAMES check is duplicated | Major | resourcemanager | Daniel Templeton | Manikandan R | | | Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem | Major | resourcemanager | Subru Krishnan | Subru Krishnan | | | Document Resource Profiles feature | Major | nodemanager, resourcemanager | Sunil Govindan | Sunil Govindan | | | Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. | Major | . | Xuan Gong | Xuan Gong | | | Improve Nodes Heatmap in new YARN UI with better color coding | Major | . | Da Ding | Da Ding | | | Introduce default and max lifetime of application at LeafQueue level | Major | capacity scheduler | Rohith Sharma K S | Rohith Sharma K S | | | SharingPolicy enhancements required to support recurring reservations in ReservationSystem | Major | resourcemanager | Subru Krishnan | Carlo Curino | | | Add a new log aggregation file format controller | Major | . | Xuan Gong | Xuan Gong | | | Additional Performance Improvement for Resource Profile Feature | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Move newly added APIs to unstable in YARN-3926 branch | Blocker | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat | Major | . | Xuan Gong | Xuan Gong | | | Update fair scheduler policies to be aware of resource types | Major | fairscheduler | Daniel Templeton | Daniel Templeton | | | Add retry logic in LogsCLI when fetch running application logs | Major | . | Xuan Gong | Xuan Gong | | | Implement zookeeper based store for scheduler configuration updates | Major | . | Wangda Tan | Jonathan Hung | | | Change hosts JSON file format | Major | . | Ming Ma | Ming Ma | | | Better documentation for maintenace mode and upgrade domain | Major | datanode, documentation | Wei-Chiu Chuang | Ming Ma | | | Add closing logic to configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Moving logging APIs over to slf4j in hadoop-mapreduce-examples | Major | . | Gergely Novk | Gergely Novk | | | ReflectionUtils should use Time.monotonicNow to mesaure duration | Minor | . | Bharat Viswanadham | Bharat Viswanadham | | | MetricsSystemImpl should use Time.monotonicNow for measuring durations | Minor | . | Chetna Chaudhari | Chetna Chaudhari | | | Documentation for API based scheduler configuration management | Major | . | Jonathan Hung | Jonathan Hung | | | WritableRpcEngine should use Time.monotonicNow | Minor | . | Chetna Chaudhari | Chetna Chaudhari | | | Add fsserver defaults call to

| Minor | webhdfs | Rushabh S Shah | Rushabh S Shah | | | Removing queue then failing over results in exception | Critical | . | Jonathan Hung | Jonathan Hung | | | Misc changes to YARN-5734 | Major | . | Jonathan Hung | Jonathan Hung | | | Add support for updateContainers when allocating using FederationInterceptor | Minor | . | Botong Huang | Botong Huang | | | Add size-based rolling policy to LogAggregationIndexedFileController | Major | . | Xuan Gong | Xuan Gong | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-app | Major | . | Jinjiang Ling | Jinjiang Ling | | | Moving logging APIs over to slf4j in hadoop-yarn-server-common | Major | . | Akira Ajisaka | Akira Ajisaka | | | [READ] Fix errors in image generation tool from latest rebase | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router | Major | . | Yeliang Cang | Yeliang Cang | | | Add work preserving restart support for Unmanaged AMs | Major | resourcemanager | Karthik Kambatla | Botong Huang | | | Placement Agent enhancements required to support recurring reservations in ReservationSystem | Blocker | resourcemanager | Subru Krishnan | Carlo Curino | | | Fix alignment issues and missing information in new YARN UI's Queue page | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | Azure wasb: getFileStatus not making any auth checks | Major | fs/azure, security | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Restrict Access to setPermission operation when authorization is enabled in WASB | Major | fs/azure | Kannapiran Srinivasan | Kannapiran Srinivasan | | | Cleanup usages of ResourceProfiles | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | convertToProtoFormat(Resource r) is not setting for all resource types | Major | . | lovekesh bansal | lovekesh bansal | | | Sticky bit implementation for rename() operation in Azure WASB | Major | fs, fs/azure | Varada Hemeswari | Varada Hemeswari | | | Add support in NodeManager to isolate GPU devices by using CGroups | Major | . | Wangda Tan | Wangda Tan | | | Log improvements for the ResourceUtils | Major | nodemanager, resourcemanager | Jian He | Sunil Govindan | | | Remove class ResourceType | Major | resourcemanager, scheduler | Yufei Gu | Sunil Govindan | | | Azure: POSIX permissions are taking effect in access() method even when authorization is enabled | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | UI and metrics changes related to absolute resource configuration | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Fix TestRMWebServicesReservation parametrization for fair scheduler | Blocker | fairscheduler, reservation system | Yufei Gu | Yufei Gu | | | [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | GPU Isolation: Incorrect minor device numbers written to devices.deny file | Major | . | Jonathan Hung | Jonathan Hung | | | Support same origin policy for cross site scripting prevention. | Major | yarn-ui-v2 | Vrushali C | Sunil Govindan | | | Make Collections.sort() more efficient by caching resource usage | Major | fairscheduler | Xianyin Xin | Yufei Gu | | |

should test all resources | Major | scheduler | Daniel Templeton | Sunil Govindan | | | Document configuration of ReservationSystem for FairScheduler | Blocker | capacity scheduler | Subru Krishnan | Yufei Gu | | | Add REST API for supporting recurring reservations | Major | resourcemanager | Sangeetha Abdu Jyothi | Sean Po | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-common | Major | client | Jinjiang Ling | Jinjiang Ling | | | Define the strings used in SLS JSON input file format | Major | scheduler-load-simulator | Yufei Gu | Gergely Novk | | | Compute effectiveCapacity per each resource vector | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Support GPU isolation for docker container | Major | . | Wangda Tan | Wangda Tan | | | Improve performance of DRF comparisons for resource types in fair scheduler | Critical | fairscheduler | Daniel Templeton | Daniel Templeton | | | Add support for individual resource types requests in MapReduce | Major | resourcemanager | Daniel Templeton | Gergo Repas | | | [API] Introduce SchedulingRequest object | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Add hadoop-aliyun as dependency of hadoop-cloud-storage | Minor | fs/oss | Genmao Yu | Genmao Yu | | | Application lifetime does not work with FairScheduler | Major | resourcemanager | Miklos Szegedi | Miklos Szegedi | | | Render cluster information on new YARN web ui | Major | webapp | Vasudevan Skm | Vasudevan Skm | | | [READ] Merge BlockFormatProvider and FileRegionProvider. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Allow client/AM update supported resource types via YARN APIs | Blocker | nodemanager, resourcemanager | Wangda Tan | Sunil Govindan | | | [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Merge code paths for Reservation/Plan queues and Auto Created queues | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | [READ] Test NameNode restarts when PROVIDED is configured | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Image generation tool does not close an opened stream | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Container REST endpoints should report resource types | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | FileNotFound handling in ResourceUtils is inconsistent | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | [READ] Increasing replication for PROVIDED files should create local replicas | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Allow cluster id to be specified to the Image generation tool | Trivial | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Reduce memory and CPU footprint for PROVIDED volumes. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Moving logging APIs over to slf4j in hadoop-yarn-api | Major | . | Yeliang Cang | Yeliang Cang | | | [YARN-7069] Limit format of resource type name | Blocker | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Improve the resource types docs | Major | docs | Daniel Templeton | Daniel Templeton | | | [API] Add Placement Constraints at the application level | Major | . | Konstantinos Karanasos | Arun Suresh | | | Inter-Queue preemption's computeFixpointAllocation need to handle absolute resources while computing normalizedGuarantee | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Make

method public to return ApplicationId for a service name | Major | . | Gour Saha | Gour Saha | | | AliyunOSS: Override listFiles and listLocatedStatus | Major | fs/oss | Genmao Yu | Genmao Yu | | | Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() | Minor | resourcemanager | Daniel Templeton | Manikandan R | | | [READ] Fix reporting of Provided volumes | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Max applications calculation per queue has to be retrospected with absolute resource support | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Race condition in service AM that can cause NPE | Major | . | Jian He | Jian He | | | Configurable heap size / JVM opts in service AM | Major | . | Jonathan Hung | Jonathan Hung | | | CapacityScheduler: Allow auto leaf queue creation after queue mapping | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | CapacityScheduler test cases cleanup post YARN-5881 | Major | test | Sunil Govindan | Sunil Govindan | | | RBF: Set MountTableResolver as default file resolver | Minor | . | igo Goiri | igo Goiri | | | Enable user re-mapping for Docker containers by default | Blocker | security, yarn | Eric Yang | Eric Yang | | | ApiServer REST API naming convention /ws/v1 is already used in Hadoop v2 | Major | api, applications | Eric Yang | Eric Yang | | | [API] Add SchedulingRequest to the AllocateRequest | Major | . | Arun Suresh | Panagiotis Garefalakis | | | TestYarnNativeServices#testRecoverComponentsAfterRMRestart() fails intermittently | Major | . | Chandni Singh | Chandni Singh | | | Add support for AMRMProxy HA | Major | amrmproxy, nodemanager | Subru Krishnan | Botong Huang | | | AliyunOSS: support user agent configuration and include that & Hadoop version information to oss server | Major | fs, fs/oss | Sammi Chen | Sammi Chen | | | [READ] Report multiple locations for PROVIDED blocks | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Allow user provided Docker volume mount list | Major | yarn | luhuichun | Shane Kumpf | | | Add support to show GPU in UI including metrics | Blocker | . | Wangda Tan | Wangda Tan | | | Fix performance regression introduced by Capacity Scheduler absolute min/max resource refactoring | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Use queue-path.capacity/maximum-capacity to specify CapacityScheduler absolute min/max resources | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Restarted RM may not inform AM about all existing containers | Major | . | Billie Rinaldi | Chandni Singh | | | [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | RBF: Add router admin commands usage in HDFS commands reference doc | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Cleanup to fix checkstyle issues of YARN-5881 branch | Minor |

| Sunil Govindan | Sunil Govindan | | | Render tooltips on columns where text is clipped in new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | NPE in scheduler UI when max-capacity is not configured | Major | capacity scheduler | Eric Payne | Sunil Govindan | | | Documentation for absolute resource support in Capacity Scheduler | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | RBF: Fix Javadoc parameter errors | Minor | . | Wei Yan | Wei Yan | | | Node updates don't update the maximum cluster capability for resources other than CPU and memory | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | Gpu Information page could be empty for nodes without GPU | Major | webapp, yarn-ui-v2 | Sunil Govindan | Sunil Govindan | | | [READ] FsVolumeImpl exception when scanning Provided storage volume | Major | . | Ewan Higgs | Virajith Jalaparti | | | [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb) | Major | . | Ewan Higgs | Ewan Higgs | | | Ensure volume to include GPU base libraries after created by plugin | Major | . | Wangda Tan | Wangda Tan | | | Add support in NodeManager to isolate FPGA devices with CGroups | Major | yarn | Zhankun Tang | Zhankun Tang | | | Uploader tool for Distributed Cache Deploy code changes | Major | . | Miklos Szegedi | Miklos Szegedi | | | [READ] Implement LevelDBFileRegionFormat | Minor | hdfs | Ewan Higgs | Ewan Higgs | | | Node information page in the old web UI should report resource types | Major | resourcemanager | Daniel Templeton | Gergely Novk | | | Skip dispatching opportunistic containers to nodes whose queue is already full | Major | . | Weiwei Yang | Weiwei Yang | | | Webhdfs file system should get delegation token from kms provider. | Major | encryption, kms, webhdfs | Rushabh S Shah | Rushabh S Shah | | | Render application specific log under application tab in new YARN UI | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | s3a troubleshooting docs to add a couple more failure modes | Minor | documentation, fs/s3 | Steve Loughran | Steve Loughran | | | Additional changes to make SchedulingPlacementSet agnostic to ResourceRequest / placement algorithm | Major | . | Wangda Tan | Wangda Tan | | | Add visibility/stability annotations | Trivial | . | Chris Douglas | Chris Douglas | | | Metrics of S3A don't print out when enable it in Hadoop metrics property file | Major | fs/s3 | Yonger | Yonger | | | [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata | Major | . | Virajith Jalaparti | Ewan Higgs | | | [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Extend Distributed Shell to support launching job with opportunistic containers | Major | applications/distributed-shell | Weiwei Yang | Weiwei Yang | | | [READ] Datanodes should use a unique identifier when reading from external stores | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-jobclient | Major | . | Akira Ajisaka | Gergely Novk | | | Moving logging APIs over to slf4j in hadoop-mapreduce-client-nativetask | Minor |

| Jinjiang Ling | Jinjiang Ling | | | [READ] Documentation for provided storage | Major | . | Chris Douglas | Virajith Jalaparti | | | Introduce AllocationTagsManager to associate allocation tags to nodes | Major | . | Wangda Tan | Wangda Tan | | | [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Support replication of Provided blocks with non-default topologies. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Add native FPGA module support to do isolation with cgroups | Major | yarn | Zhankun Tang | Zhankun Tang | | | Implement Framework and policy for capacity management of auto created queues | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | YARN UI changes to depict auto created queues | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | Queue Ordering policy changes for ordering auto created leaf queues within Managed parent Queues | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService | Major | . | Botong Huang | Botong Huang | | | Effective min and max resource need to be set for auto created leaf queues upon creation and capacity management | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | Apply erasure coding properly to framework tarball and support plain tar | Major | . | Miklos Szegedi | Miklos Szegedi | | | RBF: Complete logic for -readonly option of dfsrouteradmin add command | Major | . | Yiqun Lin | igo Goiri | | | Queue ACL validations should validate parent queue ACLs before auto-creating leaf queues | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | Allow searchable filter for Application page log viewer in new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Node resource is not parsed correctly for resource names containing dot | Major | nodemanager, resourcemanager | Jonathan Hung | Gergely Novk | | | Handle recovery of applications in case of auto-created leaf queue mapping | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | [READ] Fix configuration and implementation of LevelDB-based alias maps | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | Support multiple resource types in rmadmin updateNodeResource command | Major | nodemanager, resourcemanager | Daniel Templeton | Manikandan R | | | Fix AMRMToken rollover handling in AMRMProxy | Minor | . | Botong Huang | Botong Huang | | | Yarn service pre-maturely releases the container after AM restart | Major | . | Chandni Singh | Chandni Singh | | | Unit tests related to preemption for auto created leaf queues feature | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | Documentation for auto queue creation feature and related configurations | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | [9806] Code style cleanup | Minor | . | igo Goiri | Virajith Jalaparti | | | [READ] Fix closing streams in ImageWriter | Major |

| igo Goiri | Virajith Jalaparti | | | Add a flag in distributed shell to automatically PROMOTE opportunistic containers to guaranteed once they are started | Minor | applications/distributed-shell | Weiwei Yang | Weiwei Yang | | | RBF: Add more unit tests for router admin commands | Major | test | Yiqun Lin | Yiqun Lin | | | Allow node partition filters on Queues page of new YARN UI | Major | yarn-ui-v2 | Vasudevan Skm | Vasudevan Skm | | | Modifications to the ResourceScheduler to support SchedulingRequests | Major | . | Arun Suresh | Arun Suresh | | | [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled. | Critical | . | Rohith Sharma K S | Rohith Sharma K S | | | s3a input stream "normal" fadvise mode to be adaptive | Major | fs/s3 | Steve Loughran | Steve Loughran | | | [JDK9] Ignore com.sun.javadoc.\ and com.sun.tools.\ in animal-sniffer-maven-plugin to compile with Java 9 | Major | . | Akira Ajisaka | Akira Ajisaka | | | API and interface modifications for placement constraint processor | Major | . | Arun Suresh | Arun Suresh | | | NPE in S3A getFileStatus: null instrumentation on using closed instance | Major | fs/s3 | Steve Loughran | Steve Loughran | | | NativeAzureFileSystem file rename is not atomic | Major | fs/azure | Shixiong Zhu | Thomas Marquardt | | | Rack cardinality support for AllocationTagsManager | Major | . | Panagiotis Garefalakis | Panagiotis Garefalakis | | | Introduce Placement Constraint Manager module | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Add Processor Framework for Rich Placement Constraints | Major | . | Arun Suresh | Arun Suresh | | | Implement Basic algorithm for constraint based placement | Major | . | Arun Suresh | Panagiotis Garefalakis | | | Expose canSatisfyConstraints utility function to validate a placement against a constraint | Major | . | Arun Suresh | Panagiotis Garefalakis | | | RBF: Mount table entries not properly updated in the local cache | Major | . | igo Goiri | igo Goiri | | | It should be possible to specify resource types in the fair scheduler increment value | Critical | fairscheduler | Daniel Templeton | Gergo Repas | | | Introduce scheduler specific environment variable support in ApplicationSubmissionContext for better scheduling placement configurations | Major | . | Sunil Govindan | Sunil Govindan | | | Support to specify values of different resource types in DistributedShell for easier testing | Critical | nodemanager, resourcemanager | Wangda Tan | Gergely Novk | | | Document improvement for registry dns | Major | . | Jian He | Jian He | | | s3a: Stream and common statistics missing from metrics | Major | . | Sean Mackrory | Sean Mackrory | | | RBF: Control MountTableResolver cache size | Major | . | igo Goiri | igo Goiri | | | RBF: Federation supports global quota | Major | . | Yiqun Lin | Yiqun Lin | | | Double-check placement constraints in scheduling phase before actual allocation is made | Major | RM, scheduler | Weiwei Yang | Weiwei Yang | | | Improve handling of the Docker container life cycle | Major | yarn | Shane Kumpf | Shane Kumpf | | | Uploader tool should ignore symlinks to the same directory | Minor | . | Miklos Szegedi | Miklos Szegedi | | | yarn application status should support application name | Major | yarn-native-services | Yesha Vora | Jian He | | | Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases | Major |

| Arun Suresh | Arun Suresh | | | RBF: Display mount table quota info in Web UI and admin command | Major | . | Yiqun Lin | Yiqun Lin | | | Moving logging APIs over to slf4j the rest of all in hadoop-mapreduce | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch | Critical | . | Sean Mackrory | Steve Loughran | | | RBF: Support erasure coding methods in RouterRpcServer | Critical | . | igo Goiri | igo Goiri | | | Consider writing to both ats v1 & v2 from RM for smoother upgrades | Major | timelineserver | Vrushali C | Aaron Gresch | | | Add the ability to specify a delayed replication count | Major | . | Miklos Szegedi | Miklos Szegedi | | | Support IAM Assumed roles in S3A | Major | fs/s3 | Steve Loughran | Steve Loughran | | | AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance | Major | fs/oss | wujinhu | wujinhu | | | AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects | Major | . | Arun Suresh | Arun Suresh | | | Remove SELF from TargetExpression type | Blocker | . | Wangda Tan | Konstantinos Karanasos | | | Support anti-affinity constraint via AppPlacementAllocator | Major | . | Wangda Tan | Wangda Tan | | | Allow DistributedShell to take a placement specification for containers it wants to launch | Major | . | Arun Suresh | Arun Suresh | | | RBF: Document global quota supporting in federation | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Fix spurious TestRouterRpc#testProxyGetStats | Minor | . | igo Goiri | igo Goiri | | | some YARN container events have timestamp of -1 | Critical | . | Sangjin Lee | Haibo Chen | | | Uploader tool for Distributed Cache Deploy documentation | Major | . | Miklos Szegedi | Miklos Szegedi | | | Miscellaneous fixes to the PlacementProcessor | Blocker | . | Arun Suresh | Arun Suresh | | | Allow Constraints specified in the SchedulingRequest to override application level constraints | Blocker | . | Wangda Tan | Weiwei Yang | | | Add support for setting the PID namespace mode | Major | nodemanager | Shane Kumpf | Billie Rinaldi | | | Factor out management of temp tags from AllocationTagsManager | Major | . | Arun Suresh | Arun Suresh | | | Display allocation tags in RM web UI and expose same through REST API | Major | RM | Weiwei Yang | Weiwei Yang | | | Enable user re-mapping for Docker containers in yarn-default.xml | Blocker | security, yarn | Eric Yang | Eric Yang | | | Implement doAs for Api Service REST API | Major | . | Eric Yang | Eric Yang | | | Convert yarn app cli to call yarn api services | Major | . | Eric Yang | Eric Yang | | | RBF: Federation Router State State Store internal API | Major | . | igo Goiri | igo Goiri | | | Add validation step to ensure constraints are not violated due to order in which a request is processed | Blocker |

| Arun Suresh | Arun Suresh | | | Assume intra-app anti-affinity as default for scheduling request inside AppPlacementAllocator | Blocker | . | Wangda Tan | Wangda Tan | | | Fix jenkins issues of YARN-6592 branch | Blocker | . | Sunil Govindan | Sunil Govindan | | | RBF: Heartbeat Router State | Major | . | igo Goiri | igo Goiri | | | Refactor SLS Reservation Creation | Minor | . | Young Chen | Young Chen | | | RBF: Inconsistent Router OPTS config in branch-2 and branch-3 | Minor | . | Wei Yan | Wei Yan | | | Remove automatic mounting of the cgroups root directory into Docker containers | Major | . | Shane Kumpf | Shane Kumpf | | | Fix Cluster metrics when placement processor is enabled | Major | metrics, RM | Weiwei Yang | Arun Suresh | | | Add RMContainer recovery test to verify tag population in the AllocationTagsManager | Major | . | Konstantinos Karanasos | Panagiotis Garefalakis | | | Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. | Major | . | Sumana Sathish | Sunil Govindan | | | Docker host network can not obtain IP address for RegistryDNS | Major | nodemanager | Eric Yang | Eric Yang | | | Add CryptoInputStream to WebHdfsFileSystem read call. | Major | encryption, kms, webhdfs | Rushabh S Shah | Rushabh S Shah | | | [UI2] Add page to new YARN UI to view server side configurations/logs/JVM-metrics | Major | webapp, yarn-ui-v2 | Wangda Tan | Kai Sasaki | | | Avoid using docker volume --format option to run against older docker releases | Major | . | Wangda Tan | Wangda Tan | | | Documentation for Placement Constraints | Major | . | Arun Suresh | Konstantinos Karanasos | | | Service AM should use configured default docker network | Major | yarn-native-services | Billie Rinaldi | Billie Rinaldi | | | Constraint satisfaction checker support for composite OR and AND constraints | Major | . | Arun Suresh | Weiwei Yang | | | RBF: Add a safe mode for the Router | Major | . | igo Goiri | igo Goiri | | | YARN Service - Two different users are unable to launch a service of the same name | Major | applications | Gour Saha | Gour Saha | | | RBF: Expose the state of the Routers in the federation | Major | . | igo Goiri | igo Goiri | | | Move logging to slf4j in BlockPoolSliceStorage and Storage | Major | . | Ajay Kumar | Ajay Kumar | | | RBF: Add router admin option to manage safe mode | Major | . | igo Goiri | Yiqun Lin | | | Modify PlacementAlgorithm to Check node capacity before placing request on node | Major | . | Arun Suresh | Panagiotis Garefalakis | | | Provide improved error message when YARN service is disabled | Major | yarn-native-services | Eric Yang | Eric Yang | | | Merging of placement constraints defined at different levels | Major | . | Konstantinos Karanasos | Weiwei Yang | | | Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning | Major | . | Weiwei Yang | Sunil Govindan | | | Security check for trusted docker image | Major | . | Eric Yang | Eric Yang | | | Make the YARN mounts added to Docker containers more restrictive | Major |

| Shane Kumpf | Shane Kumpf | | | Make Hadoop compatible with Guava 21.0 | Minor | . | Igor Dvorzhak | Igor Dvorzhak | | | Allow for specifying the docker client configuration directory | Major | yarn | Shane Kumpf | Shane Kumpf | | | Support AND/OR constraints in Distributed Shell | Critical | distributed-shell | Weiwei Yang | Weiwei Yang | | | S3Guard CLI to support list/purge of pending multipart commits | Major | fs/s3 | Steve Loughran | Aaron Fabbri | | | Fix failing test TestDockerContainerRuntime#testLaunchContainerWithDockerTokens | Minor | nodemanager | Shane Kumpf | Shane Kumpf | | | Fix exit code handling for short lived Docker containers | Critical | . | Shane Kumpf | Shane Kumpf | | | Upgrade AWS SDK to 1.11.271: NPE bug spams logs w/ Yarn Log Aggregation | Blocker | fs/s3 | Aaron Fabbri | Aaron Fabbri | | | Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator | Critical | . | Sumana Sathish | Zian Chen | | | Enhance S3A troubleshooting documents and add a performance document | Blocker | documentation, fs/s3 | Steve Loughran | Steve Loughran | | | Enhance IAM Assumed Role support in S3A client | Blocker | fs/s3, test | Steve Loughran | Steve Loughran | | | Simplify configuration for PlacementConstraints | Blocker | . | Wangda Tan | Wangda Tan | | | Retrospect Resource Profile Behavior for overriding capability | Blocker | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key | Critical | fs/s3 | Steve Loughran | Steve Loughran | | | ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores | Critical | nodemanager | Daniel Templeton | lovekesh bansal | | | RBF: Manage unavailable clusters | Major | . | igo Goiri | Yiqun Lin | | | Service AM gets NoAuth with secure ZK | Blocker | yarn-native-services | Billie Rinaldi | Billie Rinaldi | | | Document GPU isolation feature | Blocker | . | Wangda Tan | Wangda Tan | | | Move commons-net up to 3.6 | Minor | fs | Steve Loughran | Steve Loughran | | | Remove call to docker logs on failure in container-executor | Major | . | Shane Kumpf | Shane Kumpf | | | YARN Service component update PUT API should not use component name from JSON body | Major | api, yarn-native-services | Gour Saha | Gour Saha | | | [GQ] Refactor preemption calculators to allow overriding for Federation Global Algos | Major | . | Carlo Curino | Carlo Curino | | | Transform a PlacementConstraint to a string expression | Major | . | Weiwei Yang | Weiwei Yang | | | RBF: Fix Routers information shown in the web UI | Minor | . | Wei Yan | Wei Yan | | | RBF: Improve the unit test TestRouterRPCClientRetries | Minor | test | Yiqun Lin | Yiqun Lin | | | Document the FPGA isolation feature | Blocker | . | Zhankun Tang | Zhankun Tang | | | Add .vm extension to

to ensure proper filtering | Critical | documentation | Weiwei Yang | Weiwei Yang | | | RBF: Fix the hdfs router page missing label icon issue | Major | federation, hdfs | maobaolong | maobaolong | | | Support to set container execution type in SLS | Major | scheduler-load-simulator | Jiandan Yang | Jiandan Yang | | | AWS "shaded" SDK 1.11.271 is pulling in netty 4.1.17 | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Docker container privileged mode and --user flag contradict each other | Major | . | Eric Yang | Eric Yang | | | Component status stays "Ready" when yarn service is stopped | Major | . | Yesha Vora | Gour Saha | | | Calling stop on an already stopped service says "Successfully stopped service" | Major | . | Gour Saha | Gour Saha | | | GPU volume creation command fails when work preserving is disabled at NM | Critical | nodemanager | Sunil Govindan | Zian Chen | | | Move hadoop-openstack to slf4j | Minor | fs/swift | Steve Loughran | fang zhenyi | | | Update metrics-core version to 3.2.4 | Major | . | Ray Chiang | Ray Chiang | | | Federation: Add more Balancer tests with federation setting | Minor | balancer & mover, test | Tsz Wo Nicholas Sze | Bharat Viswanadham | | | S3Guard: implement retries for DDB failures and throttling; translate exceptions | Blocker | fs/s3 | Aaron Fabbri | Aaron Fabbri | | | Trusted image log message repeated multiple times | Major | . | Eric Badger | Shane Kumpf | | | Add ADL troubleshooting doc | Major | documentation, fs/adl | Steve Loughran | Steve Loughran | | | Support inter-app placement constraints for allocation tags by application ID | Major | . | Weiwei Yang | Weiwei Yang | | | Remove unicode multibyte characters from JavaDoc | Major | documentation | Akira Ajisaka | Takanobu Asanuma | | | JDK9 JavaDoc build fails due to one-character underscore identifiers in hadoop-yarn-common | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | TestMiniKdc fails on Java 9 | Major | test | Akira Ajisaka | Takanobu Asanuma | | | Add a profile to allow optional compilation for ATSv2 with HBase-2.0 | Major | . | Ted Yu | Haibo Chen | | | Refactor timelineservice-hbase module into submodules | Major | timelineservice | Haibo Chen | Haibo Chen | | | RBF: Complete document of Router configuration | Major | . | Tao Jie | Yiqun Lin | | | S3A multipart upload fails when SSE-C encryption is enabled | Critical | fs/s3 | Anis Elleuch | Anis Elleuch | | | LogAggregationIndexedFileController should support read from HAR file | Major | . | Xuan Gong | Xuan Gong | | | Allow regular expression matching in container-executor.cfg for devices and named docker volumes mount | Major | . | Zian Chen | Zian Chen | | | RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns | Minor | . | Wei Yan | Chao Sun | | | RBF: MountTableResolver doesn't return the correct mount point of the given path | Major | hdfs | wangzhiyuan | wangzhiyuan | | | remove .FluentPropertyBeanIntrospector from CLI operation log output | Minor | conf | Steve Loughran | Steve Loughran | | | TestLogLevel fails on Java 9 | Major | test | Akira Ajisaka | Takanobu Asanuma | | | RBF: Fix router location cache issue | Major | federation, hdfs | Weiwei Wu | Weiwei Wu | | | RBF: ConnectionPool should return first usable connection | Minor |

| Wei Yan | Ekanth Sethuramalingam | | | RBF: Update some inaccurate document descriptions | Minor | . | Yiqun Lin | Yiqun Lin | | | Introduce description and version field in Service record | Critical | . | Gour Saha | Chandni Singh | | | Make S3A etag =\> checksum feature optional | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Many tests fails in Windows due to injecting disk failures | Major | . | Yiqun Lin | Yiqun Lin | | | Extend TestReconstructStripedFile with a random EC policy | Major | erasure-coding, test | Takanobu Asanuma | Takanobu Asanuma | | | RBF: TestRouterSafemode failed if the port 8888 is in use | Major | hdfs, test | maobaolong | maobaolong | | | RBF: Quota management incorrect parent-child relationship judgement | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Throw the exception if mount table entry validated failed | Major | hdfs | maobaolong | maobaolong | | | Extend TestFileStatusWithECPolicy with a random EC policy | Major | erasure-coding, test | Takanobu Asanuma | Takanobu Asanuma | | | Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies | Major | erasure-coding, test | Takanobu Asanuma | Takanobu Asanuma | | | Support sliding window retry capability for container restart | Major | nodemanager | Varun Vasudev | Chandni Singh | | | Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad | | | TestConfiguration fails on Windows because of paths | Major | test | igo Goiri | Xiao Liang | | | RBF: Improve State Store FS implementation | Major | . | igo Goiri | igo Goiri | | | TestUGILoginFromKeytab fails on Java9 | Major | security | Takanobu Asanuma | Takanobu Asanuma | | | Docker launch fails when user private filecache directory is missing | Major | . | Eric Yang | Jason Lowe | | | RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router | Minor | . | Wei Yan | Wei Yan | | | log s3a at info | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Resolvers to support mount points across multiple subclusters | Major | . | igo Goiri | igo Goiri | | | RBF: Move Router to its own module | Major | . | igo Goiri | Wei Yan | | | Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages. | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | RBF: Router to manage requests across multiple subclusters | Major | . | igo Goiri | igo Goiri | | | [READ] Namenode support for data stored in external stores. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | [READ] Datanode support to read from external stores. | Major | . | Virajith Jalaparti | Virajith Jalaparti | | | RBF: Fix FindBugs in hadoop-hdfs-rbf | Minor | . | igo Goiri | Ekanth Sethuramalingam | | | RBF: Test Router-based federation using HDFSContract | Major | . | igo Goiri | igo Goiri | | | HBase filters are not constructed correctly in ATSv2 | Major | ATSv2 | Haibo Chen | Haibo Chen | | | ATSv2 REST API queries do not return results for uppercase application tags | Critical |

| Charan Hebri | Charan Hebri | | | RBF: Add WebHDFS | Major | fs | igo Goiri | Wei Yan | | | Yarn Service API site doc broken due to unwanted character in YarnServiceAPI.md | Blocker | site | Gour Saha | Gour Saha | | | RBF: Implement available space based OrderResolver | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Optimize name service safe mode icon | Minor | . | liuhongtong | liuhongtong | | | RBF: Add xsl stylesheet for hdfs-rbf-default.xml | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Add config in FederationRMFailoverProxy to not bypass facade cache when failing over | Minor | . | Botong Huang | Botong Huang | | | RBF: Cache datanode reports | Minor | . | igo Goiri | igo Goiri | | | Clean up example hostnames | Major | . | Billie Rinaldi | Billie Rinaldi | | | RBF: Support NamenodeProtocol in the Router | Major | . | igo Goiri | igo Goiri | | | Update okhttp version to 2.7.5 | Major | fs/adl | Ray Chiang | Ray Chiang | | | Setting hostname of docker container breaks for --net=host in docker 1.13 | Major | yarn | Jim Brennan | Jim Brennan | | | TestDockerContainerRuntime test failures due to UID lookup of a non-existent user | Major | . | Shane Kumpf | Shane Kumpf | | | TestTrash should use proper test path to avoid failing on Windows | Minor | . | Anbang Hu | Anbang Hu | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Enable JournalNode Sync by default | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Remove the doc about Schedulable#redistributeShare() | Trivial | fairscheduler | Yufei Gu | Chetna Chaudhari | | | Add a junit test for ContainerScheduler recovery | Minor | . | kartheek muthyala | Sampada Dehankar | | | Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues | Major | . | Wangda Tan | Zian Chen | | | CryptoOutputStream should implement StreamCapabilities | Major | fs | Mike Drob | Xiao Chen | | | Add Unit Tests for ContainersLauncher | Major | . | Sampada Dehankar | Sampada Dehankar | | | Provide means for container network policy control | Major | nodemanager | Clay B. | Xuan Gong | | | FairScheduler: Deprecate continuous scheduling | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Update the release year to 2018 | Blocker | build | Akira Ajisaka | Bharat Viswanadham | | | Remove tomcat from the Hadoop-auth test bundle | Major | . | Xiao Chen | Xiao Chen | | | [Umbrella] Stabilise S3A Server Side Encryption | Major | documentation, fs/s3, test | Steve Loughran | | | | Fix TestAMRMClientPlacementConstraints | Critical | . | Botong Huang | Gergely Novk | | | WebHDFS: Add support for snasphot diff | Major | . | Lokesh Jain | Lokesh Jain | | | Document multi-URI replication Inode for ViewFS | Major | documentation, viewfs | Chris Douglas | Gera Shegalov | | | WebHDFS: Add support for getting snasphottable directory list | Major | webhdfs | Lokesh Jain | Lokesh Jain | | | RM log is getting flooded with MemoryPlacementConstraintManager info logs | Critical | . | Zian Chen | Zian Chen