Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.3.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Inconsistent log level practices in RpcProgramNfs3.java | Major | nfs | Anuhan Torgonshar | Anuhan Torgonshar | | | Erasure Coding: Decommission may generate the parity block's content with all 0 in some case | Critical | datanode, erasure-coding | Yao Guangdong | Yao Guangdong | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add base64 (urlString) support to DTUtil | Minor | . | igo Goiri | igo Goiri | | | Failover for LdapGroupsMapping | Major | common, security | Lukas Majercak | Lukas Majercak | | | [PROVIDED Storage] bootstrapStandby should manage the InMemoryAliasMap | Major | . | igo Goiri | Virajith Jalaparti | | | [Umbrella] Support Interactive Docker Shell to running Containers | Major | . | Zian Chen | Eric Yang | | | Plugin interface to support more complex usernames in Hadoop | Major | security | Eric Yang | Bolke de Bruin | | | Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API. | Major | fs, fs/azure, fs/s3 | Steve Loughran | Steve Loughran | | | Support using DNS to resolve nameservices to IP addresses | Major | . | Fengnan Li | Fengnan Li | | | Support multiple bind users in LdapGroupsMapping | Major | common, security | Lukas Majercak | Lukas Majercak | | | [Umbrella] Docker image life cycle management on HDFS | Major | . | Eric Yang | Eric Yang | | | DocumentStore as a backend for ATSv2 | Major | ATSv2 | Sushil Ks | Sushil Ks | | | Limit WebHDFS to specifc user, host, directory triples | Trivial | webhdfs | Clay B. | Clay B. | | | Support impersonation for AuthenticationFilter | Major | security | Eric Yang | Eric Yang | | | Scale testing HDFS NameNode with real metadata and workloads (Dynamometer) | Major | namenode, test | Zhe Zhang | Erik Krogen | | | [Umbrella] Support Vector Engine ( a new accelerator hardware) based on pluggable device framework | Major | nodemanager | Zhankun Tang | Peter Bacsko | | | Balancer: make balancer to be a long service process for easy to monitor it. | Major | balancer & mover | maobaolong | Chen Zhang | | | Exports Hadoop metrics to Prometheus | Major | metrics | Akira Ajisaka | Akira Ajisaka | | | Add a whitelist of endpoints to skip Kerberos authentication | Major | security | Akira Ajisaka | Akira Ajisaka | | | Add DataTransferThrottler to the Datanode transfers | Minor | datanode | igo Goiri | Lisheng Sun | | | Allow overriding application submissions based on server side configs | Major | . | Jonathan Hung | pralabhkumar | | | Zero length files in container log output haven't got a header | Major | log-aggregation, yarn | Adam Antal | Adam Antal | | | Add PathCapabilities to FS and FC to complement StreamCapabilities | Major | . | Steve Loughran | Steve Loughran | | | Incorporate Tencent Cloud COS File System Implementation | Major | fs/cos | Junping Du | Yang Yu | | | Support configuring application priorities on a workflow level | Major |

| Jonathan Hung | Varun Saxena | | | Support non-volatile storage class memory(SCM) in HDFS cache directives | Major | caching, datanode | Sammi Chen | Feilong He | | | Consistent Reads from Standby Node | Major | hdfs | Konstantin Shvachko | Konstantin Shvachko | | | Deadnode detection | Major | hdfs-client | Gang Xie | Lisheng Sun | | | Introduce HealthReporter interface to support multiple health checker files | Major | nodemanager, yarn | Adam Antal | Adam Antal | | | [Umbrella] A pluggable device plugin framework to ease vendor plugin development | Major | yarn | Zhankun Tang | Zhankun Tang | | | Application Catalog for YARN applications | Major | . | Eric Yang | Eric Yang | | | Scheduling of opportunistic containers | Major | . | Konstantinos Karanasos | | | | Batch listing of multiple directories | Major | . | Andrew Wang | Chao Sun | | | Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... | Critical | hdfs | Ramesh Mani | Wei-Chiu Chuang | | | Supports config the shuffle's path cache related parameters | Major | mrv2 | Wanqiang Ji | Wanqiang Ji | | | Support TLS 1.3 | Major | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Emit per priority RPC queue time and processing time from DecayRpcScheduler | Major | common | Fengnan Li | Fengnan Li | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve anti-affinity section description in YARN Service API doc | Major | docs, documentation | Charan Hebri | Gour Saha | | | Make HTTP timeout configurable in ADLS Connector | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | Fix wrong log statement in AbstractService | Minor | util | Szilard Nemeth | Szilard Nemeth | | | Registering MutableQuantiles via Metric annotation | Major | metrics | Sushil Ks | Sushil Ks | | | Add test coverage of custom Resource Types for the apps/\<appId\> REST API endpoint | Major | . | Szilard Nemeth | Szilard Nemeth | | | Review of DirectoryScanner Class | Major | datanode | David Mollitor | David Mollitor | | | Add unit tests of min/max allocation for custom resource types in FairScheduler | Minor | . | Szilard Nemeth | Szilard Nemeth | | | Refactor TestQueueMetrics | Minor | resourcemanager | Szilard Nemeth | Szilard Nemeth | | | ACL documentation update to indicate that ACL entries are capped by 32 | Minor | hdfs | Adam Antal | Adam Antal | | | Miscellaneous Improvements for FsVolumeSpi | Major | datanode | David Mollitor | David Mollitor | | | Improve unit test for RMAppImpl.FinalTransition | Minor | . | Szilard Nemeth | Szilard Nemeth | | | HDFS Router Quota Class Review | Minor | federation, hdfs | David Mollitor | David Mollitor | | | Upgrade BouncyCastle to 1.60 | Major | . | Robert Kanter | Robert Kanter | | | Set a maximum delay for retrying locateFollowingBlock | Major | . | Kitti Nanasi | Kitti Nanasi | | | javadocs for FileInputFormat and OutputFormat to mention DT collection | Minor | client | Steve Loughran | Steve Loughran | | | BlockReceiver Array-Based Queue | Minor | datanode | David Mollitor | David Mollitor | | | TGT renewal thread does not log IOException | Major |

| Szilard Nemeth | Szilard Nemeth | | | Include modificationTime in the toString method of CopyListingFileStatus | Minor | . | Ted Yu | Ted Yu | | | HDFS Block Placement Policy - Client Local Rack | Minor | documentation | David Mollitor | Ayush Saxena | | | Upgrade netty version to 3.10.6 | Major | . | Xiao Chen | Xiao Chen | | | Add tags and attributes in resource definition | Major | . | Weiwei Yang | Weiwei Yang | | | RBF: Review of RandomResolver Class | Minor | federation | David Mollitor | David Mollitor | | | Optimize collections used by MR JHS to reduce its memory | Major | jobhistoryserver, mrv2 | Misha Dmitriev | Misha Dmitriev | | | AuthToken Use StringBuilder instead of StringBuffer | Trivial | auth | David Mollitor | David Mollitor | | | Support to configure ftpClient.setControlKeepAliveTimeout | Minor | fs | Krishnamoorthy Dharmalingam | Adam Antal | | | TestCleanupAfterKIll is failing due to unsatisfied dependencies | Blocker | yarn-native-services | Eric Yang | Robert Kanter | | | Yarn Service: When all the components of a service have restart policy NEVER then initiation of service upgrade should fail | Major | . | Chandni Singh | Chandni Singh | | | upgrade to commons-compress 1.18 | Major | . | PJ Fanning | Akira Ajisaka | | | Define a constant "docker" string in "ContainerRuntimeConstants.java" for better maintainability | Minor | . | Zhankun Tang | Zhankun Tang | | | Fix errors in yarn-default.xml related to GPU/FPGA | Major | . | Zhankun Tang | Zhankun Tang | | | Improve DataNode BlockSender waitForMinLength | Minor | datanode | David Mollitor | David Mollitor | | | Move Hadoop YARN Registry to Hadoop Registry | Major | . | igo Goiri | igo Goiri | | | Yarn Service: Add component name to container json | Major | . | Chandni Singh | Chandni Singh | | | Cleanup references to ENV file type in YARN service code | Minor | yarn-native-services | Suma Shivaprasad | Suma Shivaprasad | | | YARN to facilitate HTTPS in AM web server | Major | yarn | Haibo Chen | Robert Kanter | | | make storageId in BlockPoolTokenSecretManager.checkAccess optional | Major | . | Ajay Kumar | Ajay Kumar | | | Overload BlockPoolTokenSecretManager.checkAccess to make storageId and storageType optional | Major | . | Ajay Kumar | Ajay Kumar | | | Sleep in TestLazyPersistFiles should be put into a loop | Trivial | hdfs | Adam Antal | Adam Antal | | | Provide auto-renewal for keytab based logins | Minor | security | Harsh J | Hrishikesh Gadre | | | Update the doc about the default value of "maximum-container-assignments" for capacity scheduler | Minor | . | Zhankun Tang | Zhankun Tang | | | NN should log snapshotdiff report | Major | namenode | Pranay Singh | Pranay Singh | | | Make HttpFS' ACLs RegEx configurable | Major | httpfs | Siyao Meng | Siyao Meng | | | Reservations list field in ReservationListInfo is not accessible | Minor | resourcemanager, restapi | Oleksandr Shevchenko | Oleksandr Shevchenko | | | Add queue and partition info to RM audit log | Major | resourcemanager | Jonathan Hung | Eric Payne | | | Credentials class should allow access to aliases | Trivial | . | Lars Francke | Lars Francke | | | AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting | Major |

| Wanqiang Ji | Wanqiang Ji | | | Provide ability for NN to re-replicate based on topology changes. | Major | . | ellen johansen | Hrishikesh Gadre | | | Refactor NameNodeHttpServer#initWebHdfs to specify local keytab | Major | . | igo Goiri | CR Hota | | | Add Serializable interface to ComponentContainers | Minor | . | Zhankun Tang | Zhankun Tang | | | Remove redundant modifiers in interface "ApplicationConstants" | Trivial | . | Zhankun Tang | Zhankun Tang | | | Add missing maven modules in BUILDING.txt | Trivial | . | Wanqiang Ji | Wanqiang Ji | | | Fast fail jobs when exceeds dfs quota limitation | Major | task | Wang Yan | Wang Yan | | | Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode | Trivial | . | Wanqiang Ji | Wanqiang Ji | | | [Submarine] Small refactors of modifier, condition check and redundant local variables | Minor | . | Zhankun Tang | Zhankun Tang | | | Refactor NameNodeWebHdfsMethods to allow better extensibility | Major | . | CR Hota | CR Hota | | | Document upgrading the section in NOTICE.txt when upgrading the version of AWS SDK | Minor | documentation | Akira Ajisaka | Dinesh Chitlangia | | | Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN | Major | datanode | Jiandan Yang | Jiandan Yang | | | distcp documentation is woefully out of date | Critical | documentation, tools/distcp | Allen Wittenauer | Dinesh Chitlangia | | | Support noredirect param for CREATE/APPEND/OPEN/GETFILECHECKSUM in HttpFS | Major | . | igo Goiri | igo Goiri | | | Federation client intercepter class contains unwanted character | Minor | router | Rakesh Shah | Abhishek Modi | | | Improve error handling in hdfsThreadDestructor in native thread local storage | Major | native | Daniel Templeton | Daniel Templeton | | | AliyunOSS: Enable Yarn to use OSS | Major | fs/oss | wujinhu | wujinhu | | | Update start-build-env.sh and build instruction for docker for Mac instead of docker toolbox. | Minor | build, documentation | Akira Ajisaka | Dinesh Chitlangia | | | WEBHDFS: Support Enable/Disable EC Policy | Major | erasure-coding, webhdfs | Ayush Saxena | Ayush Saxena | | | [UI2] YARN ui2 should use clusters/{cluster name} for all ATSv2 REST APIs | Major | . | Rohith Sharma K S | Akhil PB | | | AliyunOSS: add missing owner & group attributes for oss FileStatus | Major | fs/oss | wujinhu | wujinhu | | | Performance improvement in BlockManager Data Structures | Minor | hdfs | David Mollitor | David Mollitor | | | Performance improvement in BlockPlacementPolicyDefault | Minor | . | David Mollitor | David Mollitor | | | FileOutputCommitter does not report progress while merging paths. | Major | . | Kuhu Shukla | Kuhu Shukla | | | [Submarine] Use predefined Charset object StandardCharsets.UTF\_8 instead of String "UTF-8" | Trivial | . | Zhankun Tang | Zhankun Tang | | | Improve the assertion message in TestGPUResourceHandler | Trivial | . | Zhankun Tang | Zhankun Tang | | | Improve the GPU/FPGA module log message of container-executor | Minor | . | Zhankun Tang | Zhankun Tang | | | Fix SchedulerInfo#getSchedulerType for custom schedulers | Minor |

| Bilwa S T | Bilwa S T | | | EC: Track Erasure Coding commands in DFS statistics | Major | erasure-coding | Ayush Saxena | Ayush Saxena | | | Escape newlines in health report in YARN UI | Major | . | Jonathan Hung | Keqiu Hu | | | Refactor NamenodeFsck#copyBlock | Minor | namenode | David Mollitor | David Mollitor | | | Add a tool to check rack configuration against EC policies | Major | erasure-coding | Xiao Chen | Kitti Nanasi | | | Extend OIV to detect FSImage corruption | Major | hdfs | Adam Antal | Adam Antal | | | Improve GreedyPlanner Parameter Logging | Trivial | hdfs | David Mollitor | David Mollitor | | | Replace TreeSet in NamenodeFsck with HashSet | Trivial | . | David Mollitor | David Mollitor | | | Improve debug log in FSParentQueue when assigning container | Minor | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | EC : Add Configuration to restrict UserDefined Policies | Major | erasure-coding | Ayush Saxena | Ayush Saxena | | | Add Guaranteed and MaxCapacity to CSQueueMetrics | Major | . | Jonathan Hung | Jonathan Hung | | | EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs | Major | erasure-coding, httpfs, webhdfs | Souryakanta Dwivedy | Ayush Saxena | | | Integrate multiple CustomResourceTypesConfigurationProvider implementations into one | Minor | . | Szilard Nemeth | Szilard Nemeth | | | Extend YARN distributed shell with file localization feature | Major | yarn | Peter Bacsko | Peter Bacsko | | | Clearer error message for ReplicaNotFoundException | Major | hdfs | Adam Antal | Adam Antal | | | Use MultiMap for CacheManager Directives to simplify the code | Minor | caching, hdfs | David Mollitor | David Mollitor | | | Refactor name node to allow different token verification implementations | Major | . | CR Hota | CR Hota | | | Add table of contents to YARN Service API document | Minor | documentation | Akira Ajisaka | Zhankun Tang | | | Remove TLSv1 and SSLv2Hello from the default value of hadoop.ssl.enabled.protocols | Major | security | Akira Ajisaka | Gabor Bota | | | Removed Unused field from Resource: NUM\MANDATORY\RESOURCES | Minor | . | Szilard Nemeth | Vidura Bhathiya Mudalige | | | map-only job should ignore node lost event when task is already succeeded | Major | mrv2 | Zhaohui Xin | Lei Li | | | Add Bind\_HOST configuration for Yarn Web Proxy | Major | yarn | Rong Tang | Rong Tang | | | Upgrade to ADLS SDK which has major performance improvement for ingress/egress | Major | fs/adl | Vishwajeet Dusane | Vishwajeet Dusane | | | Fix test, checkstyle and javadoc issues in TestKerberosAuthenticationHandler | Major | test | Dinesh Chitlangia | Dinesh Chitlangia | | | Log longest FSN write/read lock held stack trace | Minor | . | Yiqun Lin | Yiqun Lin | | | The buffer size is small when unpacking tar archives | Minor | common, util | David Mollitor | David Mollitor | | | Remove StringBuffer from ManagedParentQueue.java | Trivial | resourcemanager | David Mollitor | David Mollitor | | | Improve InMemoryPlan#toString | Minor | reservation system | David Mollitor | David Mollitor | | | Performance improvement in Tailing EditLog | Major | namenode | Kenneth Yang | Kenneth Yang | | | [SPS] Add support for URI based path in satisfystoragepolicy command | Major | . | Ayush Saxena | Ayush Saxena | | | Debug Admin Command Should Support Generic Options. | Major |

| Harshakiran Reddy | Ayush Saxena | | | Optimize system credentials sent in node heartbeat responses | Major | RM | Naganarasimha G R | Manikandan R | | | KeyProvider class should implement Closeable | Major | kms | Kuhu Shukla | Kuhu Shukla | | | Consecutive StringBuilder.append can be reused | Major | . | Ayush Saxena | Ayush Saxena | | | Emit FairCallQueue stats as metrics | Major | metrics, rpc-server | Erik Krogen | Christopher Gregorian | | | Upgrade Jackson2 to 2.9.8 | Major | security | Akira Ajisaka | lqjacklee | | | Remove Jansson from BUILDING.txt | Minor | documentation | Akira Ajisaka | Dinesh Chitlangia | | | Replace Guava Optional with Java Optional | Major | . | Arpit Agarwal | Arpit Agarwal | | | Make ThrottledAsyncChecker constructor public | Major | . | Arpit Agarwal | Arpit Agarwal | | | [SPS] : Add Support for Storage Policy Satisfier in WEBHDFS | Major | . | Ayush Saxena | Ayush Saxena | | | Check if the cluster topology supports the EC policy before setting, enabling or adding it | Major | erasure-coding, hdfs | Kitti Nanasi | Kitti Nanasi | | | Cleanup method calls to static Assert methods in TestAddStripedBlocks | Minor | hdfs | Shweta | Shweta | | | Upgrade checkstyle version to 8.16 | Minor | build | Dinesh Chitlangia | Dinesh Chitlangia | | | Make warning message more clear when there are not enough data nodes for EC write | Major | erasure-coding | Kitti Nanasi | Kitti Nanasi | | | AliyunOSS: update oss-sdk version to 3.4.1 | Major | fs/oss | wujinhu | wujinhu | | | Use parameterized log format in ECTopologyVerifier | Trivial | erasure-coding | Kitti Nanasi | Kitti Nanasi | | | DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error | Major | hdfs | Kitti Nanasi | Kitti Nanasi | | | [SBN read] msync should sync with active NameNode to fetch the latest stateID | Major | namenode | Chao Sun | Chao Sun | | | Add application launch time to ATSV2 | Major | timelineserver | Kanwaljeet Sachdev | Abhishek Modi | | | RM UI should sort memory / cores numerically | Major | . | Eric Maynard | Ahmed Hussein | | | Avoid NPE when SectionName#fromString returns null | Minor | . | Xiang Li | Xiang Li | | | Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication' | Trivial | . | Szilard Nemeth | Charan Hebri | | | Replace synchronized method in BlockReceiver with atomic value | Minor | datanode | David Mollitor | David Mollitor | | | Provide proper documentation for FairCallQueue | Major | documentation, ipc | Erik Krogen | Erik Krogen | | | Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's | Major | . | Harshakiran Reddy | Ayush Saxena | | | Provide feedback on successful renameSnapshot and deleteSnapshot | Minor | hdfs, shell | Siyao Meng | Siyao Meng | | | DistributedFileSystem.create should allow an option to provide StoragePolicy | Major | hdfs | Jean-Marc Spaggiari | Ayush Saxena | | | [UI2] Improve "Resource Usage" tab error message when there are no data available. | Minor | yarn-ui-v2 | Gergely Novk | Charan Hebri | | | Support trust top-level image like "centos" when "library" is configured in "docker.trusted.registries" | Major |

| Zhankun Tang | Zhankun Tang | | | Introduce Java Concurrent Package To DataXceiverServer Class | Minor | datanode | David Mollitor | David Mollitor | | | [UI2] Yarn Component Instance page should include link to container logs | Major | yarn-ui-v2 | Yesha Vora | Akhil PB | | | Distcp to add no-rename copy option | Major | tools/distcp | Steve Loughran | Andrew Olson | | | Remove Shuffle Method From DFSUtil | Trivial | . | David Mollitor | David Mollitor | | | Prefer ArrayList over LinkedList in VolumeScanner | Minor | datanode | David Mollitor | David Mollitor | | | Improve graph text in SLS to avoid overlapping | Minor | . | Bilwa S T | Bilwa S T | | | Make hdfs ec -verifyClusterSetup command accept an erasure coding policy as a parameter | Major | erasure-coding | Kitti Nanasi | Kitti Nanasi | | | KMS Benchmark Tool | Major | . | Wei-Chiu Chuang | George Huang | | | Logging stale datanode information | Trivial | hdfs | Karthik Palanisamy | Karthik Palanisamy | | | Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon | Major | . | Surendra Singh Lilhore | Ranith Sardar | | | Add test\libhdfs\ops to libhdfs tests, mark libhdfs\_read/write.c as examples | Major | libhdfs, native, test | Sahil Takiar | Sahil Takiar | | | Refactor NameNodeWebHdfsMethods#generateDelegationToken() to allow better extensibility | Major | . | CR Hota | CR Hota | | | Jenkinsfile for Hadoop | Major | build | Allen Wittenauer | Allen Wittenauer | | | Improve log messages of ECTopologyVerifier | Minor | . | Kitti Nanasi | Kitti Nanasi | | | DistributedShell client timeout should be -1 by default | Minor | . | Zhankun Tang | Zhankun Tang | | | Support clearing namespace quota on "/" | Major | namenode | Guo Ruijing | Ayush Saxena | | | ipc.Client.stop() may sleep too long to wait for all connections | Major | ipc | Tsz-wo Sze | Tsz-wo Sze | | | Consecutive StringBuilder append should be reused | Major | . | Ayush Saxena | Ayush Saxena | | | Improve logging for initialization of Resource plugins | Major | yarn | Szilard Nemeth | Szilard Nemeth | | | Replace GpuDiscoverer.getInstance() to a readable object for easy access control | Major | . | Szilard Nemeth | Szilard Nemeth | | | Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo | Major | . | Szilard Nemeth | Szilard Nemeth | | | Simplify initializer code of GpuDiscoverer | Major | . | Szilard Nemeth | Szilard Nemeth | | | Repeat adding node description into network topology | Minor | datanode | HuangTao | HuangTao | | | RackResolver tool should accept multiple hosts | Minor | yarn | Lantao Jin | Lantao Jin | | | Datanode usage histogram is clicked to show ip list | Major | . | fengchuang | fengchuang | | | Fix -Xcheck:jni issues in libhdfs, run ctest with -Xcheck:jni enabled | Major | hdfs-client, libhdfs, native | Sahil Takiar | Sahil Takiar | | | Cleanup LineReader Unit Test | Trivial | common | David Mollitor | David Mollitor | | | Remove unused Job Summary Appender configurations from

| Major | conf | Chen Zhi | Chen Zhi | | | Fix checkstyle for NameNodeMXBean | Trivial | namenode | Danny Becker | Danny Becker | | | Implement FS placement rules using PlacementRule interface | Major | scheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Add CorruptFilesCount to JMX | Minor | fs, metrics, namenode | Danny Becker | Danny Becker | | | Improve test coverage for nvidia-smi binary execution of GpuDiscoverer | Major | . | Szilard Nemeth | Szilard Nemeth | | | Add application launch time to ATSV1 | Major | . | Kanwaljeet Sachdev | Abhishek Modi | | | Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2 | Major | ATSv2 | Sushil Ks | Sushil Ks | | | [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore | Minor | tools | Shweta | Shweta | | | JobHistoryServer should log exception when loading/parsing history file failed | Minor | mrv2 | Jiandan Yang | Jiandan Yang | | | The yarn-default.xml has two identical property named yarn.timeline-service.http-cross-origin.enabled | Trivial | yarn | jenny | Abhishek Modi | | | JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable | Major | mrv2 | Jiandan Yang | Jiandan Yang | | | Better time precision in getTimeDuration | Minor | namenode | Chao Sun | Chao Sun | | | Improve HDFS append performance | Major | hdfs | Chao Sun | Chao Sun | | | Typo in comment in SchedulerUtils | Trivial | resourcemanager | Daniel Templeton | Yesha Vora | | | [Clean-up] Remove NULL check before instanceof and fix checkstyle issue in TaskResult | Minor | . | Shweta | Shweta | | | [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey | Minor | yarn | Shweta | Shweta | | | [Clean-up] Remove NULL check before instanceof in TestGSet | Minor | . | Shweta | Shweta | | | "hadoop CLASSFILE" prints error messages on Ubuntu 18 | Major | scripts | Daniel Templeton | Daniel Templeton | | | YARN Services with simple authentication doesn't respect current UGI | Major | security, yarn-native-services | Todd Lipcon | Eric Yang | | | [Consistent Observer Reads] Allow for configurable "always msync" mode | Major | hdfs-client | Erik Krogen | Erik Krogen | | | Better logging in recoverAssignedGpus in class GpuResourceAllocator | Trivial | . | Szilard Nemeth | Yesha Vora | | | Allow CopyListing sequence file keys and values to be more easily customized | Major | tools/distcp | Andrew Olson | Andrew Olson | | | Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322 | Major | . | Szilard Nemeth | Zoltan Siegl | | | Change FairScheduler to use PlacementRule interface | Major | capacityscheduler, fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | High lock contention on hdfsHashMutex in libhdfs | Major | hdfs-client, libhdfs, native | Sahil Takiar | Sahil Takiar | | | Add Threadpool for DataTransfers | Major | datanode | David Mollitor | David Mollitor | | | Remove WARN Logging From Interrupts | Minor | hdfs-client | David Mollitor | David Mollitor | | | [Umbrella] Follow-up on IntelOpenCL FPGA plugin | Major | nodemanager | Peter Bacsko | Peter Bacsko | | | Add SleepJob additional parameter to make parallel runs distinguishable | Major | . | Adam Antal | Adam Antal | | | Add AbstractYarnScheduler#getValidQueues method to remove duplication | Major |

| Wanqiang Ji | Wanqiang Ji | | | hdfs dfs -checksum command should optionally show block size in output | Minor | hdfs | Adam Antal | Lokesh Jain | | | Using FQDN instead of IP to access servers with DNS resolving | Major | . | Fengnan Li | Fengnan Li | | | Use new API of RackResolver to get better performance | Major | yarn | Lantao Jin | Lantao Jin | | | Do Not Log InterruptedException in Client | Minor | common | David Mollitor | David Mollitor | | | Improve Logging in FSNamesystem by adding parameterized logging | Minor | hdfs | Shweta | Shweta | | | Cleanup calling of sun.security.krb5.Config | Minor | . | Kai Zheng | Akira Ajisaka | | | Add queueName info when failing with queue capacity sanity check | Trivial | capacity scheduler | Aihua Xu | Aihua Xu | | | hadoop-common pom should not depend on kerb-simplekdc | Major | common | Todd Lipcon | Todd Lipcon | | | Remove Subversion and Forrest from Dockerfile | Minor | build | Akira Ajisaka | Xieming Li | | | Change Log Level to trace in NetUtils.java | Major | . | Bharat Viswanadham | chencan | | | Upgrade checkstyle to 8.19 | Major | build | Akira Ajisaka | Akira Ajisaka | | | Make CallerContext LimitedPrivate scope to Public | Minor | ipc | Kenneth Yang | Kenneth Yang | | | KMS should log the IP address of the clients | Major | kms | Zsombor Gegesy | Zsombor Gegesy | | | Clean up and split testcases in TestNMWebServices for GPU support | Minor | . | Szilard Nemeth | Szilard Nemeth | | | When doTransition() method occurs exception, the log level practices are inconsistent | Major | nodemanager | Anuhan Torgonshar | | | | dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml | Minor | hdfs | puleya7 | puleya7 | | | Expose total number of delegation tokens in AbstractDelegationTokenSecretManager | Major | . | CR Hota | CR Hota | | | Replace incorrect use of system property user.name | Major | . | Dinesh Chitlangia | Dinesh Chitlangia | | | Update jackson from 1.9.13 to 2.x in hadoop-yarn-services-core | Major | . | Akira Ajisaka | Akira Ajisaka | | | pRead equivalent for direct read path | Major | hdfs-client, performance | Henry Robinson | Sahil Takiar | | | Add Log Level link under NameNode and DataNode Web UI Utilities dropdown | Trivial | webhdfs | Siyao Meng | Siyao Meng | | | DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured | Major | . | CR Hota | CR Hota | | | Avoid FileStream to improve performance | Major | . | Ayush Saxena | Ayush Saxena | | | Improve Bad Sequence Number Error Message | Minor | ipc | David Mollitor | Shweta | | | Use SASL Factories Cache to Improve Performance | Critical | . | Ayush Saxena | Ayush Saxena | | | Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner | Major | . | Sherwood Zheng | Sherwood Zheng | | | Log correct cpu controller path on error while initializing CGroups. | Major | nodemanager | Jonathan Hung | Jonathan Hung | | | Allow extra jsvc startup option in hadoop\start\secure\_daemon in hadoop-functions.sh | Major | scripts | Siyao Meng | Siyao Meng | | | Add the possbility to set SO\_REUSEADDR in IPC Server Listener | Minor | ipc | Peter Bacsko | Peter Bacsko | | | Clean up code long if-else chain in ApplicationCLI#run | Major |

| Szilard Nemeth | Wanqiang Ji | | | Add configuration option for YARN Native services AM classpath | Major | . | Gergely Pollak | Gergely Pollak | | | Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles | Minor | documentation | Siyao Meng | Siyao Meng | | | [Umbrella] Dynamically add or remove auxiliary services | Major | nodemanager | Billie Rinaldi | Billie Rinaldi | | | Incorrect header or version mismatch log message | Minor | ipc | David Mollitor | Shweta | | | keepResults option in NNThroughputBenchmark should call saveNamespace() | Major | benchmarks, hdfs | Konstantin Shvachko | Konstantin Shvachko | | | https everywhere in Maven settings | Minor | build | Akira Ajisaka | Akira Ajisaka | | | Resource report REST API could return NaN or Inf | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Use a random secret when a secret file doesn't exist in HttpFS. This should be default. | Major | httpfs, security | Pulkit Bhardwaj | Takanobu Asanuma | | | Use Logger format in ContainersMonitorImpl | Major | . | igo Goiri | igo Goiri | | | Create healthcheck REST endpoint for ATSv2 | Major | ATSv2 | Zoltan Siegl | Zoltan Siegl | | | Make DurationInfo "public unstable" | Minor | util | Kevin Risden | Kevin Su | | | Cleanup in TestLogAggregationIndexFileController | Major | log-aggregation, yarn | Adam Antal | Adam Antal | | | Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory | Major | ha, journal-node | Amit Anand | star | | | Remove the defunct startKdc profile from hdfs | Major | test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Auto-created leaf queues do not honor cluster-wide min/max memory/vcores | Major | capacity scheduler | Craig Condit | Craig Condit | | | Use logger format in Container Executor. | Major | . | Abhishek Modi | Abhishek Modi | | | FSImage which is saving should be clean while NameNode shutdown | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2 | Major | ATSv2, yarn-ui-v2 | Zoltan Siegl | Zoltan Siegl | | | Fix zstandard shortname misspelled as zts | Major | . | Jonathan Turner Eagles | Jonathan Turner Eagles | | | Allow block replication parameters to be refreshable | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Refactor OIV Delimited output entry building mechanism | Minor | tools | Adam Antal | Adam Antal | | | should prevent the possibility of NPE about ReconfigurableBase.java | Minor | . | hu xiaodong | hu xiaodong | | | Add doc about how to disable client socket cache | Trivial | hdfs-client | Weiwei Yang | Weiwei Yang | | | Better option for curl in hadoop-auth-examples | Minor | documentation | Jingguo Yao | Andras Bokor | | | Lower the log level when can't get postOpAttr | Minor | nfs | Anuhan Torgonshar | Anuhan Torgonshar | | | hadoop jar command has no help argument | Major | common | Adam Antal | Adam Antal | | | [UI2] Add a link in docs's top page | Major | documentation, yarn-ui-v2 | Wanqiang Ji | Wanqiang Ji | | | [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl | Minor |

| Shweta | Shweta | | | Ability to disallow safemode NN to become active | Major | auto-failover | Xiao Liang | Xiao Liang | | | Missing Space in Client Error Message | Minor | hdfs-client | David Mollitor | Shweta | | | Update HAState.java to fix typos. | Trivial | namenode | bianqi | Nikhil Navadiya | | | Review Logging of BlockPlacementPolicyDefault | Minor | . | David Mollitor | David Mollitor | | | hadoop-yarn-applications-catalog-webapp doesn't respect mvn test -D parameter | Major | . | Wei-Chiu Chuang | Eric Yang | | | HttpExceptionUtils#validateResponse swallows exceptions | Trivial | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add the documents of swebhdfs configurations on the client side | Major | documentation, webhdfs | Takanobu Asanuma | Takanobu Asanuma | | | Cost-Based RPC FairCallQueue | Major | ipc, namenode | Erik Krogen | Christopher Gregorian | | | Add more fine-grained processing time metrics to the RPC layer | Minor | ipc | Christopher Gregorian | Erik Krogen | | | Ability to tell HDFS client not to request KMS Information from NameNode | Major | common, kms | Greg Senia | | | | Allow authoritative mode on a subdirectory | Major | fs/s3 | Sean Mackrory | Sean Mackrory | | | Support configurable MIN\LOG\ROLLING\_INTERVAL | Minor | log-aggregation, nodemanager, yarn | Adam Antal | Adam Antal | | | Making md5 computing being in parallel with image loading | Major | . | zhouyingchao | Lisheng Sun | | | Allow authoritative mode on non-qualified paths | Major | fs/s3 | Sean Mackrory | Sean Mackrory | | | Reduce useless #getNumLiveDataNodes call in SafeModeMonitor | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | When decommissioning a node, log remaining blocks to replicate periodically | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | DistributedShell cannot specify LogAggregationContext | Major | distributed-shell, log-aggregation, yarn | Adam Antal | Adam Antal | | | GPU auto-discovery script runs even when the resource is given by hand | Major | yarn | Adam Antal | Adam Antal | | | Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource | Major | . | Szilard Nemeth | Peter Bacsko | | | Create more tests to verify GpuDeviceInformationParser | Major | . | Szilard Nemeth | Peter Bacsko | | | Fair Scheduler configuration defaults are not documented in case of min and maxResources | Major | docs, documentation, fairscheduler, yarn | Adam Antal | Adam Antal | | | DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set. | Major | . | Jinglun | Jinglun | | | Remove unnecessary search in INodeDirectory.addChild during image loading | Major | namenode | zhouyingchao | Lisheng Sun | | | Remove useless log in IOUtils.java and ExceptionDiags.java | Minor | . | Lisheng Sun | Lisheng Sun | | | The console log is noisy when using DNSDomainNameResolver to resolve NameNode. | Minor | . | Akira Ajisaka | Kevin Su | | | NNBench should support multi-cluster access | Major | benchmarks | Chen Zhang | Chen Zhang | | | Increase ipc.maximum.data.length default from 64MB to 128MB | Major | ipc | Wei-Chiu Chuang | Siyao Meng | | | Expose total number of DT in JMX for Namenode | Major | . | Fengnan Li | Fengnan Li | | | Avoid repeated calls to the listOpenFiles function | Minor | namenode, performance | HuangTao | HuangTao | | | WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response | Major |

| Siyao Meng | Siyao Meng | | | Use Configured in GpuDiscoverer and FpgaDiscoverer | Major | nodemanager, yarn | Adam Antal | Adam Antal | | | Remove commented code block from the beginning of TestDefaultContainerExecutor | Trivial | . | Szilard Nemeth | Vidura Bhathiya Mudalige | | | Change the logging level form DEBUG to ERROR for RuntimeErrorException in JMXJsonServlet | Major | common | Anuhan Torgonshar | Anuhan Torgonshar | | | Container-executor.c duplicates messages to stdout | Major | nodemanager, yarn | Adam Antal | Peter Bacsko | | | TestGpuResourceHandler / TestFpgaResourceHandler should be renamed | Major | . | Szilard Nemeth | Kevin Su | | | HealthMonitor connection retry times should be configurable | Major | . | Chen Zhang | Chen Zhang | | | Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du | Major | datanode, performance | Lisheng Sun | Lisheng Sun | | | DataNode#DataTransfer should be named | Minor | . | igo Goiri | igo Goiri | | | Add the warn log when the volume available space isn't enough | Minor | hdfs | liying | liying | | | Missing spaces in NMClientImpl | Trivial | client | Charles Xu | Charles Xu | | | Document the usage of the new Balancer "asService" parameter | Major | . | Chen Zhang | Chen Zhang | | | Change Log Level to warn in SlotReleaser | Minor | . | Lisheng Sun | Lisheng Sun | | | Remove unused configuration dfs.min.replication | Trivial | . | Wei-Chiu Chuang | CR Hota | | | NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. | Minor | namenode | Xudong Cao | Xudong Cao | | | Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM | Trivial | . | Szilard Nemeth | Gergely Pollak | | | Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily | Major | . | Szilard Nemeth | Gergely Pollak | | | Create an object for cgroups mount enable and cgroups mount path as they belong together | Minor | . | Szilard Nemeth | Gergely Pollak | | | Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently | Minor | . | Szilard Nemeth | Adam Antal | | | Logging fileSize of log files under NM Local Dir | Major | log-aggregation | Prabhu Joseph | Prabhu Joseph | | | OIV: print out storage policy id in oiv Delimited output | Minor | tools | Wang, Xinglong | Wang, Xinglong | | | [UI2] Fix error message for logs when ATSv2 is offline | Major | yarn-ui-v2 | Zoltan Siegl | Zoltan Siegl | | | AbstractLivelinessMonitor add serviceName to PingChecker thread | Minor | . | Bibin Chundatt | Bilwa S T | | | [UI2] yarn-container-log URI need to be encoded to avoid potential misuses | Major | . | Prabhu Joseph | Akhil PB | | | Update how exceptions are handled in NetUtils | Minor | . | Lisheng Sun | Lisheng Sun | | | Support "Pending Resource" metrics in RM's RESTful API | Major | . | Zhankun Tang | Prabhu Joseph | | | NM State store ResourceMappings serialization are tested with Strings instead of real Device objects | Major | . | Szilard Nemeth | Peter Bacsko | | | Turn on HDFS ACLs by default. | Major |

| Ajay Kumar | Siyao Meng | | | Edit log tailing fast-path should allow for backoff | Major | namenode, qjm | Erik Krogen | Erik Krogen | | | container working directory has group read permissions | Minor | yarn | Jim Brennan | Jim Brennan | | | Make DefaultAuditLogger class in FSnamesystem to Abstract | Major | . | hemanthboyina | hemanthboyina | | | More Clarity on Namenode UI Around Blocks and Replicas | Minor | . | Alan Jackoway | Siyao Meng | | | No test coverage for redefining FPGA / GPU resource types in TestResourceUtils | Major | . | Szilard Nemeth | Peter Bacsko | | | Make tests more easy to comprehend in TestGpuResourceHandler | Major | . | Szilard Nemeth | Peter Bacsko | | | Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager | Trivial | . | Szilard Nemeth | Peter Bacsko | | | Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes | Major | . | Adam Antal | Adam Antal | | | Skip YARNFeatureNotEnabledException from ClientRMService | Minor | resourcemanager | Prabhu Joseph | Prabhu Joseph | | | Regular code cleanup in TestResourcePluginManager | Major | . | Szilard Nemeth | Adam Antal | | | Increase ipc.server.listen.queue.size default from 128 to 256 | Major | . | Lisheng Sun | Lisheng Sun | | | Extract log aggregation related fields and methods from RMAppImpl | Major | . | Szilard Nemeth | Peter Bacsko | | | Add tests for GpuResourceAllocator and do minor code cleanup | Major | . | Szilard Nemeth | Peter Bacsko | | | Allow triggerBlockReport to a specific namenode | Major | datanode | Leon Gao | Leon Gao | | | Remove excess read lock for NetworkToplogy | Major | . | Wu Weiwei | Wu Weiwei | | | SpanReceiverInfo - Prefer ArrayList over LinkedList | Trivial | common | David Mollitor | David Mollitor | | | DistCp to support checksum validation when copy blocks in parallel | Major | tools/distcp | Kai Xie | Kai Xie | | | [KMS] Improve KeyAuthorizationKeyProvider#toString() | Trivial | . | Wei-Chiu Chuang | Yeliang Cang | | | Trivial test code update after HDFS-14687 | Trivial | ec | Wei-Chiu Chuang | Kevin Su | | | Report bad block to NN when transfer block encounter EIO exception | Major | datanode | Chen Zhang | Chen Zhang | | | HttpFS: LISTSTATUS response is missing HDFS-specific fields | Major | httpfs | Siyao Meng | Siyao Meng | | | Minor spell mistake in comment (PR#388) | Major | . | Wei-Chiu Chuang | | | | [SBN read] Reduce tailing overhead | Major | ha, namenode | Wei-Chiu Chuang | Ayush Saxena | | | Update Apache Yetus to 0.10.0 | Major | build | Akira Ajisaka | Akira Ajisaka | | | Provide LiveNode and DeadNode filter in DataNode UI | Major | . | Ravuri Sushma sree | hemanthboyina | | | Increase Balancer Defaults Further | Major | balancer & mover | Stephen O'Donnell | Stephen O'Donnell | | | Improve fsimage load time by writing sub-sections to the fsimage index | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Write lock held by metasave impact following RPC processing | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Log INFO mode if snapshot usage and actual usage differ | Major |

| CR Hota | CR Hota | | | RBF: Improve some RPC performance by using previous block | Minor | rbf | xuzq | xuzq | | | Create metric that sums total memory/vcores preempted per round | Major | capacity scheduler | Eric Payne | Manikandan R | | | Review getImageTxIdToRetain | Minor | namenode | David Mollitor | David Mollitor | | | Review Logging of NameNode Class | Minor | namenode | David Mollitor | David Mollitor | | | Remove low-level zookeeper test to be able to build Hadoop against zookeeper 3.5.5 | Major | test | Mate Szalay-Beko | Mate Szalay-Beko | | | Make DataNodePeerMetrics#minOutlierDetectionSamples configurable | Major | . | Lisheng Sun | Lisheng Sun | | | Ensure jar validation works on Windows. | Blocker | build | Brian Grunkemeyer | Brian Grunkemeyer | | | RBF: Add optional parameter -d for detailed listing of mount points. | Major | federation | Soumyapn | Ayush Saxena | | | Remove unnecessary if at EntityGroupFSTimelineStore#parseApplicationId | Minor | . | Prabhu Joseph | Prabhu Joseph | | | The StorageType quota and consume in QuotaFeature is not handled for rename | Major | . | Jinglun | Jinglun | | | Allow custom wrapped exception to be thrown by server if RPC call queue is filled up | Major | . | CR Hota | CR Hota | | | RBF: MountTableRefresherService should load cache when refresh | Major | . | xuzq | xuzq | | | Add queue capacity/maxcapacity percentage metrics | Major | . | Jonathan Hung | Shubham Gupta | | | Add more methods to WebHdfsTestUtil to support tests outside of package | Major | . | Chen Zhang | Chen Zhang | | | LazyPersistFileScrubber.run() should log meaningful warn message | Minor | namenode | Daniel Templeton | Daniel Green | | | Log more detail for slow RPC | Major | . | Chen Zhang | Chen Zhang | | | Print application tags in application summary | Major | . | Jonathan Hung | Manoj Kumar | | | ClusterMetrics to include AM allocation delay | Minor | . | Fengnan Li | Fengnan Li | | | Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. | Major | metrics, nodemanager, resourcemanager | zhuqi | zhuqi | | | Typo in analysesErrorMsg | Trivial | yarn | David Mollitor | Kevin Su | | | Print application submission context label in application summary | Major | . | Jonathan Hung | Manoj Kumar | | | Remove Unsupported SSL/TLS Versions from Docs/Properties | Minor | documentation, security | Daisuke Kobayashi | Daisuke Kobayashi | | | Fall back to configured queue ordering policy class name | Major | . | Jonathan Hung | Jonathan Hung | | | Add GitHub pull request template | Major | . | Akira Ajisaka | Akira Ajisaka | | | ReservationACLsTestBase fails with NPE | Minor | yarn | Ahmed Hussein | Ahmed Hussein | | | Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager | Minor | hdfs | David Mollitor | David Mollitor | | | Do Not Call Map containsKey In Conjunction with get | Minor | namenode | David Mollitor | hemanthboyina | | | Add Throttler for writing block | Minor | . | Lisheng Sun | Lisheng Sun | | | Fix some LGTM alerts | Minor | common | Malcolm Taylor | Malcolm Taylor | | | Support configure ZK\DTSM\ZK\KERBEROS\PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /\_HOST | Minor | common | luhuachao | luhuachao | | | Make buffer

BlockReaderRemote#newBlockReader#BufferedOutputStream configurable | Minor | . | Lisheng Sun | Lisheng Sun | | | Allow separate custom signing algorithms for S3 and DDB | Major | fs/s3 | Siddharth Seth | Siddharth Seth | | | Add submission context label to audit logs | Major | . | Jonathan Hung | Manoj Kumar | | | Review of Block.java | Minor | hdfs-client | David Mollitor | David Mollitor | | | Double Synchronization in BlockReportLeaseManager | Minor | . | David Mollitor | David Mollitor | | | RBF : Add Icon for ReadOnly False | Minor | . | hemanthboyina | hemanthboyina | | | Add assertion to TestDefaultNameNodePort#testGetAddressFromConf | Minor | hdfs | legend | Nikhil Navadiya | | | TestDelegationTokenRenewer throws NPE but tests pass | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable | Major | hdfs-client, libhdfs, native | Sahil Takiar | Sahil Takiar | | | Optimize FileSystemAccessService#getFileSystemConfiguration | Major | httpfs, performance | Lisheng Sun | Lisheng Sun | | | Track missing DFS operations in Statistics and StorageStatistics | Major | . | Ayush Saxena | Ayush Saxena | | | Allow a SignerInitializer to be specified along with a Custom Signer | Major | fs/s3 | Siddharth Seth | Siddharth Seth | | | RBF: Enable Parallel Test Profile for builds | Major | . | Ayush Saxena | Ayush Saxena | | | Upgrade hugo to the latest version in Dockerfile | Minor | build | Akira Ajisaka | Kevin Su | | | RBF: RouterQuotaUpdateService supports inherited rule. | Major | . | Jinglun | Jinglun | | | Add more tests to ratio method in TestResourceCalculator | Major | . | Szilard Nemeth | Zoltan Siegl | | | Use Relative URLS in Hadoop HDFS HTTP FS | Major | httpfs | David Mollitor | David Mollitor | | | Better input parsing for resource in allocation file | Minor | fairscheduler | Yufei Gu | Zoltan Siegl | | | Enable service mode for Docker containers on YARN | Major | . | Prabhu Joseph | Prabhu Joseph | | | Use Relative URLS in Hadoop HDFS RBF | Major | rbf | David Mollitor | David Mollitor | | | A log in NNThroughputBenchmark should change log level to "INFO" instead of "ERROR" | Major | . | Shen Yinjie | Shen Yinjie | | | Update netty4 to the latest 4.1.42 | Major | . | Wei-Chiu Chuang | Lisheng Sun | | | Review FSNameSystem editlog sync | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Upgrade Eclipse Jetty version to 9.4.x | Major | . | Yuming Wang | Siyao Meng | | | Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.6 in Hadoop | Major | . | Mate Szalay-Beko | Norbert Kalmr | | | Remove useless getRedundancyThread from BlockManagerTestUtil | Minor | . | Fei Hui | Fei Hui | | | Move Superuser Check Before Taking Lock For Encryption API | Major | . | Ayush Saxena | Ayush Saxena | | | Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing | Major | . | Ayush Saxena | Ayush Saxena | | | Add password check for credential provider | Major | . | hong dongdong | hong dongdong | | | Change the ICON of "Decommissioned & dead" datanode on

| Trivial | ui | Xieming Li | Xieming Li | | | Remove dead code from HealthMonitor | Minor | . | Fei Hui | Fei Hui | | | Use separate configs for free disk space checking for full and not-full disks | Minor | yarn | Jim Brennan | Jim Brennan | | | Refactor DFSNetworkTopology#isNodeInScope | Minor | . | Lisheng Sun | Lisheng Sun | | | ResourceManager JMX UI does not give HA State | Major | resourcemanager | Nishan Shetty | Rohith Sharma K S | | | [UI] Add Application Tag column to RM All Applications table | Major | . | Kinga Marton | Kinga Marton | | | Add getNumOfChildren() for interface InnerNode | Minor | . | Lisheng Sun | Lisheng Sun | | | RBF: Add metrics for async callers thread pool | Minor | rbf | Leon Gao | Leon Gao | | | Review of ArrayWritable | Minor | common | David Mollitor | David Mollitor | | | Add Timestamp for longest FSN write/read lock held log | Major | . | Chen Zhang | Chen Zhang | | | Tuning TaskRuntimeEstimator | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex | Minor | . | Lisheng Sun | Lisheng Sun | | | RBF: Update the quota in MountTable when calling setQuota on a MountTable src. | Major | . | Jinglun | Jinglun | | | Make FpgaDevice and GpuDevice classes more similar to each other | Major | . | Szilard Nemeth | Kevin Su | | | [UI2] Add Application tag to the app table and app detail page. | Major | . | Kinga Marton | Kinga Marton | | | UI: unifying the WebUI across different components. | Trivial | ui | Xieming Li | Xieming Li | | | Add LF for SetECPolicyCommand usage | Trivial | . | Fei Hui | Fei Hui | | | Add configuration to disable AM preemption | Major | fairscheduler | zhoukang | zhoukang | | | [Observer Node] Balancer should submit getBlocks to Observer Node when possible | Major | balancer & mover, hdfs | Erik Krogen | Erik Krogen | | | MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log | Major | metrics | Lukas Majercak | Lukas Majercak | | | Unify Logging in UserGroupInformation | Minor | common | David Mollitor | David Mollitor | | | Consider DataNode load when #getBlockLocation | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | RBF: Make tests use free ports | Major | . | igo Goiri | igo Goiri | | | RBF: getQuotaUsage() on mount point should return global quota. | Minor | . | Jinglun | Jinglun | | | Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml | Trivial | . | Wei-Chiu Chuang | Xieming Li | | | Queue mapping based on userid passed through application tag | Major | scheduler | Kinga Marton | Szilard Nemeth | | | Skip safemode if blockTotal is 0 in new NN | Trivial | namenode | Rajesh Balamohan | Xiaoqiao He | | | Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime() | Minor | . | Lisheng Sun | Lisheng Sun | | | Add getServerDefaults() support to HttpFS | Major |

| Kihwal Lee | hemanthboyina | | | Refactor QuotaUsage | Minor | common | David Mollitor | David Mollitor | | | RBF: Fix annotation in RouterAdmin | Trivial | . | Jinglun | Jinglun | | | Expose metrics for custom resource types in QueueMetrics | Major | . | Szilard Nemeth | Szilard Nemeth | | | Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter | Major | . | David Mollitor | Xieming Li | | | Code duplication in UserGroupMappingPlacementRule | Major | . | Szilard Nemeth | Kevin Su | | | Queue mapping based on userid passed through application tag: Change prefix to 'userid' | Major | scheduler | Szilard Nemeth | Szilard Nemeth | | | Code cleanup in TestNMLeveldbStateStoreService | Minor | . | Szilard Nemeth | Denes Gerencser | | | Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Reduce NameNode overview tab response time | Minor | namenode | HuangTao | HuangTao | | | Allow disabling Server Name Indication (SNI) for Jetty | Major | . | Siyao Meng | Aravindan Vijayan | | | Remove the invalid lock in ContainerExecutor | Major | . | Wanqiang Ji | Wanqiang Ji | | | Extract version numbers to head of pom.xml | Minor | build | Tamas Penzes | Tamas Penzes | | | FrameworkUploader: skip replication check entirely if timeout == 0 | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | Replace all MockRM submit method definitions with a builder | Minor | . | Szilard Nemeth | Szilard Nemeth | | | Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings | Major | . | Szilard Nemeth | Adam Antal | | | [SBN read] ZKFC should check the state before joining the election | Major | . | Fei Hui | Fei Hui | | | Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS\SESSION\TOKEN | Minor | documentation, fs/s3 | Mingliang Liu | Mingliang Liu | | | Keep the capacity of volume and reduce a system call | Minor | datanode | Yang Yun | Yang Yun | | | Guaranteed and max capacity queue metrics for custom resources | Major | . | Jonathan Hung | Manikandan R | | | Allow compact property description in xml in httpfs | Major | httpfs | Akira Ajisaka | Masatake Iwasaki | | | Provide a builder interface for FairScheduler allocations for use in tests | Major | fairscheduler | Karthik Kambatla | Adam Antal | | | Create improved decommission monitor implementation | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Optimize log information when DFSInputStream meet CannotObtainBlockLengthException | Major | dfsclient | Xiaoqiao He | Xiaoqiao He | | | RBF: getDatanodeReport() should return the latest update | Major | . | igo Goiri | igo Goiri | | | TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources | Major | capacity scheduler, test | Eric Payne | Eric Payne | | | Allow disabling app submission from REST endpoints | Major | . | Jonathan Hung | Jonathan Hung | | | CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. | Major | capacity scheduler, test | Eric Payne | Eric Payne | | | [UI] Finish Time is not correctly parsed in the RM Apps page | Minor | webapp | igo Goiri | igo Goiri | | | Update checkstyle to 8.26 and maven-checkstyle-plugin to

| Major | build | Andras Bokor | Andras Bokor | | | Add LOG when sendIBRs failed | Major | datanode | Fei Hui | Fei Hui | | | In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined | Critical | capacity scheduler | Eric Payne | Eric Payne | | | Install yarnpkg and upgrade nodejs in Dockerfile | Major | buid, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. | Major | erasure-coding, hdfs-client | lufei | lufei | | | BPServiceActor processes commands from NameNode asynchronously | Major | datanode | Xiaoqiao He | Xiaoqiao He | | | RBF: Make Router support storage type quota. | Major | . | Jinglun | Jinglun | | | Typo in RetryCache#waitForCompletion annotation | Trivial | namenode | Fei Hui | Fei Hui | | | DataNode.DataTransfer thread should catch all the expception and log it. | Major | datanode | Surendra Singh Lilhore | hemanthboyina | | | [SBN read] ObserverReadProxyProvider should throw InterruptException | Major | . | xuzq | xuzq | | | Recover data blocks from persistent memory read cache during datanode restarts | Major | caching, datanode | Feilong He | Feilong He | | | Add Tez to LimitedPrivate of ClusterStorageCapacityExceededException | Minor | common | Wang Yan | Wang Yan | | | Cache Admin and Quota Commands Should Check SuperUser Before Taking Lock | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: MountPoint Listing Should Return Flag Values Of Destination | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: GetServerDefaults Should be Cached At Router | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: Reuse ugi string in ConnectionPoolID | Major | . | Ayush Saxena | Ayush Saxena | | | Upgrade Kafka version in hadoop-kafka module | Major | build, security | Akira Ajisaka | Akira Ajisaka | | | dfs.client.server-defaults.validity.period.ms to support time units | Major | . | Ayush Saxena | Ayush Saxena | | | Typo in CapacityScheduler documentation: Runtime Configuration | Major | . | Szilard Nemeth | Kevin Su | | | Validate %primary\_group queue in CS queue manager | Major | . | Manikandan R | Manikandan R | | | Capacity scheduler: support u:user2:%secondary\_group queue mapping | Major | capacity scheduler, capacityscheduler | Manikandan R | Manikandan R | | | Purge log in KMS and HttpFS | Minor | httpfs, kms | Doris Gu | Doris Gu | | | RBF: Do not return FileNotFoundException when a subcluster is unavailable | Major | . | igo Goiri | igo Goiri | | | Refactor HAAdmin | Major | ha | Akira Ajisaka | Xieming Li | | | Add ability to know datanode staleness | Minor | datanode, logging, namenode | Ahmed Hussein | Ahmed Hussein | | | Improve error handling when application recovery fails with exception | Major | resourcemanager | Gergo Repas | Wilfred Spiegelenburg | | | YARN Container Phase 2 | Major | . | Eric Yang | Eric Yang | | | EC: Add getECTopologyResultForPolicies to DistributedFileSystem | Major |

| Ayush Saxena | Ayush Saxena | | | Allow expiration of cached locations in DFSInputStream | Minor | dfsclient | Ahmed Hussein | Ahmed Hussein | | | Use JUnit TemporaryFolder Rule in TestFileUtils | Minor | common, test | David Mollitor | David Mollitor | | | MRApp helpers block for long intervals (500ms) | Minor | mr-am | Ahmed Hussein | Ahmed Hussein | | | RM Renew Delegation token thread should timeout and retry | Major | . | CR Hota | Manikandan R | | | Cross origin request support for Job history server web UI | Critical | jobhistoryserver | Adam Antal | Adam Antal | | | Allow inheritance of max app lifetime / default app lifetime | Major | capacity scheduler | Eric Payne | Eric Payne | | | Support wildcard in CLASSPATH for libhdfs | Major | libhdfs | John Zhuge | Muhammad Samir Khan | | | Expose diagnostics in RMAppManager summary | Major | . | Jonathan Hung | Jonathan Hung | | | Use switch case for ProtoUtils#convertFromProtoFormat containerState | Major | . | Bibin Chundatt | Bilwa S T | | | Fix native build failure of hadoop-pipes on CentOS 8 | Major | tools/pipes | Masatake Iwasaki | Masatake Iwasaki | | | Test TestGroupsCaching fail if HashSet iterates in a different order | Minor | test | testfixer0 | testfixer0 | | | Introduce read write lock to Datanode | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Decrease lease hard limit | Minor | . | Eric Payne | hemanthboyina | | | RBF: Do not allow writes when a subcluster is unavailable for HASH\_ALL mount points. | Major | . | igo Goiri | igo Goiri | | | Remove obsolete validateTargetPath() from FrameworkUploader | Major | mrv2 | Adam Antal | Hudky Mrton Gyula | | | UIv2 build is broken in trunk | Critical | yarn, yarn-ui-v2 | Adam Antal | Adam Antal | | | Block scheduled counter never get decremet if the block got deleted before replication. | Major | 3.1.1 | Surendra Singh Lilhore | hemanthboyina | | | Support getting thread info from thread group for JvmMetrics to improve the performance | Major | metrics, performance | Tao Yang | Tao Yang | | | Supporting rack exclusion in countNumOfAvailableNodes in NetworkTopology | Major | net | igo Goiri | igo Goiri | | | InstrumentedLock should log lock queue time | Major | util | Stephen O'Donnell | Stephen O'Donnell | | | Upgrade objenesis to 2.6 | Major | build, timelineservice | Jason Darrell Lowe | Akira Ajisaka | | | Add option to disable rack local write preference | Major | balancer & mover, block placement, datanode, fs, hdfs, hdfs-client, namenode, nn, performance | Hari Sekhon | Ayush Saxena | | | Enable GcTimePercentage Metric in NameNode's JvmMetrics. | Minor | . | Jinglun | Jinglun | | | Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations | Major | . | Lisheng Sun | Lisheng Sun | | | Update netty to 4.1.42Final in yarn-csi | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path | Major | . | Kuhu Shukla | Kuhu Shukla | | | Reset LowRedundancyBlocks Iterator periodically | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Refresh BlockPlacementPolicy at runtime. | Major | . | Jinglun | Jinglun | | | HttpFS : Add Support for Storage Policy Satisfier | Major |

| hemanthboyina | hemanthboyina | | | Support to save replica cached files to other place and make expired time configurable | Minor | datanode | Yang Yun | Yang Yun | | | Add Unit test for queue ACL for both FS and CS | Major | scheduler | Kinga Marton | Kinga Marton | | | Update HdfsDesign.md to reduce ambiguity | Minor | documentation | Akshay Nehe | Akshay Nehe | | | Contract Tests to verify create, mkdirs and rename under a file is forbidden | Major | fs, fs/azure, fs/s3, fs/swift | Steve Loughran | Steve Loughran | | | DFSClient.rename() to unwrap ParentNotDirectoryException; define policy for renames under a file | Minor | fs | Steve Loughran | Steve Loughran | | | Batch listing of multiple directories to be an unstable interface | Major | fs | Steve Loughran | Steve Loughran | | | Extract version numbers to head of pom.xml (addendum) | Major | build | Tamas Penzes | Tamas Penzes | | | [Umbrella] Usability improvements for scheduler activities | Major | capacityscheduler | Tao Yang | Tao Yang | | | Prevent adding same DN multiple times in PendingReconstructionBlocks | Major | . | hemanthboyina | hemanthboyina | | | [SBN read] Change ObserverRetryOnActiveException log to debug | Minor | hdfs | Chen Liang | Chen Liang | | | Update hadoop-thirdparty dependency version to 1.0.0 | Major | . | Vinayakumar B | Vinayakumar B | | | ber-jira: S3A phase VI: Hadoop 3.3 features | Major | fs/s3 | Steve Loughran | Steve Loughran | | | The owner of folder can set quota for his sub folder | Minor | namenode | Yang Yun | Yang Yun | | | Make non-HA proxy providers pluggable | Major | . | Roger Liu | Roger Liu | | | Allow only hdfs superusers the ability to assign HDFS storage policies | Major | hdfs | Bob Cauthen | Siddharth Wagle | | | Add number of containers to RMAppManager summary | Major | . | Jonathan Hung | Jonathan Hung | | | Remove process command timing from BPServiceActor | Major | . | igo Goiri | Xiaoqiao He | | | Add a RMFailoverProxyProvider that does DNS resolution on failover | Major | . | Roger Liu | Roger Liu | | | RBFNamenodeHeartbeatService caused memory to grow rapidly | Major | . | xuzq | xuzq | | | Add button to go to the parent directory in the explorer | Major | . | igo Goiri | hemanthboyina | | | ABFS Streams to update FileSystem.Statistics counters on IO. | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | ber-JIRA: S3Guard Phase IV: Hadoop 3.3 features | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Mark Router FSCK unstable | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | Usage output of container-executor binary needs to include --http/--https argument | Minor | . | Siddharth Ahuja | Siddharth Ahuja | | | historyContext doesn't need to be a class attribute inside JobHistoryServer | Minor | jobhistoryserver | Siddharth Ahuja | Siddharth Ahuja | | | YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore | Major | . | Szilard Nemeth | Benjamin Teke | | | Create separate configuration for max global AM attempts | Major |

| Jonathan Hung | Bilwa S T | | | lock too long when fix inconsistent blocks between disk and in-memory | Major | datanode | Sean Chow | Sean Chow | | | When localizers fail due to resource timestamps being out, provide more diagnostics | Major | nodemanager | Steve Loughran | Siddharth Ahuja | | | Code cleanup in TestSchedConfCLI | Minor | . | Szilard Nemeth | Bilwa S T | | | Resources should be created with ResourceTypesTestHelper instead of TestUtils | Trivial | . | Szilard Nemeth | Andras Gyori | | | Code cleanup and improvements in ConfigurationStoreBaseTest | Minor | . | Szilard Nemeth | Benjamin Teke | | | Configurable max application tags and max tag length | Major | . | Jonathan Hung | Bilwa S T | | | Ignore AuthenticationFilterInitializer for KMSWebServer | Blocker | kms | Masatake Iwasaki | Masatake Iwasaki | | | Add explanation of unimplemented methods in InMemoryConfigurationStore | Major | . | Szilard Nemeth | Siddharth Ahuja | | | The suffix name of the unified compression class | Major | io | bianqi | bianqi | | | Code cleanup in ZKConfigurationStore | Minor | . | Szilard Nemeth | Andras Gyori | | | Code cleanup in QueueAdminConfigurationMutationACLPolicy | Major | . | Szilard Nemeth | Siddharth Ahuja | | | Adding Output Stream Counters in ABFS | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Code cleanup in LeveldbConfigurationStore | Minor | . | Szilard Nemeth | Benjamin Teke | | | TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup | Minor | . | Szilard Nemeth | Benjamin Teke | | | Code cleanup in LeveldbRMStateStore | Minor | . | Benjamin Teke | Benjamin Teke | | | Add isAbsoluteResource config for queue in scheduler response | Minor | scheduler | Prabhu Joseph | Prabhu Joseph | | | Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo | Major | . | Prabhu Joseph | Prabhu Joseph | | | Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes. | Major | rpc-server | Vinayakumar B | Vinayakumar B | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Uncaught exception in ContainersMonitorImpl during relaunch due to the process ID changing | Major | . | Shane Kumpf | Shane Kumpf | | | Specify minimum GCC version to avoid TLS support error in Build of hadoop-hdfs-native-client | Major | build, documentation, native | LiXin Ge | LiXin Ge | | | TestCGroupElasticMemoryController.testNormalExit() hangs on Linux | Major | nodemanager | Haibo Chen | Miklos Szegedi | | | RM metrics rest API throws GSSException in kerberized environment | Blocker | . | Kshitij Badani | Sunil G | | | Maven 3.3.3 unable to parse pom file | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Reuse Object Mapper in KMSJSONReader | Major | kms | Jonathan Turner Eagles | Jonathan Turner Eagles | | | Fix incorrect option used in description of InMemoryAliasMap | Minor | documentation | Yiqun Lin | Yiqun Lin | | | ApplicationAttempt stuck for ever due to UnknownHostException | Major | . | Bibin Chundatt | Bibin Chundatt | | | RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice | Major | . | Ayush Saxena | Ayush Saxena | | | mvn package -Pyarn-ui fails on JDK9 | Major |

| Akira Ajisaka | Vidura Bhathiya Mudalige | | | [UI2] Maintain constant horizontal application info bar for Application Attempt page | Major | yarn-ui-v2 | Yesha Vora | Akhil PB | | | RMWebServices returns only RUNNING apps when filtered with queue | Major | yarn | Prabhu Joseph | Szilard Nemeth | | | updateNodeResource does not support units for memory | Minor | . | igo Goiri | Manikandan R | | | Add null check for add-replica pool to avoid lock acquiring | Major | . | Yiqun Lin | Surendra Singh Lilhore | | | Skip javadoc build for auto-generated sources in hadoop-mapreduce-client | Minor | client, documentation | Mukul Kumar Singh | Mukul Kumar Singh | | | [UI2] Application Attempts tab is not shown correctly when there are no attempts | Major | yarn-ui-v2 | Charan Hebri | Akhil PB | | | Rumen crashes trying to handle MRAppMaster recovery events | Minor | tools/rumen | Jonathan Bender | Peter Bacsko | | | Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml | Minor | documentation | Toshihiro Suzuki | Toshihiro Suzuki | | | Reading values from Configuration before adding deprecations make it impossible to read value with deprecated key | Minor | conf | Szilard Nemeth | Zoltan Siegl | | | [UI2] Remove application tab from YARN Queue Page | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | [UI2] Lost nodes representation missing from Nodemanagers Chart | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | Service AM should set a finite limit on NM container max retries | Major | yarn-native-services | Suma Shivaprasad | Suma Shivaprasad | | | HDFS DelegationTokenFetcher can't print non-HDFS tokens in a tokenfile | Minor | tools | Steve Loughran | Steve Loughran | | | TestDataNodeVolumeFailure is Flaky | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: Remove FSCK from Router Web UI | Major | . | Fei Hui | Fei Hui | | | SLS tool fix node addition | Critical | . | Bibin Chundatt | Bibin Chundatt | | | YARN Service Client might not work correctly with RM REST API for Kerberos authentication | Blocker | . | Eric Yang | Eric Yang | | | TestDiskFailures.testLocalDirsFailures sometimes can fail on concurrent File modifications | Major | test, yarn | Antal Blint Steinbach | Antal Blint Steinbach | | | TestConfigurationDeprecation leaves behind a temp file, resulting in a license issue | Major | test | Robert Kanter | Ayush Saxena | | | Kerberos principal is needed when submitting a submarine job | Critical | . | Zac Zhou | Zac Zhou | | | TestDataNodeVolumeFailure#testTolerateVolumeFailuresAfterAddingMoreVolumes is flaky | Major | . | Ayush Saxena | Ayush Saxena | | | Yarn Service: discrepancy between hashcode and equals of ConfigFile | Minor | . | Chandni Singh | Chandni Singh | | | Copy of resource-types.xml is not deleted if test fails, causes other test failures | Major | yarn | Antal Blint Steinbach | Antal Blint Steinbach | | | RBF: Documentation should reflect right scripts for v3.0 and above | Major | . | CR Hota | CR Hota | | | Set HTTPOnly attribute to Cookie | Major | . | Chandni Singh | Chandni Singh | | | NM incorrectly logs container user as the user who sent a start/stop container request in its audit log | Major | nodemanager | Haibo Chen | Wilfred Spiegelenburg | | | Synchronization Issue With HashResolver | Minor | federation | David Mollitor | David Mollitor | | | Fix findbugs warning in trunk for FSImageFormatPBINode | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Web UI update to bootstrap-3.3.7 | Major | . | igo Goiri | igo Goiri | |

TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException | Major | common | Suma Shivaprasad | Suma Shivaprasad | | | Incompatible layout when generating FSImage | Major | . | igo Goiri | igo Goiri | | | HDFS bytes-default configurations should accept multiple size units | Major | . | Yiqun Lin | Yiqun Lin | | | Misleading log statement in NM when max retries is -1 | Minor | . | Chandni Singh | Chandni Singh | | | CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0 | Critical | tools/distcp | Ted Yu | Ted Yu | | | TestOfflineImageViewer crashes in windows | Major | . | Vinayakumar B | Vinayakumar B | | | Nodemanager restart docs should describe nodemanager supervised property | Major | documentation | Jason Darrell Lowe | Suma Shivaprasad | | | Renamed HADOOP\SECURITY\GROUP\SHELL\COMMAND\_TIMEOUT keys break compatibility | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix test-container-executor | Major | test | Robert Kanter | Robert Kanter | | | Fix lingering timeline collector after serviceStop in TimelineCollectorManager | Trivial | ATSv2 | Prabha Manepalli | Prabha Manepalli | | | Add JavaBeans Activation Framework API to LICENSE.txt | Major | . | Wei-Chiu Chuang | Akira Ajisaka | | | Some tests fail due to NoClassDefFoundError for OperatorCreationException | Critical | . | Takanobu Asanuma | Takanobu Asanuma | | | TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk | Major | . | Ayush Saxena | Ayush Saxena | | | MRAM web UI does not work with HTTPS | Major | mr-am | Alejandro Abdelnur | Robert Kanter | | | Yarn Service Upgrade: Update the swagger definition | Major | . | Chandni Singh | Chandni Singh | | | Trunk build fails to compile native on Windows | Blocker | native | Vinayakumar B | Vinayakumar B | | | Javadoc build fails in hadoop-yarn-csi | Major | . | Takanobu Asanuma | Weiwei Yang | | | Add service upgrade cancel and express examples to the service upgrade doc | Minor | . | Chandni Singh | Chandni Singh | | | TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently | Major | erasure-coding, test | Xiao Chen | Xiao Chen | | | RMContainerAllocator#handleJobPriorityChange expects application\_priority always | Major | . | Bibin Chundatt | Bilwa S T | | | DefaultOOMHandler should only pick running containers to kill upon oom events | Major | nodemanager | Haibo Chen | Haibo Chen | | | Delays are noticed to launch docker container | Major | . | Yesha Vora | dockerzhang | | | TestPendingReconstruction.testPendingAndInvalidate fails | Major | . | Ayush Saxena | Ayush Saxena | | | Fix findbugs warnings in hadoop-yarn-csi | Minor | . | Takanobu Asanuma | Weiwei Yang | | | CGroup-based strict container memory enforcement does not work with CGroupElasticMemoryController | Major | nodemanager | Haibo Chen | Haibo Chen | | | TestUpgradeDomainBlockPlacementPolicy is flaky | Major | . | Ayush Saxena | Ayush Saxena | | | HDFS OIV temporary dir deletes folder | Major | hdfs | Adam Antal | Adam Antal | | | DFSStripedOutputStream should implement both hsync methods | Critical | erasure-coding | Xiao Chen | Xiao Chen | | | Compilation fails with dependency convergence error for hbase.profile=2.0 | Blocker | . | Rohith Sharma K S | Rohith Sharma K S | | | Upgrade jquery datatable version references to

| Critical | . | Akhil PB | Akhil PB | | | Fix findbugs warnings in RegistryDNS.java | Major | . | Akira Ajisaka | Akira Ajisaka | | | LoadBasedRouterPolicy throws NPE in case of sub cluster unavailability | Minor | federation, router | Akshay Agarwal | Bilwa S T | | | TestHttpFSServerWebServer fails on Windows because of missing winutils.exe | Major | . | igo Goiri | igo Goiri | | | Update AWS Java SDK versions in NOTICE.txt | Major | . | Akira Ajisaka | Akira Ajisaka | | | Update JSch versions in LICENSE.txt | Major | . | Akira Ajisaka | Akira Ajisaka | | | Fix NPE when PROVIDED storage is missing | Major | . | igo Goiri | Virajith Jalaparti | | | Tolerate corrupted seen\_txid file | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. | Major | . | Sumana Sathish | Wangda Tan | | | Improve the debug message in CS#allocateContainerOnSingleNode | Trivial | . | Weiwei Yang | Zhankun Tang | | | RMStateStore contains large number of expired RMDelegationToken | Major | resourcemanager | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | DFSOutputStream close() throws exception on subsequent call after DataNode restart | Major | hdfs-client | Erik Krogen | Erik Krogen | | | NullPointerException when reaching max shuffle connections | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | Fix a parsing error for crossdomain.xml | Major | build, yarn-ui-v2 | Takanobu Asanuma | Takanobu Asanuma | | | Clarify documentation on how to enable cgroup support | Major | nodemanager | Yufei Gu | Zhankun Tang | | | ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3) | Major | erasure-coding | Xiao Chen | Kitti Nanasi | | | Allow HttpServer2 to discover resources in /static when symlinks are used | Major | . | igo Goiri | igo Goiri | | | Fix fair scheduler race condition in app submit and queue cleanup | Blocker | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Upgrade Maven Surefire plugin to 3.0.0-M1 | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | YARN Service keytab does not support s3, wasb, gs and is restricted to HDFS and local filesystem only | Major | yarn-native-services | Gour Saha | Gour Saha | | | NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Failed Storage Locations shows nothing in the Datanode Volume Failures | Major | . | Ayush Saxena | Ayush Saxena | | | create-release script should set max-cache-ttl as well as default-cache-ttl for gpg-agent | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | start-build-env.sh still creates an invalid /etc/sudoers.d/hadoop-build-${USER\_ID} file entry after HADOOP-15802 | Major | build | Akira Ajisaka | Akira Ajisaka | | | BlockDecompressorStream#decompress should not return -1 in case of IOException. | Major | . | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Inefficient Flush Logic in JobHistory EventWriter | Major | . | Zichen Sun | Zichen Sun | | | Exclude MD5 checksum files from release artifact | Critical | build | Akira Ajisaka | Akira Ajisaka | | | TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError | Major |

| Jason Darrell Lowe | Sushil Ks | | | TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky | Major | . | Zsolt Venczel | Zsolt Venczel | | | The config and log of gpg-agent are removed in create-release script | Major | build | Akira Ajisaka | Dinesh Chitlangia | | | NN UI is broken with IE11 | Minor | namenode, ui | Daisuke Kobayashi | Ayush Saxena | | | Fix error messages in HDFS-12716 | Minor | hdfs | Adam Antal | Ayush Saxena | | | Filter overlapping objenesis class in hadoop-client-minicluster | Minor | build | Xiaoyu Yao | Xiaoyu Yao | | | Bump up Atsv2 hbase versions | Major | . | Rohith Sharma K S | Vrushali C | | | AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty | Critical | . | Yang Wang | Yang Wang | | | Javadoc error in deviceplugin package | Major | . | Rohith Sharma K S | Zhankun Tang | | | SFTPConnectionPool connections leakage | Critical | fs | Mikhail Pryakhin | Mikhail Pryakhin | | | Inconsistency in get and put syntax if filename/dirname contains space | Minor | fs | vivek kumar | Ayush Saxena | | | dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException | Major | namenode | Vinayakumar B | Vinayakumar B | | | TestEvents#testEvents fails | Critical | jobhistoryserver, test | Zhaohui Xin | Zhaohui Xin | | | Yarn Service Upgrade: Instance state changes from UPGRADING to READY without performing a readiness check | Critical | . | Chandni Singh | Chandni Singh | | | If a task stucks before its first heartbeat, it never timeouts and the MR job becomes stuck | Major | . | Ankit Malhotra | Zhaohui Xin | | | Containers fail during creating a symlink which started with hyphen for a resource file | Minor | . | Oleksandr Shevchenko | Oleksandr Shevchenko | | | LogsCLI should contact ATSv2 for "-am" option | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | Fix ITestDynamoDBMetadataStore test error issues | Major | fs/s3 | Gabor Bota | Gabor Bota | | | Log aggregation changes to handle filesystems which do not support setting permissions | Major | log-aggregation | Suma Shivaprasad | Suma Shivaprasad | | | PlacementRule interface should be for all YarnSchedulers | Major | . | Bibin Chundatt | Bibin Chundatt | | | YARN Resource Manager is running OOM because of leak of Configuration Object | Major | yarn-native-services | Eric Yang | Eric Yang | | | Fix the incorrect trailing slash deletion in constructor method of CGroupsHandlerImpl | Major | . | Zhankun Tang | Zhankun Tang | | | mapred-site.xml is misformatted in single node setup document | Major | documentation | Akira Ajisaka | Zhaohui Xin | | | Terminate the namenode when failed to start log segment | Critical | . | Ayush Saxena | Ayush Saxena | | | Upgrade plexus-utils from 2.0.5 to 3.1.0 | Major | security | Akira Ajisaka | Akira Ajisaka | | | Upgrade Curator version to 2.13.0 to fix ZK tests | Major | . | Jason Darrell Lowe | Akira Ajisaka | | | NM and service AM don't have updated status for reinitialized containers | Critical |

| Billie Rinaldi | Chandni Singh | | | Fix race condition between move app and queue cleanup in Fair Scheduler | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Ratio calculation of ResourceCalculator implementations could return NaN | Major | . | Szilard Nemeth | Szilard Nemeth | | | TestFairScheduler#testChildMaxResources is flaky | Major | . | Szilard Nemeth | Szilard Nemeth | | | FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs | Minor | . | OrDTesters | OrDTesters | | | FairScheduler should not parse negative maxResources or minResources values as positive | Major | fairscheduler | Sen Zhao | Szilard Nemeth | | | TestMaintenanceState fails with ArrayIndexOutOfBound Exception | Major | . | Ayush Saxena | Ayush Saxena | | | [UI2] log service redirect url need to support user name | Major | webapp, yarn-ui-v2 | Sunil G | Akhil PB | | | Add ldap.bind.password.alias in LdapGroupsMapping to distinguish aliases when using multiple providers through CompositeGroupsMapping | Major | common | Lukas Majercak | Lukas Majercak | | | TestPred fails in Trunk | Major | . | Ayush Saxena | Ayush Saxena | | | Doc typo in PluggableShuffleAndPluggableSort.md | Minor | documentation | Zhaohui Xin | Zhaohui Xin | | | TestBlockStorageMovementAttemptedItems#testNoBlockMovementAttemptFinishedReportAdded fails sporadically in Trunk | Major | . | Ayush Saxena | Ayush Saxena | | | RequestHedgingProxyProvider can throw NullPointerException when failover due to no lock on currentUsedProxy | Major | hdfs-client | Yuxuan Wang | Yuxuan Wang | | | LevelDBCacheTimelineStore in ATS 1.5 leaks native memory | Major | timelineserver | Tarun Parimi | Tarun Parimi | | | Service Upgrade: With default readiness check, the status of upgrade is reported to be successful prematurely | Major | . | Chandni Singh | Chandni Singh | | | Ls command with e option fails when the filesystem is not HDFS | Major | erasure-coding, tools | Takanobu Asanuma | Takanobu Asanuma | | | Fix itemization in YARN service quickstart document | Minor | documentation | Akira Ajisaka | Ayush Saxena | | | Ls with -e option not giving the result in proper format | Major | . | Soumyapn | Shubham Dewan | | | In NameNode UI under DataNode tab ,the Capacity column is Non-Aligned | Minor | . | Shubham Dewan | Shubham Dewan | | | In-Maintenance ICON is missing in datanode info page | Major | datanode | Harshakiran Reddy | Ranith Sardar | | | [SPS] Remove the -w parameter from the -satisfystoragepolicy usage | Major | . | Ayush Saxena | Ayush Saxena | | | MapReduce example wordmedian should handle generic options | Major | . | Fei Hui | Fei Hui | | | Shutdown NM may cause NPE when opportunistic container scheduling is enabled | Critical | . | lujie | lujie | | | Fetching yarn logs fails for long running application if it is not present in timeline store | Major | log-aggregation | Tarun Parimi | Tarun Parimi | | | KMS client uses wrong UGI after HADOOP-14445 | Blocker | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix NetworkTopology chooseRandom function to support excluded nodes | Major | . | Sihai Ke | Sihai Ke | | | AliyunOSS: bring fixes back from HADOOP-15671 | Blocker | fs/oss | wujinhu | wujinhu | | | Fix TestRMAdminCLI#testHelp | Major | resourcemanager, test | Ayush Saxena | Ayush Saxena | |

TestSecureLogins#testValidKerberosName fails | Major | security | Akira Ajisaka | Akira Ajisaka | | | FairShare calculation broken for large values after YARN-8833 | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fix intermittent failure of TestNameNodeMetrics | Major | . | Ayush Saxena | Ayush Saxena | | | Add BlockLocation.isStriped() to determine if block is replicated or Striped | Major | hdfs | Shweta | Shweta | | | Avoid potential integer overflow when computing fair shares | Major | fairscheduler | liyakun | liyakun | | | TestSSLFactory#testServerWeakCiphers sporadically fails in precommit builds | Major | security, test | Jason Darrell Lowe | Akira Ajisaka | | | Upload and Create button doesn't get enabled after getting reset. | Major | . | Ayush Saxena | Ayush Saxena | | | DecayRpcScheduler decay thread should run as a daemon | Major | ipc | Erik Krogen | Erik Krogen | | | NPE in ITestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is not set | Major | fs/s3 | Adam Antal | Gabor Bota | | | Fix NPE in AbstractYarnScheduler#updateNewContainerInfo | Major | test | Akira Ajisaka | Akira Ajisaka | | | [UI2] YARN UI2 page loading failed due to js error under some time zone configuration | Critical | webapp | collinma | collinma | | | Fix typos in yarn-default.xml | Trivial | documentation | Rahul Padmanabhan | Rahul Padmanabhan | | | Invalid event: REGISTERED and LAUNCH\_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM | Critical | . | lujie | lujie | | | EC: Native XOR decoder should reset the output buffer before using it. | Major | ec, hdfs | Surendra Singh Lilhore | Ayush Saxena | | | NPE in service AM when failed to launch container | Major | yarn-native-services | kyungwan nam | kyungwan nam | | | RM fails to start if absolute resource is specified for partition capacity in CS queues | Blocker | yarn | Jiandan Yang | Jiandan Yang | | | ZKFC should catch exception when ha configuration missing | Major | hdfs | Fei Hui | Fei Hui | | | DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL | Major | common, kms | Xiaoqiao He | Xiaoqiao He | | | EC: Ls -e throw NPE when directory ec policy is disabled | Major | . | Surendra Singh Lilhore | Ayush Saxena | | | RM nodes web page can not display node info | Blocker | yarn | Jiandan Yang | Jiandan Yang | | | When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER\THEN\MAX\_ALLOCATION) | Critical | . | Zhankun Tang | Zhankun Tang | | | [UI2] Flow Run End Time shows 'Invalid date' | Major | . | Charan Hebri | Akhil PB | | | -Ddynamodb should be -Ddynamo in AWS SDK testing document | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Incorrect getSnapshottableDirListing() javadoc | Major | snapshots | Wei-Chiu Chuang | Dinesh Chitlangia | | | Print launchTime in ApplicationSummary | Major | . | Jonathan Hung | Jonathan Hung | | | Restart "NEVER" policy does not work with component dependency | Critical | . | Yesha Vora | Suma Shivaprasad | | | NM should ignore sending finished apps to RM during RM fail-over | Major | yarn | Jiandan Yang | Jiandan Yang | | | YARN AHS shows submitTime for jobs same as startTime | Minor |

| Prabhu Joseph | Prabhu Joseph | | | Lazy persist copy/ put fails with ViewFs | Major | fs | Hanisha Koneru | Hanisha Koneru | | | Build failure for -Dhbase.profile=2.0 | Blocker | . | Rohith Sharma K S | Rohith Sharma K S | | | GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way | Major | . | Szilard Nemeth | Szilard Nemeth | | | Fix the comment for getClass in Configuration | Trivial | . | Fengnan Li | Fengnan Li | | | TestRMAppAttemptTransitions is failing with an NPE | Critical | resourcemanager | Sunil G | lujie | | | TestDistributedShell fix timeout | Major | distributed-shell | Prabhu Joseph | Prabhu Joseph | | | libhdfs is not included in binary tarball | Critical | build, libhdfs | Akira Ajisaka | Akira Ajisaka | | | Checkpointer ignores configured time period \> 5 minutes | Minor | namenode | Timo Walter | Timo Walter | | | Disable speculative execution in TestDFSIO | Major | . | Kihwal Lee | Zhaohui Xin | | | SPNEGO+SSL Client Connections with HttpClient Broken | Major | build, security | Larry McCay | Larry McCay | | | "dfs.disk.balancer.max.disk.throughputInMBperSec" property is not working as per set value. | Major | diskbalancer | Ranith Sardar | Ranith Sardar | | | yarn container -status misses logUrl when integrated with ATSv2 | Major | . | Rohith Sharma K S | Abhishek Modi | | | NPE when executing a command yarn node -status or -states without additional arguments | Minor | client | Masahiro Tanaka | Masahiro Tanaka | | | OIV WebImageViewer: NPE when param op is not specified | Major | tools | Siyao Meng | Siyao Meng | | | [ATSv2] When passing a non-number as metricslimit, the error message is wrong | Trivial | api | Grant Sohn | Charan Hebri | | | Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04 | Blocker | . | Jack Bearden | Ayush Saxena | | | RMServerUtils does not count SHUTDOWN as an accepted state | Major | . | Kuhu Shukla | Kuhu Shukla | | | Distcp It should clear sub directory ACL before applying new ACL on it. | Major | tools/distcp | Ranith Sardar | Ranith Sardar | | | JournalNodeSyncer authentication is failing in secure cluster | Major | journal-node, security | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Distributed Shell client throws a NPE for a non-existent queue | Major | distributed-shell | Charan Hebri | Charan Hebri | | | Service AM support for decommissioning component instances | Major | . | Billie Rinaldi | Billie Rinaldi | | | Fix javadoc warnings in hadoop-aws | Minor | fs/s3 | Masatake Iwasaki | Masatake Iwasaki | | | EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. | Major | . | Harshakiran Reddy | Ayush Saxena | | | distcp / mv is not working on ftp | Major | fs | Fabian Zimmermann | | | | EC: NPE While Listing EC Policy For A Directory Following Replication Policy. | Major | erasure-coding | Souryakanta Dwivedy | Ayush Saxena | | | Remove unnecessary block file exists check

FsDatasetImpl#getBlockInputStream() | Major | datanode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link | Major | yarn-ui-v2 | Sumana Sathish | Akhil PB | | | [UI2] Fix label typo in Cluster Overview page | Trivial | yarn-ui-v2 | Charan Hebri | Charan Hebri | | | fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240 | Blocker | fairscheduler, scheduler | Akira Ajisaka | Wilfred Spiegelenburg | | | Fix the unit of yarn.service.am-resource.memory in the document | Minor | documentation, yarn-native-services | Masahiro Tanaka | Masahiro Tanaka | | | Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference | Minor | documentation | Szilard Nemeth | Adam Antal | | | Fix Spelling Mistakes - DECOMISSIONED | Trivial | . | David Mollitor | David Mollitor | | | DataXceiverServer May Double-Close PeerServer | Minor | datanode | David Mollitor | David Mollitor | | | [Timeline Server] Sorting based on FinalStatus shows pop-up message | Minor | timelineserver | Nallasivan | Bilwa S T | | | hdfs dfsadmin -metasave metasave\_test results NPE | Major | hdfs | Shweta | Shweta | | | Fix checkstyle issues in BlockLocation's method javadoc | Trivial | . | Shweta | Shweta | | | Enable more reliable SSL connection reuse | Major | common | Daryn Sharp | Daryn Sharp | | | Fix compilation issue of handling typedef an existing name by gcc compiler | Blocker | . | Wei-Chiu Chuang | Zhankun Tang | | | Avoid allocating opportunistic containers to previous/removed/non-exist application attempt | Critical | . | lujie | lujie | | | Handle exceptions with parsing user defined GPU devices in GpuDiscoverer | Major | . | Szilard Nemeth | Szilard Nemeth | | | [SBN Read] Race condition in ObserverReadProxyProvider | Major | hdfs, namenode | Erik Krogen | Erik Krogen | | | Misc. bug fixes for KMS Benchmark | Major | kms | Wei-Chiu Chuang | George Huang | | | libhdfs hdfsRead copies entire array even if its only partially filled | Major | hdfs-client, libhdfs, native | Sahil Takiar | Sahil Takiar | | | Avoid repeated YarnConfiguration#timelineServiceV2Enabled check | Major | . | Bibin Chundatt | Prabhu Joseph | | | Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler | Major | capacityscheduler | Tao Yang | Tao Yang | | | RM Web UI v1 does not show custom resource allocations for containers page | Major | . | Szilard Nemeth | Szilard Nemeth | | | updatePriority is blocked when using FairScheduler | Major | . | Jiandan Yang | Jiandan Yang | | | ViewFs: Correct error message for read only operations | Minor | federation | hu xiaodong | hu xiaodong | | | In ipc.Client, put a new connection could happen after stop | Major | ipc | Tsz-wo Sze | Tsz-wo Sze | | | ApplicationHistoryManagerImpl#getApplications doesn't honor filters | Minor | resourcemanager, yarn | Lantao Jin | Lantao Jin | | | TestRMRestart hangs due to a deadlock | Major | . | Prabhu Joseph | Prabhu Joseph | | | RMContainerImpl:Invalid event: ACQUIRED at KILLED | Major | . | lujie | lujie | | | Resources#multiplyAndRoundUp does not consider Resource Types | Major |

| Szilard Nemeth | Szilard Nemeth | | | FilterFileSystem doesn't wrap all create() or new builder calls; may skip CRC logic | Blocker | fs | Steve Loughran | Steve Loughran | | | hadoop-mapreduce-client-app build not converging due to transient dependencies | Major | build | Steve Loughran | Steve Loughran | | | YARN Service Client does not work with SPNEGO when knox is configured | Major | yarn-native-services | Tarun Parimi | Billie Rinaldi | | | FSLeafQueue#computeMaxAMResource does not override zero values for custom resources | Major | . | Szilard Nemeth | Szilard Nemeth | | | Incorrect Format Debug Statement KMSACLs | Trivial | kms | David Mollitor | Charan Hebri | | | Fix TestDataNodeVolumeFailure | Major | . | Ayush Saxena | Ayush Saxena | | | Secondary NN Web UI displays nothing, and the console log shows moment is not defined. | Major | namenode | Rui Chen | Ayush Saxena | | | [SBN read] ObserverReadProxyProvider should sync with active txnID on startup | Major | tools | Wei-Chiu Chuang | Erik Krogen | | | fullBlockReportLeaseId should be reset after registering to NN | Critical | datanode | star | star | | | Timeline Server event handler threads locked | Major | ATSv2, timelineserver | Venkata Puneet Ravuri | Prabhu Joseph | | | checksumFS doesn't wrap concat(): concatenated files don't have checksums | Major | fs | Steve Loughran | Steve Loughran | | | [UI2] Show flow runs in the order of recently created time in graph widgets | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | hdfsOpenFile on HDFS causes unnecessary IO from file offset 0 | Major | hdfs-client, libhdfs | Todd Lipcon | Sahil Takiar | | | Reentrant lock() before try | Minor | yarn | Prabhu Joseph | Prabhu Joseph | | | Standby does not trigger edit log rolling when in-progress edit log tailing is enabled | Critical | . | Ekanth Sethuramalingam | Ekanth Sethuramalingam | | | Disable wildfly logs to the console | Major | fs/azure | Denes Gerencser | Denes Gerencser | | | Restore a comment line mistakenly removed in ProtobufRpcEngine | Major | . | Konstantin Shvachko | Fengnan Li | | | Datanode fails to start if any disk has errors during Namenode registration | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | CallQueue backoff bug fixes: doesn't perform backoff when add() is used, and doesn't update backoff when refreshed | Major | ipc | Erik Krogen | Erik Krogen | | | fix wrong command in TimelineServiceV2.md | Major | timelineserver | Runlin Zhang | Runlin Zhang | | | Modify HBase Liveness monitor log to debug | Minor | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | Inherited ACL permissions masked when parent directory does not exist (mkdir -p) | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Fix SSLFactory truststore reloader thread leak in URLConnectionFactory | Major | hdfs-client, webhdfs | Takanobu Asanuma | Takanobu Asanuma | | | Fix JNI exception handling issues in libhdfs | Major | hdfs-client, libhdfs, native | Sahil Takiar | Sahil Takiar | | | TestYarnNativeServices fails sporadically with bind address in use | Major | test, yarn-native-services | Prabhu Joseph | Prabhu Joseph | | | Fix typos SPENGO into SPNEGO | Trivial | documentation | okumin | okumin | | | KMSLoadBlanceClientProvider does not select token correctly | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | DistributedShell RelativePath is not removed at end | Minor | distributed-shell | Prabhu Joseph | Prabhu Joseph | | | Fix flaky junit test fair.TestAppRunnability after YARN-8967 | Minor | fairscheduler, test | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fix links to the developer mailing lists in

| Minor | documentation | Akira Ajisaka | Wanqiang Ji | | | new Path(String str) does not remove all the trailing slashes of str | Minor | fs | Akira Ajisaka | Akira Ajisaka | | | Fix errors in the checkstyle configration xmls | Major | build | Akira Ajisaka | Wanqiang Ji | | | QueueMetrics needs to be cleared before MockRM is initialized | Major | scheduler | Daniel Templeton | Peter Bacsko | | | OsSecureRandom very slow compared to other SecureRandom implementations | Major | security | Todd Lipcon | Siyao Meng | | | getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | YARN\RM\CONTAINER\_CREATED published twice to ATS | Major | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes | Minor | . | Wanqiang Ji | Wanqiang Ji | | | Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN | Major | . | igo Goiri | Fengnan Li | | | Queue resource leak after app fail for CapacityScheduler | Major | capacityscheduler | Tao Yang | Tao Yang | | | Javadoc correction for AccessControlList#buildACL | Minor | documentation | Bibin Chundatt | Yeliang Cang | | | Fix typo in AbstractService | Trivial | documentation | Mike Drob | Gabor Liptak | | | start-build-env.sh can consume all disk space during image creation | Minor | build | Craig Condit | Craig Condit | | | Fix INodeId javadoc | Major | documentation, namenode | Wei-Chiu Chuang | Adam Antal | | | Fix typo in KeyShell console | Minor | kms | hu xiaodong | hu xiaodong | | | DistCp documentation for command line options is misaligned. | Minor | documentation | Chris Nauroth | Masatake Iwasaki | | | [JDK 11] Build fails due to hard-coded target version in hadoop-yarn-applications-catalog-webapp | Major | build | Kei Kori | Kei Kori | | | Can't specify docker runtime through environment | Minor | nodemanager | caozhiqiang | caozhiqiang | | | JobHistoryServer leaks CLOSE\_WAIT tcp connections when using LogAggregationIndexedFileController | Major | log-aggregation | Tarun Parimi | Tarun Parimi | | | Stop decommission a rack of DataNodes caused NameNode fail over to standby | Major | namenode | yunjiong zhao | yunjiong zhao | | | Remove JUnit from LICENSE.txt | Major | . | Akira Ajisaka | Kei Kori | | | NodeManager native build shouldn't link against librt on macOS | Major | nodemanager | Siyao Meng | Siyao Meng | | | Race condition in RM for publishing container events vs appFinished events causes NPE | Critical | . | Rohith Sharma K S | Prabhu Joseph | | | NodeManager native build fails due to getgrouplist not found on macOS | Major | nodemanager | Ewan Higgs | Siyao Meng | | | Fix findbugs warnings in hadoop-yarn-server-resourcemanager module | Minor | . | Tao Yang | Tao Yang | | | Configuration#getTimeDuration is not consistent between default value and manual settings. | Major | . | star | star | | | TestTrySendErrorReportWhenNNThrowsIOException fails in trunk | Major | . | Ayush Saxena | Ayush Saxena | | | ObserverReadProxyProvider is unable to properly fetch HAState from Standby NNs | Major | ha, nn | Erik Krogen | Erik Krogen | | | Apps pending metric incorrect after moving app to a new queue | Minor | . | Billie Rinaldi | Abhishek Modi | | | TestApplicationMasterServiceFair\>ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent | Minor |

| Prabhu Joseph | Prabhu Joseph | | | TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent | Minor | capacityscheduler, test | Prabhu Joseph | Prabhu Joseph | | | Change getDeclaredMethods to getMethods in FederationClientInterceptor#invokeConcurrent() | Major | federation | Shen Yinjie | Shen Yinjie | | | Remove stale eclipse templates | Minor | . | Akira Ajisaka | Wanqiang Ji | | | Dynamic refresh Disk configuration results in overwriting VolumeMap | Blocker | . | xuzq | xuzq | | | yarn.nodemanager.remote-app-log-dir structure is not scalable | Major | log-aggregation | Prabhu Joseph | Prabhu Joseph | | | RM UI progress column is of wrong type | Minor | yarn | Ahmed Hussein | Ahmed Hussein | | | Federation RMs starting up at the same time can give duplicate application IDs | Minor | . | Young Chen | Young Chen | | | Fix typo in OfflineEditsVisitorFactory | Major | hdfs | bianqi | bianqi | | | NPE while DN is shutting down | Major | . | lujie | lujie | | | TestAHSWebServices and TestLogsCLI test case failures | Major | log-aggregation, test | Prabhu Joseph | Prabhu Joseph | | | Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled | Major | capacityscheduler | Tao Yang | Tao Yang | | | [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8 | Major | test | Siyao Meng | Adam Antal | | | AuthenticationFilterInitializer doc has speudo instead of pseudo | Trivial | auth, documentation | Prabhu Joseph | Prabhu Joseph | | | Typos in Docker documentation | Major | . | Szilard Nemeth | Charan Hebri | | | [JDK 11] Build fails without specifying -Djavac.version=11 | Major | . | Akira Ajisaka | Akira Ajisaka | | | DistributedShell does not release container when failed to localize at launch | Major | . | Prabhu Joseph | Prabhu Joseph | | | With S3A Filesystem, Long Running services End up Doing lot of GC and eventually die | Major | common, fs/s3, metrics | Rajat Khandelwal | Rajat Khandelwal | | | AppBlock ignores full qualified class name of PseudoAuthenticationHandler | Minor | . | Prabhu Joseph | Prabhu Joseph | | | [UI2] Fair scheduler queue view page does not show actual capacity | Major | fairscheduler, yarn-ui-v2 | Zoltan Siegl | Zoltan Siegl | | | Scheduler Page does not display the right page by query string | Major | capacity scheduler, resourcemanager, webapp | Wanqiang Ji | Wanqiang Ji | | | NetworkTopology#getWeightUsingNetworkLocation return unexpected result | Major | net | Xiaoqiao He | Xiaoqiao He | | | TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config | Major | log-aggregation | Adam Antal | Adam Antal | | | Crash when using libhdfs with bad classpath | Major | . | Todd Lipcon | Sahil Takiar | | | YarnConfiguration areNodeLabel enabled is costly in allocation flow | Critical | . | Bibin Chundatt | Bilwa S T | | | NPE in FsUrlConnection | Major | hdfs-client | Karthik Palanisamy | Karthik Palanisamy | | | TimelineEntity DAO has java.util.Set interface which JAXB can't handle | Major | timelineservice | Prabhu Joseph | Prabhu Joseph | | | Distcp fails in Hadoop 3.x when

source webhdfs url has special characters in hdfs file path | Major | webhdfs | Srinivasu Majeti | Srinivasu Majeti | | | Fix TestYarnConfigurationFields testcase failing | Major | test, yarn-native-services | Prabhu Joseph | Prabhu Joseph | | | Treat container scheduler kill exit code as a task attempt killing event | Major | applicationmaster, mr-am, mrv2 | Wanqiang Ji | Wanqiang Ji | | | mapreduce.task.timeout=0 configuration used to disable timeout doesn't work | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Remove the defunct startKdc profile from hadoop-common | Major | test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Bucket Directories as part of ATS done accumulates | Major | . | Prabhu Joseph | Prabhu Joseph | | | DistributedShell job with localization fails in unsecure cluster | Major | distributed-shell | Prabhu Joseph | Prabhu Joseph | | | Log Aggregation testcases failing | Major | log-aggregation, test | Prabhu Joseph | Prabhu Joseph | | | NameNode StartupProgress continues to report edit log segments after the LOADING\_EDITS phase is finished | Major | namenode | Erik Krogen | Erik Krogen | | | Fix JavaDoc error in TestSchedulerOvercommit | Minor | documentation, test | Wanqiang Ji | Wanqiang Ji | | | Fix typos in ResourceModel.md | Trivial | documentation | leiqiang | leiqiang | | | webhdfs that connect secure hdfs should not use user.name parameter | Minor | webhdfs | KWON BYUNGCHANG | KWON BYUNGCHANG | | | Fix ASF License check in pom.xml | Major | . | Wanqiang Ji | Akira Ajisaka | | | ONE\_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes) | Major | . | Shen Yinjie | Ayush Saxena | | | Fix yetus-wrapper not working when HADOOP\YETUS\VERSION \>= 0.9.0 | Major | yetus | Wanqiang Ji | Wanqiang Ji | | | Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines | Major | timelineservice | Prabhu Joseph | Prabhu Joseph | | | Suppress setReplication logging. | Major | . | Kihwal Lee | Kihwal Lee | | | ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled | Major | yarn | Zhankun Tang | Zhankun Tang | | | EntityGroupFSTimelineStore fails to init LevelDBCacheTimelineStore | Major | timelineserver | Prabhu Joseph | Prabhu Joseph | | | Fix NPE in NodeManager#serviceStop on startup failure | Minor | . | Bilwa S T | Bilwa S T | | | [UI2] Active User info missing from UI2 | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | [UI2] NM hostnames not displayed correctly in Node Heatmap Chart | Major | . | Charan Hebri | Akhil PB | | | Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers | Major | capacityscheduler | Tao Yang | Tao Yang | | | FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource() | Major | nodemanager | Peter Bacsko | Peter Bacsko | | | Aggregate Resource Allocation for each job is not present in ATS | Major | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | Support self-adaption width for columns of containers table on app attempt page | Minor | webapp | Tao Yang | Tao Yang | | | Stop all DataNodes may result in NN terminate | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | The exception classes in some throw statements do not accurately describe why they are thrown | Minor | . | Haicheng Chen | Ayush Saxena | | | Potential NPE when instantiating FairCallQueue metrics | Major | ipc | Erik Krogen | Erik Krogen | | | Move Server logging of StatedId inside receiveRequestState() | Major |

| Konstantin Shvachko | Shweta | | | Fix missing break statement in ContainerScheduler#handle | Major | . | lujie | lujie | | | RMAppImpl#ranNodes not cleared on FinalTransition | Major | . | Bibin Chundatt | Bilwa S T | | | ContainerStatusPBImpl default execution type is not returned | Major | . | Bibin Chundatt | Bilwa S T | | | Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI | Major | datanode, namenode | Haibo Yan | Stephen O'Donnell | | | Update guava to 27.0-jre in hadoop-project | Critical | common, security | Gabor Bota | Gabor Bota | | | EditLogTailer shouldn't output full stack trace when interrupted | Major | . | igo Goiri | igo Goiri | | | Fix typo in FileSystemShell#test documentation | Trivial | documentation | Dinesh Chitlangia | Dinesh Chitlangia | | | Spelling Mistake "gloablly" | Trivial | hdfs-client | David Mollitor | David Mollitor | | | The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read | Major | hdfs-client | Zheng Hu | Zheng Hu | | | BlockReaderRemote.sendReadResult throws NPE | Major | hdfs-client | Wei-Chiu Chuang | Yuanbo Liu | | | Use Path instead of String in the TestHdfsAdmin.verifyOpenFiles() | Trivial | . | Oleg Danilov | Oleg Danilov | | | Typo in the TestDataNodeHotSwapVolumes.testAddOneNewVolume() | Trivial | . | Oleg Danilov | Oleg Danilov | | | Should put initializeProcessTrees method call before get pid | Critical | nodemanager | Wanqiang Ji | Wanqiang Ji | | | Pass correct DF usage to ReservedSpaceCalculator builder | Minor | . | Lukas Majercak | Lukas Majercak | | | Admin helper fails to prettify NullPointerExceptions | Major | . | Marton Elek | Marton Elek | | | Random failure of testListCorruptFilesCorruptedBlock | Major | test | Kihwal Lee | Zsolt Venczel | | | Journaled Edits Cache is not cleared when formatting the JN | Major | . | Ranith Sardar | Ranith Sardar | | | Appending to EC files crashes NameNode | Critical | erasure-coding | Surendra Singh Lilhore | Surendra Singh Lilhore | | | When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. | Major | . | Yicong Cai | Yicong Cai | | | DiskBalancer: no validations for Disk balancer commands | Major | diskbalancer | Harshakiran Reddy | Lokesh Jain | | | When nodePartition is not set in Placement Constraints, containers are allocated only in default partition | Major | capacity scheduler, scheduler | Tarun Parimi | Tarun Parimi | | | Synchronized at CompositeService#removeService is not required | Major | common | Prabhu Joseph | Prabhu Joseph | | | FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers | Major | balancer & mover, diskbalancer | liumi | liumi | | | DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. | Major | hdfs | guangyi lu | guangyi lu | | | When evictableMmapped or evictable size is zero, do not throw NoSuchElementException | Major | hdfs-client, performance | Zheng Hu | Lisheng Sun | | | NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X | Minor |

| Sherwood Zheng | Sherwood Zheng | | | Findbugs warning caused by HDFS-12487 | Minor | diskbalancer | Wei-Chiu Chuang | Xiaoqiao He | | | Build fails due to bad use of '\>' in javadoc | Critical | build | Kei Kori | Kei Kori | | | DecommissioningNodesWatcher cause memory leak | Blocker | . | Bibin Chundatt | Bilwa S T | | | HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty | Major | diskbalancer | Wei-Chiu Chuang | Xiaoqiao He | | | Fix WebAppUtils#getRMWebAppURLWithScheme ignores rm2 | Major | client | Prabhu Joseph | Prabhu Joseph | | | Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). | Critical | . | Paul Ward | Paul Ward | | | Fix typos in LocalityMulticastAMRMProxyPolicy and AbstractConfigurableFederationPolicy | Major | federation, yarn | hunshenshi | hunshenshi | | | HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. | Critical | . | Paul Ward | Paul Ward | | | Improve synchronisation in ProtoUtils#convertToProtoFormat block | Critical | . | Bibin Chundatt | Bibin Chundatt | | | AllocateResponse in FederationInterceptor lost applicationPriority | Major | federation | hunshenshi | hunshenshi | | | Namenode crashes with "RedundancyMonitor thread received Runtime exception" | Major | . | krishna reddy | Ayush Saxena | | | Fix UT failures in TestLeafQueue | Minor | . | Tao Yang | Tao Yang | | | First RMContext object is always leaked during switch over | Blocker | . | Bibin Chundatt | Bibin Chundatt | | | Property value Hard Coded in DNConf.java | Trivial | . | hemanthboyina | hemanthboyina | | | Fix javadoc warnings in hadoop-dynamometer | Minor | tools | Masatake Iwasaki | Masatake Iwasaki | | | Application fails in diskchecker when ReadWriteDiskValidator is configured. | Critical | nodemanager | Anuruddh Nayak | Bilwa S T | | | Exceptions are fatal to decommissioning monitor | Critical | namenode | Daryn Sharp | Xiaoqiao He | | | The JSON License is included in binary tarball via azure-documentdb:1.16.2 | Blocker | . | Akira Ajisaka | Sushil Ks | | | NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY | Major | hdfs | Jiandan Yang | Weiwei Yang | | | Fix checkstyle and findbugs warnings in hadoop-dynamometer | Minor | tools | Masatake Iwasaki | Erik Krogen | | | UI2 - No link to a queue on the Queues page for Fair Scheduler | Major | . | Charan Hebri | Zoltan Siegl | | | Add a regression test for HDFS-14323 | Minor | fs, test, webhdfs | Yuya Ebihara | Masatake Iwasaki | | | Misleading REM\_QUOTA value with snapshot and trash feature enabled for a directory | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown | Major | yarn | Antal Blint Steinbach | Adam Antal | | | UI2 - Fair scheduler queue apps page issues | Major | . | Charan Hebri | Zoltan Siegl | | | processMisReplicatedBlocks does not return correct processed count | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Fix Invalid event FINISHED\CONTAINERS\PULLED\BY\AM at NEW on NM restart | Major | . | krishna reddy | Bilwa S T | | | DistributedShell tests failed to bind to a local host name | Major | test | Ray Yang | Ray Yang | | | Wrong log message when finalizing the upgrade | Trivial |

| kyungwan nam | kyungwan nam | | | Fix the description of storageType option for space quota | Major | documentation, tools | Takanobu Asanuma | Takanobu Asanuma | | | MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test | Major | test | Masatake Iwasaki | Masatake Iwasaki | | | NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover | Minor | resourcemanager, test | Steve Loughran | Steve Loughran | | | Documentation correction in YarnApplicationSecurity | Trivial | . | Bibin Chundatt | Yousef Abu-Salah | | | Distcp can not preserve timestamp with -delete option | Major | . | ludun | ludun | | | TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build | Minor | test | Rushabh Shah | Kevin Su | | | UGI conf doesn't read user overridden configurations on RM and NM startup | Major | . | Jonathan Hung | Jonathan Hung | | | Update jackson-databind to 2.9.9.1 | Major | . | Wei-Chiu Chuang | Siyao Meng | | | NPE during secure namenode startup | Major | hdfs | Fengnan Li | Fengnan Li | | | NPE when trying to create an error message response of SASL RPC | Major | ipc | Steve Loughran | Steve Loughran | | | Regression: FileSystem cache lock parses XML within the lock | Major | fs | Gopal Vijayaraghavan | Gopal Vijayaraghavan | | | TestWebHdfsTimeouts Fails intermittently in trunk | Major | . | Ayush Saxena | Ayush Saxena | | | Fix comments on creating \_SUCCESS file. | Trivial | documentation | Mehul Garnara (MG) | Mehul Garnara (MG) | | | Enabling SSL within LdapGroupsMapping can break system SSL configs | Major | common, security | Erik Krogen | Erik Krogen | | | Native build fails on macos due to jlong in hdfs.c | Major | . | hunshenshi | hunshenshi | | | [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider | Major | . | Chao Sun | Chao Sun | | | Block remain in COMMITTED but not COMPLETE caused by Decommission | Major | . | Yicong Cai | Yicong Cai | | | RpcMetrics should not be retained forever | Critical | rpc-server | Zoltan Haindrich | Zoltan Haindrich | | | Javadoc for LdapAuthenticationHandler#ENABLE\START\TLS is wrong | Trivial | . | Ted Yu | Don Jeba | | | TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk | Major | . | Chen Zhang | Chen Zhang | | | ABFS: fix for Sever Name Indication (SNI) | Major | fs/azure | Thomas Marqardt | Sneha Vijayarajan | | | Result of crypto -listZones is not formatted properly | Major | . | hemanthboyina | hemanthboyina | | | QueueMetrics has incorrect metrics when labelled partitions are involved | Major | capacity scheduler | Muhammad Samir Khan | Muhammad Samir Khan | | | RBF: TestDisableRouterQuota failed because port 8888 was occupied | Minor | rbf | Wei-Chiu Chuang | Chao Sun | | | RBF: updateMountTableEntry shouldn't update mountTableEntry if targetPath not exist | Major | rbf | xuzq | xuzq | | | Fix broken current folder expansion during MR job start | Major | mrv2 | Adam Antal | Peter Bacsko | | | Fix default trash policy emptier trigger time correctly | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Connection thread's name should be updated after address changing is detected | Major | ipc | zhouyingchao | Lisheng Sun | | | Update scope of log4j in hadoop-auth to provided | Major | . | lqjack | lqjacklee | | |

HttpFSFileSystem#getErasureCodingPolicy always returns null | Major | httpfs | Siyao Meng | Siyao Meng | | | AuthenticationFilter should generate valid date format for Set-Cookie header regardless of default Locale | Minor | security | Cao Manh Dat | Cao Manh Dat | | | Modifying testcase testEnableAndDisableErasureCodingPolicy | Minor | . | chencan | chencan | | | ConcurrentModificationException in Configuration.overlay() method | Major | . | Oleksandr Shevchenko | Oleksandr Shevchenko | | | DefaultAuditLogger doesn't print CallerContext | Major | hdfs | xuzq | xuzq | | | WebHDFS throws "Error writing request body to server" instead of DSQuotaExceededException | Major | webhdfs | Erik Krogen | Simbarashe Dzinamarira | | | Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. | Minor | balancer & mover, documentation | Harshakiran Reddy | usharani | | | TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk | Minor | datanode | qiang Liu | qiang Liu | | | Update jackson-databind to 2.9.9.2 | Critical | . | Siyao Meng | Siyao Meng | | | Failed to add erasure code policies with example template | Minor | ec | Yuan Zhou | Yuan Zhou | | | Typo in documentation: Using FPGA On YARN | Major | . | Szilard Nemeth | Kevin Su | | | JournalNode error: Can't scan a pre-transactional edit log | Major | ha | Wei-Chiu Chuang | Stephen O'Donnell | | | Hadoop does not work with Kerberos config in hdfs-site.xml for simple security | Minor | . | Eric Yang | Prabhu Joseph | | | Upload button should not encode complete url | Major | . | Lokesh Jain | Lokesh Jain | | | hadoop-build-tools jar is downloaded from remote repository instead of using from local | Minor | build | Oleksandr Shevchenko | Oleksandr Shevchenko | | | The DirectoryScanner doesn't fix the wrongly placed replica. | Major | . | Jinglun | Jinglun | | | Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations | Major | federation, yarn | hunshenshi | hunshenshi | | | NPE when rendering the info table of leaf queue in non-accessible partitions | Major | capacityscheduler | Tao Yang | Tao Yang | | | ClosedChannelException silently ignored in FsVolumeList.addBlockPool() | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | yarn.system-metrics-publisher.enabled=false is not honored by RM | Major | resourcemanager, timelineclient | KWON BYUNGCHANG | KWON BYUNGCHANG | | | Rogue LocalizerRunner/ContainerLocalizer repeatedly downloading same file | Major | yarn | Jim Brennan | Jim Brennan | | | In NameNode Web UI, for Head the file (first 32K) old data is showing | Major | . | hemanthboyina | hemanthboyina | | | Document the missing EC removePolicy command | Major | documentation, erasure-coding | Wenxin He | Siyao Meng | | | DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream | Major | datanode | Yiqun Lin | Yiqun Lin | | | PlacementRule logs object ID in place of queue

| Minor | resourcemanager | Prabhu Joseph | Prabhu Joseph | | | AggregatedLogsBlock shows wrong NM http port | Minor | nodemanager | Prabhu Joseph | Prabhu Joseph | | | ApplicationPlacementContext is not required for terminated jobs during recovery | Major | resourcemanager | Prabhu Joseph | Prabhu Joseph | | | Block report leases cause missing blocks until next report | Critical | namenode | Daryn Sharp | Santosh Marella | | | Failed to restart yarn-service if it doesnt exist in RM | Major | yarn-native-services | kyungwan nam | kyungwan nam | | | HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory | Major | hdfs | Siyao Meng | Siyao Meng | | | TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk | Minor | . | Lisheng Sun | Lisheng Sun | | | RollingLevelDBTimelineStore.getEntityByTime fails with NPE | Major | timelineserver | Prabhu Joseph | Prabhu Joseph | | | Fix invalid metric types in PrometheusMetricsSink | Major | metrics | Akira Ajisaka | Akira Ajisaka | | | Reduce additional namenode call by EntityGroupFSTimelineStore#cleanLogs | Major | timelineserver | Prabhu Joseph | Prabhu Joseph | | | HDFS-11848 breaks API compatibility | Blocker | . | Wei-Chiu Chuang | Siyao Meng | | | Percent (%) and plus (+) characters no longer work in WebHDFS | Major | webhdfs | Jing Wang | Masatake Iwasaki | | | Correct the safemode threshold value in BlockManagerSafeMode | Major | namenode | Surendra Singh Lilhore | hemanthboyina | | | TestHSWebApp.testLogsViewSingle fails | Major | jobhistoryserver, test | Prabhu Joseph | Prabhu Joseph | | | TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk | Major | log-aggregation, test | Peter Bacsko | Adam Antal | | | TestRMWebServicesDelegationTokenAuthentication.testCancelledDelegationToken fails with HTTP 400 | Minor | resourcemanager, test | Peter Bacsko | Peter Bacsko | | | Duplicate values in rpcDetailedMetrics | Major | . | Bilwa S T | Bilwa S T | | | HAState#prepareToEnterState needn't a lock | Major | hdfs | hunshenshi | hunshenshi | | | Standby RM should expose jmx endpoint | Major | resourcemanager | Karthik Kambatla | Rohith Sharma K S | | | Fix spelling mistakes in BPServiceActor.java. | Major | datanode | chencan | chencan | | | Add a description about the observer read configuration | Minor | . | xiangheng | xiangheng | | | Standby Namenode never come out of safemode when EC files are being written. | Critical | ec, namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Yet another fsimage corruption related to snapshot | Critical | snapshots | Yongjun Zhang | Shashikant Banerjee | | | Upgrade JQuery to latest version for YARN UI | Major | yarn | Akhil PB | Akhil PB | | | Fix prompt message in testPolicyAndStateCantBeNull | Minor | . | chencan | chencan | | | Multi-threading conflict at layoutVersion when loading block pool storage | Major | rolling upgrades | Yicong Cai | Yicong Cai | | | Failed to start DN with ArithmeticException when NULL checksum used | Major | datanode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Apply HDDS-1870 (ConcurrentModification at PrometheusMetricsSink) to Hadoop common | Major | metrics | Akira Ajisaka | Akira Ajisaka | | | TestLogAggregationService.testFixedSizeThreadPool fails intermittently on trunk | Minor | . | Varun Saxena | Bilwa S T | | | Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing | Major | yarn | Antal Blint Steinbach | Peter Bacsko | | | HDFS cat logs an info message | Major | . | Eric Badger | Eric Badger | | | Add SHA-256 or SHA-512 checksum to release artifacts to comply with the release distribution policy | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | RBF: RecoverLease should be return false when the file is open in multiple destination | Major | rbf | xuzq | xuzq | | | FileStatus#toString() will throw IllegalArgumentException | Major |

| xuzq | xuzq | | | Fix order of arguments for assertEquals in TestSLSUtils | Minor | test | Nikhil Navadiya | Nikhil Navadiya | | | NN restart fails after RollingUpgrade from 2.x to 3.x | Blocker | hdfs | Hanisha Koneru | Fei Hui | | | Failed to load image from FSImageFile when downgrade from 3.x to 2.x | Blocker | rolling upgrades | Fei Hui | Fei Hui | | | RBF: IsFileClosed should be return false when the file is open in multiple destination | Major | rbf | xuzq | xuzq | | | RBF: MountTableResolver cannot invalidate cache correctly | Major | rbf | Yuxuan Wang | Yuxuan Wang | | | RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException | Major | rbf | xuzq | xuzq | | | Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Add GPU in the container-executor.cfg example | Trivial | nodemanager, yarn | Adam Antal | Kinga Marton | | | Fix Memory Leak in AbstractYarnScheduler caused by timer | Blocker | resourcemanager | Bibin Chundatt | Bibin Chundatt | | | NameNode can kill itself if it tries to send too many txns to a QJM simultaneously | Major | namenode, qjm | Erik Krogen | Erik Krogen | | | NN should automatically set permissions on dfs.namenode.\*.dir | Major | namenode | Aaron Myers | Siddharth Wagle | | | Revisiting LICENSE and NOTICE files | Blocker | . | Akira Ajisaka | Akira Ajisaka | | | Update jackson-databind to 2.9.9.3 | Major | . | Akira Ajisaka | Akira Ajisaka | | | launchTime not written to state store for running applications | Major | . | Jonathan Hung | Jonathan Hung | | | Slow event processing could cause too many attempt unregister events | Critical | . | Bibin Chundatt | Bibin Chundatt | | | RBF: ProxyOpComplete is not accurate in FederationRPCPerformanceMonitor | Major | rbf | xuzq | xuzq | | | FSNameSystem#logAuditEvent should be called outside the read or write locks | Major | . | Kuhu Shukla | Xiaoqiao He | | | Options.Rename.To\_TRASH is considered even when Options.Rename.NONE is specified | Major | namenode | Vinayakumar B | Vinayakumar B | | | ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby | Major | resourcemanager | Tao Yang | Tao Yang | | | Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork | Major | . | Fei Hui | Fei Hui | | | TestRMAppTransitions fails intermittently | Minor | resourcemanager, test | Prabhu Joseph | Tao Yang | | | QJM doesn't move aside stale inprogress edits files | Major | qjm | Zhe Zhang | Istvan Fajth | | | ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently | Minor | test | Tao Yang | Tao Yang | | | TestRMDelegationTokens can fail in testRemoveExpiredMasterKeyInRMStateStore | Major | test, yarn | Adam Antal | Adam Antal | | | Remove duplicate sentence from TimelineServiceV2.md | Major | ATSv2, docs | Kinga Marton | Kinga Marton | | | Add containerId to ResourceLocalizationService fetch failure log statement | Minor | nodemanager | Prabhu Joseph | Prabhu Joseph | | | Checksums are not checked if block meta file is less than 7 bytes | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Configuration.getTimeDurationHelper() should not log time unit warning in info

| Minor | hdfs | Surendra Singh Lilhore | hemanthboyina | | | LeafQueue#activateApplications should use resourceCalculator#fitsIn | Blocker | . | Bibin Chundatt | Bilwa S T | | | Exclude submarine from hadoop source build | Major | . | Nanda kumar | Nanda kumar | | | SetTimes updates all negative values apart from -1 | Major | . | Harshakiran Reddy | Ayush Saxena | | | Fix DominantResourceCalculator when one resource is zero | Blocker | . | Bilwa S T | Bilwa S T | | | RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set | Major | . | Ranith Sardar | Ranith Sardar | | | Yarn REST API, services endpoint remote command ejection | Major | . | Eric Yang | Eric Yang | | | dfs.ha.zkfc.port property duplicated in hdfs-default.xml | Major | hdfs | Renukaprasad C | Renukaprasad C | | | Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero | Major | test | Prabhu Joseph | Prabhu Joseph | | | RM does not start on JDK11 when UIv2 is enabled | Critical | resourcemanager, yarn | Adam Antal | Adam Antal | | | mvn javadoc:javadoc fails in hadoop-sls | Major | documentation | Akira Ajisaka | Abhishek Modi | | | org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue is flaky | Major | test | Kinga Marton | Kinga Marton | | | RM logs InvalidStateTransitionException when app is submitted | Critical | . | Rohith Sharma K S | Prabhu Joseph | | | mvn javadoc:javadoc fails in hadoop-aws | Major | documentation | Akira Ajisaka | Xieming Li | | | ResourceManager REST API can produce an illegal xml response | Major | api, resourcemanager | Thomas | Prabhu Joseph | | | RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Display RPC (instead of HTTP) Port Number in RBF web UI | Minor | rbf, ui | Xieming Li | Xieming Li | | | EntityGroupFSTimelineStore#scanActiveLogs fails when undesired files are present under /ats/active. | Major | timelineserver | Prabhu Joseph | Prabhu Joseph | | | Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold | Critical | ec | Zhao Yi Ming | Zhao Yi Ming | | | Synchronize invalidateBlocks in DatanodeDescriptor | Minor | namenode | David Mollitor | hemanthboyina | | | Make HDFS-14617 (fsimage sub-sections) off by default | Blocker | namenode | Stephen O'Donnell | Stephen O'Donnell | | | check block directory logic not correct when there is only meta file, print no meaning warn log | Minor | datanode, hdfs | qiang Liu | qiang Liu | | | Race condition when DirectoryCollection.checkDirs() runs during container launch | Major | . | Peter Bacsko | Peter Bacsko | | | YARN Service fails to fetch status for Stopped apps with bigger spec files | Major | yarn-native-services | Tarun Parimi | Tarun Parimi | | | YARN Audit logging not added to log4j.properties | Major | . | Varun Saxena | Aihua Xu | | | JobHistoryServer can't delete aggregated files, if remote app root directory is created by NodeManager | Minor | log-aggregation, yarn | Adam Antal | Adam Antal | | | FileIoProvider should not increase FileIoErrors metric in datanode volume metric | Minor |

| Aiphago | Aiphago | | | libhdfs tests are failing on trunk due to jni usage bugs | Major | libhdfs, native | Sahil Takiar | Sahil Takiar | | | LocalFileSystem's mkdirs() does not work as expected under viewfs. | Major | . | Kihwal Lee | Kihwal Lee | | | RBF: Security should use common AuthenticationFilter | Major | . | CR Hota | Chen Zhang | | | ValueQueue does not trigger an async refill when number of values falls below watermark | Major | common, kms | Yuval Degani | Yuval Degani | | | NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present | Major | . | Ranith Sardar | Ranith Sardar | | | DistCp job fails when new data is appended in the file while the distCp copy job is running | Critical | distcp | Mukund Thakur | Mukund Thakur | | | RBF: Fix typo in TestRouterQuota | Trivial | . | Jinglun | Jinglun | | | EC: Improper size values for corrupt ec block in LOG | Major | ec | Harshakiran Reddy | Ayush Saxena | | | Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs | Critical | httpfs | Akira Ajisaka | Prabhu Joseph | | | mvn package fails in hadoop-aws | Major | documentation | Xieming Li | Xieming Li | | | Fix TestHDFSCLI and TestDFSShell test break because of logging change in mkdir | Major | hdfs | Gabor Bota | Gabor Bota | | | Fix dfsadmin doc for triggerBlockReport | Major | documentation | Fei Hui | Fei Hui | | | Erasure Coding: the internal block is replicated many times when datanode is decommissioning | Major | ec, erasure-coding | HuangTao | HuangTao | | | Remove unused imports from TestBlockMissingException.java and TestClose.java | Minor | test | Lisheng Sun | Lisheng Sun | | | Optimize RMContext getExclusiveEnforcedPartitions | Major | . | Jonathan Hung | Jonathan Hung | | | Snapshot memory leak | Major | snapshots | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove redundant super user priveledge checks from namenode. | Major | . | Ayush Saxena | Ayush Saxena | | | yarn-services-core test timeout | Major | . | Prabhu Joseph | Prabhu Joseph | | | UI: Fix a typo on WebUI of DataNode. | Trivial | datanode, ui | Xieming Li | Xieming Li | | | Upgrade jackson and jackson-databind to 2.9.10 | Major | . | Siyao Meng | Siyao Meng | | | NullPointerException happens in NamenodeWebHdfs | Critical | . | lujie | lujie | | | NPE in TestAdlSdkConfiguration failing in yetus | Major | fs/adl | Steve Loughran | Sneha Vijayarajan | | | Safemode 'forceExit' option, doesnt shown in help message | Major | . | Renukaprasad C | Renukaprasad C | | | Namenode may not replicate blocks to meet the policy after enabling upgradeDomain | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Header was wrong in Snapshot web UI | Major | . | hemanthboyina | hemanthboyina | | | [SBN Read] Namenode crashes if one of The JN is down | Critical | . | Harshakiran Reddy | Ayush Saxena | | | Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when

is not zero | Major | hdfs | Srinivasu Majeti | Srinivasu Majeti | | | Fix documentation about NodeHealthScriptRunner | Major | documentation, nodemanager | Peter Bacsko | Peter Bacsko | | | FairScheduler: NODE\_UPDATE can cause NoSuchElementException | Major | fairscheduler | Peter Bacsko | Peter Bacsko | | | Erasure Coding : The number of Under-Replicated Blocks never reduced | Critical | ec | hemanthboyina | hemanthboyina | | | Fix build failure of hadoop-hdfs-native-client | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | Class cast error in GetGroups with ObserverReadProxyProvider | Major | . | Shen Yinjie | Erik Krogen | | | EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe | Critical | ec, hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x | Blocker | . | Yuxuan Wang | Yuxuan Wang | | | Fix TestWebDelegationToken#testKerberosDelegationTokenAuthenticator on Windows | Major | security | Xiaoyu Yao | Kitti Nanasi | | | In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec | Major | . | hemanthboyina | hemanthboyina | | | Additional Unit tests to verify queue limit and max-limit with multiple resource types | Major | capacity scheduler | Sunil G | Adam Antal | | | Setting permissions on name directory fails on non posix compliant filesystems | Blocker | . | hirik | Siddharth Wagle | | | "hadoop.ssl.enabled.protocols" should be considered in httpserver2 | Major | security | Brahma Reddy Battula | Brahma Reddy Battula | | | Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException | Major | common | Adam Antal | Adam Antal | | | RBF: LS command for mount point shows wrong owner and permission information. | Major | . | xuzq | Jinglun | | | DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Remove unnecessary InnerNode check in NetworkTopology#add() | Minor | . | Lisheng Sun | Lisheng Sun | | | RBF: In Router Web UI, Observer Namenode Information displaying as Unavailable | Major | . | hemanthboyina | hemanthboyina | | | Erasure Coding: Blocks are over-replicated while EC decommissioning | Critical | ec | Fei Hui | Fei Hui | | | RBF: line breaks are missing from the output of 'hdfs dfsrouteradmin -ls' | Minor | rbf, ui | Xieming Li | Xieming Li | | | Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType() | Major | . | Ayush Saxena | Ayush Saxena | | | Fix FindBug issue in QueueMetrics | Minor | . | Prabhu Joseph | Prabhu Joseph | | | DN may not send block report to NN after NN restart | Major | datanode | TanYuxin | Xiaoqiao He | | | INode access time is ignored because of race between open and rename | Major | . | Jinglun | Jinglun | | | S3A connector does not support including the token renewer in the token identifier | Major | fs/s3 | Philip Zampino | Philip Zampino | | | Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. | Major | . | Tarun Parimi | Tarun Parimi | | | Missing leveldbjni package of aarch64 platform | Major | . | liusheng | | | | Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. | Blocker |

| igo Goiri | Wei-Chiu Chuang | | | Fixing a typo in documentation of Observer NameNode | Trivial | documentation | Xieming Li | Xieming Li | | | DFSStripedInputStream curStripeBuf is not freed by unbuffer() | Major | ec | Joe McDonnell | Zhao Yi Ming | | | hdfs crypto commands limit column width | Major | . | Eric Badger | Eric Badger | | | Remove unused configuration dfs.web.authentication.filter | Major | . | Chen Zhang | Chen Zhang | | | Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission | Major | ec | Fei Hui | Fei Hui | | | EC : Busy DN replica should be consider in live replica check. | Major | datanode, erasure-coding, hdfs, namenode | guojh | guojh | | | BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false | Major | . | hu xiaodong | hu xiaodong | | | rename operation should check nest snapshot | Major | namenode | Junwang Zhao | Junwang Zhao | | | Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo | Minor | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Revise PacketResponder's log. | Minor | datanode | Xudong Cao | Xudong Cao | | | Erasure Coding: Block recovery failed during decommissioning | Major | . | Fei Hui | Fei Hui | | | Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType() | Major | . | Lisheng Sun | Lisheng Sun | | | mvn javadoc:javadoc fails in hadoop-aws | Major | documentation | Xieming Li | Xieming Li | | | When lastLocatedBlock token expire, it will take 1~3s second to refetch it. | Major | hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Correct the sequence of statistics & exit message in balencer | Major | balancer & mover | Renukaprasad C | Renukaprasad C | | | Bootstrap standby may fail if used in-progress tailing | Major | namenode | Chen Liang | Chen Liang | | | Potential editlog race condition can cause corrupted file | Major | namenode | Chen Liang | Chen Liang | | | TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup | Minor | hdfs | Jim Brennan | Jim Brennan | | | DataNode shouldn't report block as bad block if the block length is Long.MAX\_VALUE. | Major | datanode | Surendra Singh Lilhore | hemanthboyina | | | RBF: ConnectionPool#newConnection() error log wrong protocol class | Minor | rbf | Yuxuan Wang | Yuxuan Wang | | | Fix broken links in site index | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Move documentation of hadoop-cos to under src directory. | Major | tools | Masatake Iwasaki | Masatake Iwasaki | | | Filesystems to be closed if they failed during initialize() | Major | fs, fs/s3 | Steve Loughran | Steve Loughran | | | Prevent snapshot modification time got change on startup | Major | . | hemanthboyina | hemanthboyina | | | Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. | Minor | common | Xudong Cao | Xudong Cao | | | [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write | Major | . | Ranith Sardar | Ranith Sardar | | | Add sanity check that zone key equals feinfo key while setting Xattrs | Major | encryption, hdfs | Mukul Kumar Singh | Mukul Kumar Singh | | | AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path | Minor | fs, test |

| Xieming Li | | | The feature of protect directories should be used in RenameOp | Major | hdfs | Fei Hui | Fei Hui | | | Fix Container API example link in NodeManager REST API doc | Trivial | documentation | Charan Hebri | Charan Hebri | | | TestWebHDFS fails in Windows | Major | . | Renukaprasad C | Renukaprasad C | | | Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set | Major | auxservices, nodemanager | Prabhu Joseph | Prabhu Joseph | | | FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | signalToContainer REST API does not work even if requested by the app owner | Major | restapi | kyungwan nam | kyungwan nam | | | TestOfflineEditsViewer is failing in Trunk | Major | . | hemanthboyina | hemanthboyina | | | Typo in YARN Service overview documentation | Trivial | documentation | Denes Gerencser | Denes Gerencser | | | Remove the disallowed element config within maven-checkstyle-plugin | Major | . | Wanqiang Ji | Wanqiang Ji | | | RpcQueueTime may be negative when the response has to be sent later | Minor | . | xuzq | xuzq | | | Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr | Major | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix jsvc startup command in hadoop-functions.sh due to jsvc \>= 1.0.11 changed default current working directory | Major | scripts | Siyao Meng | Siyao Meng | | | RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations | Major | ec, rbf | Ayush Saxena | Ayush Saxena | | | HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB | Minor | balancer & mover | Souryakanta Dwivedy | hemanthboyina | | | RenameSnapshot not updating new modification time | Major | . | hemanthboyina | hemanthboyina | | | RBF: Exceptions are conflicting when creating the same mount entry twice | Major | federation | Soumyapn | Ranith Sardar | | | Fix resource inconsistency for queues when moving app with reserved container to another queue | Critical | capacity scheduler | jiulongzhu | jiulongzhu | | | Public Localizer is exiting in NodeManager due to NullPointerException | Major | nodemanager | Tarun Parimi | Tarun Parimi | | | Race condition during decommissioning | Major | nodemanager | Peter Bacsko | Peter Bacsko | | | Balancer getBlocks RPC dispersal does not function properly | Major | balancer & mover | Erik Krogen | Erik Krogen | | | FileSystem#listStatusIterator does not check if given path exists | Major | fs | Sahil Takiar | Sahil Takiar | | | Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler | Major | . | Prabhu Joseph | Prabhu Joseph | | | Exception ' Invalid event: TA\TOO\MANY\FETCH\FAILURE at SUCCESS\FINISHING\CONTAINER' cause job error | Critical | . | luhuachao | luhuachao | | | ReplicaCachingGetSpaceUsed throws ConcurrentModificationException | Major | datanode, performance | Ryan Wu | Aiphago | | | Invalid event TA\TOO\MANY\FETCH\FAILURE at SUCCESS\CONTAINER\CLEANUP causes job failure | Critical | applicationmaster, mrv2 | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | [SBN read] Prevent ZKFC changing Observer Namenode state | Major | . | igo Goiri | Ayush Saxena | | | BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method | Major | datanode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Testcase fails with "Insufficient configured threads: required=16 \< max=10" | Major |

| Prabhu Joseph | Prabhu Joseph | | | FSCK "-list-corruptfileblocks" return Invalid Entries | Major | . | hemanthboyina | hemanthboyina | | | HTTPFS - CHECKACCESS operation missing | Major | . | Bert Hekman | hemanthboyina | | | TestPendingReconstruction#testPendingReconstruction() fail in trunk | Minor | . | Lisheng Sun | Lisheng Sun | | | if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.\* | Major | build | KWON BYUNGCHANG | KWON BYUNGCHANG | | | Improve yarn.scheduler.capacity.queue-mappings documentation | Major | . | Manikandan R | Manikandan R | | | Validate Parent Queue for QueueMapping contains dynamic group as parent queue | Major | . | Manikandan R | Manikandan R | | | Fix building instruction to enable zstd | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | IOException used in place of YARNException in CapacitySheduler | Minor | capacity scheduler | Prabhu Joseph | Adam Antal | | | Synchronize on diffs in DirectoryScanner | Minor | . | Lisheng Sun | Lisheng Sun | | | Unsupported "transitionToObserver" option displaying for rmadmin command | Minor | RM, yarn | Souryakanta Dwivedy | Ayush Saxena | | | RBF: Secured Router should not run when SecretManager is not running | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | TestContinuousScheduling fails sporadically with NPE | Major | fairscheduler | Jason Darrell Lowe | Wilfred Spiegelenburg | | | DataStreamer#createBlockOutputStream() should log exception in warn. | Major | dfsclient | Surendra Singh Lilhore | Ravuri Sushma sree | | | Fix docker failed to build yetus/hadoop | Blocker | build | Kevin Su | Kevin Su | | | Balancer crashes when it fails to contact an unavailable NN via ObserverReadProxyProvider | Major | balancer & mover | Erik Krogen | Erik Krogen | | | Active NameNode should not silently fail the image transfer | Major | namenode | Konstantin Shvachko | Chen Liang | | | Fix broken link in single node cluster setup documentation | Trivial | documentation | Denes Gerencser | Denes Gerencser | | | Fix findbug in DirectoryScanner | Major | . | Takanobu Asanuma | Masatake Iwasaki | | | Fix curator dependencies for gradle projects using hadoop-minicluster | Major | . | Mate Szalay-Beko | Mate Szalay-Beko | | | LeaseManager should check parent-child relationship when filter open files. | Minor | . | Jinglun | Jinglun | | | NameQuota is not update after concat operation, so namequota is wrong | Major | . | Ranith Sardar | Ranith Sardar | | | Fix build instruction of hadoop-yarn-ui | Minor | yarn-ui-v2 | Masatake Iwasaki | Masatake Iwasaki | | | NN fails to parse Edit logs after applying HDFS-13101 | Blocker | nn | Eric Lin | Shashikant Banerjee | | | Upgrade build tools for YARN Web UI v2 | Major | build, security, yarn-ui-v2 | Akira Ajisaka | Masatake Iwasaki | | | Uupgrade grpc-xxx depdencies to 1.26.0 | Major | . | liusheng | liusheng | | | TestDiskChecker and TestReadWriteDiskValidator fails when run with -Pparallel-tests | Major | . | Vinayakumar B | Vinayakumar B | | | Upgrade yarn to the latest version in Dockerfile | Major | build, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | bower install fails | Blocker | build, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | Fix tests that hold FSDirectory lock, without holding FSNamesystem

| Major | test | Konstantin Shvachko | Konstantin Shvachko | | | Replace curator-shaded guava import with the standard one | Minor | hdfs-client | Akira Ajisaka | Chandra Sanivarapu | | | Upgrade the dependencies managed by yarnpkg | Major | build, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | Delete Snapshot not updating new modification time | Major | . | hemanthboyina | hemanthboyina | | | Update the link to HadoopJavaVersion | Minor | documentation | Akira Ajisaka | Chandra Sanivarapu | | | Should not use AbstractPath to create unix domain socket | Major | test | zhao bo | liusheng | | | [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0 | Major | . | Takanobu Asanuma | Ayush Saxena | | | HttpFS : getFileStatus doesn't return ecPolicy | Major | . | hemanthboyina | hemanthboyina | | | Placement rules do not use correct group service init | Major | yarn | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | DataNode could meet deadlock if invoke refreshVolumes when register | Major | datanode | Xiaoqiao He | Aiphago | | | RBF: SmallFix for RBFMetrics in doc | Trivial | documentation, rbf | luhuachao | luhuachao | | | Fix typo in MapReduce documentaion example | Trivial | documentation | Sergey Pogorelov | Sergey Pogorelov | | | Improve connection error message for YARN ApiServerClient | Major | . | Eric Yang | Prabhu Joseph | | | Fix duplicate assertj-core dependency in hadoop-common module. | Minor | build | Akira Ajisaka | Xieming Li | | | HDFS MiniCluster fails to start when run in directory path with a % | Minor | . | Geoffrey Jacoby | Masatake Iwasaki | | | Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Fix the issue in reading persistent memory cached data with an offset | Major | caching, datanode | Feilong He | Feilong He | | | org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently | Major | . | Miklos Szegedi | Jim Brennan | | | TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out | Major | nodemanager | Jason Darrell Lowe | Chandni Singh | | | INodeReference Space Consumed was not same in QuotaUsage and ContentSummary | Major | namenode | hemanthboyina | hemanthboyina | | | Handling 0 progress in SimpleExponential task runtime estimator | Minor | . | Ahmed Hussein | Ahmed Hussein | | | HttpFS: put requests are not supported for path "/" | Major | . | hemanthboyina | hemanthboyina | | | HttpFS: POST requests are not supported for path "/" | Major | . | hemanthboyina | hemanthboyina | | | Configuration parsing of CDATA values are blank | Major | conf | Jonathan Turner Eagles | Daryn Sharp | | | Fix accidental comment in flaky test TestDecommissioningStatus | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | IBM Java has deprecated OS login module classes and OS principal classes. | Major | security | Nicholas Marion | | | | container-executor: misc improvements in child processes and exec() calls | Minor | nodemanager | Peter Bacsko | Peter Bacsko | | | [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException on ObserverNode | Major | namenode | Konstantin Shvachko | Chen Liang | | | RBF: MembershipNamenodeResolver should invalidate cache incase of active namenode update | Major | . | Ayush Saxena | Ayush Saxena | | | AvailableSpaceBlockPlacementPolicy always prefers local node | Major | block placement | Wei-Chiu Chuang | Ayush Saxena | | | u:user2:%primary\_group is not working as expected | Major |

| Manikandan R | Manikandan R | | | Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException | Major | common | Adam Antal | Adam Antal | | | Fix javadoc error in SimpleExponentialSmoothing | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | RM Received RMFatalEvent of type CRITICAL\THREAD\CRASH | Major | fairscheduler, resourcemanager | Girish Bhat | Wilfred Spiegelenburg | | | [JDK11] TestAuxServices#testCustomizedAuxServiceClassPath fails because of ClassCastException. | Major | test | Siyao Meng | Akira Ajisaka | | | Modify HistoryServerRest.html content,change The job attempt ids datatype from string to int | Major | documentation | zhaoshengjie | | | | NPE if no rule is defined and application-tag-based-placement is enabled | Major | . | Kinga Marton | Kinga Marton | | | Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping | Major | . | Manikandan R | Manikandan R | | | Update decimal values for queue capacities shown on queue status CLI | Major | client | Prabhu Joseph | Prabhu Joseph | | | Exception message from ClientRMProxy#getRMAddress is misleading | Trivial | yarn | Adam Antal | Ravuri Sushma sree | | | Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin | Minor | build | Akira Ajisaka | Xieming Li | | | Remove WARN log when ipc connection interrupted in Client#handleSaslConnectionFailure() | Minor | . | Lisheng Sun | Lisheng Sun | | | TestDatanodeRegistration#testForcedRegistration fails intermittently | Major | . | Ahmed Hussein | Ahmed Hussein | | | TestResourceTrackerService.testNodeRemovalGracefully fails sporadically | Minor | resourcemanager, test | Prabhu Joseph | Prabhu Joseph | | | Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero | Major | . | kyungwan nam | kyungwan nam | | | Unit test failing to clean testing data and crashed future Maven test run due to failure in TestDataNodeVolumeFailureToleration | Critical | hdfs, test | Ctest | Ctest | | | LocatedStripedBlock returns wrong block type | Major | . | Ayush Saxena | Ayush Saxena | | | checkDiskError doesn't work during datanode startup | Major | datanode | Yang Yun | Yang Yun | | | HttpFS: getAclStatus() returns permission as null | Major | . | hemanthboyina | hemanthboyina | | | TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently | Critical | fs | Gabor Bota | Ahmed Hussein | | | testSpeculateSuccessfulWithUpdateEvents fails Intermittently | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Invoking NMWebServices#getNMResourceInfo tries to execute gpu discovery binary even if auto discovery is turned off | Major | . | Szilard Nemeth | Szilard Nemeth | | | JobHistory#ServiceStop implementation is incorrect | Major | . | Jason Darrell Lowe | Ahmed Hussein | | | [SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. | Major | hdfs-client | Konstantin Shvachko | Chen Liang | | | [JDK11] TestTimelineWebServices.testContextFactory fails | Major | timelineservice | Adam Antal | Akira Ajisaka | | | Client-side SocketTimeoutException during Fsck | Major | namenode | Carl Steinbach | Stephen O'Donnell | | | Replace com.sun.istack.Nullable with javax.annotation.Nullable in DNS.java | Minor | build | Akira Ajisaka | Xieming Li | | | LOG flooding in secure mode when Cookies are not set in request header | Major | . | Renukaprasad C | Renukaprasad C | | | Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug | Major |

| wangzhixiang | wangzhixiang | | | The number of failed volumes mismatch with volumeFailures of Datanode metrics | Minor | datanode | Yang Yun | Yang Yun | | | unused import in Configuration class | Trivial | conf | runzhou wu | Jan Hentschel | | | start-build-env.sh behaves incorrectly when username is numeric only | Minor | build | Jihyun Cho | Jihyun Cho | | | cmake is missing in the CentOS 8 section of BUILDING.txt | Minor | build, documentation | Akira Ajisaka | Masatake Iwasaki | | | When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close() | Major | . | Lisheng Sun | Lisheng Sun | | | RM failed to start due to system services | Major | . | kyungwan nam | kyungwan nam | | | [Router] : Application metrics are hardcode as N/A in UI. | Major | federation, yarn | Sushanta Sen | Bilwa S T | | | Fix TestDelegationTokensWithHA | Major | . | Ayush Saxena | Ayush Saxena | | | ipc.Server readAndProcess threw NullPointerException | Major | rpc-server | Tsz-wo Sze | Tsz-wo Sze | | | In Du missed calling getAttributesProvider | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Cannot reset the AM failure count for YARN Service | Major | . | kyungwan nam | kyungwan nam | | | Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure | Major | build | Akira Ajisaka | Akira Ajisaka | | | WebHDFS getTrashRoot leads to OOM due to FileSystem object creation | Major | webhdfs | Wei-Chiu Chuang | Masatake Iwasaki | | | FPGA plugin can't find the localized aocx file | Major | nodemanager | Peter Bacsko | Peter Bacsko | | | StartupProgress reports edits segments until the entire startup completes | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | | | TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk | Minor | . | Lisheng Sun | Lisheng Sun | | | Remove redundant field fStream in ByteStringLog | Major | . | Konstantin Shvachko | Xieming Li | | | CORRUPT replica mismatch between namenodes after failover | Critical | . | Ayush Saxena | Ayush Saxena | | | YARN-10101 broke Yarn logs CLI | Blocker | yarn | Adam Antal | Adam Antal | | | The description of hadoop.http.authentication.signature.secret.file contains outdated information | Minor | documentation | Akira Ajisaka | Xieming Li | | | TestStreamingStatus fails | Major | . | Andras Bokor | Andras Bokor | | | Fix typo 'complaint' which means quite different in Federation.md | Minor | documentation, federation | Sungpeo Kook | Sungpeo Kook | | | Updating scheduler conf with comma in config value fails | Major | . | Anthony Hsu | Tanu Ajmera | | | Interceptor in FederationInterceptorREST doesnt update on RM switchover | Major | federation, restapi | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Fix findbugs warnings in hadoop-yarn-applications-mawo-core module | Major | . | Akira Ajisaka | Akira Ajisaka | | | Block Report Interval shouldn't be reset apart from first Block Report | Major | . | Ayush Saxena | Ayush Saxena | | | LazyPersistTestCase wait logic is error-prone | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Support Fuse with Users from multiple Security Realms | Critical | fuse-dfs | Sailesh Patel | Istvan Fajth | | | Crashing bugs in NameNode when using a valid configuration for

| Critical | namenode | Ctest | Ctest | | | TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk | Major | yarn | Adam Antal | Manikandan R | | | stopStandbyServices() should log which service state it is transitioning from. | Major | hdfs, logging | Konstantin Shvachko | Xieming Li | | | NPE in BlockSender | Major | . | Ayush Saxena | Ayush Saxena | | | Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures | Major | ATSv2, distributed-shell, test | Prabhu Joseph | Prabhu Joseph | | | Upgrade jackson-databind to 2.9.10.3 | Blocker | . | Siyao Meng | Siyao Meng | | | TestDeadNodeDetection test cases time-out | Major | datanode | Ahmed Hussein | Lisheng Sun | | | Sort fields in ReflectionUtils.java | Minor | test, util | cpugputpu | cpugputpu | | | Documentation typos: fs.s3a.experimental.fadvise -\> fs.s3a.experimental.input.fadvise | Major | documentation, fs/s3 | Josh Rosen | Josh Rosen | | | TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | Quota Usage and Content summary are not same in Truncate with Snapshot | Major | . | hemanthboyina | hemanthboyina | | | Make pid file generation timeout configurable in case of reacquired container | Minor | yarn | Adam Antal | Adam Antal | | | TestEncryptionZones.testVersionAndSuiteNegotiation fails in trunk | Major | . | Ayush Saxena | Ayush Saxena | | | AliyunOSS: getFileStatus throws FileNotFoundException in versioning bucket | Major | fs/oss | wujinhu | wujinhu | | | Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled | Major | . | Szilard Nemeth | Andras Gyori | | | TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically | Major | scheduler, test | Prabhu Joseph | Ahmed Hussein | | | EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. | Major | erasure-coding | Surendra Singh Lilhore | Ravuri Sushma sree | | | Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId | Major | . | Erik Krogen | Ravuri Sushma sree | | | SlowDiskReport won't update when SlowDisks is always empty in heartbeat | Major | . | Haibin Huang | Haibin Huang | | | writeIoRate of DataNodeVolumeMetrics is never used | Major | hdfs | Haibin Huang | Haibin Huang | | | Wrong Use Case of -showprogress in fsck | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | Encryption zone file copy failure leaks temp file .\COPYING\ and wrapped stream | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out | Minor | namenode, test | John Zhuge | Ayush Saxena | | | Dependency divergence building Timeline Service on HBase 2.2.0 and above | Major | build, timelineservice | Wei-Chiu Chuang | Wei-Chiu Chuang | | | The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big | Major | . | Lisheng Sun | Lisheng Sun | | | EC: File write hangs during close in case of Exception during updatePipeline | Critical | . | Ayush Saxena | Ayush Saxena | | | Provide Non DFS Used per DataNode in DataNode UI | Major | . | Lisheng Sun | Lisheng Sun | | | Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs | Trivial | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Allocation tags are not removed when node decommission | Major | . | kyungwan nam | kyungwan nam | | | FSCK fails if one namenode is not available | Major |

| Ayush Saxena | Ayush Saxena | | | Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage | Critical | . | Ayush Saxena | Ayush Saxena | | | WebHDFS: Add snapshot counts to Content Summary | Major | . | hemanthboyina | hemanthboyina | | | NPE if the last block changes from COMMITTED to COMPLETE during FSCK | Major | . | krishna reddy | Ayush Saxena | | | [managedParent].%primary\_group mapping rule doesn't work after YARN-9868 | Major | capacity scheduler | Peter Bacsko | Peter Bacsko | | | Missing IBR when NameNode restart if open processCommand async feature | Blocker | datanode | Xiaoqiao He | Xiaoqiao He | | | Fix libhdfspp test failures with GCC 7 | Major | native, test | Akira Ajisaka | Akira Ajisaka | | | SnapshotCounter hits MaxSnapshotID limit | Major | snapshots | Karthik Palanisamy | Karthik Palanisamy | | | DFS Client will stuck when ResponseProcessor.run throw Error | Major | hdfs-client | zhengchenyu | zhengchenyu | | | The Timestamp for longest write/read lock held log is wrong | Major | . | Toshihiro Suzuki | Toshihiro Suzuki | | | Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API | Blocker | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Improving the error message for missing \`dfs.namenode.rpc-address.$NAMESERVICE\` | Minor | hdfs | Ctest | Ctest | | | Fix documentation about NodeAttributes. | Minor | documentation | Sen Zhao | Sen Zhao | | | Fix TestMapreduceConfigFields | Major | mrv2 | Ayush Saxena | Wanqiang Ji | | | pylint fails in the build environment | Critical | build | Akira Ajisaka | Akira Ajisaka | | | TestNetworkedJob fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Upgrade maven-clean-plugin to 3.1.0 | Major | build | Allen Wittenauer | Akira Ajisaka | | | CLOSE\_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI | Major | yarn | Siddharth Ahuja | Siddharth Ahuja | | | NPE in Capacity Scheduler while using %primary\_group queue mapping | Critical | capacity scheduler | Peter Bacsko | Peter Bacsko | | | NameNode should check the authorization API version only once during initialization | Blocker | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | distcp copy calls getFileStatus() needlessly and can fail against S3 | Minor | fs/s3, tools/distcp | Thangamani Murugasamy | Steve Loughran | | | RM HA tests failed intermittently on trunk | Major | client | Wangda Tan | Ahmed Hussein | | | CacheDirective becomes invalid when NN restart or failover | Major | caching, hdfs | Wang XL | Wang XL | | | TaskAttemptListenerImpl excessive log messages | Major | . | Ahmed Hussein | Ahmed Hussein | | | YARN service placement constraints is broken | Blocker | . | Eric Yang | Eric Yang | | | [YARN UI2] No Logs were found in "YARN Daemon Logs" page | Blocker | yarn-ui-v2 | Akhil PB | Akhil PB | | | Handle release package related issues | Major | . | Vinayakumar B | Vinayakumar B | | | Cache pool MAXTTL is not persisted and restored on cluster restart | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | RBF: MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. | Major | rbf | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Use Yetus

in GitHub PR | Major | build | Akira Ajisaka | Akira Ajisaka | | | Concat on INodeRefernce fails with illegal state exception | Critical | . | hemanthboyina | hemanthboyina | | | ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679 | Major | common | Gopal Vijayaraghavan | Gopal Vijayaraghavan | | | Duplicate jersey-test-framework-core dependency in yarn-server-common | Minor | build | Akira Ajisaka | Akira Ajisaka | | | hadoop-cos fails to build | Major | fs/cos | Wei-Chiu Chuang | Yang Yu | | | revert YARN-4946 | Blocker | log-aggregation, resourcemanager | Steven Rand | Steven Rand | | | Concat on a same files deleting the file | Critical | . | hemanthboyina | hemanthboyina | | | statfs function in hdfs-fuse is not working | Major | fuse-dfs, libhdfs | Aryan Gupta | Aryan Gupta | | | INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | StandbyNode fails transition to active due to insufficient transaction tailing | Major | namenode, qjm | Konstantin Shvachko | Konstantin Shvachko | | | TestConfiguredFailoverProxyProvider is failing | Major | test | Akira Ajisaka | Akira Ajisaka | | | ViewFileSystem.close doesn't close child filesystems and causes FileSystem objects leak. | Major | . | Jinglun | Jinglun | | | Dependency netty-all-4.1.27.Final doesn't support ARM platform | Major | . | liusheng | liusheng | | | Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue | Major | . | liusheng | liusheng | | | YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource | Minor | api | Gergely Pollak | Gergely Pollak | | | TestFsck testFsckListCorruptSnapshotFiles is failing in trunk | Major | . | hemanthboyina | hemanthboyina | | | Relax the condition for accepting a fsimage when receiving a checkpoint | Critical | namenode | Chen Liang | Chen Liang | | | Yarn Service fails if am java opts contains ZK authentication file path | Major | . | Bilwa S T | Bilwa S T | | | Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false | Major | task | Tarun Parimi | Tarun Parimi | | | YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars | Blocker | yarn | Vinayakumar B | Vinayakumar B | | | IBR leak causes standby NN to be stuck in safe mode | Blocker | namenode | Kihwal Lee | Akira Ajisaka | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Modify a logging message in TestCapacityScheduler | Trivial | . | Zhankun Tang | Zhankun Tang | | | TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey | Minor | test | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896 | Minor | capacity scheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | TestConfigurationNodeAttributesProvider fails after Mockito updated | Minor | . | Weiwei Yang | Weiwei Yang | | | TestCapacitySchedulerMetrics fails intermittently | Minor | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently | Minor | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11 | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Timeline related testcases are failing | Major | . | Prabhu Joseph | Abhishek Modi | | | TestTimelineReaderWhitelistAuthorizationFilter ignores Http Errors | Major |

| Prabhu Joseph | Prabhu Joseph | | | TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent | Major | resourcemanager | Prabhu Joseph | Prabhu Joseph | | | TestYarnNativeServices#testExpressUpgrade fails intermittent | Major | yarn-native-services | Prabhu Joseph | Prabhu Joseph | | | TestQueueManagementDynamicEditPolicy fails intermittent | Minor | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Disable libhdfs zerocopy test on Mac | Minor | libhdfs | John Zhuge | Akira Ajisaka | | | Add testcase for ensuring that FsShell cann't move file to the target directory that file exists | Minor | test | legend | legend | | | TestCSAllocateCustomResource failures | Major | yarn | Jim Brennan | Jim Brennan | | | TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails | Minor | test | Fei Hui | Fei Hui | | | TestRouterWebServicesREST is corrupting STDOUT | Minor | yarn | Jim Brennan | Jim Brennan | | | TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature | Major | fs, test | John Zhuge | Jim Brennan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Changes to the NameNode to support reads from standby | Major | namenode | Konstantin Shvachko | Chao Sun | | | Add stateId to RPC headers. | Major | ipc, namenode | Konstantin Shvachko | Plamen Jeliazkov | | | Add lastSeenStateId to RpcRequestHeader. | Major | . | Plamen Jeliazkov | Plamen Jeliazkov | | | libhdfs++: Get minidfscluster tests running under valgrind | Major | hdfs-client | James Clampffer | Anatoli Shein | | | Add haadmin commands to transition between standby and observer | Major | ha, hdfs, namenode | Chao Sun | Chao Sun | | | Add ReadOnly annotation to methods in ClientProtocol | Major | . | Chao Sun | Chao Sun | | | Make Client field AlignmentContext non-static. | Major | . | Plamen Jeliazkov | Plamen Jeliazkov | | | [Edit Tail Fast Path Pt 1] Enhance JournalNode with an in-memory cache of recent edit transactions | Major | ha, journal-node | Erik Krogen | Erik Krogen | | | [Edit Tail Fast Path Pt 2] Add ability for JournalNode to serve edits via RPC | Major | . | Erik Krogen | Erik Krogen | | | [Edit Tail Fast Path Pt 3] NameNode-side changes to support tailing edits via RPC | Major | ha, namenode | Erik Krogen | Erik Krogen | | | ClientGCIContext should be correctly named ClientGSIContext | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Introduce ObserverReadProxyProvider | Major | hdfs-client | Konstantin Shvachko | Chao Sun | | | Move RPC response serialization into Server.doResponse | Major | . | Plamen Jeliazkov | Plamen Jeliazkov | | | S3Guard DDB retryBackoff to be more informative on limits exceeded | Major | fs/s3 | Steve Loughran | Gabor Bota | | | [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync | Major | ha, journal-node, namenode | Erik Krogen | Erik Krogen | | | [Edit Tail Fast Path] Allow SbNN to tail in-progress edits from JN via RPC | Major | ha, hdfs, journal-node, namenode | Erik Krogen | Erik Krogen | | | Introduce msync API call | Major | . | Chen Liang | Chen Liang | | | Reduce logging frequency of QuorumJournalManager#selectInputStreams | Trivial | namenode, qjm | Erik Krogen | Erik Krogen | | | Add msync server

| Major | namenode | Chen Liang | Chen Liang | | | Remove AlignmentContext from AbstractNNFailoverProxyProvider | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | ObserverReadProxyProvider should work with IPFailoverProxyProvider | Major | test | Konstantin Shvachko | Konstantin Shvachko | | | Implement performFailover logic for ObserverReadProxyProvider. | Major | hdfs-client | Konstantin Shvachko | Erik Krogen | | | s3guard set-capacity command to fail fast if bucket is unguarded | Minor | fs/s3 | Steve Loughran | Gabor Bota | | | remove obsolete S3A test ITestS3ACredentialsInURL | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add mechanism to allow certain RPC calls to bypass sync | Major | namenode | Chen Liang | Chen Liang | | | TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. | Major | test | Konstantin Shvachko | Plamen Jeliazkov | | | Use getServiceStatus to discover observer namenodes | Major | . | Chao Sun | Chao Sun | | | Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode | Major | . | Chao Sun | Chao Sun | | | Limit logging frequency of edit tail related statements | Major | hdfs, qjm | Erik Krogen | Erik Krogen | | | [JDK10] Building native package on JDK10 fails due to missing javah | Major | native | Takanobu Asanuma | Takanobu Asanuma | | | [JDK10] Fix javadoc errors in hadoop-hdfs-rbf module | Major | documentation | Akira Ajisaka | igo Goiri | | | S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing | Major | fs/s3 | Aaron Fabbri | Gabor Bota | | | HttpFS: Implement GETSNAPSHOTDIFF | Major | httpfs | Siyao Meng | Siyao Meng | | | TestObserverNode refactoring | Major | test | Konstantin Shvachko | Konstantin Shvachko | | | Add WebSocket logic to the Node Manager web server to establish servlet | Major | . | Zian Chen | Zian Chen | | | [JDK9] Add missing javax.activation-api dependency | Critical | test | Akira Ajisaka | Akira Ajisaka | | | NPE in DynamoDBMetadataStore.lambda$listChildren for root + auth S3Guard | Blocker | fs/s3 | Steve Loughran | Gabor Bota | | | Handle AM register requests asynchronously in FederationInterceptor | Major | amrmproxy, federation | Subramaniam Krishnan | Botong Huang | | | ABFS: Enable some tests for namespace not enabled account using OAuth | Major | fs/azure | Da Zhou | Da Zhou | | | [JDK10] Javadoc build fails on JDK 10 in hadoop-common | Major | build, documentation | Takanobu Asanuma | Dinesh Chitlangia | | | Container Executor C binary change to execute interactive docker command | Major |

| Zian Chen | Eric Yang | | | Serve aggregated logs of historical apps from timeline service | Critical | timelineserver | Varun Saxena | Rohith Sharma K S | | | Provide Java client for fetching Yarn specific entities from TimelineReader | Critical | timelinereader | Rohith Sharma K S | Abhishek Modi | | | [Storage implementation] Create HDFS backing storage implementation for ATS reads | Major | timelineserver | Tsuyoshi Ozawa | Abhishek Modi | | | Support observer nodes in MiniDFSCluster | Major | namenode, test | Erik Krogen | Konstantin Shvachko | | | RBF: Add multiple paths for dfsrouteradmin "rm" and "clrquota" commands | Major | federation | Soumyapn | Ayush Saxena | | | @Retries annotation of putObject() call & uses wrong | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | AM HTTPS Support for AM communication with RMWeb proxy | Major | . | Robert Kanter | Robert Kanter | | | Unit Test for transitioning between different states | Major | test | Sherwood Zheng | Sherwood Zheng | | | Document YARN support for HTTPS in AM Web server | Major | docs | Robert Kanter | Robert Kanter | | | RM HA for AM web server HTTPS Support | Major | . | Robert Kanter | Robert Kanter | | | [YARN-8811] Add CSI java-based client library | Major | . | Weiwei Yang | Weiwei Yang | | | RBF: Add more information to HdfsFileStatus for a mount point | Major | . | Akira Ajisaka | Akira Ajisaka | | | Add configurable additional RPC listener to NameNode | Major | ipc | Chen Liang | Chen Liang | | | Handle BlockMissingException when reading from observer | Major | . | Chao Sun | Chao Sun | | | Change junit dependency in parent pom file to junit 5 while maintaining backward compatibility to junit4. | Major | build | Ajay Kumar | Akira Ajisaka | | | ABFS: Stop requiring client ID and tenant ID for MSI | Major | . | Sean Mackrory | Da Zhou | | | AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy | Major | fs/oss | wujinhu | wujinhu | | | Create an interface to provide cluster information to application | Major | . | Eric Yang | Eric Yang | | | RBF: The default MountTableResolver should fail resolving multi-destination paths | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | Document behavior of YARN-5742 | Major | . | Vrushali C | Suma Shivaprasad | | | [Atsv2] Update document for running v1 and v2 TS | Major | . | Rohith Sharma K S | Suma Shivaprasad | | | [JDK10] Fix javadoc errors in hadoop-hdfs module | Major | documentation | Akira Ajisaka | Dinesh Chitlangia | | | [libhdfs++] Disable libhdfs++ build on systems that do not support thread\_local | Major | . | Anatoli Shein | Anatoli Shein | | | ObserverReadProxyProvider should enable observer read by default | Major | . | Chen Liang | Chen Liang | | | libhdfs++: Fix compilation errors and warnings when compiling with Clang | Blocker | hdfs-client | Anatoli Shein | Anatoli Shein | | | [JDK9+] Add missing javax.annotation-api dependency to hadoop-yarn-csi | Major | build | Takanobu Asanuma | Takanobu Asanuma | | | RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService | Major | . | CR Hota | CR Hota | | | [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client | Major | amrmproxy, federation | Botong Huang | Botong Huang | | | RBF: Connection creator thread should catch Throwable | Critical | . | CR Hota | CR Hota | | | [Router] Add JvmMetricsInfo and pause monitor | Minor | . | Bibin Chundatt | Bilwa S T | | | [JDK 11] Javadoc build failed due to "bad use of '\>'" in hadoop-hdfs | Major | build | Takanobu Asanuma | Takanobu Asanuma | | | [JDK 11] Specify the HTML version of Javadoc to 4.01 | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | [libhdfs++] Fix hdfsGetLastExceptionRootCause bug in

| Major | libhdfs, native | Anatoli Shein | Anatoli Shein | | | RBF: Support for Kerberos authentication | Major | security | Zhe Zhang | Sherwood Zheng | | | Phase 1 - Add configurations for pluggable plugin framework | Major | . | Zhankun Tang | Zhankun Tang | | | Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled | Major | . | Weiwei Yang | Tao Yang | | | [CSI] Add volume manager that manages CSI volume lifecycle | Major | . | Weiwei Yang | Weiwei Yang | | | Usability improvements node-attributes CLI | Critical | . | Weiwei Yang | Bibin Chundatt | | | [CSI] Extend service spec to allow setting resource attributes | Major | . | Weiwei Yang | Weiwei Yang | | | Container Executor change to create stdin/stdout pipeline | Major | . | Zian Chen | Eric Yang | | | RBF: The DN\REPORT\TIME\OUT and DN\REPORT\CACHE\EXPIRE should be configured in RBFConfigKeys. | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | AliyunOSS: fix incorrect ReadOps and WriteOps in statistics | Major | fs/oss | wujinhu | wujinhu | | | NN status discovery does not leverage delegation token | Major | . | Chen Liang | Chen Liang | | | [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '\_' as identifier | Major | build | Devaraj Kavali | Zsolt Venczel | | | YarnClient should contact TimelineReader for application/attempt/container report | Critical | . | Rohith Sharma K S | Abhishek Modi | | | ObserverReadProxyProviderWithIPFailover should work with HA configuration | Major | . | Chen Liang | Chen Liang | | | [YARN-8851] Add basic pluggable device plugin framework | Major | . | Zhankun Tang | Zhankun Tang | | | Add Command Line interface to invoke interactive docker shell | Major | . | Zian Chen | Eric Yang | | | [CSI] CSI driver adaptor module support in NodeManager | Major | . | Weiwei Yang | Weiwei Yang | | | Add security check for container user is same as websocket user | Major | nodemanager | Eric Yang | Eric Yang | | | RBF: Add Storage policies related ClientProtocol APIs | Major | . | Dibyendu Karmakar | Dibyendu Karmakar | | | RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService | Minor | . | Ranith Sardar | Ranith Sardar | | | Allow manual failover between standby and observer | Major | . | Chao Sun | Chao Sun | | | Fix the order of logging arguments in ObserverReadProxyProvider. | Major | logging | Konstantin Shvachko | Ayush Saxena | | | Fix FederationStateStoreFacade#buildGetSubClustersCacheRequest | Major | federation | Bibin Chundatt | Bibin Chundatt | | | [CSI] YARN service fail to launch due to CSI changes | Blocker | . | Eric Yang | Weiwei Yang | | | publish all exposed ports to random ports when using bridge network | Minor | yarn | dockerzhang | dockerzhang | | | ApplicationCLI should have option to take clusterId | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | LocalMetadataStore put() does not retain isDeleted in parent listing | Major | fs/s3 | Gabor Bota | Gabor Bota | | | S3A log message on rm s3a://bucket/ not intuitive | Trivial | fs/s3 | Steve Loughran | Gabor Bota | | | [YARN-8851] Add a shared device mapping manager (scheduler) for device plugins | Major |

| Zhankun Tang | Zhankun Tang | | | Move DockerCommandPlugin volume related APIs' invocation from DockerLinuxContainerRuntime#prepareContainer to #launchContainer | Major | . | Zhankun Tang | Zhankun Tang | | | Add specification of Multipart Upload API to FS specification, with contract tests | Blocker | fs, test | Steve Loughran | Ewan Higgs | | | ITestS3GuardTool failures in testDestroyNoBucket() | Minor | fs/s3 | Aaron Fabbri | Gabor Bota | | | Add ingress port based sasl resolver | Major | security | Chen Liang | Chen Liang | | | ORFPP should also clone DT for the virtual IP | Major | . | Chen Liang | Chen Liang | | | RBF: LS command for root shows wrong owner and permission information. | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: MIN\ACTIVE\RATIO should be configurable | Major | . | Fei Hui | Fei Hui | | | [CSI] CSI jar file should not bundle third party dependencies | Blocker | build | Eric Yang | Weiwei Yang | | | Add xtermjs to YARN UI2 | Major | yarn-ui-v2 | Eric Yang | Eric Yang | | | Create user guide for "Consistent reads from Observer" feature. | Major | documentation | Konstantin Shvachko | Chao Sun | | | s3guard bucket-info will create s3guard table if FS is set to do this automatically | Major | fs/s3 | Steve Loughran | Gabor Bota | | | Should be able to set empty directory flag to TRUE in DynamoDBMetadataStore#innerGet when using authoritative directory listings | Major | fs/s3 | Gabor Bota | Gabor Bota | | | Move ipfailover config key out of HdfsClientConfigKeys | Minor | . | Chen Liang | Chen Liang | | | [DevicePlugin] Support NM APIs to query device resource allocation | Major | . | Zhankun Tang | Zhankun Tang | | | [DevicePlugin] Add an interface for device plugin to provide customized scheduler | Major | . | Zhankun Tang | Zhankun Tang | | | Add ability to use interactive shell with normal yarn container | Major | . | Eric Yang | Eric Yang | | | ObserverNode should reject read requests when it is too far behind. | Major | hdfs-client, namenode | Konstantin Shvachko | Konstantin Shvachko | | | Description errors in the comparison logic of transaction ID | Minor | . | xiangheng | xiangheng | | | Handle exception from internalQueueCall | Critical | ipc | Chao Sun | Chao Sun | | | Support sh shell for interactive container shell at command line | Blocker | yarn | Eric Yang | Eric Yang | | | Add Terminal Link to Service component instance page for UI2 | Major | yarn-ui-v2 | Eric Yang | Eric Yang | | | Add flag to disable interactive shell | Major | . | Eric Yang | Eric Yang | | | Improve terminal message when connection is refused | Major | . | Eric Yang | Eric Yang | | | RBF: Fix a typo in RouterAdmin usage | Major | . | Takanobu Asanuma | Ayush Saxena | | | RBF: Handle NPE

NamenodeBeanMetrics#getFederationMetrics | Major | namenode | Surendra Singh Lilhore | Ranith Sardar | | | [SPS] : Add Support for Storage Policy Satisfier in ViewFs | Major | federation | Ayush Saxena | Ayush Saxena | | | ABFS: getNamespaceEnabled can fail blocking user access thru ACLs | Major | fs/azure | Da Zhou | Da Zhou | | | ABFS: reduce list page size to to 500 | Major | fs/azure | Da Zhou | Da Zhou | | | ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException | Major | fs/azure | Da Zhou | Da Zhou | | | Carriage Return character in launch command cause node manager to become unhealthy | Major | . | Eric Yang | Billie Rinaldi | | | Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. | Major | hdfs-client | Chen Liang | Chao Sun | | | Adjust annotations on new interfaces/classes for SBN reads. | Major | . | Konstantin Shvachko | Chao Sun | | | RBF: Make the read-only column of Mount Table clearly understandable | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Web browser close without proper exit can leak shell process | Major | . | Eric Yang | Eric Yang | | | Container shell does not work when using yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user is set | Major | . | Eric Yang | Eric Yang | | | Dynamically add or remove auxiliary services | Major | nodemanager | Billie Rinaldi | Billie Rinaldi | | | RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. | Major | fs | Mohammad Arshad | Mohammad Arshad | | | Container reinit always fails in branch-3.2 and trunk | Major | . | Eric Yang | Chandni Singh | | | ObserverReadInvocationHandler should implement RpcInvocationHandler | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Ensure flush after printing to log plus additional cleanup | Major | . | Billie Rinaldi | Eric Yang | | | Interactive docker shell | Major | . | Eric Yang | Zian Chen | | | [JDK 11] Update maven.plugin-tools.version to 3.6.0 | Major | build | Devaraj Kavali | Dinesh Chitlangia | | | Document dfs.ha.tail-edits.period in user guide. | Major | documentation | Chao Sun | Chao Sun | | | Add bouncycastle jars to hadoop-aws as test dependencies | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | Document usage of Dynamic auxiliary services | Major | . | Eric Yang | Billie Rinaldi | | | Auxiliary service REST API query does not return running services | Major | . | Eric Yang | Billie Rinaldi | | | Updating distributed node attributes only when necessary | Major | resourcemanager | Tao Yang | Tao Yang | | | Add port mapping handling when docker container use bridge network | Major | . | Jun Gong | Xun Liu | | | Fix white spaces related to SBN reads. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Replace the url of the repository in Apache Hadoop source code | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | ABFS: Throw IllegalArgumentException when Directory/File name ends with a period(.) | Major | fs/azure | Sean Mackrory | Shweta | | | RBF: Add stale nodes to federation metrics | Major | . | igo Goiri | igo Goiri | | | RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection | Major | . | Fei Hui | Fei Hui | | | AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore | Major | fs/oss | wujinhu | wujinhu | | | Auxiliary manifest file deleted from HDFS does not trigger service to be removed | Major |

| Eric Yang | Billie Rinaldi | | | [CSI] Add ability to publish/unpublish volumes on node managers | Major | . | Weiwei Yang | Weiwei Yang | | | Allow port range to be specified while starting NM Timeline collector manager. | Major | timelineserver | Varun Saxena | Abhishek Modi | | | Fix logging for preemption of Opportunistic containers for Guaranteed containers. | Minor | . | Abhishek Modi | Abhishek Modi | | | [JDK 11] Update maven-shade-plugin.version to 3.2.1 | Major | build | Devaraj Kavali | Dinesh Chitlangia | | | Add metrics for queued opportunistic and guaranteed containers. | Major | . | Abhishek Modi | Abhishek Modi | | | Nvidia-docker v2 support for YARN GPU feature | Critical | . | Zhankun Tang | dockerzhang | | | [CSI] Ignore volume resource in resource calculators based on tags | Major | . | Weiwei Yang | Sunil G | | | RBF: Quotas of the sub-cluster should be removed when removing the mount point | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Remove hard coded router status from FederationMetrics. | Major | . | Ranith Sardar | Ranith Sardar | | | [DOC] Effective use of FS instances during S3A integration tests | Major | fs/s3 | Gabor Bota | Gabor Bota | | | RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | Don't run TestDU on Windows | Trivial | common, test | Lukas Majercak | Lukas Majercak | | | S3A to support Delegation Tokens | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Cleanup quota modules | Major | . | igo Goiri | igo Goiri | | | RBF: Create new policy provider for router | Major | namenode | Surendra Singh Lilhore | Ranith Sardar | | | [JDK 11] Compilation failure: package com.sun.jndi.ldap is not visible | Major | common | Uma Maheswara Rao G | Takanobu Asanuma | | | RBF: Inconsistency with the Default Namespace | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: rollEdit() command fails with Router | Major | . | Harshakiran Reddy | Shubham Dewan | | | [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/\\ classes for \>= Java 9 | Major | build | Devaraj Kavali | Devaraj Kavali | | | [JDK11] TestIPC.testRTEDuringConnectionSetup fails | Major | . | Akira Ajisaka | Zsolt Venczel | | | REST API to trigger storing auxiliary manifest file and publish to NMs | Major | . | Eric Yang | Billie Rinaldi | | | Add UT to verify node-attributes in RM nodes rest API | Minor | resourcemanager, restapi | Weiwei Yang | Prabhu Joseph | | | RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable | Major | . | Shubham Dewan | Shubham Dewan | | | Capacity Scheduler: implements queue level maximum-allocation inheritance | Major | capacity scheduler | Aihua Xu | Aihua Xu | | | Retrieve the status of resource localization | Major | yarn | Chandni Singh | Chandni Singh | | | RBF: Add configuration documents for using multiple sub-clusters | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Add a flag to enable dynamic auxiliary service feature | Major | . | Eric Yang | Billie Rinaldi | | | RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations | Major |

| Ayush Saxena | Ayush Saxena | | | RBF: Remove dependency on availability of default namespace | Major | . | Ayush Saxena | Ayush Saxena | | | Docker container rm command should be executed after stop | Major | . | Zhaohui Xin | Zhaohui Xin | | | [CSI] Run csi-driver-adaptor as aux service | Major | . | Weiwei Yang | Weiwei Yang | | | Move Mockito up to version 2.23.4 | Major | test | Steve Loughran | Akira Ajisaka | | | UserAgent string for ABFS | Major | fs/azure | Shweta | Shweta | | | [JDK 11] Remove animal-sniffer-maven-plugin to fix build | Major | build | Devaraj Kavali | Dinesh Chitlangia | | | RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace | Minor | federation | Surendra Singh Lilhore | Ranith Sardar | | | Add link to NodeAttributes doc in PlacementConstraints document | Minor | documentation | Weiwei Yang | Masatake Iwasaki | | | Have a way to turn off container metrics from NMs | Major | timelineserver | Vrushali C | Abhishek Modi | | | RBF : Exceptions are exposing the actual sub cluster path | Major | . | Ayush Saxena | Ayush Saxena | | | ABFS: Enable owner and group conversion for MSI and login user using OAuth | Major | fs/azure | junhua gu | Da Zhou | | | Add UT to verify Placement Constraint in Distributed Shell | Major | . | Prabhu Joseph | Prabhu Joseph | | | Allocation Tag Namespace support in Distributed Shell | Major | distributed-shell | Prabhu Joseph | Prabhu Joseph | | | Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options | Minor | . | Weiwei Yang | Prabhu Joseph | | | REST API support for node resource configuration | Major | graceful, nodemanager, scheduler | Junping Du | igo Goiri | | | Document docker registry deployment with NFS Gateway | Major | . | Eric Yang | Eric Yang | | | Add support for S3 Select to S3A | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Throw RetriableException instead of IOException when no namenodes available | Major | . | Fei Hui | Fei Hui | | | Docker run doesn't pull down latest image if the image exists locally | Major | nodemanager | Zhaohui Xin | Zhaohui Xin | | | Allow wrapping NN QOP into token in encrypted message | Major | . | Chen Liang | Chen Liang | | | RBF: Support for Delegation Token (RPC) | Major | . | Sherwood Zheng | CR Hota | | | [SBN read] Unclear Log.WARN message in GlobalStateIdContext | Major | hdfs | Shweta | Shweta | | | Optimize MockAMLauncher event handling | Major | . | Bibin Chundatt | Bibin Chundatt | | | RBF: Setting attributes should set on all subclusters' directories. | Major | . | Takanobu Asanuma | Ayush Saxena | | | RBF: Fix the location of the DNs in getDatanodeReport() | Major | . | igo Goiri | igo Goiri | | | [YARN-8851] Phase 1 - Support device isolation and use the Nvidia GPU plugin as an example | Major |

| Zhankun Tang | Zhankun Tang | | | s3guard bucket-info command to not print a stack trace on bucket-not-found | Minor | fs/s3 | Steve Loughran | Adam Antal | | | Wasb tests to downgrade to skip when test a/c is namespace enabled | Major | fs/azure, test | Steve Loughran | Masatake Iwasaki | | | RBF: Tooling to identify the subcluster location of a file | Major | . | igo Goiri | igo Goiri | | | Support to specify allocation tags without constraint in distributed shell CLI | Major | distributed-shell | Prabhu Joseph | Prabhu Joseph | | | [YARN-8851] Improve debug message in device plugin method compatibility check of ResourcePluginManager | Trivial | . | Zhankun Tang | Zhankun Tang | | | Documentation of the pluggable device framework | Major | documentation | Zhankun Tang | Zhankun Tang | | | Document docker registry deployment with direct S3 driver | Major | . | Eric Yang | Suma Shivaprasad | | | [YARN-8851] GPU hierarchy/topology scheduling support based on pluggable device framework | Major | . | Zhankun Tang | Zhankun Tang | | | Make ZKFC ObserverNode aware | Major | ha | Konstantin Shvachko | xiangheng | | | [YARN-8851] Fix a bug that lacking cgroup initialization when bootstrap DeviceResourceHandlerImpl | Major | . | Zhankun Tang | Zhankun Tang | | | Move DurationInfo from hadoop-aws to hadoop-common org.apache.hadoop.util | Minor | fs/s3, util | Steve Loughran | Abhishek Modi | | | Improve the documentation for the docker.trusted.registries configuration | Major | . | Simon Prewo | Eric Yang | | | ABFS: Should only transform username to short name | Major | . | Da Zhou | Da Zhou | | | Add support for Docker Images command | Major | yarn | Chandni Singh | Chandni Singh | | | Limit the flow name size & consider cleanup for hex chars | Major | timelineserver | Vrushali C | Sushil Ks | | | [Storage implementation] Adding retry semantics to HDFS backing storage | Major | timelineserver | Tsuyoshi Ozawa | Abhishek Modi | | | ABFS Authentication and Delegation Token plugins to optionally be bound to specific URI of the store | Major | fs/azure | Steve Loughran | Steve Loughran | | | Privileged, trusted containers should be supported only in ENTRYPOINT mode | Major | . | Eric Badger | Eric Yang | | | RBF: Fix safemode message for Router | Major | . | igo Goiri | Ranith Sardar | | | RBF: Add maintenance nodes to federation metrics | Major | . | Ayush Saxena | Ayush Saxena | | | Moving logging APIs over to slf4j in hadoop-yarn-common | Major | . | Yeliang Cang | Prabhu Joseph | | | RBF: IOE While Removing Mount Entry | Major | rbf | Surendra Singh Lilhore | Ayush Saxena | | | RBF: Fix heartbeat typos in the Router. | Trivial | . | CR Hota | CR Hota | | | Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager | Major | . | Yeliang Cang | Prabhu Joseph | | | Erasure Coding: Append on striped file | Major | . | Jing Zhao | Ayush Saxena | | | NPE in setup/teardown of ITestAbfsDelegationTokens | Major | fs/azure, test | Steve Loughran | Steve Loughran | | | RBF: Use human readable format for long numbers in the Router UI | Major | . | igo Goiri | igo Goiri | | | Document docker registry deployment with Ozone CSI driver | Major |

| Eric Yang | Eric Yang | | | Adding a NoOp timeline writer and reader plugin classes for ATSv2 | Minor | ATSv2, timelineclient, timelineserver | Prabha Manepalli | Prabha Manepalli | | | FPGA plugin fails to recognize Intel Processing Accelerator Card | Critical | . | Peter Bacsko | Peter Bacsko | | | Docker image life cycle management on HDFS | Major | yarn | Eric Yang | Eric Yang | | | RBF: Fix renaming folders spread across multiple subclusters | Major | . | igo Goiri | Ayush Saxena | | | Automatically convert the launch command to the exec form when using entrypoint support | Major | . | Shane Kumpf | Eric Yang | | | [SBN Read] StateId and TrasactionId not present in Trace level logging | Trivial | namenode | Shweta | Shweta | | | General improvements in IntelFpgaOpenclPlugin | Major | . | Peter Bacsko | Peter Bacsko | | | Parquet reading S3AFileSystem causes EOF | Blocker | fs/s3 | Dave Christianson | Steve Loughran | | | Separate white list for docker.trusted.registries and docker.privileged-container.registries | Major | . | Eric Yang | Eric Yang | | | S3A input stream to use etags/version number to detect changed source files | Major | fs/s3 | Brahma Reddy Battula | Ben Roling | | | Refactor MappableBlock to align with the implementation of SCM cache | Major | caching, datanode | Feilong He | Feilong He | | | Replace isDebugEnabled with SLF4J parameterized log messages | Major | . | Prabhu Joseph | Prabhu Joseph | | | Update abfs storage back-end with "close" flag when application is done writing to a file | Major | fs/azure | Vishwajeet Dusane | Vishwajeet Dusane | | | Replace isDebugEnabled with SLF4J parameterized log messages for remaining code | Minor | yarn | Prabhu Joseph | Prabhu Joseph | | | Extend documentation in testing.md about endpoint constants | Trivial | fs/s3 | Adam Antal | Adam Antal | | | Remove commons-logging dependency from remaining hadoop-yarn | Major | . | Prabhu Joseph | Prabhu Joseph | | | AliyunOSS: improvements for copyFile/copyDirectory and logging | Major | fs/oss | wujinhu | wujinhu | | | RBF: Optimize configuration item resolving for monitor namenode | Major | rbf | Xiaoqiao He | Xiaoqiao He | | | Update document for ATS HBase Custom tablenames (-entityTableName) | Critical | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | FlowActivity and FlowRun table prefix is wrong | Minor | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | Javadoc error on FPGA related java files | Major | . | Eric Yang | Peter Bacsko | | | General improvements in FpgaResourceHandlerImpl | Major | . | Peter Bacsko | Peter Bacsko | | | Opportunistic containers should not be scheduled on Decommissioning nodes. | Major | . | Abhishek Modi | Abhishek Modi | | | S3AFileSystem#innerMkdirs builds needless lists | Trivial | fs/s3 | Lokesh Jain | Lokesh Jain | | | RBF: Prevent loading metric system when disabled | Major | . | igo Goiri | igo Goiri | | | Disable PATH variable to be passed to Docker container | Major | . | Eric Yang | Jim Brennan | | | General improvements in FpgaDevice | Major | . | Peter Bacsko | Peter Bacsko | | | Minor cleanup in FpgaResourceAllocator | Minor | . | Peter Bacsko | Peter Bacsko | | | S3Guard: NPE in

| Major | fs/s3 | Steve Loughran | Gabor Bota | | | S3Guard: Better support for out-of-band operations | Major | fs/s3 | Sean Mackrory | Gabor Bota | | | Refactor FsDatasetCache for SCM cache implementation | Major | . | Rakesh Radhakrishnan | Rakesh Radhakrishnan | | | S3A tests to include Terasort | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | Minor cleanup in TestFpgaDiscoverer | Minor | . | Peter Bacsko | Peter Bacsko | | | Application Catalog initial project setup and source | Major | applications | Eric Yang | Eric Yang | | | Build issues on hadoop-yarn-application-catalog-webapp | Major | . | Eric Yang | Eric Yang | | | RBF: Support unavailable subclusters for mount points with multiple destinations | Major | . | igo Goiri | igo Goiri | | | Implement HDFS cache on SCM by using pure java mapped byte buffer | Major | caching, datanode | Feilong He | Feilong He | | | Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project | Major | . | Gabor Bota | Gabor Bota | | | Improve recommend applications order | Major | . | Eric Yang | Eric Yang | | | ATSV2 /apps/appId/entities/YARN\_CONTAINER rest api does not show metrics | Critical | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | findbugs warning of null param to non-nullable method in Configuration with Guava update | Minor | build | Steve Loughran | Steve Loughran | | | Username splits won't help timelineservice.app\_flow table | Major | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | Update guava to 27.0-jre in hadoop-project trunk | Critical | build | Gabor Bota | Gabor Bota | | | Component name should start with Apache Hadoop for consistency | Minor | . | Weiwei Yang | Weiwei Yang | | | S3AUtils.translateException to map CredentialInitializationException to AccessDeniedException | Major | . | Steve Loughran | Steve Loughran | | | RBF: RouterAdmin update cmd is overwriting the entry not updating the existing | Major | . | Dibyendu Karmakar | Ayush Saxena | | | Publish container killed, paused and resumed events to ATSv2. | Major | . | Abhishek Modi | Abhishek Modi | | | [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls | Major | . | Vrushali C | Abhishek Modi | | | Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities | Major | . | Tao Yang | Tao Yang | | | RBF: Fix trailing "/" for webhdfs | Major | . | CR Hota | Akira Ajisaka | | | In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. | Major | graceful, nodemanager, scheduler | Junping Du | igo Goiri | | | Add Opportunistic Scheduler metrics in ResourceManager. | Major | . | Abhishek Modi | Abhishek Modi | | | S3A MarshalledCredentials.toString() doesn't print full date/time of expiry | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP | Major | . | Chen Liang | Chen Liang | | | S3AInputStream to implement CanUnbuffer | Major | fs/s3 | Steve Loughran | Sahil Takiar | | | Fix new findbugs issues after update guava to 27.0-jre in hadoop-project trunk | Critical | . | Gabor Bota | Gabor Bota | | | Add express upgrade button to Appcatalog UI | Major |

| Eric Yang | Eric Yang | | | Remove hard coded sleep from Opportunistic Scheduler tests. | Major | . | Abhishek Modi | Abhishek Modi | | | Support asynchronized scheduling mode and multi-node lookup mechanism for app activities | Major | capacityscheduler | Tao Yang | Tao Yang | | | Force removal of docker containers that do not get removed on first try | Major | . | Eric Badger | Eric Yang | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-api | Major | test | Akira Ajisaka | Akira Ajisaka | | | Update AssertJ to 3.12.2 | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Router shouldn't allow READ operations in safe mode | Major | . | igo Goiri | igo Goiri | | | Add security filters to Application catalog | Major | security, yarn-native-services | Eric Yang | Eric Yang | | | App catalog navigation stylesheet does not display correctly in Safari | Major | . | Eric Yang | Eric Yang | | | [hadoop-yarn-project] Fix order of actual and expected expression in assert statements | Major | . | Prabhu Joseph | Prabhu Joseph | | | Fix Opportunistic Scheduling for node local allocations. | Major | . | Abhishek Modi | Abhishek Modi | | | Externalize Solr data storage | Major | . | Eric Yang | Eric Yang | | | Limit the number of opportunistic container allocated per AM heartbeat | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Abhishek Modi | | | Build Common module for MaWo application | Major | . | Yesha Vora | Yesha Vora | | | Create basic VE plugin | Major | nodemanager | Peter Bacsko | Peter Bacsko | | | Use configurable dynamo table name prefix in S3Guard tests | Major | fs/s3 | Ben Roling | Ben Roling | | | RBF: Support for Delegation Token (WebHDFS) | Major | . | igo Goiri | CR Hota | | | Fix new deprecations after guava 27.0 update in trunk | Major | . | Gabor Bota | Gabor Bota | | | RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin' | Major | rbf | luhuachao | luhuachao | | | Docker container exited with failure does not get clean up correctly | Major | . | Eric Yang | Eric Yang | | | ABFS: add bufferpool to AbfsOutputStream | Major | fs/azure | Da Zhou | Da Zhou | | | Create unit tests for VE plugin | Major | . | Peter Bacsko | Peter Bacsko | | | S3Guard: fail write that doesn't update metadata store | Major | fs/s3 | Ben Roling | Ben Roling | | | RBF: getContentSummary() should allow non-existing folders | Major | . | igo Goiri | igo Goiri | | | Improve diagnostics for scheduler and app activities | Major | capacityscheduler | Tao Yang | Tao Yang | | | [JDK 11] Remove unintentional override of the version of Maven Dependency Plugin | Major | build | Akira Ajisaka | Xieming Li | | | Refine the implementation for HDFS cache on SCM | Major | caching, datanode | Feilong He | Feilong He | | | ABFS: add listFileStatus with StartFrom | Major | fs/azure | Da Zhou | Da Zhou | | | Support filtering by request-priorities and allocation-request-ids for query results of app activities | Major |

| Tao Yang | Tao Yang | | | Improve cleanup process of app activities and make some conditions configurable | Major | capacityscheduler | Tao Yang | Tao Yang | | | RBF: Add delegation token total count as one of the federation metrics | Major | . | Fengnan Li | Fengnan Li | | | AliyunOSS: Remove temporary files when upload small files to OSS | Major | fs/oss | wujinhu | wujinhu | | | RBF: ACL commands should work over all the destinations | Major | . | Shubham Dewan | Ayush Saxena | | | RBF: Remove unnecessary quota checks | Major | . | Ayush Saxena | Ayush Saxena | | | S3A SSL connections should use OpenSSL | Major | fs/s3 | Justin Uang | Sahil Takiar | | | RBF: Router should support RefreshUserMappingsProtocol | Major | rbf | Shen Yinjie | Shen Yinjie | | | Add container allocation latency for Opportunistic Scheduler | Major | . | Abhishek Modi | Abhishek Modi | | | S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite | Major | fs/s3 | Ben Roling | Ben Roling | | | RBF: Security documentation | Major | . | CR Hota | CR Hota | | | KerberosAuthenticationHandler Trusted Proxy Support for Knox | Major | auth | Prabhu Joseph | Prabhu Joseph | | | RBF: Optimize the file write process in case of multiple destinations. | Major | . | Ayush Saxena | Ayush Saxena | | | Use FileChannel.transferTo() method for transferring block to SCM cache | Major | caching, datanode | Feilong He | Feilong He | | | Support grouping by diagnostics for query results of scheduler and app activities | Major | . | Tao Yang | Tao Yang | | | RBF: Fail when try to remove mount point paths | Major | . | Wu Weiwei | Akira Ajisaka | | | Remove S3A's depedency on http core | Critical | build, fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Add Cache pools and directives related ClientProtocol APIs | Major | . | Dibyendu Karmakar | Ayush Saxena | | | Add signalToContainer REST API for RMWebServices | Major | restapi | Tao Yang | Tao Yang | | | RBF: Create hdfs-rbf-site.xml for RBF specific properties | Major | rbf | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Add Snapshot related ClientProtocol APIs | Major | federation | Ranith Sardar | igo Goiri | | | fs -expunge to take a filesystem | Minor | fs | Steve Loughran | Shweta | | | RBF: Expose router security enabled status on the UI | Major | . | CR Hota | CR Hota | | | RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key | Major | . | maobaolong | Ayush Saxena | | | S3Guard to support on-demand DDB tables | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Clean-up and refactor UI components | Minor | . | CR Hota | Takanobu Asanuma | | | [Atsv2] load hbase configuration from filesystem rather than URL | Major | . | Rohith Sharma K S | Prabhu Joseph | | | Replace \`mapreduce.job.counters.limit\` with \`mapreduce.job.counters.max\` in mapred-default.xml | Major | . | Wanqiang Ji | Wanqiang Ji | | | RBF: Update the document of RBF related metrics | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | RBF:

fails | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | Make sure all end point URL is covered by the same AuthenticationFilter | Major | security | Eric Yang | Prabhu Joseph | | | Implement HDFS cache on SCM with native PMDK libs | Major | caching, datanode | Feilong He | Feilong He | | | Update AWS SDK to 1.11.563 | Major | build, fs/s3 | Steve Loughran | Steve Loughran | | | Correct incompatible, incomplete and redundant activities | Major | . | Tao Yang | Tao Yang | | | Remove unused properties from TTConfig.java | Major | . | Akira Ajisaka | Akira Ajisaka | | | RBF: Failed to get statistics from NameNodes before 2.9.0 | Major | . | Akira Ajisaka | Xiaoqiao He | | | S3Guard to support creating on-demand DDB tables | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Make queue size of BlockReportProcessingThread configurable | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Remove unused pieces related to \`mapreduce.job.userlog.retain.hours\` | Major | . | Wanqiang Ji | Wanqiang Ji | | | Enable AuthFilter as default for WebHdfs | Major | security | Prabhu Joseph | Prabhu Joseph | | | ApplicationHistoryServer related testcases failing | Major | security, test | Prabhu Joseph | Prabhu Joseph | | | Add limit/actions/summarize options for app activities REST API | Major | capacityscheduler | Tao Yang | Tao Yang | | | Fix TimelineReaderServer ignores ProxyUserAuthenticationFilterInitializer | Major | security | Prabhu Joseph | Prabhu Joseph | | | RBF: Router should support GetUserMappingsProtocol | Major | . | igo Goiri | Ayush Saxena | | | ATS v2 Generic TimelineStorageMonitor | Major | ATSv2 | Sunil G | Prabhu Joseph | | | S3Guard: Implement time-based (TTL) expiry for entries (and tombstones) | Major | fs/s3 | Gabor Bota | Gabor Bota | | | ABFS: Override access() to no-op for now | Major | fs/azure | Da Zhou | Da Zhou | | | ArtifactId of MaWo application is wrong | Major | . | Wanqiang Ji | Wanqiang Ji | | | ABFS driver continues to retry on IOException responses from REST operations | Major | fs/azure | Robert Levas | Robert Levas | | | S3AInputStream#unbuffer should merge input stream stats into fs-wide stats | Major | fs/s3 | Sahil Takiar | Sahil Takiar | | | S3Guard store becomes inconsistent after partial failure of rename | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Memory leak in S3AOutputStream | Major | fs/s3 | Piotr Nowojski | Steve Loughran | | | S3Guard table destroy to map IllegalArgumentExceptions to IOEs | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Bulk commits of S3A MPUs place needless excessive load on S3 & S3Guard | Major | fs/s3 | Gabor Bota | Steve Loughran | | | S3A list operation doesn't pick up etags from results | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | HBaseTimelineWriterImpl sync writes has to avoid thread blocking if storage down | Major | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | S3Guard DDB store prune() doesn't translate AWS exceptions to IOEs | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | [SBN Read] Add the document link to the top page | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Implement VE discovery using libudev | Major |

| Peter Bacsko | Peter Bacsko | | | ContainersMonitorImpl need be adjusted when NM resource changed. | Major | graceful, nodemanager, scheduler | Junping Du | igo Goiri | | | RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default | Major | . | igo Goiri | Takanobu Asanuma | | | Auto adjust max queue length of app activities to make sure activities on all nodes can be covered | Major | . | Tao Yang | Tao Yang | | | Restructure DockerLinuxContainerRuntime to extend a new OCIContainerRuntime | Major | . | Eric Badger | Eric Badger | | | RBF: Fix 'not a super user' error when disabling a namespace in kerberos with superuser principal | Major | . | luhuachao | luhuachao | | | [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used | Major | tools | TanYuxin | Erik Krogen | | | Enhance documentation of Docker on YARN support | Major | documentation, nodemanager | Peter Bacsko | Peter Bacsko | | | [Dynamometer] Fix TestDynamometerInfra failures | Major | test, tools | Erik Krogen | Erik Krogen | | | Make Dynamometer documentation properly compile onto the Hadoop site | Major | . | Erik Krogen | Erik Krogen | | | TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists | Minor | fs/s3 | Prabhu Joseph | Steve Loughran | | | Move handshake secret field from Token to BlockAccessToken | Blocker | hdfs | Chen Liang | Chen Liang | | | S3A: Avoid inconsistencies between DDB and S3 | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Hadoop S3Guard Prune command to support a -tombstone option. | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | ITestDynamoDBMetadataStore.testProvisionTable times out intermittently | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | Report pmem stats to namenode | Major | . | Feilong He | Feilong He | | | Update documentation for HDFS cache on SCM support | Major | . | Feilong He | Feilong He | | | RBF: Implement deletion feature for expired records in State Store | Major | rbf | Takanobu Asanuma | Takanobu Asanuma | | | Pass ITtlTimeProvider instance in initialize method in MetadataStore interface | Major | . | Gabor Bota | Gabor Bota | | | RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: FederationUtil#newInstance should allow constructor without context | Major | . | CR Hota | CR Hota | | | S3Guard testConcurrentTableCreations to set r & w capacity == 0 | Major | fs/s3, test | Steve Loughran | lqjacklee | | | S3A tombstones can confuse empty directory status | Blocker | fs/s3, test | Steve Loughran | Steve Loughran | | | Upgrade JUnit5 from 5.3.1 to 5.5.1 to support global timeout | Major | test | Akira Ajisaka | Kevin Su | | | RBF: Create secret manager instance using FederationUtil#newInstance. | Major | . | CR Hota | CR Hota | | | [Dynamometer] Unnecessary duplicate bin directory appears in dist layout | Major | namenode, test | Erik Krogen | Erik Krogen | | | findbugs warning on LocalMetadataStore.ttlTimeProvider sync | Major | build, fs/s3 | Steve Loughran | Steve Loughran | | | Upgrade Mockito to the latest version | Minor | test | Akira Ajisaka | Kevin Su | | | ABFS

returns localized time instead of UTC | Major | fs/azure | Joan Sala Reixach | Bilahari T H | | | Support getQuotaUsage API in WebHDFS | Major | fs, webhdfs | Erik Krogen | Chao Sun | | | Clean up pmem cache before setting pmem cache capacity | Minor | caching, datanode | Feilong He | Feilong He | | | ABFS: transform full UPN for named user in AclStatus | Major | fs/azure | Da Zhou | Da Zhou | | | S3A retry policy to be exponential | Critical | fs/s3 | Steve Loughran | Steve Loughran | | | ITestS3GuardDDBRootOperations.test\300\MetastorePrune needs to set region | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | UI always show default-rack for all the nodes while running SLS. | Major | . | Abhishek Modi | Abhishek Modi | | | [hadoop-mapreduce-project] Fix order of actual and expected expression in assert statements | Major | . | Adam Antal | Adam Antal | | | Junit not found in hadoop-dynamometer-infra | Major | . | Kevin Su | Kevin Su | | | DecommissioningNodesWatcher should get lists of running applications on node from RMNode. | Major | . | Abhishek Modi | Abhishek Modi | | | S3ADelegationTokens to only log at debug on startup | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Remove reapDockerContainerNoPid left behind by YARN-9074 | Trivial | yarn | Adam Antal | Kevin Su | | | RBF: RouterAdmin supports refreshRouterArgs command but not on display | Major | . | wangzhaohui | wangzhaohui | | | SLS runner crashes when run with metrics turned off. | Major | . | Abhishek Modi | Abhishek Modi | | | Add support for allocation id in SLS. | Major | . | Abhishek Modi | Abhishek Modi | | | RBF: implement getReplicatedBlockStats interface | Major | . | Chen Zhang | Chen Zhang | | | RBF: getQuotaUsage may ignore some folders | Major | . | Chen Zhang | Chen Zhang | | | RBF: Non secured routers should not log in error mode when UGI is default. | Major | . | CR Hota | CR Hota | | | Fix package name of audit log class in Dynamometer document | Major | documentation, tools | Takanobu Asanuma | Takanobu Asanuma | | | Make last AWS credential provider in default auth chain EC2ContainerCredentialsProviderWrapper | Major | fs/s3 | Steve Loughran | Steve Loughran | | | [Dynamometer] Hadoop-2 DataNode fail to start | Major | tools | Takanobu Asanuma | Takanobu Asanuma | | | Typo in s3a committers.md doc | Minor | documentation, fs/s3 | Steve Loughran | | | | [SBN read] Got an unexpected txid when tail editlog | Blocker | . | wangzhaohui | wangzhaohui | | | RBF: MountTableStoreImpl#getMountTableEntries returns extra entry | Major | . | Chen Zhang | Chen Zhang | | | RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore | Major | restapi | Prabhu Joseph | Prabhu Joseph | | | RM fails to start with FileSystemBasedConfigurationProvider | Major | resourcemanager | Prabhu Joseph | Prabhu Joseph | | | Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns | Major | . | Jonathan Hung | Jonathan Hung | | | mark DynamoDBMetadataStore.deleteTrackingValueMap as final | Trivial | fs/s3 | Steve Loughran | Kevin Su | | | WebHDFS : Support setQuota | Major | . | nijel | Chao Sun | | | Add support for arbitrary DAG AM Simulator. | Major |

| Abhishek Modi | Abhishek Modi | | | Improve response of scheduler/app activities for better understanding | Major | . | Tao Yang | Tao Yang | | | Queue Management API - rephrase error messages | Major | . | Akhil PB | Prabhu Joseph | | | RBF: RBFMetrics throws NullPointerException if stateStore disabled | Major | . | Chen Zhang | Chen Zhang | | | Queue Mutation API does not allow to remove a config | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Do ACLs check while retrieving entity-types per application | Major | . | Rohith Sharma K S | Prabhu Joseph | | | RBF: TestRouterRpc#testNamenodeMetrics is flaky | Major | . | Takanobu Asanuma | Chen Zhang | | | Update ATSv2 document for latest feature supports | Blocker | . | Rohith Sharma K S | Rohith Sharma K S | | | [Dynamometer] Cannot parse audit logs with = in unexpected places when starting a workload. | Major | . | Soya Miyoshi | Soya Miyoshi | | | [Dynamometer] start-dynamometer-cluster.sh shows its usage even if correct arguments are given. | Major | tools | Soya Miyoshi | Soya Miyoshi | | | NM hangs at serviceStop when ATSV2 Backend Hbase is Down | Major | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | S3AFilesystem.delete to incrementally update s3guard with deletions | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Avoid/handle cached 404s during S3A file creation | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Make TestOpportunisticContainerAllocatorAMService more resilient. | Major | . | Abhishek Modi | Abhishek Modi | | | [pb-upgrade] Update docker image to have 3.7.1 protoc executable | Major | . | Vinayakumar B | Vinayakumar B | | | RM crashes due to runtime errors in TimelineServiceV2Publisher | Major | . | Tarun Parimi | Tarun Parimi | | | Option to disable GCM for SSL connections when running on Java 8 | Major | fs/s3 | Sahil Takiar | Sahil Takiar | | | [SBN read] Revisit GlobalStateIdContext locking when getting server state id | Major | hdfs | Chen Liang | Chen Liang | | | [pb-upgrade] Upgrade protobuf.version to 3.7.1 | Major | . | Vinayakumar B | Vinayakumar B | | | RBF: Router Update Doesn't Sync Quota | Major | . | Ayush Saxena | Ayush Saxena | | | [pb-upgrade] Update docker image to make 3.7.1 protoc as default | Major | . | Vinayakumar B | Vinayakumar B | | | Check native pmdk lib by 'hadoop checknative' command | Minor | native | Feilong He | Feilong He | | | [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes | Major | common | Vinayakumar B | Vinayakumar B | | | Region must be provided when requesting session credentials or SdkClientException will be thrown | Major | fs/s3 | Gabor Bota | Gabor Bota | | | hadoop fs mkdir / of nonexistent abfs container raises NPE | Minor | fs/azure | Steve Loughran | Gabor Bota | | | S3A ITest\*MRjob failures | Major | fs/s3 | Siddharth Seth | Siddharth Seth | | | [YARN] use protobuf-maven-plugin to generate protobuf classes | Major | . | Vinayakumar B | Duo Zhang | | | [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes | Major |

| Vinayakumar B | Duo Zhang | | | RBF: Fix intermittently failing kerberos related unit test | Major | . | CR Hota | Xiaoqiao He | | | [SBN read] Change client logging to be less aggressive | Major | hdfs | Chen Liang | Chen Liang | | | Refactor OpportunisticContainerAllocator | Major | . | Abhishek Modi | Abhishek Modi | | | Format CS Configuration present in Configuration Store | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | SchedConfCli does not work with https mode | Major | . | Prabhu Joseph | Prabhu Joseph | | | LocatedFileStatusFetcher scans failing intermittently against S3 store | Major | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: fileSystemExists() should not call container level apis | Major | fs/azure | Da Zhou | Sneha Vijayarajan | | | Remove unused function from OpportunisticContainerAllocatorAMService | Minor | . | Abhishek Modi | Abhishek Modi | | | Document examples of SchedulerConf with Node Labels | Minor | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Add S3A implementation of FSMainOperationsBaseTest | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | [SBN read] Allow configurably enable/disable AlignmentContext on NameNode | Major | hdfs | Chen Liang | Chen Liang | | | [pb-upgrade] Remove protocol buffers 3.7.1 from requirements in BUILDING.txt | Minor | build, documentation | Akira Ajisaka | Akira Ajisaka | | | [s3a] stop treat fs.s3a.max.threads as the long-term minimum | Major | fs/s3 | Sean Mackrory | Sean Mackrory | | | StandbyNode should upload FsImage to ObserverNode after checkpointing. | Major | hdfs | Konstantin Shvachko | Chen Liang | | | Avoid DNS resolution while running SLS. | Major | . | Abhishek Modi | Abhishek Modi | | | Improved S3A MR tests | Critical | fs/s3, test | Steve Loughran | Steve Loughran | | | S3A committers leak threads/raises OOM on job/task commit at scale | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A ITestRestrictedReadAccess fails | Major | fs/s3 | Siddharth Seth | Steve Loughran | | | [hadoop-tools] Fix order of actual and expected expression in assert statements | Major | . | Adam Antal | Kevin Su | | | Make AAD endpoint configurable on all Auth flows | Minor | fs/azure | Bilahari T H | Bilahari T H | | | Mutation API Config Change need to update Version Number | Major | . | Prabhu Joseph | Prabhu Joseph | | | Balancer should work with ObserverNode | Major | . | Konstantin Shvachko | Erik Krogen | | | ITestS3AClosedFS failing -junit test thread | Blocker | fs/s3, test | Steve Loughran | Steve Loughran | | | Migration tool that help to generate CS config based on FS config [Phase 1] | Major |

| Wanqiang Ji | Peter Bacsko | | | S3A innerGetFileStatus s"directories only" scan still does a HEAD | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | S3A ITest failures without S3Guard | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | Capacity scheduler: add support for Secondary Group rule mapping | Major | capacity scheduler | Peter Bacsko | Manikandan R | | | S3 getBucketLocation() can return "US" for us-east | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3Guard bucket-info fails if the bucket location is denied to the caller | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add QueueMetrics for Custom Resources | Major | . | Manikandan R | Manikandan R | | | Capacity scheduler: add support for combined %user + %primary\_group mapping | Major | capacity scheduler | Peter Bacsko | Manikandan R | | | Make container-executor mount logic modular | Major | . | Eric Badger | Eric Badger | | | FSSchedulerConfigurationStore fails to update with hdfs path | Major | capacityscheduler | Prabhu Joseph | Prabhu Joseph | | | [Dynamometer] Fix scripts to refer to current build structure | Major | namenode, test | Erik Krogen | Takanobu Asanuma | | | [Dynamometer] DataNode can't find junit jar when using Hadoop-3 binary | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | [Dynamometer] Dynamometer in org.apache.hadoop.tools does not output the benchmark results. | Major | . | Soya Miyoshi | Takanobu Asanuma | | | [hadoop-common] Fix order of actual and expected expression in assert statements | Major | . | Adam Antal | Adam Antal | | | Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | RBF: namenode links in NameFederation Health page (federationhealth.html) cannot use https scheme | Major | rbf, ui | Xieming Li | Xieming Li | | | Capacity scheduler: add support for combined %user + %secondary\_group mapping | Major | . | Manikandan R | Manikandan R | | | Efficient allocation of Opportunistic containers. | Major | . | Abhishek Modi | Abhishek Modi | | | S3A delegation token tests fail if fs.s3a.encryption.key set | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | Implement DeadNodeDetector basic model | Major | . | Lisheng Sun | Lisheng Sun | | | Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | S3A to warn or fail if S3Guard is disabled | Minor | fs/s3 | Steve Loughran | Gabor Bota | | | Add Java changes for the new RuncContainerRuntime | Major | . | Eric Badger | Eric Badger | | | Move remaining log4j APIs over to slf4j in hadoop-common. | Major | . | Minni Mittal | Minni Mittal | | | Upgrade to yetus 0.11.1 and use emoji vote on github pre commit | Major | build | Duo Zhang | Duo Zhang | | | Offline format of YarnConfigurationStore | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Speculating & Partitioned S3A magic committers can leave pending files under \\magic | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | General usability improvements in showSimulationTrace.html | Minor | scheduler-load-simulator | Adam Antal | Adam Antal | | | NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup | Major | auth, fs/azure, security | Steve Loughran | Steve Loughran | | | ABFS: Fix testcase added for HADOOP-16138 for namespace enabled account | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | DeadNodeDetector checks dead node periodically | Major | . | Lisheng Sun | Lisheng Sun | | | HadoopExecutors cleanup to only log at debug | Minor | util | Steve Loughran | David Mollitor | | | Migration tool that help to generate CS config based on FS config [Phase 2] | Major | . | Szilard Nemeth | Peter Bacsko | | | Add suspect probe for DeadNodeDetector | Major |

| Lisheng Sun | Lisheng Sun | | | ABFS: Implement FileSystem.access() method | Minor | fs/azure | Bilahari T H | Bilahari T H | | | ABFS: Make RetryCount in ExponentialRetryPolicy Configurable | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Refactor the unit test of TestDeadNodeDetection | Minor | . | Yiqun Lin | Lisheng Sun | | | [Dynamometer] Workload doesn't start unless an absolute path of Mapper class given | Major | . | Soya Miyoshi | Takanobu Asanuma | | | RBF: Race condition between router admin quota update and periodic quota update service | Major | . | Dibyendu Karmakar | Jinglun | | | SchedConfCli to get current stored scheduler configuration | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Disable Option for Write Ahead Logs of LogMutation | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Auto-configuring rollover-size of IFile format for non-appendable filesystems | Major | log-aggregation, yarn | Adam Antal | Adam Antal | | | Add C changes for the new RuncContainerRuntime | Major | . | Eric Badger | Eric Badger | | | Track Azure Blob File System client-perceived latency | Major | fs/azure, hdfs-client | Jeetesh Mangwani | Jeetesh Mangwani | | | RBF: The detail of the Exception is not shown in ZKDelegationTokenSecretManagerImpl | Major | . | Akira Ajisaka | Akira Ajisaka | | | Refine testing.md to tell user better how to use auth-keys.xml | Minor | fs/s3 | Mingliang Liu | Mingliang Liu | | | RBF: Add dfsrouteradmin -refreshSuperUserGroupsConfiguration command option | Minor | rbf | Akira Ajisaka | Xieming Li | | | [Dynamometer] Show the line of audit log when parsing it unsuccessfully | Major | tools | Takanobu Asanuma | Takanobu Asanuma | | | Increase timeout unit test rule for MetadataStoreTestBase | Major | fs/s3 | Mingliang Liu | Mingliang Liu | | | Rewrite Python example codes using Python3 | Minor | documentation | Kengo Seki | Kengo Seki | | | DurationInfo text parsing/formatting should be moved out of hotpath | Minor | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Add ability to filter the Cluster Applications API request by name | Major | yarn | Adam Antal | Adam Antal | | | HttpFS: Implement setErasureCodingPolicy , unsetErasureCodingPolicy , getErasureCodingPolicy | Major | . | hemanthboyina | hemanthboyina | | | S3A Delegation Token extension point to use StoreContext | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: Enhance driver debug logs | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | TimelineV2Client may leak file descriptors creating ClientResponse objects. | Critical | ATSv2 | Anand Srinivasan | Anand Srinivasan | | | ITestDynamoDBMetadataStoreScale fails when

| Major | fs/s3, test | Steve Loughran | Steve Loughran | | | RBF: Print stacktrace when DFSRouter fails to fetch/parse JMX output from NameNode | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | Sync Mockito version with other modules | Major | build, test | Akira Ajisaka | Adam Antal | | | audit/tune s3a authoritative flag in s3guard DDB Table | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add dry-run feature to FS-CS converter tool | Major | . | Peter Bacsko | Peter Bacsko | | | Pull out common code pieces from ATS v1.5 and v2 | Major | ATSv2, yarn | Adam Antal | Adam Antal | | | Add dockerfile for ARM builds | Major | . | Vinayakumar B | Vinayakumar B | | | Queue Management API does not support parallel updates | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | NativeAzureFileSystem does not support setXAttr | Major | fs/azure | Clemens Wolff | Clemens Wolff | | | Integrate the new abstract log servlet to the JobHistory server | Major | yarn | Adam Antal | Adam Antal | | | FS-CS converter: disable terminal placement rule checking | Critical | . | Peter Bacsko | Peter Bacsko | | | [pb-upgrade] Remove Protobuf classes from signatures of Public APIs | Critical | common | Steve Loughran | Vinayakumar B | | | IFile format is not working against s3a remote folder | Major | log-aggregation | Adam Antal | Adam Antal | | | Stabilize S3A OpenSSL support | Blocker | fs/s3 | Steve Loughran | Sahil Takiar | | | Filesystem openFile() builder to take a FileStatus param | Minor | fs, fs/azure, fs/s3 | Steve Loughran | Steve Loughran | | | Provide utility to ask whether an application is in final status | Minor | . | Adam Antal | Adam Antal | | | Let s3 clients configure request timeout | Major | fs/s3 | Mustafa Iman | Mustafa Iman | | | TestHarFileSystem.testInheritedMethodsImplemented broken | Major | fs, test | Steve Loughran | Steve Loughran | | | S3A empty dir markers are not created in s3guard as authoritative | Critical | fs/s3 | Steve Loughran | Steve Loughran | | | FS-CS converter: remove mixed ordering policy check | Critical | . | Peter Bacsko | Peter Bacsko | | | FS-CS converter: dry run should work without output defined | Major | . | Peter Bacsko | Peter Bacsko | | | Correct the sample command in SLS README file | Trivial | yarn | Aihua Xu | Aihua Xu | | | FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues | Major | . | Peter Bacsko | Peter Bacsko | | | [Router] Add locality policy | Major | . | Giovanni Matteo Fumarola | Young Chen | | | S3Guard to support encrypted DynamoDB table | Major | fs/s3 | Mingliang Liu | Mingliang Liu | | | ITestAzureBlobFileSystemCheckAccess failing | Major | fs/azure, test | Steve Loughran | Bilahari T H | | | ITestAbfsClient.testContinuationTokenHavingEqualSign failing | Major | fs/azure, test | Sneha Vijayarajan | Sneha Vijayarajan | | | [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency | Major | . | Vinayakumar B | Vinayakumar B | | | Allow stop and convert from leaf to parent queue in a single Mutation API call | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Support listing of aggregated logs for containers belonging to an application attempt | Major | log-aggregation, yarn | Adam Antal | Adam Antal | | | FSQueueConverter should not set App Ordering Policy to Parent Queue | Major | . | Prabhu Joseph | Peter Bacsko | | | Create RM Rest API to validate a CapacityScheduler Configuration | Major | . | Kinga Marton | Kinga Marton | | | RBF: Add FSCK to the Router | Major |

| igo Goiri | Akira Ajisaka | | | Add option to UIv2 to get container logs from the new JHS API | Major | yarn | Adam Antal | Adam Antal | | | Large DeleteObject requests are their own Thundering Herd | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' | Minor | documentation, rbf | panlijie | panlijie | | | S3A committers: make sure there's regular progress() calls | Minor | fs/s3 | Steve Loughran | lqjacklee | | | ValidateAndGetSchedulerConfiguration API fails when cluster max allocation \> default 8GB | Major | . | Prabhu Joseph | Prabhu Joseph | | | ITestClientUrlScheme fails for accounts which don't support HTTP | Minor | fs/azure, test | Steve Loughran | Steve Loughran | | | S3A bucket existence checks to support v2 API and "no checks at all" | Minor | fs/s3 | Rajesh Balamohan | Mukund Thakur | | | Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes() | Major | . | Lisheng Sun | Lisheng Sun | | | Make MAX\LOCK\HOLD\_MS and full queue size configurable | Major | namenode | zhuqi | zhuqi | | | ITestS3GuardOutOfBandOperations failing on versioned S3 buckets | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | FS-CS converter: initPropertyActions() is not called without rules file | Major | . | Peter Bacsko | Peter Bacsko | | | FS-CS converter tool: issue warning on dynamic auto-create mapping rules | Major | . | Peter Bacsko | Peter Bacsko | | | FS-CS converter: Do not allow output dir to be the same as input dir | Major | . | Szilard Nemeth | Adam Antal | | | [FGL] Remove redundant locking on NameNode. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | | | S3AInputStream reopening does not handle non IO exceptions properly | Major | . | Sergei Poganshev | Sergei Poganshev | | | FS-CS converter: only convert placement rules if a cmd line switch is defined | Major | . | Peter Bacsko | Peter Bacsko | | | FS-CS Converter: Need to validate c-s.xml after converting | Major | . | Wangda Tan | Peter Bacsko | | | Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Metrics for Federation AMRMProxy | Major | . | Giovanni Matteo Fumarola | Young Chen | | | FS-CS Converter: tool doesn't handle min/max resource conversion correctly | Blocker | . | Wangda Tan | Peter Bacsko | | | ABFS: Change in expiry calculation for MSI token provider | Minor | fs/azure | Bilahari T H | Bilahari T H | | | FS-CS converter: call System.exit function call for every code path in main method | Blocker | . | Peter Bacsko | Peter Bacsko | | | FS-CS converter: fix incorrect capacity conversion | Blocker |

| Peter Bacsko | Peter Bacsko | | | In Federation Secure cluster Application submission fails when authorization is enabled | Blocker | federation | Sushanta Sen | Bilwa S T | | | Document scheduler/app activities and REST APIs | Major | documentation | Tao Yang | Tao Yang | | | Add diagnostics for outstanding resource requests on app attempts page | Major | capacityscheduler | Tao Yang | Tao Yang | | | S3Guard fsck: Add option to remove orphaned entries | Major | fs/s3 | Gabor Bota | Gabor Bota | | | RBF: Correct annotation typo of RouterPermissionChecker#checkPermission | Trivial | rbf | Xiaoqiao He | Xiaoqiao He | | | Allow multiple leaf queues with the same name in CapacityScheduler | Major | . | Gergely Pollak | Gergely Pollak | | | FS-CS converter: fix emitted ordering policy string and max-am-resource percent value | Major | . | Peter Bacsko | Peter Bacsko | | | FairOrderingPolicy Improvements | Major | . | Manikandan R | Manikandan R | | | RBF: Add Browse the Filesystem button to the UI | Major | . | igo Goiri | igo Goiri | | | hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | fs.s3a.authoritative.path should support multiple FS URIs | Major | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: Add unbuffer support to AbfsInputStream | Major | fs/azure | Sahil Takiar | Sahil Takiar | | | In Federation Router Nodes/Applications/About pages throws 500 exception when https is enabled | Critical | federation | Sushanta Sen | Bilwa S T | | | Tune S3AFileSystem.listLocatedStatus | Major | fs/s3 | Steve Loughran | Mukund Thakur | | | FS-CS converter: don't enable auto-create queue property for root | Critical | . | Peter Bacsko | Peter Bacsko | | | log DNS addresses on s3a init | Minor | fs/s3 | Steve Loughran | Mukund Thakur | | | Resolve hadoop-cos dependency conflict | Major | build, fs/cos | Yang Yu | Yang Yu | | | s3a to not need wildfly on the classpath | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF:TestRouterFaultTolerant tests are flaky | Major | test | Chen Zhang | Akira Ajisaka | | | S3A reverts KMS encryption to the bucket's default KMS key in rename/copy | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | ABFS: Make list page size configurable | Minor | fs/azure | Bilahari T H | Bilahari T H | | | YARN RMWebServices /scheduler-conf/validate leaks ZK Connections | Blocker | capacityscheduler | Akhil PB | Prabhu Joseph | | | Endpoint for obtaining direct URL for the logs | Major | yarn | Adam Antal | Andras Gyori | | | Erasure Coding: metrics xmitsInProgress become to negative. | Major | datanode, erasure-coding | maobaolong | maobaolong | | | HADOOP-16953. tune s3guard disabled warnings | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | catch and downgrade all exceptions trying to load openssl native libs through wildfly | Minor | fs/azure, fs/s3 | Steve Loughran | Steve Loughran | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Update BUILDING.txt for building native libraries | Critical | build, documentation, native | Takanobu Asanuma | Takanobu Asanuma | | | libhdfs++: Use hdfs::IoService object rather than asio::io\_service | Critical | . | James Clampffer | James Clampffer | | | libhdfs++: Fix GCC7 build | Major | . | James Clampffer | James Clampffer | | | Upgrade Apache Zookeeper version due to security concerns | Major | . | Boris Vulikh | Akira Ajisaka | | | Upgrade maven-shade-plugin from 2.4.3 to 3.2.0 | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Upgrade Eclipse Jetty version to 9.3.24 | Major |

| Boris Vulikh | Boris Vulikh | | | WebHDFS: Document ALLOWSNAPSHOT and DISALLOWSNAPSHOT API doc | Minor | documentation, webhdfs | Siyao Meng | Siyao Meng | | | Need to support "dominant" component concept inside YARN service | Major | yarn-native-services | Wangda Tan | Zac Zhou | | | Document special paths in HDFS | Major | documentation | Christopher Douglas | Masatake Iwasaki | | | Add cli option in DS to support enforceExecutionType in resource requests. | Major | . | Abhishek Modi | Abhishek Modi | | | Add metrics for paused containers in NodeManager | Major | . | Abhishek Modi | Abhishek Modi | | | Add -std=c99 / -std=gnu99 to libhdfs compile flags | Major | hdfs-client, libhdfs, native | Sahil Takiar | Sahil Takiar | | | Fix typos in hadoop-functions.sh | Trivial | . | Ray Chiang | Ray Chiang | | | Remove unused constants from RMAuditLogger | Minor | yarn | Adam Antal | Igor Rudenko | | | Remove the extra empty space in the DataStreamer logging | Trivial | hdfs | Yishuang Lu | Yishuang Lu | | | Fix typo in YarnConfiguration: physical memory | Trivial | yarn | Adam Antal | Igor Rudenko | | | Update BUILDING.txt with macOS native build instructions | Minor | . | Siyao Meng | Siyao Meng | | | Upgrade jackson-databind to 2.9.9 | Major | build | Shweta | Shweta | | | TestContainerSchedulerQueuing#testQueueShedding fails intermittently. | Minor | . | Abhishek Modi | Abhishek Modi | | | Create AbstractContainersLauncher for pluggable ContainersLauncher logic | Major | . | Jonathan Hung | Jonathan Hung | | | Upgrade jetty version to 9.3.27 | Major | . | Hrishikesh Gadre | Hrishikesh Gadre | | | Change ":" to ApplicationConstants.CLASS\PATH\SEPARATOR | Trivial | common | Kevin Su | Kevin Su | | | Upgrade Bootstrap and jQuery versions used in HDFS UIs | Major | ui | Vivek Ratnavel Subramanian | Vivek Ratnavel Subramanian | | | Introduce a config to control SSL Channel mode in Azure DataLake Store Gen1 | Major | fs/adl | Sneha Vijayarajan | Sneha Vijayarajan | | | Update commons-beanutils version to 1.9.4 | Major | . | Wei-Chiu Chuang | Kevin Su | | | Update commons-compress to 1.19 | Major | . | Wei-Chiu Chuang | YiSheng Lien | | | Support forcing configured partitions to be exclusive based on app node label | Major | . | Jonathan Hung | Jonathan Hung | | | Upgrade jackson-databind to 2.9.10.1 | Blocker | security | Wei-Chiu Chuang | Lisheng Sun | | | Document FairCallQueue configs in core-default.xml | Major | conf, documentation | Siyao Meng | Siyao Meng | | | [SBNN read] access time should be turned off | Major | documentation | Wei-Chiu Chuang | Chao Sun | | | Delete hadoop-ozone and hadoop-hdds subprojects from apache trunk | Major | . | Marton Elek | Sandeep Nemuri | | | Document the new decommission monitor (HDFS-14854) | Major | documentation | Wei-Chiu Chuang | Masatake Iwasaki | | | Update the year to 2020 | Major | . | Ayush Saxena | Ayush Saxena | | | Add an ARM CI for Hadoop | Major | build | Zhenyu Zheng | Zhenyu Zheng | | | Upgrade jackson-databind to 2.9.10.2 | Blocker | security | Akira Ajisaka | Masatake Iwasaki | | | Stripping Submarine code from Hadoop codebase. | Blocker | . | Wei-Chiu Chuang | Zhankun Tang | | | Upgrade Netty