Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.3.1.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc | Major | fs/s3 | Yongjun Zhang | Yongjun Zhang | | | RBF: Could not fetch real remote IP in RouterWebHdfsMethods | Major | webhdfs | tomscut | tomscut | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | ABFS: Delegation SAS generator for integration with Ranger | Minor | fs/azure | Thomas Marqardt | Thomas Marqardt | | | Standby NameNode process getBlocks request to reduce Active load | Major | balancer & mover, namenode | Xiaoqiao He | Xiaoqiao He | | | ABFS: Delegation SAS Generator Updates | Minor | fs/azure | Thomas Marqardt | Thomas Marqardt | | | Provide Regex Based Mount Point In Inode Tree | Major | viewfs | zhenzhao wang | zhenzhao wang | | | Using snappy-java in SnappyCodec | Major | common | DB Tsai | L. C. Hsieh | | | Using lz4-java in Lz4Codec | Major | common | L. C. Hsieh | L. C. Hsieh | | | Add Metrics to HttpFS Server | Major | httpfs | Ahmed Hussein | Ahmed Hussein | | | LocatedFileStatusFetcher to collect/publish IOStatistics | Minor | client | Steve Loughran | Steve Loughran | | | Add Public IOStatistics API | Major | fs, fs/s3 | Steve Loughran | Steve Loughran | | | EC: Verify EC reconstruction correctness on DataNode | Major | datanode, ec, erasure-coding | Toshihiko Uchida | Toshihiko Uchida | | | ber-jira: S3A Hadoop 3.3.1 features | Major | fs/s3 | Steve Loughran | Steve Loughran | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve JournalNode web UI | Major | journal-node, ui | Jianfei Jiang | Jianfei Jiang | | | Add .diff to gitignore | Minor | . | Ayush Saxena | Ayush Saxena | | | Add -S option in "Count" command to show only Snapshot Counts | Major | . | Hemanth Boyina | Hemanth Boyina | | | RBF: Provide Non DFS Used per DataNode in DataNode UI | Major | . | Ayush Saxena | Lisheng Sun | | | HsJobsBlock reuse JobACLsManager for checkAccess | Minor | . | Bibin Chundatt | Bilwa S T | | | AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. | Minor | . | Jinglun | Jinglun | | | Update Dockerfile to use Bionic | Major | build, test | Akira Ajisaka | Akira Ajisaka | | | Add isAbsoluteResource config for queue in scheduler response | Minor | scheduler | Prabhu Joseph | Prabhu Joseph | | | Add hadoop.http.idle\_timeout.ms to core-default.xml | Major | . | Wei-Chiu Chuang | Lisheng Sun | | | DFSInputStream to prefer cached replica | Major | . | Wei-Chiu Chuang | Lisheng Sun | | | listOpenFiles() should throw InvalidPathException in case of invalid paths | Minor | . | Jinglun | Jinglun | | | Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo | Major | . | Prabhu Joseph | Prabhu Joseph | | | Consider StorageType when DatanodeManager#sortLocatedBlock() | Major | . | Lisheng Sun | Lisheng Sun | | | Allow transitioning queue from DRAINING to RUNNING state | Major |

| Jonathan Hung | Bilwa S T | | | TestFTPFileSystem failing as ftp server dir already exists | Minor | fs, test | Steve Loughran | Mikhail Pryakhin | | | Unify configuration \`dfs.ha.allow.stale.reads\` to DFSConfigKeys | Major | hdfs | Xiaoqiao He | Xiaoqiao He | | | RBF: Unify router datanode UI with namenode datanode UI | Major | . | Ayush Saxena | Ayush Saxena | | | Hadoop distcp throws "ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found" | Minor | tools/distcp | Aki Tanaka | Aki Tanaka | | | HDFS-client: boost ShortCircuit Cache | Minor | dfsclient | Danil Lipovoy | Danil Lipovoy | | | VolumeScanner skip to scan blocks accessed during recent scan peroid | Minor | datanode | Yang Yun | Yang Yun | | | Avoid Potential Infinite Loop in DFSNetworkTopology | Major | . | Ayush Saxena | Ayush Saxena | | | SlotReleaser is not fast enough | Major | hdfs-client | Gang Xie | Lisheng Sun | | | Refactor method VolumeScanner#runLoop() | Minor | datanode | Yang Yun | Yang Yun | | | Make copyFromLocal's -t option available for put as well | Major | . | Andras Bokor | Andras Bokor | | | RedundantEditLogInputStream should log caught exceptions | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Generate queue metrics for each partition | Major | capacity scheduler | Jonathan Hung | Manikandan R | | | Adding Common Counters in ABFS | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Zookeeper Delegation Token Manager fetch sequence number by batch | Major | . | Fengnan Li | Fengnan Li | | | Add seek support for SFTP FileSystem | Minor | fs | Azhagu Selvan SP | Mikhail Pryakhin | | | TODO comments exist in trunk while the related issues are already fixed. | Trivial | . | Rungroj Maipradit | Rungroj Maipradit | | | Improve RawFileSystem Performance | Minor | fs | Rajesh Balamohan | Mehakmeet Singh | | | Improve the speed of Datanode Block Scan | Major | . | Hemanth Boyina | Hemanth Boyina | | | Increase precommit job timeout from 5 hours to 20 hours | Major | build | Akira Ajisaka | Akira Ajisaka | | | Update Dockerfile\_aarch64 to use Bionic | Major | build, test | RuiChen | zhaorenhai | | | RMWebApp make external class pluggable | Minor | . | Bibin Chundatt | Bilwa S T | | | TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently | Major | . | Jonathan Hung | Jim Brennan | | | Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime | Minor | common | Jim Brennan | Jim Brennan | | | ShellCommandFencer should expose info about source | Major | . | Chen Liang | Chen Liang | | | Dead link in hadoop-kms/index.md.vm | Minor | documentation, kms | Akira Ajisaka | Xieming Li | | | Adding ReadAhead Counters in ABFS | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. | Major | . | Eric Payne | Eric Payne | | | Add Localization overhead metrics to NM | Major | nodemanager | Gera Shegalov | Jim Brennan | | | Make custom DAO classes configurable into RMWebApp#JAXBContextResolver | Major | . | Prabhu Joseph | Bilwa S T | | | Show extended resources on legacy RM UI. | Major |

| Eric Payne | Eric Payne | | | Update block map and name cache in parallel while loading fsimage. | Major | namenode | Chengwei Wang | Chengwei Wang | | | ABFS driver enhancement - Allow customizable translation from AAD SPNs and security groups to Linux user and group | Major | fs/azure | Karthik Amarnath | Karthik Amarnath | | | Adding Context class for AbfsClient to pass AbfsConfigurations to limit number of parameters | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Adding Network Counters in ABFS | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Make UGI support forceful relogin from keytab ignoring the last login time | Major | security | Sandeep Guggilam | Sandeep Guggilam | | | Add phantomjsdriver.log to gitignore | Minor | . | Takanobu Asanuma | Takanobu Asanuma | | | Log vcores used and cumulative cpu in containers monitor | Minor | yarn | Jim Brennan | Jim Brennan | | | Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG | Minor | yarn | Jim Brennan | Jim Brennan | | | LeafQueue: retain user limits cache across assignContainers() calls | Major | capacity scheduler, capacityscheduler | Muhammad Samir Khan | Muhammad Samir Khan | | | Remove unnecessary sort of block list in DirectoryScanner | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Fix testCompressorDecompressorWithExeedBufferLimit to cover the intended scenario | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Access Controlled HTTPFS Proxy | Minor | httpfs | Richard | Richard | | | Hadoop - Upgrade to JQuery 3.5.1 | Major | . | Aryan Gupta | Aryan Gupta | | | Add debug-level logs in Filesystem#close | Minor | fs | Karen Coppage | Karen Coppage | | | Support BCFKS keystores for Hadoop Credential Provider | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | Reduce locking in Datanode DirectoryScanner | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. | Major | . | Eric Payne | Eric Payne | | | Add concat fs command | Minor | fs | Jinglun | Jinglun | | | Expose Mini MR Cluster attribute for testing | Minor | test | Swaroopa Kadam | Swaroopa Kadam | | | [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. | Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko | | | Avoid redundant RPC calls for getDiskStatus | Major | dfsclient | Ayush Saxena | Ayush Saxena | | | Add cpu and memory utilization per node and cluster-wide metrics | Minor | yarn | Jim Brennan | Jim Brennan | | | Upgrade to jQuery 3.5.1 in hadoop-sls | Major | . | Aryan Gupta | Aryan Gupta | | | Make block size from NNThroughputBenchmark configurable | Minor | benchmarks | Hui Fei | Hui Fei | | | Scale RM-NM heartbeat interval based on node utilization | Minor | yarn | Jim Brennan | Jim Brennan | | | Balancer logging improvement | Major | balancer & mover | Konstantin Shvachko | Konstantin Shvachko | | | Creating a token identifier should not do kerberos name resolution | Major | common | Jim Brennan | Jim Brennan | | | RMProxy should retry on SocketTimeout Exceptions | Major | yarn | Jim Brennan | Jim Brennan | | | Respect configured values of

| Major | hdfs | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Bump up snappy-java to 1.1.8.1 | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | replace href tags with ng-href | Trivial | . | Gabriel Medeiros Coelho | Gabriel Medeiros Coelho | | | Add InetAddress api to ProxyUsers.authorize | Major | performance, security | Ahmed Hussein | Ahmed Hussein | | | Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class | Major | mrv2 | Daniel Fritsi | Zoltn Erdmann | | | Improve performance of reading resource request for mapper/reducers from config | Major | applicationmaster | Wangda Tan | Peter Bacsko | | | Avoid calling UpdateHeartBeatState inside DataNodeDescriptor | Major | . | Ahmed Hussein | Ahmed Hussein | | | Don't generate edits for set operations that are no-op | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Remote exception messages should not include the exception class | Major | ipc | Ahmed Hussein | Ahmed Hussein | | | HttpFS: Log more information on request failures | Major | httpfs | Ahmed Hussein | Ahmed Hussein | | | KMS should log full UGI principal | Major | . | Ahmed Hussein | Ahmed Hussein | | | Bump up snappy-java to 1.1.8.2 | Minor | . | L. C. Hsieh | L. C. Hsieh | | | namenode audit async logger should add some log4j config | Minor | hdfs | Max Xie | | | | Improve fsck logging | Major | logging, namenode | Ahmed Hussein | Ahmed Hussein | | | Mitigate lease monitor's rapid infinite loop | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Speed up the Storage#doRecover during datanode rolling upgrade | Major | . | Hemanth Boyina | Hemanth Boyina | | | Add documentation for msync() API to filesystem.md | Major | documentation | Konstantin Shvachko | Konstantin Shvachko | | | [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 | Major | . | Gabor Bota | Gabor Bota | | | Add recommissioning nodes to the list of updated nodes returned to the AM | Major | . | Srinivas S T | Srinivas S T | | | capture the performance metrics of ZKRMStateStore | Minor | resourcemanager | Max Xie | Max Xie | | | Optimize NetworkTopology while sorting of block locations | Major | common, net | Ahmed Hussein | Ahmed Hussein | | | Diagnostics for localization timeouts is lacking | Major | . | Chang Li | Chang Li | | | Follow up changes for YARN-9833 | Major | yarn | Jim Brennan | Jim Brennan | | | Speed up BlockPlacementPolicyRackFaultTolerant#verifyBlockPlacement | Major | block placement | Akira Ajisaka | Akira Ajisaka | | | Refactor QueueMetricsForCustomResources class to move to yarn-common package | Major | . | Minni Mittal | Minni Mittal | | | Typo in hadop-aws index.md | Trivial | documentation, fs/s3 | Maksim | Maksim | | | Improve the description of hadoop.http.authentication.signature.secret.file | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Add latency information in

| Minor | client | Jungtaek Lim | Jungtaek Lim | | | Lease renewal does not require namesystem lock | Major | hdfs | Jim Brennan | Jim Brennan | | | Fix logging typo in ShutdownHookManager | Major | common | Konstantin Shvachko | Fengnan Li | | | Move Jenkinsfile outside of the root directory | Major | build | Akira Ajisaka | Akira Ajisaka | | | Simplify dependency installation instructions | Trivial | documentation | Gautham Banasandra | Gautham Banasandra | | | Parallelize building of dependencies | Minor | build | Gautham Banasandra | Gautham Banasandra | | | Make DisallowedDatanodeException terse | Minor | hdfs | Richard | Richard | | | DataStreamer: keep sending heartbeat packets while streaming | Major | hdfs | Jim Brennan | Jim Brennan | | | Log list of mappers at trace level in ShuffleHandler audit log | Minor | yarn | Jim Brennan | Jim Brennan | | | roll out StreamCapabilities across output streams of all filesystems | Major | fs, fs/adl, fs/azure, fs/oss, fs/s3, fs/swift | Steve Loughran | Steve Loughran | | | Add metrics for in-service datanodes | Minor | . | Zehao Chen | Zehao Chen | | | Log resource allocation in NM log at container start time | Major | . | Eric Badger | Eric Badger | | | if required storageType are unavailable, log the failed reason during choosing Datanode | Minor | block placement | Yang Yun | Yang Yun | | | Support to make dfs.image.parallel.load reconfigurable | Major | namenode | Hui Fei | Hui Fei | | | Erasure coding: Add/remove logs for the better readability/debugging | Minor | erasure-coding, hdfs | Bhavik Patel | Bhavik Patel | | | Solve the problem of incorrect progress of delegation tokens when loading FsImage | Major | . | JiangHua Zhu | JiangHua Zhu | | | [READ] DirectoryScanner#scan need not check StorageType.PROVIDED | Minor | datanode | Yuxuan Wang | Yuxuan Wang | | | Add kms-default.xml and httpfs-default.xml to site index | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF | Minor | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | Fixed the findbugs issues introduced by YARN-10647. | Major | . | Qi Zhu | Qi Zhu | | | ClientHSSecurityInfo class is in wrong META-INF file | Major | . | Eric Badger | Eric Badger | | | Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md | Minor | . | Ravuri Sushma sree | Ravuri Sushma sree | | | Allow parameter expansion in NM\ADMIN\USER\_ENV | Major | yarn | Jim Brennan | Jim Brennan | | | Apply YETUS-1102 to re-enable GitHub comments | Major | build | Akira Ajisaka | Akira Ajisaka | | | DistCp: Expose the JobId for applications executing through run method | Major | . | Ayush Saxena | Ayush Saxena | | | Reduce Memory Overhead of AclFeature by avoiding AtomicInteger | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Provide blocks moved count in Balancer iteration result | Major | balancer & mover | Viraj Jasani | Viraj Jasani | | | BlockPoolManager should log stack trace if unable to get Namenode addresses | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | DistCp: Reduce memory usage on copying huge directories | Critical | . | Ayush Saxena | Ayush Saxena | | | Exclude slow nodes when choose targets for blocks | Major |

| tomscut | tomscut | | | Use spotbugs-maven-plugin instead of findbugs-maven-plugin | Major | build | Akira Ajisaka | Akira Ajisaka | | | Create socket address leveraging URI cache | Major | common, hdfs-client | fanrui | fanrui | | | Improve the balancer error message when process exits abnormally. | Major | . | Renukaprasad C | Renukaprasad C | | | Automatic keystore reloading for HttpServer2 | Major | . | Kihwal Lee | Borislav Iordanov | | | Fix non-static inner classes for better memory management | Major | . | Viraj Jasani | Viraj Jasani | | | Bump Jetty to the latest version 9.4.35 | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Increase Quota initialization threads | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Log not flushed fully when daemon shutdown | Major | common | Renukaprasad C | Renukaprasad C | | | Reduce memory used during datanode layout upgrade | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Building native code fails on Fedora 33 | Major | build, common | Kengo Seki | Masatake Iwasaki | | | Please upgrade json-smart dependency to the latest version | Major | auth, build | helen huang | Viraj Jasani | | | DistCp: Use Iterator for listing target directory as well | Major | . | Ayush Saxena | Ayush Saxena | | | Add a policy for not aggregating for containers which are killed because exceeding container log size limit. | Major | . | Qi Zhu | Qi Zhu | | | Improve the log for Short Circuit Local Reads | Minor | . | Bhavik Patel | Bhavik Patel | | | LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException | Major | common | Tamas Mate | Istvn Fajth | | | Remove workaround for Curator's ChildReaper requiring Guava 15+ | Major | . | Robert Kanter | Viraj Jasani | | | ProcessReport print invalidatedBlocks should judge debug level at first | Minor | namanode | lei w | lei w | | | Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException | Major | . | junwen yang | Viraj Jasani | | | ADLFS: Update SDK version from 2.3.6 to 2.3.9 | Minor | fs/adl | Bilahari T H | Bilahari T H | | | Provide source artifacts for hadoop-client-api | Major | . | Karel Kolman | Karel Kolman | | | Allow ProtobufRpcEngine to be extensible | Major | common | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Add metrics for 'ApplicationsRunning' in NodeManager | Minor | nodemanager | ANANDA G B | ANANDA G B | | | Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist | Critical | . | David Mollitor | Vinayakumar B | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: RouterRpcServer getListing cannot list large dirs correctly | Critical | . | Fengnan Li | Fengnan Li | | | HttpFS: setWorkingDirectory should not accept invalid paths | Major | . | Hemanth Boyina | Hemanth Boyina | | | ThrottledAsyncChecker is not thread-safe. | Major | federation | Toshihiro Suzuki | Toshihiro Suzuki | | | Add missing DFSOps Statistics in WebHDFS | Major | . | Ayush Saxena | Ayush Saxena | | | HttpFS: Response of Create was not correct with noredirect and data are true | Major | . | Hemanth Boyina | Hemanth Boyina | | | ZKFC ignores dfs.namenode.rpc-bind-host and uses

to bind to host address | Major | ha, namenode | Dhiraj Hegde | Dhiraj Hegde | | | TestNNHandlesBlockReportPerStorage::blockReport\_02 fails intermittently in trunk | Major | datanode, test | Mingliang Liu | Ayush Saxena | | | EC : File write hanged when DN is shutdown by admin command. | Major | ec | Surendra Singh Lilhore | Surendra Singh Lilhore | | | The same distance and load nodes don't shuffle when consider DataNode load | Major | . | Lisheng Sun | Lisheng Sun | | | HttpFS: validate content-type in HttpFSUtils | Major | . | Hemanth Boyina | Hemanth Boyina | | | StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter | Major | webhdfs | Akira Ajisaka | Akira Ajisaka | | | Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic | Major | . | Ahmed Hussein | Ahmed Hussein | | | Account for \*env == NULL in hdfsThreadDestructor | Major | . | Babneet Singh | Babneet Singh | | | Remove invalid exclusions that minicluster dependency on HDFS | Major | . | Wanqiang Ji | Wanqiang Ji | | | TestContainerResizing fails randomly | Minor | . | Bibin Chundatt | Ahmed Hussein | | | Quota Space consumed was wrong in truncate with Snapshots | Major | . | Hemanth Boyina | Hemanth Boyina | | | PlacementRule order is not maintained in CS | Major | . | Bibin Chundatt | Bilwa S T | | | Fix invalid metastore configuration in S3GuardTool tests | Minor | fs/s3, test | Masatake Iwasaki | Masatake Iwasaki | | | TestHDFSCLI fails for user names with the dot/dash character | Major | test | Yan Xiaole | Yan Xiaole | | | Setting \`dfs.client.use.datanode.hostname\` to true can crash the system because of unhandled UnresolvedAddressException | Major | . | Ctest | Ctest | | | SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data | Major | io, test | zhao bo | Akira Ajisaka | | | FTPFileSystem attempts to close the outputstream even when it is not initialised | Major | hdfs-client | Sudharsan Sampath | Mikhail Pryakhin | | | Add an option to prevent sub-directories of protected directories from deletion | Major | 3.1.1 | liuyanyu | liuyanyu | | | EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads | Major | ec | Guo Lei | Guo Lei | | | Too many InvalidStateTransitionException with SLS | Major | . | Bibin Chundatt | Bilwa S T | | | RBF: updateActiveNamenode() is invalid when RPC address is IP | Major | . | xuzq | xuzq | | | Deletion failure should not remove directory from snapshottables | Major | . | Hemanth Boyina | Hemanth Boyina | | | PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value | Minor | . | Akshay Agarwal | Bilwa S T | | | Revert "HADOOP-8143. Change distcp to have -pb on by default" | Major | tools/distcp | Steve Loughran | Steve Loughran | | | ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). | Major | fs, viewfs | Uma Maheswara Rao G | Abhishek Das | | | Job fails with InvalidStateTransitonException: Invalid event: JOB\TASK\COMPLETED at SUCCEEDED/COMMITTING | Major | . | Varun Saxena | Bilwa S T | | | Very large files can be truncated when written through S3AFileSystem | Major | fs/s3 | Andrew Olson | Mukund Thakur | | | javax.activation-api and

define overlapping classes | Major | build | Akira Ajisaka | Akira Ajisaka | | | Fix intermittent failure of ITestBlockingThreadPoolExecutorService | Minor | fs/s3, test | Masatake Iwasaki | Masatake Iwasaki | | | BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup | Major | . | Hemanth Boyina | Hemanth Boyina | | | RENAME.TO\_TRASH is ignored When RENAME.OVERWRITE is specified | Major | . | Harshakiran Reddy | Ayush Saxena | | | Fix DataNode's xceiver count calculation | Major | datanode, hdfs | Lukas Majercak | Lisheng Sun | | | FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks | Major | . | Hemanth Boyina | Hemanth Boyina | | | Wrong description of copyFromLocal and copyToLocal in documentation | Minor | . | Jingguo Yao | Andras Bokor | | | NetUtils.connect() throws unchecked exception (UnresolvedAddressException) causing clients to abort | Major | net | Dhiraj Hegde | Dhiraj Hegde | | | Intermittent failing of ITestAbfsStreamStatistics in ABFS | Minor | fs/azure, test | Mehakmeet Singh | Mehakmeet Singh | | | CapacityScheduler incorrect User Group Mapping after leaf queue change | Major | . | Gergely Pollk | Gergely Pollk | | | Fix shelldocs path in Jenkinsfile | Major | build | Akira Ajisaka | Akira Ajisaka | | | shelldoc fails in hadoop-common | Major | build | Akira Ajisaka | Akira Ajisaka | | | PendingContainers bugs in the scheduler outputs | Critical | . | Adam Antal | Andras Gyori | | | Fix TestViewFileSystemOverloadSchemeHdfsFileSystemContract#testListStatusRootDir | Major | . | Ayush Saxena | Ayush Saxena | | | ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories | Major | . | Toshihiro Suzuki | Toshihiro Suzuki | | | EC: hdfs client hangs due to exception during addBlock | Critical | ec, hdfs-client | Hongbing Wang | Hongbing Wang | | | appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat | Major | . | Eric Badger | Eric Badger | | | ArrayIndexOfboundsException in ViewFileSystem#listStatus | Major | viewfs | Hemanth Boyina | Hemanth Boyina | | | Make ContainerPBImpl#getId/setId synchronized | Minor | . | Benjamin Teke | Benjamin Teke | | | listStatus and getFileStatus behave inconsistent in the case of ViewFs implementation for isDirectory | Major | viewfs | Srinivasu Majeti | Uma Maheswara Rao G | | | Add support for yarn logs -logFile to retain backward compatibility | Major | client | Jim Brennan | Jim Brennan | | | Blocks scheduled count was wrong on truncate | Major | . | Hemanth Boyina | Hemanth Boyina | | | NPE in FileIoProvider#transferToSocketFully | Major | . | Hemanth Boyina | Hemanth Boyina | | | Files in snapshots no longer see attribute provider permissions | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Fix NoClassDefFoundError on 'mapred minicluster' | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | ViewFS does not return correct user/group and ACL | Major | fs, viewfs | Abhishek Das | Abhishek Das | | | Document block placement policies | Major | . | igo Goiri | Amithsha | | | client fails forever when namenode ipaddr changed | Major | hdfs-client | Sean Chow | Sean Chow | | | WASB: Update azure-storage-java SDK | Critical | fs/azure | Thomas Marqardt | Thomas Marqardt | | | TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk | Major | . | Hemanth Boyina | Hemanth Boyina | | | Support reservations continue looking for Node Labels | Major | . | Tarun Parimi | Jim Brennan | | | Upgrade node.js to

| Critical | build, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | Handle an internal dir in viewfs having multiple children mount points pointing to different filesystems | Major | fs, viewfs | Abhishek Das | Abhishek Das | | | ApplicationHistory Web UI incorrect column indexing | Minor | yarn | Andras Gyori | Andras Gyori | | | Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule | Major | capacity scheduler, capacityscheduler, test | Peter Bacsko | Peter Bacsko | | | CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with error java.io.FileNotFoundException: Directory does not exist: /.reserved/raw/path | Major | hdfs | Srinivasu Majeti | Stephen O'Donnell | | | MetricsSystem doesn't start the sink adapters on restart | Minor | metrics | Madhusoodan | Madhusoodan | | | Restarting name node stuck in safe mode when using provided storage | Major | namenode | shanyu zhao | shanyu zhao | | | Fix typos in hadoop-aws documentation | Trivial | documentation, fs/s3 | Sebastian Nagel | Sebastian Nagel | | | Fix failure of docker image creation due to pip2 install error | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | Sync netty versions in hadoop-yarn-csi | Major | build | Akira Ajisaka | Akira Ajisaka | | | Yarn Service Container Completed event doesn't get processed | Critical | . | Bilwa S T | Bilwa S T | | | WASB : NativeAzureFsOutputStream#close() throwing IllegalArgumentException | Major | fs/azure | Anoop Sam John | Anoop Sam John | | | Allow RM to always cancel tokens after app completes | Major | yarn | Jim Brennan | Jim Brennan | | | TestCombineFileInputFormat#testMissingBlocks fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | LeaseManager can loop forever on the file for which create has failed | Major | namenode | Sergey Shelukhin | Stephen O'Donnell | | | Configuration.getValByRegex() shouldn't update the results while fetching. | Major | common | Mukund Thakur | Mukund Thakur | | | RBF: Add test for MountTableRefresherService failed to refresh other router MountTableEntries in secure mode | Major | rbf | zhengchenyu | zhengchenyu | | | Jetty upgrade to 9.4.x causes MR app fail with IOException | Major | . | Bilwa S T | Bilwa S T | | | ArrayIndexOfboundsException in BlockManager CreateLocatedBlock | Major | . | Hemanth Boyina | Hemanth Boyina | | | Fix spotbugs warnings surfaced after upgrade to 4.0.6 | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | Some containers can be skipped during log aggregation after NM restart | Major | nodemanager | Jason Darrell Lowe | Jim Brennan | | | Failed to get nodejs 10.21.0 when building docker image | Blocker | build, webapp | Akira Ajisaka | Akira Ajisaka | | | Fix typo in MultipleOutputFormat | Trivial | . | ywheel | ywheel | | | Ensure inodes in active filesystem are not deleted during snapshot delete | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | missing libhdfspp libs in dist-package | Major | build, libhdfs++ | Yuan Zhou | Yuan Zhou | | | Log container report only if list is not empty | Minor |

| Bilwa S T | Bilwa S T | | | [Federation] Client should be able to submit application to RM directly using normal client conf | Major | amrmproxy, federation | JohnsonGuo | Bilwa S T | | | File and directory permissions are not able to be modified from WebUI | Major | . | Hemanth Boyina | Hemanth Boyina | | | Add --mvn-custom-repos parameter to yetus calls | Major | build | Mingliang Liu | Mingliang Liu | | | Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion | Major | httpfs | Mingliang Liu | Mingliang Liu | | | Fixing javadoc in ListingOperationCallbacks | Major | build, documentation | Akira Ajisaka | Mukund Thakur | | | UGI loginUserFromKeytab doesn't set the last login time | Major | security | Sandeep Guggilam | Sandeep Guggilam | | | ApplicationResourceUsageReport should return ALL reserved resource | Major | . | Bibin Chundatt | Bibin Chundatt | | | RMNode updatedCapability flag not set while RecommissionNodeTransition | Major | resourcemanager | Pranjal Protim Borah | Pranjal Protim Borah | | | Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. | Major | datanode | AMC-team | AMC-team | | | [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | [JDK 11] Fix javadoc errors in hadoop-hdfs module | Major | documentation | Akira Ajisaka | Xieming Li | | | [JDK 11] Fix javadoc errors in hadoop-hdfs-client module | Major | documentation | Akira Ajisaka | Xieming Li | | | Fix C/C++ standard warnings | Major | build | Gautham Banasandra | Gautham Banasandra | | | Fix typo in Hadoop KMS document | Trivial | documentation, kms | Akira Ajisaka | Xieming Li | | | ITestS3AHugeFilesSSECDiskBlock failing because of bucket overrides | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | RM page should throw exception when command injected in RM REST API to get applications | Major | . | Rajshree Mishra | Bilwa S T | | | Setting dfs.mover.retry.max.attempts to negative value will retry forever. | Major | balancer & mover | AMC-team | AMC-team | | | --module-gpu functionality is broken in container-executor | Major | nodemanager | Eric Badger | Eric Badger | | | Bug in preserving Directory Attributes in DistCp with Atomic Copy | Major | tools/distcp | Swaminathan Balachandran | | | | Rename with Snapshots does not honor quota limit | Major | . | Shashikant Banerjee | Hemanth Boyina | | | Erasure Coding: Native library memory leak | Major | native | Sean Chow | Sean Chow | | | MetricsConfig incorrectly loads the configuration whose value is String list in the properties file | Major | metrics | Jiayi Liu | Jiayi Liu | | | Upgrade slf4j to 1.7.30 ( To Address: CVE-2018-8088) | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Removing from LowRedundancyBlocks does not remove the block from all queues | Major | namenode | Hui Fei | Hui Fei | | | RBF: Clear Quota in Router was not consistent | Critical | . | Hemanth Boyina | Hemanth Boyina | | | RBF: Quota and Content Summary was not correct in Multiple Destinations | Critical |

| Hemanth Boyina | Hemanth Boyina | | | Directories protected from delete can still be moved to the trash | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Validating storage keys in ABFS correctly | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | TestHDFSContractMultipartUploader fails on trunk | Major | test | Ahmed Hussein | Steve Loughran | | | NPE in HttpServer during NameNode startup | Major | namenode | Konstantin Shvachko | Simbarashe Dzinamarira | | | Test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Test failure as failed request body counted in byte received metric - ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics | Major | fs/azure, test | Sneha Vijayarajan | Mehakmeet Singh | | | SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints | Minor | . | Bilwa S T | Bilwa S T | | | Only log warning if considerLoad and considerStorageType are both true | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Log improvements in NodeStatusUpdaterImpl | Minor | nodemanager | Bilwa S T | Bilwa S T | | | Fix build the hadoop-build Docker image failed | Major | build | Wanqiang Ji | Wanqiang Ji | | | Setting dfs.disk.balancer.max.disk.errors = 0 will fail the block copy | Major | balancer & mover | AMC-team | AMC-team | | | Typo in rename spec pseudocode | Major | documentation | Rae Marks | | | | Failed to load XInclude files with relative path. | Minor | conf | Yushi Hayasaka | Yushi Hayasaka | | | Only application master should upload resource to Yarn Shared Cache | Major | mrv2 | zhenzhao wang | zhenzhao wang | | | Correct spelling errors for separator | Trivial | common | Hui Fei | Hui Fei | | | Upgrade to jQuery 3.5.1 in hadoop-yarn-common | Major | . | Wei-Chiu Chuang | Aryan Gupta | | | RBF: Fix webHdfs file display error | Major | . | wangzhaohui | wangzhaohui | | | Fix wrong comment in LongLong.java | Trivial | documentation, examples | Akira Ajisaka | Wanqiang Ji | | | NMs should supply a health status when registering with RM | Major | . | Eric Badger | Eric Badger | | | FileSystem.DirListingIterator.next() call should return NoSuchElementException | Major | common, fs | Mukund Thakur | Mukund Thakur | | | MR job live lock caused by completed state container leak in heartbeat between node manager and RM | Major | nodemanager, yarn | zhenzhao wang | Jim Brennan | | | Set default throttle value on dfs.image.transfer.bandwidthPerSec | Major | namenode | Karthik Palanisamy | Karthik Palanisamy | | | Reduce datanode upgrade/hardlink thread | Major | datanode | Karthik Palanisamy | Karthik Palanisamy | | | TestNMProxy.testNMProxyRPCRetry is not consistent | Major | . | Ahmed Hussein | Ahmed Hussein | | | TestExternalStoragePolicySatisfier fails intermittently | Major | . | Ahmed Hussein | Leon Gao | | | update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 | Blocker | . | Pranav Bheda | Pranav Bheda | | | HttpFS server throws NPE if a file is a symlink | Major | fs, httpfs | Ahmed Hussein | Ahmed Hussein | | | Audit log deletes before collecting blocks | Major | logging, namenode | Ahmed Hussein | Ahmed Hussein | | | Javadoc warnings and errors are ignored in the precommit jobs | Major | build, documentation | Akira Ajisaka | Akira Ajisaka | | | Compute datanode load based on StoragePolicy | Major | hdfs, namenode | Karthik Palanisamy | Ayush Saxena | | | Touch command with -c option is broken | Major | . | Ayush Saxena | Ayush Saxena | | | TestWebHDFS.testLargeDirectory failing | Major | test, webhdfs | Mukund Thakur | Mukund Thakur | | | Backslash in username causes build failure in the environment started by

| Minor | build | Takeru Kuramoto | Takeru Kuramoto | | | [JDK 11] Fix Javadoc errors in hadoop-hdfs-client | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Deleted blocks linger in the replications queue | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | WASB : PageBlobOutputStream succeeding hflush even when underlying flush to storage failed | Critical | . | Anoop Sam John | Anoop Sam John | | | DataNode could meet deadlock if invoke refreshNameNode | Critical | . | Hongbing Wang | Hongbing Wang | | | LazyPersist Overwrite fails in direct write mode | Major | . | Ayush Saxena | Ayush Saxena | | | Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail | Major | test | Peter Bacsko | Peter Bacsko | | | Failed volumes can cause DNs to stop block reporting | Major | block placement, datanode | Ahmed Hussein | Ahmed Hussein | | | Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640 | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers | Major | resourcemanager | Haibo Chen | Haibo Chen | | | mvn site commands fails due to MetricsSystemImpl changes | Major | . | Xiaoqiao He | Xiaoqiao He | | | Client could not obtain block when DN CommandProcessingThread exit | Major | . | Yiqun Lin | Aiphago | | | TestLdapGroupsMapping failing -string mismatch in exception validation | Major | test | Steve Loughran | Steve Loughran | | | Audit log record the unexpected allowed result when delete called | Major | hdfs | Baolong Mao | Baolong Mao | | | Update PATCH\NAMING\RULE in the personality file | Minor | build | Akira Ajisaka | Akira Ajisaka | | | Hive On Tez queries fails upon submission to dynamically created pools | Major | resourcemanager | Anand Srinivasan | Peter Bacsko | | | Fix outdated properties of JournalNode when performing rollback | Minor | . | Deegue | Deegue | | | ZStandardCompressor throws java.lang.InternalError: Error (generic) | Major | io | Stephen Jung (Stripe) | Stephen Jung (Stripe) | | | NPE when starting MiniYARNCluster from hadoop-client-minicluster | Critical | . | Chao Sun | | | | Don't relocate org.bouncycastle in shaded client jars | Critical | . | Chao Sun | Chao Sun | | | hadoop-client-integration-tests doesn't work when building with skipShade | Major | . | Chao Sun | Chao Sun | | | Contract test for renaming over existing file is too lenient | Minor | test | Attila Doroszlai | Attila Doroszlai | | | Improve excessive reloading of Configurations | Major | conf | Ahmed Hussein | Ahmed Hussein | | | Fix the documentation for dfs.namenode.replication.max-streams in hdfs-default.xml | Major | . | Xieming Li | Xieming Li | | | Doing hadoop ls on Har file triggers too many RPC calls | Major | fs | Ahmed Hussein | Ahmed Hussein | | | TimelineConnector swallows InterruptedException | Major | . | Ahmed Hussein | Ahmed Hussein | | | Log the remote address for authentication success | Minor | ipc | Ahmed Hussein | Ahmed Hussein | | | [JDK 14] TestConfiguredFailoverProxyProvider#testResolveDomainNameUsingDNS fails | Major |

| Akira Ajisaka | Akira Ajisaka | | | [JDK 11] TestMRJobsWithProfiler fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Max applications calculation per queue disregards queue level settings in absolute mode | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Skip license check on lz4 code files | Major | build | Zhihua Deng | Zhihua Deng | | | Potential thread leak in LocatedFileStatusFetcher | Major | job submission | Zhihua Deng | Zhihua Deng | | | Fair call queue is defeated by abusive service principals | Major | common, ipc | Ahmed Hussein | Ahmed Hussein | | | When building new web ui with root user, the bower install should support it. | Major | build, yarn-ui-v2 | Qi Zhu | Qi Zhu | | | Skipping network I/O in S3A getFileStatus(/) breaks some tests | Major | fs/s3, test | Mukund Thakur | Mukund Thakur | | | Fix the typo of dfshealth.html after HDFS-15358 | Trivial | namenode | Hui Fei | Hui Fei | | | Fix Yarn CapacityScheduler Markdown document | Trivial | documentation | zhaoshengjie | zhaoshengjie | | | lz4 sources missing for native Visual Studio project | Major | native | Gautham Banasandra | Gautham Banasandra | | | NN should not let the balancer run in safemode | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Update yarn.nodemanager.env-whitelist value in docs | Minor | documentation | Andrea Scarpino | Andrea Scarpino | | | hadoop-aws does not work with hadoop-client-api | Major | fs/s3 | Keith Turner | Chao Sun | | | StorageTypeProto is not compatiable between 3.x and 2.6 | Major | . | Ryan Wu | Ryan Wu | | | NNTop counts don't add up as expected | Major | hdfs, metrics, namenode | Ahmed Hussein | Ahmed Hussein | | | EC: Socket file descriptor leak in StripedBlockChecksumReconstructor | Major | datanode, ec, erasure-coding | Yushi Hayasaka | Yushi Hayasaka | | | make the rpath of container-executor configurable | Major | yarn | angerszhu | angerszhu | | | Erasure Coding: dirty buffer causes reconstruction block error | Blocker | datanode, erasure-coding | HuangTao | HuangTao | | | Fix deprecation warnings in SLSWebApp.java | Minor | build | Akira Ajisaka | Ankit Kumar | | | ServerSocketUtil.getPort() should use loopback address, not 0.0.0.0 | Major | . | Eric Badger | Eric Badger | | | Lease Recovery never completes for a committed block which the DNs never finalize | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | EC: Block gets marked as CORRUPT in case of failover and pipeline recovery | Critical | erasure-coding | Ayush Saxena | Ayush Saxena | | | Client in distributedShell swallows interrupt exceptions | Major | client, distributed-shell | Ahmed Hussein | Ahmed Hussein | | | Fix -Pdist build failure of hadoop-hdfs-native-client | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | TestDistributedShell leaks resources on timeout/failure | Major | distributed-shell, test, yarn | Ahmed Hussein | Ahmed Hussein | | | Fix failure of TestDistributedShell#testDSShellWithOpportunisticContainers | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout | Critical | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2 | Major | webapp, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | ADLFS: Update SDK version from 2.3.6 to 2.3.9 | Minor | fs/adl | Bilahari T H | Bilahari T H | | | maxAMShare should only be accepted for leaf queues, not parent queues | Major |

| Siddharth Ahuja | Siddharth Ahuja | | | Increase docker memory limit in Jenkins | Major | build, scripts, test, yetus | Ahmed Hussein | Ahmed Hussein | | | Clear the fileMap in JHEventHandlerForSigtermTest | Minor | test | Zhengxi Li | Zhengxi Li | | | Stale record should be remove when MutableRollingAverages generating aggregate data. | Major | . | Haibin Huang | Haibin Huang | | | Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags | Major | test | Peter Bacsko | Peter Bacsko | | | Install Intel ISA-L library in Dockerfile | Blocker | . | Takanobu Asanuma | Takanobu Asanuma | | | AbstractContractDeleteTest should set recursive parameter to true for recursive test cases. | Major | . | Konstantin Shvachko | Anton Kutuzov | | | MagicS3GuardCommitter fails with \`pendingset\` already exists | Major | fs/s3 | Dongjoon Hyun | Dongjoon Hyun | | | The DeadNodeDetector shouldn't be shared by different DFSClients. | Major | . | Jinglun | Jinglun | | | Intermittent test failure org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength.testSnapshotfileLength | Major | hdfs, snapshots | Hanisha Koneru | Jim Brennan | | | Fix typo in BUILDING.txt | Trivial | documentation | Gautham Banasandra | Gautham Banasandra | | | Possible Resource Leak in FSImageFormatProtobuf | Major | namenode | Narges Shadab | Narges Shadab | | | EC: Wrong checksum when reconstruction was failed by exception | Major | datanode, ec, erasure-coding | Yushi Hayasaka | Yushi Hayasaka | | | EC: fix NPE caused by StripedWriter.clearBuffers during reconstruct block | Major | . | Hongbing Wang | Hongbing Wang | | | S3A FileSystem does not correctly delete directories with fake entries | Major | fs/s3 | Kaya Kupferschmidt | | | | EC: Reconstruct task failed, and It would be XmitsInProgress of DN has negative number | Major | . | Haiyang Hu | Haiyang Hu | | | User environment is unable to prepend PATH when mapreduce.admin.user.env also sets PATH | Major | . | Eric Badger | Eric Badger | | | ClasscastException while loading FSImage | Major | nn | Renukaprasad C | Renukaprasad C | | | Upgrade ant to 1.10.9 | Major | . | Akira Ajisaka | Akira Ajisaka | | | TestDelegationTokenRenewer fails intermittently | Major | test | Akira Ajisaka | Masatake Iwasaki | | | DeadNodeDetector should close all the threads when it is closed. | Major | . | Jinglun | Jinglun | | | Upgrade Jackson databind to 2.10.5.1 | Major | build | Adam Roberts | Akira Ajisaka | | | Remove job\history\summary.py | Major | . | Akira Ajisaka | Akira Ajisaka | | | Fix TestRMNodeLabelsManager failed after YARN-10501. | Major | . | Qi Zhu | Qi Zhu | | | Not closing an SFTP File System instance prevents JVM from exiting. | Major | . | Mikhail Pryakhin | Mikhail Pryakhin | | | Hadoop prints sensitive Cookie information. | Major | . | Renukaprasad C | Renukaprasad C | | | Reported IBR is partially replaced with stored info when queuing. | Critical | namenode | Kihwal Lee | Stephen O'Donnell | | | CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource() | Major | . | Haibo Chen | Haibo Chen | | | ClusterMapReduceTestCase does not clean directories | Major | . | Ahmed Hussein | Ahmed Hussein | | | Skip any credentials stored in HDFS when starting ZKFC | Major | hdfs | Krzysztof Adamski | Stephen O'Donnell | | | ExpiredHeartbeats metric should be of

| Major | metrics | Konstantin Shvachko | Qi Zhu | | | Fix RMNodeImpl.updateExistContainers leak | Major | resourcemanager | Max Xie | Max Xie | | | All testcases in TestReservations are flaky | Major | . | Szilard Nemeth | Szilard Nemeth | | | skip-dir option is not processed by Yetus | Major | build, precommit, yetus | Ahmed Hussein | Ahmed Hussein | | | Fix Typo in TestSchedulingRequestContainerAllocation | Minor | . | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Check whether file is being truncated before truncate | Major | . | Hui Fei | Hui Fei | | | Replace GitHub App Token with GitHub OAuth token | Major | build | Akira Ajisaka | Akira Ajisaka | | | Add option to disable/enable free disk space checking and percentage checking for full and not-full disks | Major | nodemanager | Qi Zhu | Qi Zhu | | | Upgrade org.codehaus.woodstox:stax2-api to 4.2.1 | Major | . | Ayush Saxena | Ayush Saxena | | | Correct timestamp format in the docs for the touch command | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | DeadNodeDetector doesn't remove live nodes from dead node set. | Major | . | Jinglun | Jinglun | | | Yarn Job execution get failed when LZ4 Compression Codec is used | Major | common | Bhavik Patel | Bhavik Patel | | | Percentage of queue and cluster is zero in WebUI | Major | . | Bilwa S T | Bilwa S T | | | revisiting TestMRIntermediateDataEncryption | Major | job submission, security, test | Ahmed Hussein | Ahmed Hussein | | | Fix potential null pointer error of gpuNodeResourceUpdateHandler in NodeResourceMonitorImpl. | Major | . | Qi Zhu | Qi Zhu | | | Possible Resource Leak in EditLogFileOutputStream | Major | . | Narges Shadab | Narges Shadab | | | Fix the wrong CIDR range example in Proxy User documentation | Minor | documentation | Kwangsun Noh | Kwangsun Noh | | | Upgrade com.github.eirslett:frontend-maven-plugin to 1.11.2 | Major | buid | Mingliang Liu | Mingliang Liu | | | Intermediate data encryption is broken in LocalJobRunner | Major | job submission, security | Ahmed Hussein | Ahmed Hussein | | | Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal | Major | . | Narges Shadab | Narges Shadab | | | Replace bzero with explicit\_bzero for better safety | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Resources are displayed in bytes in UI for schedulers other than capacity | Major | . | Bilwa S T | Bilwa S T | | | Upgrade JUnit to 4.13.1 | Major | build, security, test | Ahmed Hussein | Ahmed Hussein | | | RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode | Major | rbf | Harunobu Daikoku | Harunobu Daikoku | | | Can't remove all node labels after add node label without nodemanager port | Critical | yarn | caozhiqiang | caozhiqiang | | | Destroy yarn service if any YarnException occurs during submitApp | Minor | yarn-native-services | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Yarn Service AM listens on all IP's on the machine | Minor | security, yarn-native-services | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Add support for hadoop.http.rmwebapp.scheduler.page.class | Major | scheduler | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Fix NullPointerException in yarn-services

| Minor | . | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Fix typo in ContainerRuntime | Trivial | documentation | Wanqiang Ji | xishuhai | | | TestReplicaCachingGetSpaceUsed#testReplicaCachingGetSpaceUsedByRBWReplica Fails on Windows | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | DelegationTokenAuthenticator prints token information | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | Kinit with keytab should not display the keytab file's full path in any logs | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | Remove unused hdfs.proto import | Major | hdfs-client | Gautham Banasandra | Gautham Banasandra | | | Some tests in TestBlockRecovery are consistently failing | Major | . | Viraj Jasani | Viraj Jasani | | | Fix integer overflow | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | hadoop-auth to remove jetty-server dependency | Major | auth | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix test4tests for libhdfspp | Critical | build, libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Incorrect representation of RESPONSE for Get Key Version in KMS index.md.vm file | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | HadoopPipes task may fail when linux kernel version change from 3.x to 4.x | Major | pipes | chaoli | chaoli | | | Fix TestKMS failure | Major | kms | Akira Ajisaka | Akira Ajisaka | | | Unreleased volume references cause an infinite loop | Major | datanode | Shuyan Zhang | Shuyan Zhang | | | Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail | Major | nodemanager, test | Peter Bacsko | Peter Bacsko | | | ITestWasbUriAndConfiguration.testCanonicalServiceName() failing now mockaccount exists | Minor | fs/azure, test | Steve Loughran | Steve Loughran | | | RBF: Unable to display the datanode UI of the router | Major | rbf, ui | Xiangyi Zhu | Xiangyi Zhu | | | Upgrade Jetty to 9.4.40 | Blocker | . | Akira Ajisaka | Akira Ajisaka | | | Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647 | Major | . | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0 | Blocker | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Datanode DirectoryScanner uses excessive memory | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Shaded guava not found when compiling with profile hbase2.0 | Blocker | timelineserver | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Interrupt DataStreamer thread | Minor | datanode | Karthik Palanisamy | | | | RBF: RBFMetrics's TotalCapacity out of bounds | Major | . | Xiaoxing Wei | Fengnan Li | | | SequeneFile.Writer should implement StreamCapabilities | Major | . | Kishen Das | Kishen Das | | | Remove additional junit 4.11 dependency from javadoc | Major | build, test, timelineservice | ANANDA G B | Akira Ajisaka | | | Fix the error of TestDynamometerInfra | Major | test | Akira Ajisaka | Takanobu Asanuma | | | TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes | Blocker | hdfs | Konstantin Shvachko | Akira Ajisaka | | | Fix outdated properties of journal node when perform rollback | Minor |

| Deegue | | | | hadoop-azure parallel tests not working on recent JDKs | Major | build, fs/azure | Steve Loughran | Steve Loughran | | | Missing access check before getAppAttempts | Critical | webapp | lujie | lujie | | | checkcompatibility.py errors out when specifying annotations | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | HDFS-15245 breaks source code compatibility between 3.3.0 and 3.3.1. | Blocker | journal-node, ui | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Backport YARN-10120 to branch-3.3 | Major | . | Bilwa S T | Bilwa S T | | | The yarn.resource-types should support multi types without trimmed. | Major | . | Qi Zhu | Qi Zhu | | | Explicitly set locale in the Dockerfile | Blocker | build | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestFrameworkUploader#testNativeIO fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | [build] fix the Dockerfile for ARM | Blocker | build | Wei-Chiu Chuang | Wei-Chiu Chuang | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Fix TestLongLong#testRightShift | Minor | . | Wanqiang Ji | Wanqiang Ji | | | Remove useless dfs.webhdfs.enabled | Minor | test | Hui Fei | Hui Fei | | | Move personality file from Yetus to Hadoop repository | Major | test, yetus | Chao Sun | Chao Sun | | | Add Test annotation for TestPersistBlocks#testRestartDfsWithSync | Minor | hdfs | Hui Fei | Hui Fei | | | TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently | Major | yarn | Prabhu Joseph | Peter Bacsko | | | Add lz4-java as hadoop-hdfs test dependency | Major | . | L. C. Hsieh | L. C. Hsieh | | | ADL Gen1: Fix the test case failures which are failing after the contract test update in hadoop-common | Minor | fs/adl | Bilahari T H | Bilahari T H | | | Test case TestOfflineImageViewer fails | Minor | . | Hui Fei | Hui Fei | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | ABFS: Optimize HttpRequest retry triggers | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Avoid storage calls to check if the account is HNS enabled or not | Minor | fs/azure | Bilahari T H | Bilahari T H | | | Endpoint for obtaining direct URL for the logs | Major | yarn | Adam Antal | Andras Gyori | | | Erasure Coding: metrics xmitsInProgress become to negative. | Major | datanode, erasure-coding | Baolong Mao | Baolong Mao | | | Extend ViewFS and provide ViewFSOverloadScheme implementation with scheme configurable. | Major | fs, hadoop-client, hdfs-client, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement | Major | capacityscheduler | Prabhu Joseph | Prabhu Joseph | | | Make mount-table to read from central place ( Let's say from HDFS) | Major | configuration, hadoop-client | Uma Maheswara Rao G | Uma Maheswara Rao G | | | distcp -update to S3A; abfs, etc always overwrites due to block size mismatch | Major | fs/s3, tools/distcp | Daisuke Kobayashi | Steve Loughran | | | Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are

| Major | fs, nflyFs, viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping | Major | . | Prabhu Joseph | Gergely Pollk | | | ABFS: Send error back to client for Read Ahead request failure | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: FS initialize fails for incompatible account-agnostic Token Provider setting | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Make DFSAdmin tool to work with ViewFSOverloadScheme | Major | dfsadmin, fs, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | S3A FullCredentialsTokenBinding fails if local credentials are unset | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add lazy initialization of LogAggregationFileControllerFactory in LogServlet | Major | log-aggregation, yarn | Adam Antal | Adam Antal | | | Document the ViewFSOverloadScheme details in ViewFS guide | Major | viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme | Major | dfsadmin, viewfsOverloadScheme | Ayush Saxena | Ayush Saxena | | | Add all available fs.viewfs.overload.scheme.target.\<scheme\>.impl classes in core-default.xml bydefault. | Major | configuration, viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FSUsage$DF should consider ViewFSOverloadScheme in processPath | Minor | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler | Major | fairscheduler | Benjamin Teke | Benjamin Teke | | | ABFS: Improve the ABFS driver documentation | Minor | fs/azure | Bilahari T H | Bilahari T H | | | ViewFileSystemOverloadScheme should represent mount links as non symlinks | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Support max running app logic for CapacityScheduler | Major | capacity scheduler, capacityscheduler | zhoukang | Peter Bacsko | | | Merged ListStatus with Fallback target filesystem and InternalDirViewFS. | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: convert maxAppsDefault, maxRunningApps settings | Major | . | Peter Bacsko | Peter Bacsko | | | ABFS: Fix idempotency test failures when SharedKey is set as AuthType | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | S3A to support additional token issuers | Minor | fs/s3 | Gabor Bota | Steve Loughran | | | ABFS: Make PUT and POST operations idempotent | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | mkdirs should work when parent dir is internalDir and fallback configured. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Default mount table name used by ViewFileSystem should be configurable | Major | viewfs, viewfsOverloadScheme | Virajith Jalaparti | Virajith Jalaparti | | | job commit failure in S3A MR magic committer test | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Document max-parallel-apps for Capacity Scheduler | Major | capacity scheduler, capacityscheduler | Peter Bacsko | Peter Bacsko | | | ABFS: Adding metrics to AbfsInputStream (AbfsInputStreamStatistics) | Major | fs/azure | Gabor Bota | Mehakmeet Singh | | | ABFS: Fix the parsing errors in ABFS Driver with creation Time (being returned in ListPath) | Major | fs/azure | Ishani | Bilahari T H | | | create should work when parent dir is internalDir and fallback configured. | Major |

| Uma Maheswara Rao G | Uma Maheswara Rao G | | | Fix NN trash emptier to work if ViewFSOveroadScheme enabled | Major | namenode, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Replace Guava Optional with Java8+ Optional | Major | . | Ahmed Hussein | Ahmed Hussein | | | Optionally ignore port number in mount-table name when picking from initialized uri | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Support for Appendblob in abfs driver | Major | fs/azure | Ishani | Ishani | | | Add fs.viewfs.overload.scheme.target.ofs.impl to core-default.xml | Major | configuration, viewfs, viewfsOverloadScheme | Siyao Meng | Siyao Meng | | | ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links | Major | viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | S3AFS globStatus attempts to resolve symlinks | Minor | fs/s3 | Jimmy Zuber | Jimmy Zuber | | | Tune S3A listFiles() api. | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | Replace Guava Function with Java8+ Function | Major | . | Ahmed Hussein | Ahmed Hussein | | | Replace Guava Predicate with Java8+ Predicate | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Remove unnecessary ABFS toString() invocations | Minor | fs/azure | Jeetesh Mangwani | Bilahari T H | | | ITestS3ADirectoryPerformance.testListOperations failing | Minor | fs/s3, test | Mukund Thakur | Mukund Thakur | | | When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target fs. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Replace Guava Supplier with Java8+ Supplier in Hadoop | Major | . | Ahmed Hussein | Ahmed Hussein | | | ABFS: Fix For Idempotency code | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Long waits and unintended retries when multiple threads try to fetch token using ClientCreds | Major | fs/azure | Sneha Vijayarajan | Bilahari T H | | | Refactor S3A Listing code for better isolation | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic | Minor | fs/azure, test | Sneha Vijayarajan | Bilahari T H | | | ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey | Minor | fs/azure | Sneha Vijayarajan | Bilahari T H | | | ABFS: Add debug log for rename failures | Major | fs/azure | Bilahari T H | Bilahari T H | | | mkdirs on fallback should throw IOE out instead of suppressing and returning false | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | S3A to optionally retain directory markers | Major | fs/s3 | Aaron Fabbri | Steve Loughran | | | S3AFileSystem silently deletes "fake" directories when writing a file. | Minor | fs, fs/s3 | Joel Baranick | | | | ABFS: Upgrade Store REST API Version to 2019-12-12 | Major | fs/azure | Ishani | Sneha Vijayarajan | | | Provide DFS API compatible class(ViewDistributedFileSystem), but use ViewFileSystemOverloadScheme inside | Major | dfs, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Optimise s3a Listing to be fully

| Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | getChildFilesystems should include fallback fs as well | Critical | viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS | Minor | fs/s3 | Steve Loughran | Mukund Thakur | | | improve s3guard markers command line tool | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RESOURCE\_UPDATE event was repeatedly registered in DECOMMISSIONING state | Minor | resourcemanager | yehuanhuan | yehuanhuan | | | ViewDistributedFileSystem#recoverLease should call super.recoverLease when there are no mounts configured | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Handle transient stream read failures in FileSystem contract tests | Minor | fs/s3 | Steve Loughran | | | | Tiny Improve for DeadNode detector | Minor | hdfs-client | dark\_num | imbajin | | | RBF: Refresh cacheNS when SocketException occurs | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | listFiles on root/InternalDir will fail if fallback root has file | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Fix the rename issues with fallback fs enabled | Major | viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | ViewDFS#getDelegationToken should not throw UnsupportedOperationException. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | ABFS: Support for conditional overwrite | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | RBF: TestRouterRpc#testErasureCoding is flaky | Major | rbf | Chen Zhang | Chen Zhang | | | Tune listStatus() api of s3a. | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | ABFS: Test testNegativeScenariosForCreateOverwriteDisabled fails for non-HNS account | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Enable checkaccess API | Major | fs/azure | Bilahari T H | Bilahari T H | | | RBF: Router FSCK fails after HDFS-14442 | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | Implement FileSystem.listStatusIterator() in S3AFileSystem | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | S3A to always probe S3 in S3A getFileStatus on non-auth paths | Major | fs/s3 | Steve Loughran | Steve Loughran | | | RBF: Fix test failures after HADOOP-17281 | Major | rbf, test | Akira Ajisaka | Akira Ajisaka | | | s3a rename() now requires s3:deleteObjectVersion permission | Major | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance | Major | . | Bilahari T H | Bilahari T H | | | ABFS: configure output stream thread pool | Minor | fs/azure | Bilahari T H | Bilahari T H | | | ABFS: read-ahead error reporting breaks buffer management | Critical | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Use shaded guava from thirdparty | Major | . | Ayush Saxena | Ayush Saxena | | | TestBlockTokenWithDFSStriped fails intermittently | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | TestDFSClientRetries#testGetFileChecksum fails intermittently | Major | dfsclient, test | Ahmed Hussein | Ahmed Hussein | | | TestHAAppend#testMultipleAppendsDuringCatchupTailing is flaky | Major | . | Vinayakumar B | Ahmed Hussein | | | RBF: TestRouter#testNamenodeHeartBeatEnableDefault fails by BindException | Major | rbf, test | Akira Ajisaka | Akira Ajisaka | | | ITestCustomSigner fails with gcs s3 compatible endpoint. | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | EC: Fix checksum computation in case of native encoders | Blocker |

| Ahmed Hussein | Ayush Saxena | | | Harmonize guava version and shade guava in yarn-csi | Major | . | Wei-Chiu Chuang | Akira Ajisaka | | | ITestS3AContractRename failing against stricter tests | Major | fs/s3, test | Steve Loughran | Attila Doroszlai | | | AbstractS3ATokenIdentifier to set issue date == now | Major | fs/s3 | Steve Loughran | Jungtaek Lim | | | HADOOP-17244. S3A directory delete tombstones dir markers prematurely. | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | AbstractS3ATokenIdentifier to issue date in UTC | Major | . | Steve Loughran | Jungtaek Lim | | | Upgrade aws-java-sdk to 1.11.901 | Minor | build, fs/s3 | Dongjoon Hyun | Steve Loughran | | | WASB: Test failures | Major | fs/azure, test | Sneha Vijayarajan | Steve Loughran | | | s3a getFileStatus("/") to skip IO | Minor | fs/s3 | Steve Loughran | Mukund Thakur | | | ABFS: Logs should redact SAS signature | Major | fs/azure, security | Sneha Vijayarajan | Bilahari T H | | | FileSystem.get to support slow-to-instantiate FS clients | Major | fs, fs/azure, fs/s3 | Steve Loughran | Steve Loughran | | | [JDK 11] mvn package -Pdocs fails | Major | build, documentation | Akira Ajisaka | Akira Ajisaka | | | ABFS: testRenameFileOverExistingFile Fails after Contract test update | Major | fs/azure, test | Sneha Vijayarajan | Sneha Vijayarajan | | | S3A committer to support concurrent jobs with same app attempt ID & dest dir | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | ITestS3ADeleteCost.testDirMarkersFileCreation failure | Major | fs/s3 | Steve Loughran | Steve Loughran | | | tag S3GuardTool entry points as limitedPrivate("management-tools")/evolving | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | S3A marker tool mixes up -min and -max | Trivial | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: SAS Test updates for version and permission update | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | TestURLConnectionFactory fails by NoClassDefFoundError in branch-3.3 and branch-3.2 | Blocker | test | Akira Ajisaka | Chao Sun | | | TestUpgradeDomainBlockPlacementPolicy flaky | Major | namenode, test | Ahmed Hussein | Ahmed Hussein | | | ABFS: Set default ListMaxResults to max server limit | Major | fs/azure | Sumangala Patki | Thomas Marqardt | | | hadoop-common to add IOStatistics API | Major | fs | Steve Loughran | Steve Loughran | | | S3A statistics to support IOStatistics | Major | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: Optimise read for small files/tails of files | Major | fs/azure | Bilahari T H | Bilahari T H | | | ABFS Streams to support IOStatistics API | Major | fs/azure | Steve Loughran | Mehakmeet Singh | | | Intermittent failure of S3A tests which make assertions on statistics/IOStatistics | Major | fs/s3 | Steve Loughran | Steve Loughran | | | TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStream fails intermittently | Minor | . | Toshihiko Uchida | Toshihiko Uchida | | | TestBalancerWithMultipleNameNodes#testBalancingBlockpoolsWithBlockPoolPolicy fails on trunk | Major | . | Ahmed Hussein | Masatake Iwasaki | | | s3a to instrument duration of HTTP calls | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | S3A ITestPartialRenamesDeletes.testPartialDirDelete[bulk-delete=true] failure | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | [s3a] Intermittent failure of

| Major | fs/s3, test | Gabor Bota | Steve Loughran | | | Skipping network I/O in S3A getFileStatus(/) breaks ITestAssumeRole | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | ABFS: Allow Random Reads to be of Buffer Size | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Release Elastic ByteBuffer pool memory at outputStream close | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Delete Idempotency handling can lead to NPE | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Piggyback flush on Append calls for short writes | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | S3A docs to state s3 is consistent, deprecate S3Guard | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Magic committer files don't have the count of bytes written collected by spark | Major | fs/s3 | Steve Loughran | Steve Loughran | | | renaming S3A Statistic DELEGATION\TOKENS\ISSUED to DELEGATION\TOKEN\ISSUED broke tests downstream | Major | fs/s3 | Steve Loughran | Steve Loughran | | | magic committer to be enabled for all S3 buckets | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A NetworkBinding has a runtime class dependency on a third-party shaded class | Blocker | fs/s3 | Chris Wensel | Steve Loughran | | | ABFS : add high performance listStatusIterator | Major | fs/azure | Bilahari T H | Bilahari T H | | | [JDK 16] KerberosUtil#getOidInstance is broken by JEP 396 | Major | auth | Akira Ajisaka | Akira Ajisaka | | | Add OutputStream + Syncable to the Filesystem Specification | Major | fs | Steve Loughran | Steve Loughran | | | RBF: Fix contract tests after HADOOP-13327 | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | Support disabling buffered reads in ABFS positional reads | Major | . | Anoop Sam John | Anoop Sam John | | | ABFS checkException to map 403 to AccessDeniedException | Blocker | fs/azure | Steve Loughran | Steve Loughran | | | create client protocol: add ecPolicyName & storagePolicy param to debug statement string | Minor | . | Bhavik Patel | Bhavik Patel | | | Migrate to Python 3 and upgrade Yetus to 0.13.0 | Major | . | Akira Ajisaka | Akira Ajisaka | | | Add some Abortable.abort() interface for streams etc which can be terminated | Blocker | fs, fs/azure, fs/s3 | Steve Loughran | Jungtaek Lim | | | typo in MagicCommitTracker | Trivial | fs/s3 | Pierrick HYMBERT | Pierrick HYMBERT | | | ABFS: Run the integration tests with various combinations of configurations and publish consolidated results | Minor | fs/azure, test | Bilahari T H | Bilahari T H | | | Improve S3A rename resilience | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | ABFS: Toggle Store Mkdirs request overwrite parameter | Major | fs/azure | Sumangala Patki | Sumangala Patki | | | Correct abfs test assertion reversed in HADOOP-13327 | Major | fs/azure, test | Sumangala Patki | Sumangala Patki | | | Improve the Logs for File Concat Operation | Minor | namenode | Bhavik Patel | Bhavik Patel | | | TestBalancer#testMaxIterationTime fails sporadically | Major | . | Jason Darrell Lowe | Toshihiko Uchida | | | ClusterMetrics should support GPU capacity related metrics. | Major | metrics, resourcemanager | Qi Zhu | Qi Zhu | | | Add Node GPU Utilization and apply to NodeMetrics. | Major |

| Qi Zhu | Qi Zhu | | | Improve the log for HTTPFS server operation | Minor | httpfs | Bhavik Patel | Bhavik Patel | | | ITestAssumeRole.testAssumeRoleBadInnerAuth failure | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | Collect AwsSdkMetrics in S3A FileSystem IOStatistics | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | ClusterMetrics should support custom resource capacity related metrics. | Major | . | Qi Zhu | Qi Zhu | | | Improve the log for the Storage Policy Operations | Minor | namenode | Bhavik Patel | Bhavik Patel | | | Add cluster metric for amount of CPU used by RM Event Processor | Minor | yarn | Jim Brennan | Jim Brennan | | | Support queue capacity in terms of absolute resources with custom resourceType. | Critical | . | Qi Zhu | Qi Zhu | | | [JDK 15] TestPrintableString fails due to Unicode 13.0 support | Major | . | Akira Ajisaka | Akira Ajisaka | | | ABFS: Disable throttling update for auth failures | Major | fs/azure | Sumangala Patki | Sumangala Patki | | | Change CS nodes page in UI to support custom resource. | Major | . | Qi Zhu | Qi Zhu | | | ABFS: Support infinite lease dirs | Minor | . | Billie Rinaldi | Billie Rinaldi | | | ABFS to collect IOStatistics | Major | fs/azure | Steve Loughran | Mehakmeet Singh | | | ABFS: ITestAzureBlobFileSystemCheckAccess test failure if test doesn't have oauth keys | Major | fs/azure | Steve Loughran | Steve Loughran | | | whitespace not allowed in paths when saving files to s3a via committer | Blocker | fs/s3 | Krzysztof Adamski | Krzysztof Adamski | | | Add option to downgrade S3A rejection of Syncable to warning | Minor | . | Steve Loughran | Steve Loughran | | | mvn versions:set fails to parse pom.xml | Blocker | build | Wei-Chiu Chuang | Wei-Chiu Chuang | | | ABFS: Suport for customer provided encryption key | Minor | fs/azure | Bilahari T H | Bilahari T H | | | Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. | Major | yarn | Qi Zhu | Qi Zhu | | | Do not use guava's Files.createTempDir() | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestRouterRpcMultiDestination#testProxyGetTransactionID and testProxyVersionRequest are flaky | Major | rbf | Harunobu Daikoku | Akira Ajisaka | | | Possible NPE in S3A MultiObjectDeleteSupport error handling | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add back the exceptions removed by HADOOP-17432 for compatibility | Blocker | bulid | Akira Ajisaka | Quan Li | | | Race condition: AsyncDispatcher can get stuck by the changes introduced in YARN-8995 | Critical | resourcemanager | zhengchenyu | zhengchenyu | | | Add dispatcher metrics to RM | Major | . | Jonathan Hung | Qi Zhu | | | Multipart uploaders to be created through API call to FileSystem/FileContext, not service loader | Major | fs, fs/s3, hdfs | Steve Loughran | Steve Loughran | | | Ignore missing keystore configuration in reloading mechanism | Major | . | Borislav Iordanov | Borislav Iordanov | | | Remove useless property hadoop.assemblies.version in pom file | Trivial | build | Wei-Chiu Chuang | Akira Ajisaka | | | Update LICENSE for 3.3.1 | Blocker |

| Wei-Chiu Chuang | Wei-Chiu Chuang | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Restore ability to set Text to empty byte array | Minor | common | gaozhan ding | gaozhan ding | | | Remove unused configuration dfs.namenode.stripe.min | Minor | . | tomscut | tomscut | | | Add metrics for FSNamesystem read/write lock hold long time | Major | hdfs | tomscut | tomscut | | | Add namenode address in logs for block report | Minor | datanode, hdfs | tomscut | tomscut | | | Close FSImage and FSNamesystem after formatting is complete | Minor | . | tomscut | tomscut | | | Add metric for editPendingQ in FSEditLogAsync | Minor | . | tomscut | tomscut | | | Remove unused parameters in NameNodeProxiesClient | Minor | . | tomscut | tomscut | | | Use LongAdder instead of AtomicLong | Minor | . | tomscut | tomscut | | | Print network topology on the web | Minor | . | tomscut | tomscut | | | Add location into datanode info for NameNodeMXBean | Minor | . | tomscut | tomscut | | | Remove residual code of Ozone | Major | . | Wanqiang Ji | Wanqiang Ji | | | Merge QueueMapping and QueueMappingEntity | Major | yarn | Gergely Pollk | Gergely Pollk | | | Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule | Major | . | Gergely Pollk | Gergely Pollk | | | Avoid unnecessary QueueMappingEntity creations | Minor | . | Gergely Pollk | Hudky Mrton Gyula | | | CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy | Major | . | Gergely Pollk | Szilard Nemeth | | | Update Mockserver | Major | . | Wei-Chiu Chuang | Attila Doroszlai | | | CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review | Major | . | Gergely Pollk | Szilard Nemeth | | | Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes | Critical | webapp | Sunil G | Jim Brennan | | | Update the year to 2021 | Major | . | Xiaoqiao He | Xiaoqiao He | | | Reduce threadCount for unit tests to reduce the memory usage | Major | build, test | Akira Ajisaka | Akira Ajisaka | | | Upgrade com.fasterxml.woodstox:woodstox-core for security reasons | Major | . | Viraj Jasani | Viraj Jasani | | | DFSAdmin#printOpenFiles has redundant String#format usage | Minor | . | Viraj Jasani | Viraj Jasani | | | Removed duplicate dependency of hadoop-annotations | Minor | . | Viraj Jasani | Viraj Jasani | | | Bump netty to the latest 4.1.61 | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Avoid usage of deprecated IOUtils#cleanup API | Minor | . | Viraj Jasani | Viraj Jasani | | | Remove any rocksdb exclusion code | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Update to Jetty 9.4.39 | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Split TestBalancer into two classes | Major | . | Viraj Jasani | Viraj Jasani | | | Restrict imports from org.apache.curator.shaded | Major | . | Viraj Jasani | Viraj Jasani | | | Update commons-io to 2.8.0 | Major | . | Wei-Chiu Chuang | Akira Ajisaka | | | Upgrade to hadoop-thirdparty-1.1.0 | Major | . | Ayush Saxena | Wei-Chiu Chuang |