Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.4.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: Could not fetch real remote IP in RouterWebHdfsMethods | Major | webhdfs | Tao Li | Tao Li | | | Make some parameters configurable for DataNodeDiskMetrics | Major | hdfs | Tao Li | Tao Li | | | Refactor HDFS tool tests for better reuse | Blocker | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Replace all default Charset usage with UTF-8 | Major | common | Viraj Jasani | Viraj Jasani | | | Make HDFS snapshot tools cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Make HDFS ownership tools cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Make HDFS data transfer tools cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Improve lock type for ReplicaMap under fine-grain lock mode. | Major | hdfs | Mingxiang Li | Mingxiang Li | | | Split datanode block pool locks to volume grain. | Major | datanode | Mingxiang Li | Mingxiang Li | | | Fix shadedclient test failure | Blocker | test | Gautham Banasandra | Akira Ajisaka | | | CryptoOutputStream::close leak when encrypted zones + quota exceptions | Critical | fs | Colm Dougan | Colm Dougan | | | YARN Federation improvements | Major | federation | Subramaniam Krishnan | Subramaniam Krishnan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add queue capacity weights support in FairCallQueue | Major | ipc | Fengnan Li | Fengnan Li | | | Add Available Space Rack Fault Tolerant BPP | Major | block placement | Ayush Saxena | Ayush Saxena | | | Standby NameNode process getBlocks request to reduce Active load | Major | balancer & mover, namenode | Xiaoqiao He | Xiaoqiao He | | | Add a tool to validate FsImage | Major | namenode | Tsz-wo Sze | Tsz-wo Sze | | | Implement service-user feature in DecayRPCScheduler | Major | rpc-server | Takanobu Asanuma | Takanobu Asanuma | | | Provide Regex Based Mount Point In Inode Tree | Major | viewfs | zhenzhao wang | zhenzhao wang | | | Applying NVDIMM storage media to HDFS | Major | datanode, hdfs | YaYun Wang | YaYun Wang | | | Add SM4 encryption method for HDFS | Major | hdfs | liusheng | liusheng | | | Using snappy-java in SnappyCodec | Major | common | DB Tsai | L. C. Hsieh | | | Federation balance tool | Major | rbf, tools | Jinglun | Jinglun | | | Using lz4-java in Lz4Codec | Major | common | L. C. Hsieh | L. C. Hsieh | | | RBF: Improved isolation for downstream name

{Static} | Major | rbf | CR Hota | Fengnan Li | | | Add Metrics to HttpFS Server | Major | httpfs | Ahmed Hussein | Ahmed Hussein | | | Support HuaweiCloud Object Storage as a Hadoop Backend File System | Major | fs | zhongjun | lixianwei | | | EC: Verify EC reconstruction correctness on DataNode | Major | datanode, ec, erasure-coding | Toshihiko Uchida | Toshihiko Uchida | | | Print network topology on the web | Minor | namanode, ui | Tao Li | Tao Li | | | RBF: Print network topology on the router web | Minor | rbf | Tao Li | Tao Li | | | Distcp SnapshotDiff to support WebHDFS | Major | distcp, webhdfs | Xun REN | Xun REN | | | Discover datanodes with unbalanced block pool usage by the standard deviation | Major | datanode, ui | Tao Li | Tao Li | | | Add a method appendIfAbsent for CallerContext | Minor | common | Tao Li | Tao Li | | | Show start time of Datanode on Web | Minor | datanode, ui | Tao Li | Tao Li | | | Make dfs.blockreport.intervalMsec reconfigurable | Major | datanode | Tao Li | Tao Li | | | Exclude slow disks when choosing volume | Major | datanode | Tao Li | Tao Li | | | Async Profiler endpoint for Hadoop daemons | Major | common | Viraj Jasani | Viraj Jasani | | | Reconfig DataXceiver parameters for datanode | Major | datanode | Tao Li | Tao Li | | | Reconfig cache report parameters for datanode | Major | datanode | Tao Li | Tao Li | | | Reconfig block report parameters for datanode | Major | datanode | Tao Li | Tao Li | | | RBF: Add search box for Router's tab-mounttable web page | Minor | rbf | Max Xie | Max Xie | | | Reconfig slow peer parameters for datanode | Major | datanode | Tao Li | Tao Li | | | Reconfig slow disk parameters for datanode | Major | datanode | Tao Li | Tao Li | | | Introduce new config to specify AM default node-label when not specified | Major | nodeattibute | Junfan Zhang | Junfan Zhang | | | Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues | Major | capacity scheduler, yarn | Tamas Domok | Tamas Domok | | | Reconfig dfs usage parameters for datanode | Major | datanode | Tao Li | Tao Li | | | DFS API to retrieve slow datanodes | Major | datanode, dfsclient | Viraj Jasani | Viraj Jasani | | | dfsadmin -reconfig option to start/query reconfig on all live datanodes | Major | dfsadmin | Viraj Jasani | Viraj Jasani | | | Expose aggregate latency of slow node as perceived by the reporting node | Major | metrics | Viraj Jasani | Viraj Jasani | | | Slow peer metrics - add median, mad and upper latency limits | Major | metrics | Viraj Jasani | Viraj Jasani | | | Enhance client protocol to propagate last seen state IDs for multiple nameservices. | Major | common | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Add uncleaning option for local app log file with log-aggregation enabled | Major | log-aggregation | Ashutosh Gupta | Ashutosh Gupta | | | Support loading alternative docker client config from system environment | Major | yarn | Ashutosh Gupta | Ashutosh Gupta | | | Dynamically adjust max slow disks to exclude | Major | datanode | dingshun | dingshun | | | Add recoverLease(), setSafeMode(), isFileClosed() APIs to FileSystem | Major | fs | Wei-Chiu Chuang | Tak-Lon (Stephen) Wu | | | Add switch to decide whether to enable native

| Minor | erasure-coding | WangYuanben | WangYuanben | | | Make Manifest Committer the default for abfs and gcs | Major | client | Steve Loughran | Steve Loughran | | | Reconfig transfer and write bandwidth for datanode. | Major | datanode | WangYuanben | WangYuanben | | | Extend add-opens flag to container launch commands on JDK17 nodes | Major | build, mrv2 | Peter Szucs | Peter Szucs | | | Add add-opens flag to container launch commands on JDK17 nodes | Major | mr-am, mrv2 | Benjamin Teke | Benjamin Teke | | | Support to configure different capacity reserved for each disk of DataNode. | Minor | datanode, hdfs | Jiale Qi | Jiale Qi | | | Reconfigure the scheduling cycle of the slowPeerCollectorDaemon thread. | Major | configuration | Zhaobo Huang | Zhaobo Huang | | | Add read and write dataXceiver threads count metrics to datanode. | Major | datanode | Zhaobo Huang | Zhaobo Huang | | | Setup pre-commit CI for Windows 10 | Critical | build | Gautham Banasandra | Gautham Banasandra | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve JournalNode web UI | Major | journal-node, ui | Jianfei Jiang | Jianfei Jiang | | | FileInputFormat listStatus with less memory footprint | Major | job submission | Zhihua Deng | Zhihua Deng | | | Add metrics for operations hold lock times of FsDatasetImpl | Major | datanode | Xiaoqiao He | Xiaoqiao He | | | Add

to gitignore | Minor | build | Ayush Saxena | Ayush Saxena | | | Add -S option in "Count" command to show only Snapshot Counts | Major | common | Hemanth Boyina | Hemanth Boyina | | | Usage output of container-executor binary needs to include --http/--https argument | Minor | yarn | Siddharth Ahuja | Siddharth Ahuja | | | Create separate configuration for max global AM attempts | Major | am | Jonathan Hung | Bilwa S T | | | RBF: Add Block Related Metrics | Major | metrics, rbf | Ayush Saxena | Ayush Saxena | | | RBF: Provide Non DFS Used per DataNode in DataNode UI | Major | datanode, rbf, ui | Ayush Saxena | Lisheng Sun | | | When localizers fail due to resource timestamps being out, provide more diagnostics | Major | nodemanager | Steve Loughran | Siddharth Ahuja | | | Code cleanup in TestSchedConfCLI | Minor | test | Szilard Nemeth | Bilwa S T | | | Resources should be created with ResourceTypesTestHelper instead of TestUtils | Trivial | resourcemanager | Szilard Nemeth | Andras Gyori | | | Code cleanup and improvements in ConfigurationStoreBaseTest | Minor | test, yarn | Szilard Nemeth | Benjamin Teke | | | Parent directory in the explorer does not support all path formats | Minor | ui | Jianfei Jiang | Jianfei Jiang | | | Tidy Up Text and ByteWritables Classes | Minor | common | David Mollitor | David Mollitor | | | Configurable max application tags and max tag length | Major | resourcemanager | Jonathan Hung | Bilwa S T | | | Add explanation of unimplemented methods in InMemoryConfigurationStore | Major | capacity scheduler | Szilard Nemeth | Siddharth Ahuja | | | HsJobsBlock reuse JobACLsManager for checkAccess | Minor | mrv2 | Bibin Chundatt | Bilwa S T | | | Add more information to longest write/read lock held log | Major | namanode | Toshihiro Suzuki | Toshihiro Suzuki | | | The suffix name of the unified compression class | Major | io | bianqi | bianqi | | | Code cleanup in ZKConfigurationStore | Minor | resourcemanager | Szilard Nemeth | Andras Gyori | | | Code cleanup in QueueAdminConfigurationMutationACLPolicy | Major | resourcemanager | Szilard Nemeth | Siddharth Ahuja | | | Code cleanup in LeveldbConfigurationStore | Minor | resourcemanager | Szilard Nemeth | Benjamin Teke | | | TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup | Minor | test | Szilard Nemeth | Benjamin Teke | | | AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. | Minor | block placement | Jinglun | Jinglun | | | Update Dockerfile to use Bionic | Major | build, test | Akira Ajisaka | Akira Ajisaka | | | Code cleanup in LeveldbRMStateStore | Minor | resourcemanager | Benjamin Teke | Benjamin Teke | | | Add hadoop.http.idle\_timeout.ms to core-default.xml | Major | conf | Wei-Chiu Chuang | Lisheng Sun | | | Use DFSConfigKeys MONITOR\CLASS\DEFAULT constant | Minor | hdfs | bianqi | bianqi | | | DFSInputStream to prefer cached replica | Major | hdfs-client | Wei-Chiu Chuang | Lisheng Sun | | | Replace the deprecated method shaHex | Minor | balancer & mover | bianqi | bianqi | | | listOpenFiles() should throw InvalidPathException in case of invalid paths | Minor | namanode | Jinglun | Jinglun | | | Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo | Major | capacity scheduler, webapp | Prabhu Joseph | Prabhu Joseph | | | Set dfs.client.failover.random.order to true as default | Major | hdfs-client | Takanobu Asanuma | Takanobu Asanuma | | | Consider StorageType when DatanodeManager#sortLocatedBlock() | Major | hdfs | Lisheng Sun | Lisheng Sun | | | RBF: RouterPermissionChecker#checkSuperuserPrivilege should use UGI#getGroups after HADOOP-13442 | Major | rbf | Xiaoyu Yao | Xiaoyu Yao | | | Allow transitioning queue from DRAINING to RUNNING state | Major | capacity scheduler | Jonathan Hung | Bilwa S T | | | DataNode#checkSuperuserPrivilege should use UGI#getGroups after HADOOP-13442 | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Add a Distcp option to preserve Erasure Coding attributes | Major | tools/distcp | Wei-Chiu Chuang | Ayush Saxena | | | Introduce StreamContext for Abfs Input and Output streams. | Major | fs/azure | Mukund Thakur | Mukund Thakur | | | TestFTPFileSystem failing as ftp server dir already exists | Minor | fs, test | Steve Loughran | Mikhail Pryakhin | | | Unify configuration \`dfs.ha.allow.stale.reads\` to DFSConfigKeys | Major | hdfs | Xiaoqiao He | Xiaoqiao He | | | RBF: Unify router datanode UI with namenode datanode UI | Major | datanode, rbf, ui | Ayush Saxena | Ayush Saxena | | | Hadoop distcp throws "ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found" | Minor | tools/distcp | Aki Tanaka | Aki Tanaka | | | HDFS-client: boost ShortCircuit Cache | Minor | dfsclient | Danil Lipovoy | Danil Lipovoy | | | VolumeScanner skip to scan blocks accessed during recent scan peroid | Minor | datanode | Yang Yun | Yang Yun | | | Avoid Potential Infinite Loop in DFSNetworkTopology | Major | dfs | Ayush Saxena | Ayush Saxena | | | Use sudo instead of su to allow nologin user for secure DataNode | Major | datanode, security | Akira Ajisaka | Kei Kori | | | SlotReleaser is not fast enough | Major | hdfs-client | Gang Xie | Lisheng Sun | | | Refactor

VolumeScanner#runLoop() | Minor | datanode | Yang Yun | Yang Yun | | | Make the default block storage policy ID configurable | Minor | block placement, namenode | Yang Yun | Yang Yun | | | TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally | Major | balancer, test | Xiaoqiao He | Xiaoqiao He | | | Make copyFromLocal's -t option available for put as well | Major | common | Andras Bokor | Andras Bokor | | | RedundantEditLogInputStream should log caught exceptions | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Generate queue metrics for each partition | Major | capacity scheduler | Jonathan Hung | Manikandan R | | | Zookeeper Delegation Token Manager fetch sequence number by batch | Major | security | Fengnan Li | Fengnan Li | | | TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology | Minor | hdfs | Jim Brennan | Jim Brennan | | | EC: Allow closing a file with committed blocks | Major | erasure-coding | Ayush Saxena | Ayush Saxena | | | TODO comments exist in trunk while the related issues are already fixed. | Trivial | fs | Rungroj Maipradit | Rungroj Maipradit | | | Update the error about command line POST in httpfs documentation | Major | httpfs | bianqi | bianqi | | | Improve the speed of Datanode Block Scan | Major | datanode | Hemanth Boyina | Hemanth Boyina | | | Embrace Immutability of Java Collections | Minor | common | David Mollitor | David Mollitor | | | QueueACLsManager and ReservationsACLManager should not use instanceof checks | Major | resourcemanager | Szilard Nemeth | Bilwa S T | | | Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios | Minor | test | Szilard Nemeth | Szilard Nemeth | | | RBF: Disable watch in ZKDelegationSecretManager for performance | Major | rbf | Fengnan Li | Fengnan Li | | | Improve DataStorage#addStorageLocations() for empty locations | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | Increase precommit job timeout from 5 hours to 20 hours | Major | build | Akira Ajisaka | Akira Ajisaka | | | Update Dockerfile\_aarch64 to use Bionic | Major | build, test | RuiChen | zhaorenhai | | | Apply umask when creating directory by WebHDFS | Minor | webhdfs | Ye Ni | Ye Ni | | | Review Logging of DFSClient | Minor | dfsclient | Hongbing Wang | Hongbing Wang | | | RMWebApp make external class pluggable | Minor | resourcemanager, webapp | Bibin Chundatt | Bilwa S T | | | YarnClient obtain Delegation Token for Log Aggregation Path | Major | log-aggregation | Prabhu Joseph | Prabhu Joseph | | | Optimize UGI#getGroups by adding UGI#getGroupsSet | Major | build | Xiaoyu Yao | Xiaoyu Yao | | | TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently | Major | fairscheduler, test | Jonathan Hung | Jim Brennan | | | Nonstandard characters exist in NameNode.java | Minor | namanode | JiangHua Zhu | Zhao Yi Ming | | | Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime | Minor | common | Jim Brennan | Jim Brennan | | | Upgrade boost library to 1.72 | Critical | build, libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Add

to hadoop-aws docs | Minor | documentation, fs/s3 | Nicholas Chammas | Nicholas Chammas | | | Make AsyncStream class' executor\_ member private | Minor | build, libhdfs++ | Suraj Naik | Suraj Naik | | | Fix typo corrputBlocksFiles to corruptBlocksFiles | Trivial | hdfs | bianqi | bianqi | | | ShellCommandFencer should expose info about source | Major | ha, tools | Chen Liang | Chen Liang | | | Dead link in hadoop-kms/index.md.vm | Minor | documentation, kms | Akira Ajisaka | Xieming Li | | | Adding ReadAhead Counters in ABFS | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | Record Last N Scheduler Activities from ActivitiesManager | Major | activitiesmanager, resourcemanager, router | Prabhu Joseph | Prabhu Joseph | | | Add Capability To Get Text Length | Minor | common, io | David Mollitor | David Mollitor | | | Add capacityScheduler metric for NODE\_UPDATE interval | Minor | capacity scheduler, metrics | Pranjal Protim Borah | Pranjal Protim Borah | | | Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. | Major | resourcemanager, ui | Eric Payne | Eric Payne | | | Add Localization overhead metrics to NM | Major | nodemanager | Gera Shegalov | Jim Brennan | | | Send out application attempt state along with other elements in the application attempt object returned from appattempts REST API call | Minor | yarn-ui-v2 | Siddharth Ahuja | Siddharth Ahuja | | | Make custom DAO classes configurable into RMWebApp#JAXBContextResolver | Major | resourcemanager, webapp | Prabhu Joseph | Bilwa S T | | | Remove smallBufferSize in DFSClient | Minor | dfsclient | Takanobu Asanuma | Takanobu Asanuma | | | Show extended resources on legacy RM UI. | Major | . | Eric Payne | Eric Payne | | | Use visitor pattern to visit namespace tree | Major | namenode | Tsz-wo Sze | Tsz-wo Sze | | | Option to override RMWebServices with custom WebService class | Major | resourcemanager, webservice | Prabhu Joseph | Tanu Ajmera | | | Update block map and name cache in parallel while loading fsimage. | Major | namenode | Chengwei Wang | Chengwei Wang | | | Add python2 to required package on CentOS 8 for building documentation | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Check inaccessible INodes in FsImageValidation | Major | tools | Tsz-wo Sze | Tsz-wo Sze | | | Refactor NodeQueueLoadMonitor class to make it extendable | Minor | resourcemanager | Zhengbo Li | Zhengbo Li | | | Remove duplicate BlockPoolManager starting when run DataNode | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | Make UGI support forceful relogin from keytab ignoring the last login time | Major | security | Sandeep Guggilam | Sandeep Guggilam | | | Erasure Coding: Typo in document | Trivial | documentation | Hui Fei | Hui Fei | | | Remove unused imports from TestFileTruncate.java | Minor | test | Ravuri Sushma sree | Ravuri Sushma sree | | | [UI1] Provide a way to hide Tools section in Web UIv1 | Minor | ui | Andras Gyori | Andras Gyori | | | Add

to gitignore | Minor | yarn | Takanobu Asanuma | Takanobu Asanuma | | | Erasure Coding: Remove dead code from common side | Minor | erasure-coding | Hui Fei | Hui Fei | | | getNMResourceInfo NodeManager REST API method is not documented | Major | documentation, nodemanager | Szilard Nemeth | Hudky Mrton Gyula | | | Log vcores used and cumulative cpu in containers monitor | Minor | yarn | Jim Brennan | Jim Brennan | | | Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG | Minor | yarn | Jim Brennan | Jim Brennan | | | Call recoverLease on DFSOutputStream close exception | Major | hdfs-client | Chen Zhang | Lisheng Sun | | | LeafQueue: retain user limits cache across assignContainers() calls | Major | capacity scheduler, capacityscheduler | Muhammad Samir Khan | Muhammad Samir Khan | | | Remove unnecessary sort of block list in DirectoryScanner | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | LoadBalanceKMSClientProvider#deleteKey should invalidateCache via all KMSClientProvider instances | Major | common | Xiaoyu Yao | Xiaoyu Yao | | | Fix testCompressorDecompressorWithExeedBufferLimit to cover the intended scenario | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | RBF: The constructor of PathLocation may got some misunderstanding | Minor | rbf | Janus Chow | Janus Chow | | | RBF: force router check file existence in destinations before adding/updating mount points | Minor | rbf | Fengnan Li | Fengnan Li | | | Allow SSLFactory fallback to input config if ssl-\*.xml fail to load from classpath | Major | common | Xiaoyu Yao | Xiaoyu Yao | | | Access Controlled HTTPFS Proxy | Minor | httpfs | Richard | Richard | | | Log the reason why a storage log file can't be deleted | Minor | hdfs | Ye Ni | Ye Ni | | | Fair scheduler docs should explain meaning of weight=0 for a queue | Major | docs | Daniel Templeton | Takeru Kuramoto | | | Hadoop - Upgrade to JQuery 3.5.1 | Major | build, common | Aryan Gupta | Aryan Gupta | | | libzstd-dev should be used instead of libzstd1-dev on Ubuntu 18.04 or higher | Minor | common | Takeru Kuramoto | Takeru Kuramoto | | | Lazy calculate live datanodes in safe mode tip | Minor | namenode | Ye Ni | Ye Ni | | | Refactor TestTracing | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Fix typo in DFS\ROUTER\QUOTA\CACHE\UPDATE\_INTERVAL var definition | Minor | rbf | Sha Fanghao | Sha Fanghao | | | Fix Typo for HdfsDataNodeAdminGuide doc | Trivial | documentation | Hui Fei | Hui Fei | | | RBF: Fix getLocationsForPath twice in create operation | Major | rbf | Zhaohui Wang | Zhaohui Wang | | | Support BCFKS keystores for Hadoop Credential Provider | Major | security | Xiaoyu Yao | Xiaoyu Yao | | | Service-user cost shouldn't be accumulated to totalDecayedCallCost and totalRawCallCost. | Major | ipc | Jinglun | Jinglun | | | Reduce locking in Datanode DirectoryScanner | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Support new Instance by non default constructor by ReflectionUtils | Major | common | Baolong Mao | Baolong Mao | | | Extend CallerContext to make it include many items | Major | common, ipc | Hui Fei | Hui Fei | | | RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. | Major | resourcemanager, ui | Eric Payne | Eric Payne | | | Expose Mini MR Cluster attribute for testing | Minor | test | Swaroopa Kadam | Swaroopa Kadam | | | [SBN Read] HDFS should expose msync() API to allow downstream applications call it

| Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko | | | KMS ACL: Allow DeleteKey Operation to Invalidate Cache | Major | kms | Xiaoyu Yao | Xiaoyu Yao | | | Avoid redundant RPC calls for getDiskStatus | Major | dfsclient | Ayush Saxena | Ayush Saxena | | | Add cpu and memory utilization per node and cluster-wide metrics | Minor | yarn | Jim Brennan | Jim Brennan | | | Update Hadoop's lz4 to v1.9.2 | Major | build, common | Hemanth Boyina | Hemanth Boyina | | | Add seqno when warning slow mirror/disk in BlockReceiver | Major | datanode | Haibin Huang | Haibin Huang | | | Upgrade to jQuery 3.5.1 in hadoop-sls | Major | build, common | Aryan Gupta | Aryan Gupta | | | Make block size from NNThroughputBenchmark configurable | Minor | benchmarks | Hui Fei | Hui Fei | | | Scale RM-NM heartbeat interval based on node utilization | Minor | yarn | Jim Brennan | Jim Brennan | | | Balancer logging improvement | Major | balancer & mover | Konstantin Shvachko | Konstantin Shvachko | | | Creating a token identifier should not do kerberos name resolution | Major | common | Jim Brennan | Jim Brennan | | | RMProxy should retry on SocketTimeout Exceptions | Major | yarn | Jim Brennan | Jim Brennan | | | Respect configured values of rpc.engine | Major | hdfs | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | RBF: Fix RouterRPCMetrics annocation and document misplaced error | Minor | documentation | Hongbing Wang | Hongbing Wang | | | Bump up snappy-java to 1.1.8.1 | Minor | common | Masatake Iwasaki | Masatake Iwasaki | | | replace href tags with ng-href | Trivial | applications-catalog, webapp | Gabriel Medeiros Coelho | Gabriel Medeiros Coelho | | | Rename variable DistCp#CLEANUP | Trivial | distcp | JiangHua Zhu | JiangHua Zhu | | | Dynamically configure the size of PacketReceiver#MAX\PACKET\SIZE | Major | hdfs-client | JiangHua Zhu | JiangHua Zhu | | | Add InetAddress api to ProxyUsers.authorize | Major | performance, security | Ahmed Hussein | Ahmed Hussein | | | Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class | Major | mrv2 | Daniel Fritsi | Zoltn Erdmann | | | EC: Call recoverLease on DFSStripedOutputStream close exception | Major | dfsclient, ec | Hongbing Wang | Hongbing Wang | | | Improve performance of reading resource request for mapper/reducers from config | Major | applicationmaster | Wangda Tan | Peter Bacsko | | | Avoid calling UpdateHeartBeatState inside DataNodeDescriptor | Major | datanode | Ahmed Hussein | Ahmed Hussein | | | Add Option to let Balancer prefer highly utilized nodes in each iteration | Major | balancer & mover | Leon Gao | Leon Gao | | | Fix a typo in SecondaryNameNode.java | Trivial | hdfs | Sixiang Ma | Sixiang Ma | | | Don't generate edits for set operations that are no-op | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Remote exception messages should not include the exception class | Major | ipc | Ahmed Hussein | Ahmed Hussein | | | HttpFS: Log more information on request failures | Major | httpfs | Ahmed Hussein | Ahmed Hussein | | | KMS should log full UGI principal | Major | kms | Ahmed Hussein | Ahmed Hussein | | | Add checking of effective filesystem during initializing storage locations | Minor | datanode | Yang Yun | Yang Yun | | | Upgrade googletest to 1.10.0 | Critical | build, libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Bump up snappy-java to 1.1.8.2 | Minor | build, common | L. C. Hsieh | L.

Hsieh | | | Improve fsck logging | Major | logging, namenode | Ahmed Hussein | Ahmed Hussein | | | Update description of dfs.datanode.handler.count in hdfs-default.xml | Minor | configuration | liuyan | liuyan | | | Mitigate lease monitor's rapid infinite loop | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Add seqno in log when BlockReceiver receive packet | Minor | datanode | Haibin Huang | Haibin Huang | | | Add option to make balancer prefer to get cold blocks | Minor | balancer & mover | Yang Yun | Yang Yun | | | Speed up the Storage#doRecover during datanode rolling upgrade | Major | datanode | Hemanth Boyina | Hemanth Boyina | | | Make size of editPendingQ can be configurable | Major | hdfs | Baolong Mao | Baolong Mao | | | Make DataNodePeerMetrics#LOW\THRESHOLD\MS and MIN\OUTLIER\DETECTION\_NODES configurable | Major | datanode, metrics | Haibin Huang | Haibin Huang | | | Add documentation for msync() API to filesystem.md | Major | documentation | Konstantin Shvachko | Konstantin Shvachko | | | Create packet metrics for DataNode | Minor | datanode | Fengnan Li | Fengnan Li | | | Add recommissioning nodes to the list of updated nodes returned to the AM | Major | resourcemanager | Srinivas S T | Srinivas S T | | | capture the performance metrics of ZKRMStateStore | Minor | resourcemanager | Max Xie | Max Xie | | | Optimize NetworkTopology while sorting of block locations | Major | common, net | Ahmed Hussein | Ahmed Hussein | | | Add timeout to RouterWebServiceUtil#invokeRMWebService | Major | router, webservice | igo Goiri | Minni Mittal | | | Diagnostics for localization timeouts is lacking | Major | nodemanager | Chang Li | Chang Li | | | Follow up changes for YARN-9833 | Major | yarn | Jim Brennan | Jim Brennan | | | Fix typos in MutableMetric | Trivial | metrics | Haibin Huang | Haibin Huang | | | Speed up BlockPlacementPolicyRackFaultTolerant#verifyBlockPlacement | Major | block placement | Akira Ajisaka | Akira Ajisaka | | | Refactor QueueMetricsForCustomResources class to move to yarn-common package | Major | metrics | Minni Mittal | Minni Mittal | | | "yarn top" command not quitting completely with ctrl+c | Minor | yarn | Agshin Kazimli | Agshin Kazimli | | | Improve the description of hadoop.http.authentication.signature.secret.file | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Upgrade guice to

| Major | build, common | Yuming Wang | Yuming Wang | | | Update Dockerfile to use Focal | Major | build, test | Gautham Banasandra | Gautham Banasandra | | | Lease renewal does not require namesystem lock | Major | hdfs | Jim Brennan | Jim Brennan | | | Make basename cross-platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix logging typo in ShutdownHookManager | Major | common | Konstantin Shvachko | Fengnan Li | | | Move Jenkinsfile outside of the root directory | Major | build | Akira Ajisaka | Akira Ajisaka | | | Simplify dependency installation instructions | Trivial | documentation | Gautham Banasandra | Gautham Banasandra | | | Parallelize building of dependencies | Minor | build | Gautham Banasandra | Gautham Banasandra | | | EC: Remove unnecessary method (getWeight) in StripedReconstructionInfo | Trivial | erasure-coding | Haiyang Hu | Haiyang Hu | | | Make DisallowedDatanodeException terse | Minor | hdfs | Richard | Richard | | | Fix a codestyle issue for TestQuotaByStorageType | Trivial | hdfs | Baolong Mao | Baolong Mao | | | Add queuePath to RESTful API for CapacityScheduler consistent with FairScheduler queuePath | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | DataStreamer: keep sending heartbeat packets while streaming | Major | hdfs | Jim Brennan | Jim Brennan | | | Set thread names for CapacityScheduler AsyncScheduleThread | Minor | capacity scheduler | Bibin Chundatt | Amogh Desai | | | Log list of mappers at trace level in ShuffleHandler audit log | Minor | yarn | Jim Brennan | Jim Brennan | | | Add metrics for in-service datanodes | Minor | metrics | Zehao Chen | Zehao Chen | | | FairScheduler: add global flag to disable AM-preemption | Major | fairscheduler | Peter Bacsko | Peter Bacsko | | | Log resource allocation in NM log at container start time | Major | nodemanager | Eric Badger | Eric Badger | | | if required storageType are unavailable, log the failed reason during choosing Datanode | Minor | block placement | Yang Yun | Yang Yun | | | Support to make dfs.image.parallel.load reconfigurable | Major | namenode | Hui Fei | Hui Fei | | | Erasure coding: Add/remove logs for the better readability/debugging | Minor | erasure-coding, hdfs | Bhavik Patel | Bhavik Patel | | | Solve the problem of incorrect progress of delegation tokens when loading FsImage | Major | namanode | JiangHua Zhu | JiangHua Zhu | | | [READ] DirectoryScanner#scan need not check StorageType.PROVIDED | Minor | datanode | Yuxuan Wang | Yuxuan Wang | | | Add kms-default.xml and httpfs-default.xml to site index | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF | Minor | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | Fixed the findbugs issues introduced by YARN-10647. | Major | test | Qi Zhu | Qi Zhu | | | Make write pipeline retry times configurable. | Minor | hdfs-client | Qi Zhu | Qi Zhu | | | ClientHSSecurityInfo class is in wrong META-INF file | Major | mapreduce-client | Eric Badger | Eric Badger | | | Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md | Minor | documentation | Ravuri Sushma sree | Ravuri Sushma sree | | | Capacity scheduler should support refresh queue automatically by a thread policy. | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Change ipc.client.rpc-timeout.ms from 0 to 120000 by default to avoid potential hang | Major | ipc | Haoze Wu | Haoze Wu | | | Document getLocatedBlocks(String src, long start) of DFSClient only return partial blocks | Minor | documentation | Yang Yun | Yang Yun | | | CapacityScheduler QueueInfo add queue path field to avoid ambiguous

| Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Allow parameter expansion in NM\ADMIN\USER\_ENV | Major | yarn | Jim Brennan | Jim Brennan | | | Apply YETUS-1102 to re-enable GitHub comments | Major | build | Akira Ajisaka | Akira Ajisaka | | | Remove trace subcommand from hadoop CLI | Minor | scripts | Masatake Iwasaki | Masatake Iwasaki | | | Remove Commons Logger from FileSystem Class | Minor | common | David Mollitor | David Mollitor | | | Fix incorrectly initializing RandomAccessFile based on configuration options | Major | namanode | Xie Lei | Xie Lei | | | [libhdfs++] Make write cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix an issue in CapacityScheduler which fails to delete queues | Major | capacity scheduler | Wangda Tan | Wangda Tan | | | DistCp: Expose the JobId for applications executing through run method | Major | tools/distcp | Ayush Saxena | Ayush Saxena | | | Queue metrics for Unmanaged applications | Minor | resourcemanager | Cyrus Jackson | Cyrus Jackson | | | Remove unnecessary Lease Renew in FSNamesystem#internalReleaseLease | Major | namenode | Lisheng Sun | Lisheng Sun | | | Refactor X-Platform library | Minor | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Remove NULL checks before instanceof | Minor | common | Jiajun Jiang | Jiajun Jiang | | | Remove useless NULL checks before instanceof | Minor | hdfs | Jiajun Jiang | Jiajun Jiang | | | Reduce Memory Overhead of AclFeature by avoiding AtomicInteger | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Provide blocks moved count in Balancer iteration result | Major | balancer & mover | Viraj Jasani | Viraj Jasani | | | BlockPoolManager should log stack trace if unable to get Namenode addresses | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Implement HttpServer2 metrics | Major | httpfs, kms | Akira Ajisaka | Akira Ajisaka | | | DistCp: Reduce memory usage on copying huge directories | Critical | tools/distcp | Ayush Saxena | Ayush Saxena | | | Exclude slow nodes when choose targets for blocks | Major | block placement | Tao Li | Tao Li | | | Notify Namenode missing or new block on disk as soon as possible | Minor | datanode | Yang Yun | Yang Yun | | | Use spotbugs-maven-plugin instead of findbugs-maven-plugin | Major | build | Akira Ajisaka | Akira Ajisaka | | | Create socket address leveraging URI cache | Major | common, hdfs-client | Rui Fan | Rui Fan | | | AMParams.java having un-necessary access identifier static final | Trivial | resourcemanager | ANANDA G B | ANANDA G B | | | Improve the balancer error message when process exits abnormally. | Major | balancer | Renukaprasad C | Renukaprasad C | | | RBF: Validation message to be corrected in FairnessPolicyController | Minor | rbf | Renukaprasad C | Renukaprasad C | | | Automatic keystore reloading for HttpServer2 | Major | common | Kihwal Lee | Borislav Iordanov | | | Log the size of DelegationTokenRenewer event queue in case of too many pending events | Major | resourcemanager | Qi Zhu | Qi Zhu | | | Fix non-static inner classes for better memory management | Major | hdfs | Viraj Jasani | Viraj Jasani | | | Bump Jetty to the latest version

| Major | build, common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Increase Quota initialization threads | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Make fnmatch cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Log not flushed fully when daemon shutdown | Major | common | Renukaprasad C | Renukaprasad C | | | Reduce memory used during datanode layout upgrade | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Make explicit\_bzero cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Make strcasecmp cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Building native code fails on Fedora 33 | Major | build, common | Kengo Seki | Masatake Iwasaki | | | Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs | Major | auth, build | helen huang | Viraj Jasani | | | DistCp: Use Iterator for listing target directory as well | Major | tools/distcp | Ayush Saxena | Ayush Saxena | | | Add a policy for not aggregating for containers which are killed because exceeding container log size limit. | Major | nodemanager | Qi Zhu | Qi Zhu | | | Solve DatanodeManager#getBlockRecoveryCommand() printing IOException | Trivial | namanode | JiangHua Zhu | JiangHua Zhu | | | Improve the log for Short Circuit Local Reads | Minor | datanode | Bhavik Patel | Bhavik Patel | | | LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException | Major | common | Tamas Mate | Istvn Fajth | | | Make DirectoryScanner reconcile blocks batch size and interval between batch configurable. | Major | datanode, diskbalancer | Qi Zhu | Qi Zhu | | | Remove workaround for Curator's ChildReaper requiring Guava 15+ | Major | common | Robert Kanter | Viraj Jasani | | | Improve the log for The DecayRpcScheduler | Minor | ipc | Bhavik Patel | Bhavik Patel | | | ProcessReport print invalidatedBlocks should judge debug level at first | Minor | namanode | lei w | lei w | | | Dockerfile for building on Centos 7 | Major | build | Gautham Banasandra | Gautham Banasandra | | | Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException | Major | hdfs | junwen yang | Viraj Jasani | | | Provide source artifacts for hadoop-client-api | Major | build | Karel Kolman | Karel Kolman | | | Dockerfile for building on Centos 8 | Major | build | Gautham Banasandra | Gautham Banasandra | | | Increase the job name max length in mapred job -list | Major | mapreduce-client | Ayush Saxena | Ayush Saxena | | | Fix typos in CapacityScheduler#schedule. | Minor | capacity scheduler | Qi Zhu | Qi Zhu | | | Improve the readability of diagnostics log in yarn-ui2 web page. | Minor | yarn-ui-v2 | huangkunlun | huangkunlun | | | Allow ProtobufRpcEngine to be extensible | Major | common | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Add the number of containers assigned per second metrics to ClusterMetrics | Minor | metrics | chaosju | chaosju | | | BlockReconstructionWork should resetTargets()

BlockManager#validateReconstructionWork return false | Minor | block placement | Haiyang Hu | Haiyang Hu | | | Add metrics for 'ApplicationsRunning' in NodeManager | Minor | nodemanager | ANANDA G B | ANANDA G B | | | RBF: Improving Router Connection Management | Major | rbf | Fengnan Li | Fengnan Li | | | Optimize the display of hdfs "count -e" or "count -t" command | Minor | dfsclient | Hongbing Wang | Hongbing Wang | | | Remove the old hamlet package | Major | webapp | Akira Ajisaka | Akira Ajisaka | | | Error message around yarn app -stop/start can be improved to highlight that an implementation at framework level is needed for the stop/start functionality to work | Minor | client, documentation | Siddharth Ahuja | Siddharth Ahuja | | | Document the removal of FS default queue creation | Major | fairscheduler | Benjamin Teke | Benjamin Teke | | | Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist | Critical | ipc | David Mollitor | Vinayakumar B | | | RBF: Rename data to the Trash should be based on src locations | Major | rbf | Xiangyi Zhu | Xiangyi Zhu | | | Make mkstemp cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix java doc in FSPermissionChecker | Minor | documentation | Tao Li | Tao Li | | | Modularize docker images | Major | build | Gautham Banasandra | Gautham Banasandra | | | Set Completed AppAttempt LogsLink to Log Server Url | Major | webapp | Prabhu Joseph | Abhinaba Sarkar | | | Increase precommit job timeout from 20 hours to 24 hours. | Major | build | Takanobu Asanuma | Takanobu Asanuma | | | Change Capacity Scheduler minimum-user-limit-percent to accept decimal values | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB | Minor | hdfs-client | lei w | lei w | | | Remove an expensive debug string concatenation | Major | hdfs-client | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add Dockerfile for Debian 10 | Major | build | Gautham Banasandra | Gautham Banasandra | | | HDFS mover to emit metrics | Major | balancer & mover | Leon Gao | Leon Gao | | | RBF: Invoking method in all locations should break the loop after successful result | Minor | rbf | Viraj Jasani | Viraj Jasani | | | Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects | Major | hdfs | Viraj Jasani | Viraj Jasani | | | Use empty array constants present in TaskCompletionEvent to avoid creating redundant objects | Minor | mrv2 | Viraj Jasani | Viraj Jasani | | | Avoid non-atomic operations on exceptionsSinceLastBalance and failedTimesSinceLastSuccessfulBalance in Balancer | Major | balancer | Viraj Jasani | Viraj Jasani | | | CI for Debian 10 | Major | build | Gautham Banasandra | Gautham Banasandra | | | Avoid using slow DataNodes for reading by sorting locations | Major | hdfs | Tao Li | Tao Li | | | Move the getPermissionChecker out of the read lock | Minor | namanode | Tao Li | Tao Li | | | Intra-queue preemption: apps that don't use defined custom resource won't be preempted. | Major | scheduler preemption | Eric Payne | Eric Payne | | | Update clover-maven-plugin version from 3.3.0 to 4.4.1 | Major | build, common | Wanqiang Ji | Wanqiang Ji | | | Delete useless

DirectoryWithQuotaFeature#setQuota | Major | hdfs | Xiangyi Zhu | Xiangyi Zhu | | | Fine grained locking for datanodeNetworkCounts | Major | datanode | Viraj Jasani | Viraj Jasani | | | CI for Centos 8 | Major | build | Gautham Banasandra | Gautham Banasandra | | | Add volume information to datanode log for tracing | Minor | datanode | Tao Li | Tao Li | | | [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler | Major | capacity scheduler | Weiwei Yang | Weiwei Yang | | | Remove unuse variable and IoException in ProvidedStorageMap | Minor | namenode | lei w | lei w | | | Remove lock contention in SelectorPool of SocketIOWithTimeout | Major | common | Xuesen Liang | Xuesen Liang | | | the balancer parameters print error | Minor | balancer | jiaguodong | jiaguodong | | | Remove JavaScript package from Docker environment | Major | build | Masatake Iwasaki | Masatake Iwasaki | | | Standby NameNode process getLiveDatanodeStorageReport request to reduce Active load | Major | namanode | Tao Li | Tao Li | | | Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS | Major | documentation, kms, security | Akira Ajisaka | Akira Ajisaka | | | Fix DistCpContext#toString() | Minor | distcp | Tao Li | Tao Li | | | Document hadoop.security.token.service.use\_ip | Major | documentation | Arpit Agarwal | Akira Ajisaka | | | Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes | Major | datanode | Leon Gao | Leon Gao | | | Remove an invalid comment content in the FileContext class | Major | common | JiangHua Zhu | JiangHua Zhu | | | RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry | Major | resourcemanager | Eric Payne | Eric Payne | | | Make the socket timeout for computing checksum of striped blocks configurable | Minor | datanode, ec, erasure-coding | Yushi Hayasaka | Yushi Hayasaka | | | [UI2] YARN-10826 breaks Queue view | Major | yarn-ui-v2 | Andras Gyori | Masatake Iwasaki | | | Enable RpcMetrics units to be configurable | Major | ipc, metrics | Erik Krogen | Viraj Jasani | | | Make max container per heartbeat configs refreshable | Major | capacity scheduler | Eric Badger | Eric Badger | | | Checkstyle - Allow line length: 100 | Major | common | Akira Ajisaka | Viraj Jasani | | | start balancer with parameters -hotBlockTimeInterval xxx is invalid | Minor | balancer | jiaguodong | jiaguodong | | | Improve the comments related to FairCallQueue#queues | Minor | ipc | JiangHua Zhu | JiangHua Zhu | | | ABFS ExponentialRetryPolicy doesn't pick up configuration values | Minor | documentation, fs/azure | Brian Frank Loss | Brian Frank Loss | | | Add extensions to ProtobufRpcEngine RequestHeaderProto | Major | common | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Solve BlockSender#sendPacket() does not record SocketTimeout exception | Minor | datanode | JiangHua Zhu | JiangHua Zhu | | | Add node usage metrics in SLS | Major | scheduler-load-simulator | VADAGA ANANYO RAO | VADAGA ANANYO RAO | | | Add runningApps stats in SLS | Major | yarn | VADAGA ANANYO RAO | VADAGA ANANYO RAO | | | Prevent ATS v2 health check REST API call if the ATS service itself is disabled. | Major | yarn-ui-v2 | Siddharth Ahuja | Benjamin Teke | | | Run CI for Centos 7 | Critical | build | Gautham Banasandra | Gautham Banasandra | | | Support marking inactive node as untracked without configured include path | Major | resourcemanager | Tao Yang | Tao Yang | | | Improve the parameter annotation in FairCallQueue#priorityLevels | Minor | ipc | JiangHua Zhu | JiangHua Zhu | | | Refactor NM ContainerLaunch#getEnvDependencies's unit tests | Minor | yarn | Tamas Domok | Tamas Domok | | | All three replicas are lost due to not adding a new DataNode in time | Major | dfsclient | Shuyan Zhang | Shuyan Zhang | | | Refactor NM ContainerLaunch.java#orderEnvByDependencies | Minor | yarn | Benjamin Teke | Tamas Domok | | | Clarify testcase documentation for TestServiceAM#testContainersReleasedWhenPreLaunchFails | Minor | test | Szilard Nemeth | Szilard Nemeth | | | Avoid evaluation of

statement in QuorumJournalManager | Trivial | journal-node | Zhaohui Wang | Zhaohui Wang | | | TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely | Minor | journal-node | Zhaohui Wang | Zhaohui Wang | | | Make it easier to debug UnknownHostExceptions from NetUtils.connect | Minor | common | Bryan Beaudreault | Bryan Beaudreault | | | Refactor fetching of credentials in Jenkins | Major | build | Gautham Banasandra | Gautham Banasandra | | | Make mkdtemp cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Avoid locking entire blockPinningFailures map | Major | balancer | Viraj Jasani | Viraj Jasani | | | Add BuiltInGzipCompressor | Major | common | L. C. Hsieh | L. C. Hsieh | | | Improve DFSUtil#checkProtectedDescendants() related parameter comments | Major | documentation | JiangHua Zhu | JiangHua Zhu | | | Improve the parameter annotation in DatanodeProtocol#sendHeartbeat | Minor | datanode | Tao Li | Tao Li | | | FsVolumeImpl.nextBlock should consider that the block meta file has been deleted. | Minor | datanode | Max Xie | Max Xie | | | Improve the configurable value of Server #PURGE\INTERVAL\NANOS | Major | ipc | JiangHua Zhu | JiangHua Zhu | | | Improve CopyCommands#Put#executor queue configurability | Major | fs | JiangHua Zhu | JiangHua Zhu | | | Extend QueueInfo with max-parallel-apps in CapacityScheduler | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | Mark KeyProvider as Stable | Major | security | Akira Ajisaka | Akira Ajisaka | | | Empty the statistical parameters when emptying the redundant queue | Minor | hdfs | zhanghuazong | zhanghuazong | | | Use constants HdfsClientConfigKeys.Failover.PREFIX instead of "dfs.client.failover." | Minor | hdfs-client | Weison Wei | Weison Wei | | | BlockReportProcessingThread exit doesn't print the actual stack | Major | block placement | Renukaprasad C | Renukaprasad C | | | Improve FSDirEncryptionZoneOp related parameter comments | Major | documentation | JiangHua Zhu | JiangHua Zhu | | | Add description for dfs.namenode.caching.enabled | Major | documentation | Tao Li | Tao Li | | | Allow nested blocks in switch case in checkstyle settings | Minor | build | Masatake Iwasaki | Masatake Iwasaki | | | Add documentation for YARN-10623 auto refresh queue conf in CS | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Check real user ACLs in addition to proxied user ACLs | Major | security | Eric Payne | Eric Payne | | | Remove GzipOutputStream | Major | common | L. C. Hsieh | L. C. Hsieh | | | RBF: Add metrics to record Router's operations | Major | rbf | Janus Chow | Janus Chow | | | RBF: Router to support resolving monitored namenodes with DNS | Minor | rbf | Leon Gao | Leon Gao | | | RBF: Add the option of refreshCallQueue to RouterAdmin | Major | rbf | Janus Chow | Janus Chow | | | ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Simplify getting NNStorage in FSNamesystem | Major | namenode | JiangHua Zhu | JiangHua Zhu | | | Support default queue properties of capacity scheduler to simplify configuration management | Major | capacity scheduler | Weihao Zheng | Weihao Zheng | | | RBF: Add usage of refreshCallQueue for Router | Major | rbf | Janus Chow | Janus Chow | | | AvailableSpaceRackFaultTolerantBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better

| Major | block placement | Ayush Saxena | Ayush Saxena | | | Move ClusterStorageCapacityExceededException to Public from LimitedPrivate | Major | common, hdfs-client | Ayush Saxena | Ayush Saxena | | | Solve the problem that the value of SafeModeMonitor#RECHECK\_INTERVAL can be configured | Major | block placement | JiangHua Zhu | JiangHua Zhu | | | Fix typo for FederationTestUtils | Minor | rbf | Tao Li | Tao Li | | | Filter deps with release labels | Blocker | build | Gautham Banasandra | Gautham Banasandra | | | Print RPC response length in the exception message | Minor | ipc | Tao Li | Tao Li | | | Remove the use of obsolete BLOCK\DELETION\INCREMENT | Trivial | documentation, namenode | JiangHua Zhu | JiangHua Zhu | | | Improve PrometheusSink for Namenode TopMetrics | Major | metrics | Max Xie | Max Xie | | | Maven-eclipse-plugin is no longer needed since Eclipse can import Maven projects by itself. | Minor | documentation | Rintaro Ikeda | Rintaro Ikeda | | | Add toString to EditLogFileInputStream | Minor | namanode | David Mollitor | Dionisii Iuzhakov | | | AM Total Queue Limit goes below per-user AM Limit if parent is full. | Major | capacity scheduler, capacityscheduler | Eric Payne | Eric Payne | | | Fix java doc for BlockReaderRemote#newBlockReader | Minor | documentation | Tao Li | Tao Li | | | Support building on Apple Silicon | Major | build, common | Dongjoon Hyun | Dongjoon Hyun | | | Record the BPServiceActor information that communicates with Standby | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | Update xerces to 2.12.1 | Minor | build, common | Zhongwei Zhu | Zhongwei Zhu | | | Modify Text.ensureCapacity() to efficiently max out the backing array size | Major | io | Peter Bacsko | Peter Bacsko | | | Print lockWarningThreshold in InstrumentedLock#logWarning and InstrumentedLock#logWaitWarning | Minor | common | Tao Li | Tao Li | | | Improve comments related to EncryptionZoneManager | Minor | documentation, encryption, namenode | JiangHua Zhu | JiangHua Zhu | | | JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics. | Minor | journal-node | Max Xie | Max Xie | | | RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit | Major | rbf | Haiyang Hu | Haiyang Hu | | | Refactor AllowSnapshotMock using GMock | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Correct docs for dfs.http.client.retry.policy.spec | Major | documentation | Stephen O'Donnell | Stephen O'Donnell | | | Make hdfs\_cat tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Add CMakeLists for hdfs\_allowSnapshot | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Standby close reconstruction thread | Major | namanode | zhanghuazong | zhanghuazong | | | When adding block keys, the records come from the specific Block Pool | Minor | datanode | JiangHua Zhu | JiangHua Zhu | | | Make hdfs\_deleteSnapshot tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Make hdfs\_df tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Improve error msg for FSNamesystem#startFileInt | Minor | namanode | Tao Li | Tao Li | | | The error of Constant annotation in

| Minor | fs/azure | guoxin | guoxin | | | Improve decision in AvailableSpaceBlockPlacementPolicy | Major | block placement | guophilipse | guophilipse | | | WASB : Support disabling buffered reads in positional reads | Major | fs/azure | Anoop Sam John | Anoop Sam John | | | Duplicate generic usage information to hdfs debug command | Minor | tools | daimin | daimin | | | Provide optional means for a scheduler to check real user ACLs | Major | capacity scheduler, scheduler | Eric Payne | Eric Payne | | | Print detail datanode info when process first storage report | Minor | datanode | Tao Li | Tao Li | | | WebHDFS should support getSnapshotDiffReportListing | Major | webhdfs | Masatake Iwasaki | Masatake Iwasaki | | | Make log more standardized when executing verifyAndSetNamespaceInfo() | Minor | datanode | JiangHua Zhu | JiangHua Zhu | | | Debug tool to verify the correctness of erasure coding on file | Minor | erasure-coding, tools | daimin | daimin | | | Add remote port information to HDFS audit log | Major | ipc, namanode | Tao Li | Tao Li | | | Make the comment of INode#ReclaimContext more standardized | Minor | documentation, namenode | JiangHua Zhu | JiangHua Zhu | | | Remove invalid DataNode#CONFIG\PROPERTY\SIMULATED | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | RBF: RouterRpcFairnessPolicyController add denied permits for each nameservice | Major | rbf | Janus Chow | Janus Chow | | | RBF: RouterRpcFairnessPolicyController add availableHandleOnPerNs metrics | Major | rbf | Xiangyi Zhu | Xiangyi Zhu | | | RBF: RouterRpcFairnessPolicyController record requests handled by each nameservice | Major | rbf | Janus Chow | Janus Chow | | | Improve HdfsBlockPlacementPolicies docs readability | Minor | documentation | guophilipse | guophilipse | | | Fix bug for TestDataNodeVolumeMetrics#verifyDataNodeVolumeMetrics | Minor | datanode, test | Tao Li | Tao Li | | | Improve BenchmarkThroughput#SIZE naming standardization | Minor | benchmarks, test | JiangHua Zhu | JiangHua Zhu | | | Record the remote NameNode address when the rolling log is triggered | Major | namenode | JiangHua Zhu | JiangHua Zhu | | | Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable | Major | datanode | Haiyang Hu | Haiyang Hu | | | Revisit allocation and reservation logging | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Fix invalid config in TestAvailableSpaceRackFaultTolerantBPP | Minor | test | guophilipse | guophilipse | | | Add docs on removing node label mapping from a node | Minor | documentation | Manu Zhang | Manu Zhang | | | Add metrics related to Transfer and NativeCopy for DataNode | Major | datanode, metrics | Tao Li | Tao Li | | | RBF: Add client port to CallerContext for Router | Major | rbf | Tao Li | Tao Li | | | Datanode retrieve slownode information from NameNode | Major | datanode | Janus Chow | Janus Chow | | | Allow get command to run with multi threads. | Major | fs | Chengwei Wang | Chengwei Wang | | | Improve DirectoryScanner.Stats#toString | Major | . | Tao Li | Tao Li | | | Allow cp command to run with multi

| Major | fs | Chengwei Wang | Chengwei Wang | | | Update CompressionCodecFactory to handle uppercase file extensions | Minor | common, io, test | Desmond Sisson | Desmond Sisson | | | HttpFS implementation for getSnapshotDiffReportListing | Major | httpfs | Viraj Jasani | Viraj Jasani | | | Remove unnecessary brackets in NameNodeRpcServer#L453 | Trivial | namanode | Zhaohui Wang | Zhaohui Wang | | | Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable | Major | block placement | Haiyang Hu | Haiyang Hu | | | Fix error configuration message in FSImage | Minor | hdfs | guophilipse | guophilipse | | | Add path exception information in FSNamesystem | Minor | hdfs | guophilipse | guophilipse | | | Add description of GETSNAPSHOTDIFFLISTING to WebHDFS doc | Minor | documentation, webhdfs | Masatake Iwasaki | Masatake Iwasaki | | | Fix test cases fail in TestBlockStoragePolicy | Major | build | guophilipse | guophilipse | | | Bump mina-core from 2.0.16 to 2.1.5 in /hadoop-project | Major | build | Ayush Saxena | Ayush Saxena | | | Update to Jetty 9.4.44 | Major | build, common | Yuan Luo | Yuan Luo | | | Use maven.test.failure.ignore instead of ignoreTestFailure | Major | build | Akira Ajisaka | Akira Ajisaka | | | WASB : Make metadata checks case insensitive | Major | fs/azure | Anoop Sam John | Anoop Sam John | | | OpensslCipher initialization error should log a WARN message | Trivial | kms, security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix jetty version in LICENSE-binary | Major | build, common | Yuan Luo | Yuan Luo | | | Make dfs.namenode.max.slowpeer.collect.nodes reconfigurable | Major | namenode | Tao Li | Tao Li | | | Add datanode address to BlockReportLeaseManager logs | Minor | datanode | Tao Li | Tao Li | | | The FBR lease ID should be exposed to the log | Major | datanode | Tao Li | Tao Li | | | Add tests that shows how to delete config values with Mutation API | Minor | capacity scheduler, restapi | Szilard Nemeth | Szilard Nemeth | | | return the real datanode numBlocks in #getDatanodeStorageReport | Major | datanode | qinyuren | qinyuren | | | Typo in method name: RMWebServiceProtocol#removeFromCluserNodeLabels | Trivial | resourcemanager, webservice | Szilard Nemeth | Szilard Nemeth | | | Reduce DataNode load when FsDatasetAsyncDiskService is working | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | Avoid evaluation of LOG.debug statement in NameNodeHeartbeatService | Trivial | rbf | Zhaohui Wang | Zhaohui Wang | | | Improve RM system metrics publisher's performance by pushing events to timeline server in batch | Critical | resourcemanager, timelineserver | Hu Ziqian | Ashutosh Gupta | | | Support Apple Silicon in start-build-env.sh | Major | build | Akira Ajisaka | Akira Ajisaka | | | Mark slownode as badnode to recover pipeline | Major | datanode | Janus Chow | Janus Chow | | | RPCMetrics increases the number of handlers in processing | Major | rpc-server | JiangHua Zhu | JiangHua Zhu | | | Make hdfs\_du tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | DistCp: Filter duplicates in the source paths | Major | tools/distcp | Ayush Saxena | Ayush Saxena | | | Fix typo for CachingGetSpaceUsed | Minor | fs | Tao Li | Tao Li | | | Hadoop - Upgrade to JQuery

| Major | build, common | Yuan Luo | Yuan Luo | | | Add markedDeleteBlockScrubberThread to delete blocks asynchronously | Major | hdfs, namanode | Xiangyi Zhu | Xiangyi Zhu | | | fix nextBlockReportTime when trigger full block report force | Major | datanode | qinyuren | qinyuren | | | Validate maximum blocks in EC group when adding an EC policy | Minor | ec, erasure-coding | daimin | daimin | | | Improve FUSE IO performance by supporting FUSE parameter max\_background | Minor | fuse-dfs | daimin | daimin | | | Async refresh of cached locations in DFSInputStream | Major | hdfs-client | Bryan Beaudreault | Bryan Beaudreault | | | Remove the worthless DatasetVolumeChecker#numAsyncDatasetChecks | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java | Trivial | test | Xing Lin | Xing Lin | | | balancer should not get blocks on stale storages | Major | balancer & mover | qinyuren | qinyuren | | | Show start time of JournalNode on Web | Major | journal-node | Tao Li | Tao Li | | | containerLaunchedOnNode method not need to hold schedulerApptemt lock | Major | scheduler | Ryan Wu | Minni Mittal | | | Make HDFS count, mkdir, rm cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | RBF: Registry HDFS Router's RPCServer & RPCClient metrics for PrometheusSink | Minor | rbf | Max Xie | Max Xie | | | ViewFileSystem: Add Support for Localized Trash Root | Major | common | Xing Lin | Xing Lin | | | RBF: Support router get HAServiceStatus with Lifeline RPC address | Minor | rbf | YulongZ | YulongZ | | | Add an option to preserve root directory permissions | Minor | tools | Mohanad Elsafty | Mohanad Elsafty | | | RBF: register RBFMetrics in MetricsSystem for promethuessink | Minor | rbf | Max Xie | Max Xie | | | Expose JournalNode storage info in the jmx metrics | Major | journal-node, metrics | Viraj Jasani | Viraj Jasani | | | Fix some issues in TestRMWebServicesCapacitySchedDynamicConfig | Minor | resourcemanager, webservice | Szilard Nemeth | Tamas Domok | | | Allow configuration of zookeeper server principal | Major | auth | Owen O'Malley | Owen O'Malley | | | Fix typo: indicies -\> indices | Minor | block placement | Jiale Qi | Jiale Qi | | | outputstream.md typo issue | Major | documentation | leo sun | leo sun | | | Upgrade jQuery version in Yarn UI2 | Major | yarn-ui-v2 | Tamas Domok | Tamas Domok | | | Make HDFS get tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | MutableConfScheduler is referred as plain String instead of class name | Minor | resourcemanager | Szilard Nemeth | Szilard Nemeth | | | Split one FsDatasetImpl lock to volume grain locks. | Major | datanode | Mingxiang Li | Mingxiang Li | | | RBF should prepend the client ip rather than append it. | Major | rbf | Owen O'Malley | Owen O'Malley | | | getTrashRoot/s in ViewFileSystem should return viewFS path, not targetFS path | Major | common | Xing Lin | Xing Lin | | | Removed reuse

AvailableSpaceVolumeChoosingPolicy#initLocks() | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | Make HDFS find tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Add parameter for NameNode to process getBloks request | Minor | balancer & mover, namanode | Max Xie | Max Xie | | | Add space in debug log of ParentQueue | Minor | capacity scheduler | Junfan Zhang | Junfan Zhang | | | Make HDFS ls tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Decouple job parsing logic from SLSRunner | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Eliminate code duplication in SLSCapacityScheduler and SLSFairScheduler | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Follow up changes for YARN-10547 | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Add opname to read/write lock for remaining operations | Major | block placement | Tao Li | Tao Li | | | Decouple AM runner logic from SLSRunner | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Improve code quality in TestRMWebServicesNodeLabels | Minor | test | Szilard Nemeth | Szilard Nemeth | | | Decouple RM runner logic from SLSRunner | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Decouple NM runner logic from SLSRunner | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Introduce the config to control the AM allocated to non-exclusive nodes | Major | capacity scheduler | Junfan Zhang | Junfan Zhang | | | SLS cleanup after previously merged SLS refactor jiras | Minor | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Make HDFS setrep tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Add global timeout rule for TestRouterDistCpProcedure | Minor | test | Tao Li | Tao Li | | | Remove unnecessary setObserverRead in TestConsistentReadsObserver | Trivial | test | Zhaohui Wang | Zhaohui Wang | | | setReplication debug log creates a new string even if debug is disabled | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Support touch command for directory | Major | common | Akira Ajisaka | Viraj Jasani | | | Make fs.getspaceused.classname reconfigurable | Major | namenode | yanbin.zhang | yanbin.zhang | | | Add debug log for BlockManager#chooseExcessRedundancyStriped | Minor | erasure-coding | Tao Li | Tao Li | | | EC: Add param comment for liveBusyBlockIndices with HDFS-14768 | Minor | erasure-coding, namanode | caozhiqiang | caozhiqiang | | | Make HDFS stat tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | fix filesystemshell wrong params | Minor | documentation | guophilipse | guophilipse | | | Make HDFS tail tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Improve NNThroughputBenchmark test mkdirs | Major | benchmarks, namenode | JiangHua Zhu | JiangHua Zhu | | | Building MapReduce NativeTask fails on Fedora 34+ | Major | build, nativetask | Kengo Seki | Kengo Seki | | | Improve the description of dfs.block.scanner.volume.bytes.per.second | Minor | documentation, hdfs | guophilipse | guophilipse | | | Refactor tests in TestFileUtil | Trivial | common | Gautham Banasandra | Gautham Banasandra | | | Replace log4j 1.x with reload4j | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Print the exception when reporting a bad block | Major | datanode | qinyuren | qinyuren | | | Make asynchronous blocks deletion lock and unlock durtion threshold configurable | Major | namanode | Chengwei Wang | Chengwei Wang | | | Upgrade maven-site-plugin to 3.11.0 | Major | build, common | Akira Ajisaka | Ashutosh Gupta | | | Update

| Minor | build, documentation | Steve Loughran | Steve Loughran | | | Add throttler to EC reconstruction | Minor | datanode, ec | daimin | daimin | | | Enhance openFile() for better read performance against object stores | Major | fs, fs/s3, tools/distcp | Steve Loughran | Steve Loughran | | | Remove unused configuration dfs.namenode.block.deletion.increment. | Major | namenode | Chengwei Wang | Chengwei Wang | | | RBF: Support refreshing/changing router fairness policy controller without rebooting router | Minor | rbf | Felix N | Felix N | | | Fix checkstyle for the length of BlockManager construction method over limit. | Major | namenode | Chengwei Wang | Chengwei Wang | | | Migrate Times util from SimpleDateFormat to thread-safe DateTimeFormatter class | Minor | utils, yarn-common | Jonathan Turner Eagles | Jonathan Turner Eagles | | | Upgrade moment.min.js to 2.29.2 | Major | build, common | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Define ssize\_t for Windows | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Improve EC pread: avoid potential reading whole block | Major | dfsclient, ec, erasure-coding | daimin | daimin | | | RMContainerRequestor#makeRemoteRequest has confusing log message | Trivial | mrv2 | Szilard Nemeth | Ashutosh Gupta | | | Add metrics to track delegation token secret manager operations | Major | metrics, security | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | RMWebServices returns only apps matching exactly the submitted queue name | Major | capacity scheduler, webapp | Szilard Nemeth | Szilard Nemeth | | | Support nested mount points in INodeTree | Major | viewfs | Lei Yang | Lei Yang | | | Remove redundant strings.h inclusions | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Support show app id on localizer thread pool | Major | nodemanager | zhoukang | Ashutosh Gupta | | | Record StandbyNameNode information when Balancer is running | Major | balancer & mover, namenode | JiangHua Zhu | JiangHua Zhu | | | Change scope of getRootFallbackLink for InodeTree to make them accessible from outside package | Minor | fs | Xing Lin | Xing Lin | | | Fix getUri() in HttpRequest has been deprecated | Major | common | Shilun Fan | Shilun Fan | | | Upgrade Yetus to 0.14.0 | Major | build | Akira Ajisaka | Ashutosh Gupta | | | Add @VisibleForTesting in Dispatcher.java after HDFS-16268 | Trivial | balancer | Wei-Chiu Chuang | Ashutosh Gupta | | | Fix typo in hadoop-hdfs-rbf module | Minor | rbf | Shilun Fan | Shilun Fan | | | Remove unused Imports in Hadoop YARN project | Minor | yarn | Ashutosh Gupta | Ashutosh Gupta | | | Improve DatanodeHttpServer With Netty recommended method | Minor | datanode | Shilun Fan | Shilun Fan | | | Make fsck read timeout configurable | Major | hdfs-client | Stephen O'Donnell | Stephen O'Donnell | | | Remove unused imports in HDFS project | Minor | hdfs | Ashutosh Gupta | Ashutosh Gupta | | | Remove unused JNStorage#getCurrentDir() | Minor | journal-node, qjm | JiangHua Zhu | JiangHua Zhu | | | Make dirent cross platform compatible | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | |

BPServiceActor#register log to add NameNode address | Minor | hdfs | Shilun Fan | Shilun Fan | | | Fix Flakes Junit Tests that often report timeouts | Major | test | Shilun Fan | Shilun Fan | | | Refactor LogAggregationFileControllerFactory | Minor | log-aggregation | Szilard Nemeth | Szilard Nemeth | | | Refactor TestAggregatedLogDeletionService | Minor | log-aggregation | Szilard Nemeth | Szilard Nemeth | | | Locate protoc-gen-hrpc across platforms | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | EC: Improve performance of decommissioning dn with many ec blocks | Major | ec, erasure-coding, namenode | caozhiqiang | caozhiqiang | | | [JDK 11] Fix javadoc warnings in hadoop-hdfs module | Minor | hdfs | Shilun Fan | Shilun Fan | | | Fix testDelegationToken | Major | test | Chenyu Zheng | Chenyu Zheng | | | Refactor TestAggregatedLogDeletionService: 2nd phase | Minor | log-aggregation | Szilard Nemeth | Szilard Nemeth | | | Remove unused Imports in Hadoop Common project | Minor | common | Ashutosh Gupta | Ashutosh Gupta | | | Total requests and total requests per sec served by RPC servers | Major | rpc-server | Viraj Jasani | Viraj Jasani | | | Add some description for PowerShellFencer | Major | ha | JiangHua Zhu | JiangHua Zhu | | | Remove Unnecessary semicolon ';' | Minor | common | Shilun Fan | Shilun Fan | | | Optimize ClientRMService.getApplications | Major | yarn | Tamas Domok | Tamas Domok | | | Delete unused NameNode#FS\HDFS\IMPL\_KEY | Minor | namenode | JiangHua Zhu | JiangHua Zhu | | | Add isDebugEnabled check for debug blockLogs in BlockManager | Trivial | namenode | dzcxzl | dzcxzl | | | Upgrade dependency-check-maven to 7.1.1 | Minor | security | Ashutosh Gupta | Ashutosh Gupta | | | Implement Linux permission flags on Windows | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Make Job History File Permissions configurable | Major | jobhistoryserver | Prabhu Joseph | Ashutosh Gupta | | | Ensure build folder exists before writing checksum file.ProtocRunner#writeChecksums | Minor | common | Ashutosh Gupta | Ashutosh Gupta | | | tag FSDataInputStream.getWrappedStream() @Public/@Stable | Minor | fs | Steve Loughran | Ashutosh Gupta | | | Enhancing IPC client throughput via multiple connections per user | Minor | ipc | Jason Kace | igo Goiri | | | Remove rs-api dependency by downgrading jackson to 2.12.7 | Major | build | PJ Fanning | PJ Fanning | | | Pass CMake args for Windows in pom.xml | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Create only libhdfspp static libraries for Windows | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | RBF: Show datanode IP list when click DN histogram in Router | Minor | rbf | Zhaohui Wang | Zhaohui Wang | | | Improve Code With Lambda in hadoop-hdfs-rbf module | Minor | rbf | Shilun Fan | Shilun Fan | | | Ensure Protobuf generated headers are included first | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | OIV: print out erasure coding policy name in oiv Delimited output | Minor | erasure-coding | Max Xie | Max Xie | | | Improve Code With Lambda in IPCLoggerChannel class | Minor | journal-node | ZanderXu | ZanderXu | | | Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import. | Major | hdfs | Shilun Fan | Shilun Fan | | | BlockManager should output some logs when logEveryBlock is

| Minor | block placement | ZanderXu | ZanderXu | | | RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout | Major | rbf | ZanderXu | ZanderXu | | | Support auto queue creation template wildcards for arbitrary queue depths | Major | capacity scheduler | Andras Gyori | Bence Kosztolnik | | | Update commons-math3 from 3.1.1 to 3.6.1. | Minor | build, common | Shilun Fan | Shilun Fan | | | Upgrade commons-io to 2.11.0 | Minor | build, common | Ashutosh Gupta | Ashutosh Gupta | | | Fix incorrect placeholder in DataNode.java | Major | datanode | ZanderXu | ZanderXu | | | Refactor AMRMProxy Service code and Added Some Metrics | Major | amrmproxy | Minni Mittal | Shilun Fan | | | [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock | Major | ha, namanode | ZanderXu | ZanderXu | | | Normalize the usage of debug logs in NameNode | Minor | namanode | ZanderXu | ZanderXu | | | Remove redundant cast in FSEditLogOp.class | Major | namanode | ZanderXu | ZanderXu | | | impove JobEndNotifier#httpNotification With recommended methods | Minor | mrv1 | Shilun Fan | Shilun Fan | | | MiniDFSCluster should report cause of exception in assertion error | Minor | hdfs | Steve Vaughan | Steve Vaughan | | | Replace incorrect SafeModeException with StandbyException in RouterRpcServer.class | Major | rbf | ZanderXu | ZanderXu | | | RBF supports disable getNodeUsage() in RBFMetrics | Major | rbf | ZanderXu | ZanderXu | | | Datanode return empty response instead of NPE for GetVolumeInfo during restarting | Major | datanode | ZanderXu | ZanderXu | | | Make FederationStateStoreFacade#getApplicationHomeSubCluster use JCache | Major | federation | chaosju | Shilun Fan | | | RBF: Support healthMonitor timeout configurable and cache NN and client proxy in NamenodeHeartbeatService | Major | rbf | ZanderXu | ZanderXu | | | Updated addresses are still accessed using the old IP address | Major | common | Steve Vaughan | Steve Vaughan | | | Replace NPE with IOException in DataNode.class | Major | datanode | ZanderXu | ZanderXu | | | RouterFsckServlet replicates code from DfsServlet base class | Major | federation | Steve Vaughan | Steve Vaughan | | | hadoop-client-runtime impact by CVE-2022-2047 CVE-2022-2048 due to shaded jetty | Major | build | phoebe chen | Ashutosh Gupta | | | Update commons-net from 3.6 to 3.8.0. | Minor | common | Shilun Fan | Shilun Fan | | | Adds alignment context to call path for creating RPC proxy with multiple connections per user. | Major | ipc | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Add Configuration to delegationToken RemoverScanInterval | Major | resourcemanager | Shilun Fan | Shilun Fan | | | Exclude self from JournalNodeSyncer when using a bind host | Major | journal-node | Steve Vaughan | Steve Vaughan | | | Reduce the number of HeartbeatManager loops | Major | datanode, namanode | Shuyan Zhang | Shuyan Zhang | | | NUMA Awareness support in DefaultContainerExecutor | Major | nodemanager | Prabhu Joseph | Samrat Deb | | | Make shuffle key length configurable | Major | mrv2 | Andrs Gyri | Ashutosh Gupta | | | Remove org.apache.hadoop.maven.plugin.shade.resource.ServicesResourceTransformer | Major | common | PJ Fanning | PJ Fanning | | | Allow dynamic groupSearchFilter in LdapGroupsMapping | Major | security | Ayush Saxena | Ayush Saxena | | | Improve ZKDelegationTokenSecretManager#startThead With recommended

| Minor | common | Shilun Fan | Shilun Fan | | | Ambiguous error message in mutation API | Major | capacity scheduler | Andrs Gyri | Ashutosh Gupta | | | Improve Nodemanager#NodeStatusUpdaterImpl Log | Minor | nodemanager | Shilun Fan | Shilun Fan | | | Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable | Minor | resourcemanager | Shilun Fan | Shilun Fan | | | EC: Allow block reconstruction pending timeout refreshable to increase decommission performance | Major | ec, namenode | caozhiqiang | caozhiqiang | | | [Documentation] RBF: Duplicate statement to be removed for better readabilty | Minor | documentation, rbf | Renukaprasad C | Renukaprasad C | | | GetJournalEditServlet fails to authorize valid Kerberos request | Major | journal-node | Steve Vaughan | Steve Vaughan | | | Upgrade Apache Kerby version to 2.0.x | Major | security | Jiajia Li | Colm O hEigeartaigh | | | Fix KMS Accept Queue Size default value to 500 | Minor | common | guophilipse | guophilipse | | | Avoid stopContainer() on dead node | Major | mrv2 | Ashutosh Gupta | Ashutosh Gupta | | | Add a re-queue metric to RpcMetrics.java to quantify the number of re-queue RPCs | Minor | metrics | ZanderXu | ZanderXu | | | Upgrade jquery ui to 1.13.2 | Major | security | D M Murali Krishna Reddy | Ashutosh Gupta | | | Fix BlockPlacementPolicy details in hdfs defaults | Minor | documentation | guophilipse | guophilipse | | | Update hsqldb.version from 2.3.4 to 2.5.2 | Major | common | Shilun Fan | Shilun Fan | | | LocalDirAllocator to provide diagnostics when file creation fails | Minor | util | Ramesh Kumar Thangarajan | Ashutosh Gupta | | | upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 | Major | common | PJ Fanning | PJ Fanning | | | Erasure Coding: The length of targets should be checked when DN gets a reconstruction task | Major | erasure-coding | Ruinan Gu | Ruinan Gu | | | JN should tersely print logs about NewerTxnIdException | Major | journal-node | ZanderXu | ZanderXu | | | Limit the findbugs suppression IS2\INCONSISTENT\SYNC to S3AFileSystem field | Minor | fs/s3 | Viraj Jasani | Viraj Jasani | | | Refactor NM#FederationInterceptor#recover Code | Major | federation, nodemanager | Shilun Fan | Shilun Fan | | | Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster | Minor | federation | Shilun Fan | Shilun Fan | | | Fix incorrect placeholder in yarn-module | Minor | yarn | Shilun Fan | Shilun Fan | | | container-executor message on empty configuration file can be improved | Trivial | container-executor | Miklos Szegedi | Riya Khandelwal | | | Remove WhiteBox in yarn module. | Minor | test | Shilun Fan | Shilun Fan | | |

MultipleOutputs#close call | Major | mapreduce-client | Prabhu Joseph | Ashutosh Gupta | | | Add XMLUtils methods to centralise code that creates secure XML parsers | Major | common | PJ Fanning | PJ Fanning | | | Remove the hadoop-openstack module | Major | build, fs, fs/swift | Steve Loughran | Steve Loughran | | | upgrade jettison json jar due to fix CVE-2022-40149 | Major | build | PJ Fanning | PJ Fanning | | | Add helper method in FairSchedulerAppsBlock to print app info | Minor | webapp | Daniel Templeton | Riya Khandelwal | | | Improve async delete replica on datanode | Major | datanode | Haiyang Hu | Haiyang Hu | | | Lock File System Creator Semaphore Uninterruptibly | Minor | fs | David Mollitor | David Mollitor | | | Exclude Dockerfile\windows\10 from hadolint | Major | common | Gautham Banasandra | Gautham Banasandra | | | Add Dockerfile for Windows 10 | Critical | build | Gautham Banasandra | Gautham Banasandra | | | Update commons-csv from 1.0 to 1.9.0. | Minor | common | Shilun Fan | Shilun Fan | | | update jackson-databind 2.12.7.1 due to CVE fixes | Major | common | PJ Fanning | PJ Fanning | | | InstrumentedWriteLock should consider Reentrant case | Major | common | ZanderXu | ZanderXu | | | Add GETFILEBLOCKLOCATIONS operation to HttpFS | Major | httpfs | Gao Zhong Liang | Ashutosh Gupta | | | Update Bouncy Castle to 1.68 or later | Major | build | Takanobu Asanuma | PJ Fanning | | | Upgrade commons-text version to fix CVE-2022-42889 | Major | build | Xiaoqiao He | PJ Fanning | | | Refactoring part of the code of SQLFederationStateStore | Major | federation, router | Shilun Fan | Shilun Fan | | | Improve some annotations in hdfs module | Major | documentation, namenode | JiangHua Zhu | JiangHua Zhu | | | Use secure XML parser utils in hdfs classes | Major | hdfs | PJ Fanning | PJ Fanning | | | Upgrade maven-shade-plugin to 3.3.0 | Minor | build | Willi Raschkowski | Willi Raschkowski | | | Update build instructions for Windows using VS2019 | Major | build, documentation | Gautham Banasandra | Gautham Banasandra | | | Use secure XML parser utils in YARN | Major | yarn | PJ Fanning | PJ Fanning | | | Use secure XML parser utils in MapReduce | Major | mrv1, mrv2 | PJ Fanning | PJ Fanning | | | Upgrade DataTables to 1.11.5 to fix CVEs | Major | yarn | Bence Kosztolnik | Bence Kosztolnik | | | Remove useless DataNode lock related configuration | Major | datanode | Haiyang Hu | Haiyang Hu | | | Print options when accessing ClientProtocol#rename2() | Minor | namenode | JiangHua Zhu | JiangHua Zhu | | | Add number of decommissioning/shutdown nodes to YARN cluster metrics. | Major | client, resourcemanager | Chris Nauroth | Chris Nauroth | | | Upgrade to snakeyaml 1.33 | Major | common | PJ Fanning | PJ Fanning | | | upgrade woodstox-core to 5.4.0 for security fix | Major | common | phoebe chen | PJ Fanning | | | Remove unused TimelineVersionWatcher and TimelineVersion from hadoop-yarn-server-tests | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | Docker Container to accept docker Image name with sha256 digest | Major | yarn | Ashutosh Gupta | Ashutosh Gupta | | | Support DecommissionBackoffMonitor parameters reconfigurable | Major | datanode, namanode | Haiyang Hu | Haiyang Hu | | | ABFS: Add fs.azure.enable.readahead option to disable readahead | Major | fs/azure | Steve Loughran | Steve Loughran | | | Hadoop metrics should return 0 when there is no change | Major | metrics | leo sun | leo sun | | | Fix main thread name. | Major | common, ipc | Chenyu Zheng | Chenyu Zheng | | | Code improvements in MutableCSConfigurationProvider | Minor | capacity scheduler | Szilard Nemeth | Peter Szucs | | | Remove WhiteBox in mapreduce module. | Minor | mrv2 | Shilun Fan | Shilun Fan | | | Replace and deprecate mapred.tasktracker.indexcache.mb | Major | mapreduce-client | Sandy Ryza | Ashutosh Gupta | | | upgrade hsqldb to v2.7.1 due to CVE | Major | common | PJ Fanning | Ashutosh Gupta | | |

throws an IllegalArgumentException with large uids after 1.21 | Major | client | Benjamin Teke | Benjamin Teke | | | [RBF] The routers should be resiliant against exceptions from StateStore | Major | rbf | Owen O'Malley | Owen O'Malley | | | Remove parameter validation logic such as dfs.namenode.decommission.blocks.per.interval in DatanodeAdminManager#activate | Major | datanode | Haiyang Hu | Haiyang Hu | | | Enhance the function of DebugAdmin#VerifyECCommand | Major | erasure-coding | Haiyang Hu | Haiyang Hu | | | Enhance the usage description about oiv in HDFSCommands.md and OfflineImageViewerPB | Major | hdfs | Haiyang Hu | Haiyang Hu | | | Add ErasureCodingPolicy information to the response description for GETFILESTATUS in WebHDFS.md | Major | webhdfs | ZanderXu | ZanderXu | | | Fix hadoop-yarn-common module Java Doc Errors | Major | yarn | Shilun Fan | Shilun Fan | | | Fix hadoop-yarn-api module Java Doc Errors | Major | yarn | Shilun Fan | Shilun Fan | | | EC: Only EC blocks should be effected by max-streams-hard-limit configuration | Major | erasure-coding | caozhiqiang | caozhiqiang | | | RBF: Add a utility to dump the StateStore | Major | rbf | Owen O'Malley | Owen O'Malley | | | It should consider EC reconstruction work when we determine if a node is busy | Major | ec, erasure-coding | Ruinan Gu | Ruinan Gu | | | Upgrade moment.min.js to 2.29.4 | Major | build, ui | D M Murali Krishna Reddy | Anurag Parvatikar | | | Fix hadoop-yarn-server-common module Java Doc Errors | Minor | yarn | Shilun Fan | Shilun Fan | | | Improve error reporting on non-standard kerberos names | Blocker | security | Steve Loughran | Steve Loughran | | | CVE-2021-37533 on commons-net is included in hadoop common and hadoop-client-runtime | Blocker | build | phoebe chen | Steve Loughran | | | FileStatus compareTo does not specify ordering | Trivial | documentation | DDillon | DDillon | | | Upgrade kafka to 2.8.2 | Major | build | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Upgrade jquery datatable version references to v1.10.19 | Major | ui | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Fs2cs could be extended to set ULF to -1 upon conversion | Major | yarn | Susheel Gupta | Susheel Gupta | | | EC : Fsck -blockId shows number of redundant internal block replicas for EC Blocks | Major | erasure-coding | Haiyang Hu | Haiyang Hu | | | Duplicate field name in hdfs-default.xml | Minor | documentation | YUBI LEE | YUBI LEE | | | Log start and end of phase/step in startup progress | Minor | namenode | Viraj Jasani | Viraj Jasani | | | Fix Typo of ResourceManager#webapp module | Minor | resourcemanager | Shilun Fan | Shilun Fan | | | Fix "the the" and friends typos | Minor | common | Shilun Fan | Nikita Eshkeev | | | Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel | Major | namenode | Viraj Jasani | Viraj Jasani | | | Standardize the usage of DFSClient debug log | Minor | dfsclient | Hualong Zhang | Hualong Zhang | | | Add compile platform in the hadoop version output | Major | build, common | Ayush Saxena | Ayush Saxena | | | BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier

PendingReconstructionBlocks#timeout should be volatile | Major | block placement | Haiyang Hu | Haiyang Hu | | | Sasl connection failure should log remote address | Major | ipc | Viraj Jasani | Viraj Jasani | | | Remove redundant exception capture in NMClientAsyncImpl and improve readability in ContainerShellWebSocket, etc | Minor | client | jingxiong zhong | jingxiong zhong | | | RBF: Improve StateStoreZookeeperImpl | Major | rbf | Sun Hao | Sun Hao | | | Fix javadoc of Class LightWeightResizableGSet | Trivial | datanode, hdfs | farmmamba | farmmamba | | | Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat | Major | datanode | farmmamba | farmmamba | | | Fix method name of RPC.Builder#setnumReaders | Minor | ipc | Haiyang Hu | Haiyang Hu | | | ShuffleHandler is not working correctly in SSL mode after the Netty 4 upgrade | Major | mrv2 | Tamas Domok | Tamas Domok | | | RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath | Minor | rbf | farmmamba | farmmamba | | | Add LastHeartbeatResponseTime for BP service actor | Major | datanode | Viraj Jasani | Viraj Jasani | | | Server connection should log host name before returning VersionMismatch error | Minor | ipc | Viraj Jasani | Viraj Jasani | | | [Federation] Improve Router Handler FinishApps | Major | federation, router, yarn | Shilun Fan | Shilun Fan | | | Federation: Improve Yarn Router Web Page | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Router Supports SubClusterCleaner | Major | federation | Shilun Fan | Shilun Fan | | | Add some logs for updateBlockForPipeline RPC. | Minor | namanode | farmmamba | farmmamba | | | Enhance WritableName to be able to return aliases for classes that use serializers | Minor | common | Bryan Beaudreault | Bryan Beaudreault | | | Fix Typo of hadoop-yarn-ui README.md | Minor | yarn-ui-v2 | Shilun Fan | Shilun Fan | | | Improve the use of JUnit Test in DFSClient | Minor | dfsclient | Hualong Zhang | Hualong Zhang | | | Upgrade ant to 1.10.13 | Major | common | Aleksandr Nikolaev | Aleksandr Nikolaev | | | Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. | Major | resourcemanager | Shilun Fan | Shilun Fan | | | Implement token storage solution based on MySQL | Major | common | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Refactor MemoryFederationStateStore code. | Major | federation, router | Shilun Fan | Shilun Fan | | | Provide keytab file key name with ServiceStateException | Minor | common | Viraj Jasani | Viraj Jasani | | | Upgrade Netty to 4.1.89.Final | Major | build | Aleksandr Nikolaev | Aleksandr Nikolaev | | | Fix bin/hadoop usage script terminology | Blocker | scripts | Steve Loughran | Steve Loughran | | | RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store | Major | rbf | Viraj Jasani | Viraj Jasani | | | RBF: Mount table store APIs should update cache only if state store record is successfully updated | Major | rbf | Viraj Jasani | Viraj Jasani | | | Add bswap support for LoongArch | Major | native | zhaixiaojuan | zhaixiaojuan | | | Update log

BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer | Major | erasure-coding | Ruinan Gu | Ruinan Gu | | | Improve processing of excess redundancy after failover | Major | block placement | Shuyan Zhang | Shuyan Zhang | | | Improve YARN NodeLabel Memory Display | Major | resourcemanager | Shilun Fan | Shilun Fan | | | AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS | Major | fs/oss | wujinhu | wujinhu | | | RBF: State store cache loading metrics | Major | rbf | Viraj Jasani | Viraj Jasani | | | [Federation] Add missing REST APIs for Router | Major | federation | Bilwa S T | Shilun Fan | | | RBF: File based state stores should allow concurrent access to the records | Major | rbf | Viraj Jasani | Viraj Jasani | | | S3A filesystem to support binding to other URI schemes | Major | common | Harshit Gupta | Harshit Gupta | | | [Federation] MemoryFederationStateStore Support Version. | Major | federation | Shilun Fan | Shilun Fan | | | RBF: MountTableResolver cache size lookup should take read lock | Major | rbf | Viraj Jasani | Viraj Jasani | | | Remove unnecessary dependency on json-smart | Major | auth | Michiel de Jong | Michiel de Jong | | | Support getLinkTarget API in WebHDFS | Minor | webhdfs | Hualong Zhang | Hualong Zhang | | | Add read time metrics for remote reads in Statistics | Minor | hdfs | Melissa You | Melissa You | | | Consider volumes average load of each DataNode when choosing target. | Major | datanode | Shuyan Zhang | Shuyan Zhang | | | Publish SBOM artifacts | Major | build | Dongjoon Hyun | Dongjoon Hyun | | | Improved YarnClient Log Format | Minor | client | Lu Yuan | Lu Yuan | | | [Federation] ZookeeperFederationStateStore Support Version. | Major | federation | Shilun Fan | Shilun Fan | | | Simplify single node instructions for creating directories for Map Reduce | Trivial | documentation | Nikita Eshkeev | Nikita Eshkeev | | | Add a CallerContext getter on the Schedulable interface | Major | common | Christos Bisias | Christos Bisias | | | Node Labels root directory creation doesn't have a retry logic | Major | capacity scheduler | Benjamin Teke | Ashutosh Gupta | | | Add RPC metrics for response time | Minor | metrics | liuguanghua | liuguanghua | | | Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project | Major | common | Ayush Saxena | Ayush Saxena | | | Improve NameServices info at JournalNode web UI | Minor | journal-node, ui | Zhaohui Wang | Zhaohui Wang | | | Support getFileLinkStatus API in WebHDFS | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | [Federation] SQLFederationStateStore Support Version. | Major | federation | Shilun Fan | Shilun Fan | | | S3A to support upload of files greater than 2 GB using DiskBlocks | Major | fs/s3 | Harshit Gupta | Harshit Gupta | | | The yarn queue list is displayed on the CLI | Minor | client | Lu Yuan | Lu Yuan | | | Remove unused parameters

NameNodeHttpServer#initWebHdfs | Minor | webhdfs | Zhaohui Wang | Zhaohui Wang | | | RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus | Minor | rbf | Jiale Qi | Jiale Qi | | | Avoid cross-platform build for irrelevant Dockerfile changes | Major | build | Gautham Banasandra | Gautham Banasandra | | | Fix Typo of hadoop-yarn-common | Minor | yarn | Shilun Fan | Shilun Fan | | | Improvements for TestYarnConfigurationFields and TestConfigurationFieldsBase | Minor | test | Szilard Nemeth | Szilard Nemeth | | | Set the locale to avoid printing useless logs in BlockSender | Major | block placement | Shuyan Zhang | Shuyan Zhang | | | Federation: Add missing FederationClientInterceptor APIs | Major | federation, router | D M Murali Krishna Reddy | Shilun Fan | | | Setup Jenkins nightly CI for Windows 10 | Critical | build | Gautham Banasandra | Gautham Banasandra | | | FederationStateStoreFacade Cache Support Guava Cache | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] MemoryFederationStateStore Support Store ApplicationSubmitData | Major | federation | Shilun Fan | Shilun Fan | | | Update commons-cli from 1.2 to 1.5. | Major | common | Shilun Fan | Shilun Fan | | | [Federation] ZookeeperFederationStateStore Support Store ApplicationSubmitData | Major | federation | Shilun Fan | Shilun Fan | | | HttpFS Add Support getFileLinkStatus API | Major | httpfs | Hualong Zhang | Hualong Zhang | | | [Federation] Router Support Calls SubCluster's RMAdminRequest | Major | federation, router | Shilun Fan | Shilun Fan | | | RBF: Admin command to support bulk add of mount points | Minor | rbf | Viraj Jasani | Viraj Jasani | | | Support getStatus API in WebHDFS | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | Fix typos in hadoop-yarn-server-web-proxy | Minor | webapp | Shilun Fan | Shilun Fan | | | Typos in HDFS Documents | Minor | configuration | Liang Yan | Liang Yan | | | RBF: state store putAll should also return failed records | Major | rbf | Viraj Jasani | Viraj Jasani | | | Remove unused DFSConfigKeys#DFS\DATANODE\PMEM\CACHE\DIRS\_DEFAULT | Minor | datanode, hdfs | JiangHua Zhu | JiangHua Zhu | | | RBF: Add dfsrouter port in hdfsauditlog | Major | rbf | liuguanghua | liuguanghua | | | Improve error messages in ShortCircuitCache | Minor | dfsadmin | ECFuzz | ECFuzz | | | HttpFS Add Support getStatus API | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | Improve TimelineService log format | Minor | timelineservice | Xianming Lei | Xianming Lei | | | Improve ReplicaMap#mergeAll method. | Minor | datanode | farmmamba | farmmamba | | | RBF: mount table addAll should print failed records in std error | Major | rbf | Viraj Jasani | Viraj Jasani | | | Fix javadoc of field IncrementalBlockReportManager#readyToSend. | Major | datanode | farmmamba | farmmamba | | | Add lru cache for RMWebServices.getApps | Minor | resourcemanager | Xianming Lei | Xianming Lei | | | RBF: NamenodeHeartbeatService should update JMX report with configurable frequency | Major | rbf | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | RBF: Reduce repeated code in RouterRpcServer | Minor | rbf | Chengwei Wang | Chengwei Wang | | | Add curator based ZooKeeper communication support over SSL/TLS into the common library | Major | common | Ferenc Erdelyi | Ferenc Erdelyi | | | trigger deletion of log-dir by size for NonAggregatingLogHandler | Minor | nodemanager | Xianming Lei | Xianming Lei | | | RBF: Optimize debug logs of class ConnectionPool and other related

| Minor | rbf | farmmamba | farmmamba | | | Support removal of only selective node states in untracked removal flow | Major | resourcemanager | Mudit Sharma | Mudit Sharma | | | Support getECPolices API in WebHDFS | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | FsVolumeImpl#getActualNonDfsUsed may return negative value | Minor | datanode | farmmamba | farmmamba | | | [Umbrella] Upgrade to netty 4 | Major | build, common | Haohui Mai | Haohui Mai | | | Consider nonDfsUsed when running balancer | Major | balancer & mover | Shuyan Zhang | Shuyan Zhang | | | [Federation] YARN Federation Supports Non-HA mode. | Major | federation, nodemanager | Shilun Fan | Shilun Fan | | | Improve updateTestDataAutomatically in TestRMWebServicesCapacitySched | Major | yarn | Tamas Domok | Tamas Domok | | | Limit wait time for getHAServiceState in ObserverReaderProxy | Minor | hdfs | Xing Lin | Xing Lin | | | Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode | Major | hdfs | Xing Lin | Xing Lin | | | HttpFS implementation for getAllErasureCodingPolicies | Major | httpfs | Hualong Zhang | Hualong Zhang | | | Add .vscode to gitignore | Major | common | Xiaoqiao He | Xiaoqiao He | | | The formatted yarn queue list is displayed on the command line | Minor | yarn | Lu Yuan | Lu Yuan | | | Optimize method BlockInfoStriped#findSlot to reduce time complexity. | Trivial | hdfs | farmmamba | farmmamba | | | Improve TestRMWebServices test config and data | Major | capacityscheduler | Tamas Domok | Bence Kosztolnik | | | Update solr from 8.8.2 to 8.11.2 | Minor | common | Xuesen Liang | Xuesen Liang | | | RBF: Add DataNode maintenance states to Federation UI | Major | rbf | Haiyang Hu | Haiyang Hu | | | Export HAState as a metric from Namenode for monitoring | Minor | hdfs | Xing Lin | Xing Lin | | | Set size of non-exist block to NO\_ACK when process FBR or IBR to avoid useless report from DataNode | Major | namenode | Haiyang Hu | Haiyang Hu | | | Remove ozone from hadoop dev support | Trivial | common | Xiaoqiao He | Xiaoqiao He | | | Fix typos in hadoop-hdfs-project | Minor | hdfs | Zhaohui Wang | Zhaohui Wang | | | Improve hadoop-function.sh#status script | Major | common | Shilun Fan | Shilun Fan | | | Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute | Major | hdfs | Haiyang Hu | Haiyang Hu | | | Remove unused import in DataNodeMetricHelper.java. | Trivial | datanode | farmmamba | farmmamba | | | Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter | Major | balancer, documentation | Haiyang Hu | Haiyang Hu | | | Update fsck to display stale state info of blocks accurately | Minor | datanode, namanode | WangYuanben | WangYuanben | | | Remove the unused method isSlownodeByNameserviceId in DataNode | Major | datanode | Haiyang Hu | Haiyang Hu | | | ipc.server.handler.queue.size missing from core-default.xml | Major | rpc-server | WangYuanben | WangYuanben | | | Add documentation for provisionSnapshotTrash command to HDFSCommands.md and HdfsSnapshots.md | Major | documentation | Haiyang Hu | Haiyang Hu | | | Support getErasureCodeCodecs API in WebHDFS | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | Datanode should record last directory scan time. | Minor | datanode | farmmamba | farmmamba | | | Fix the parameter settings

TestDiskspaceQuotaUpdate#updateCountForQuota. | Major | test | Haiyang Hu | Haiyang Hu | | | Delete path directly when it can not be parsed in trash | Major | common | farmmamba | farmmamba | | | Reconfig disk balancer parameters for datanode | Major | datanode | Haiyang Hu | Haiyang Hu | | | Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks | Major | hdfs | WangYuanben | WangYuanben | | | Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md | Major | dfsadmin | Haiyang Hu | Haiyang Hu | | | Fix typo: form -\> from | Trivial | nodemanager | Seokchan Yoon | Seokchan Yoon | | | Remove incorrect comment in TestRedudantBlocks#setup | Trivial | test | farmmamba | farmmamba | | | Show decommission duration in JMX and HTML | Major | namenode | Shuyan Zhang | Shuyan Zhang | | | RBF: Logger fix for StateStoreMySQLImpl | Trivial | rbf | Zhaohui Wang | Zhaohui Wang | | | HttpFS Add Support getErasureCodeCodecs API | Major | httpfs | Hualong Zhang | Hualong Zhang | | | Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync. | Major | namenode | Haiyang Hu | Haiyang Hu | | | RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService. | Major | rbf | Haiyang Hu | Haiyang Hu | | | Update fsck -blockId to display excess state info of blocks | Major | namnode | Haiyang Hu | Haiyang Hu | | | Fix annotation description and typo in BlockPlacementPolicyDefault Class | Minor | block placement | Zhaobo Huang | Zhaobo Huang | | | FS2CS should use CapacitySchedulerConfiguration in FSQueueConverterBuilder | Major | capacity scheduler | Benjamin Teke | Susheel Gupta | | | Fix minor checkstyle warnings in TestObserverReadProxyProvider | Trivial | hdfs | Xing Lin | Xing Lin | | | Some properties are missing from hadoop-policy.xml. | Major | common, documentation, security | WangYuanben | WangYuanben | | | Document missing a lot of properties in core-default.xml | Major | common, documentation | WangYuanben | WangYuanben | | | Remove incorrect comment in method storeAllocatedBlock | Trivial | namenode | farmmamba | farmmamba | | | Standby/Observer NameNode skip to handle redundant replica block logic when set decrease replication. | Major | namenode | Haiyang Hu | Haiyang Hu | | | Add enQueue time to RpcMetrics | Minor | rpc-server | Liangjun He | Liangjun He | | | Fix description of property dfs.namenode.file.close.num-committed-allowed. | Trivial | documentation | farmmamba | farmmamba | | | RBF: SQLDelegationTokenSecretManager must cleanup expired tokens in SQL | Major | rbf | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Add Throttler for datanode reading block | Major | datanode | Haiyang Hu | Haiyang Hu | | | RBF: Add missing comments in StateStoreService | Minor | rbf | TIsNotT | TIsNotT | | | S3A supports S3 on Outposts | Major | fs/s3 | Sotetsu Suzugamine | Sotetsu Suzugamine | | | Support getTrashRoots API in WebHDFS | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | Add some rpc related metrics to Metrics.md | Major | documentation | Haiyang Hu | Haiyang Hu | | | Revisit the BPOfferService.reportBadBlocks() method. | Minor | datanode | Liangjun He | Liangjun He | | | Fix wrong config in yarn-default.xml | Major | router | Chenyu Zheng | Chenyu Zheng | | | ErasureCodingWork reconstruct ignore the block length is

| Major | erasure-coding | Haiyang Hu | Haiyang Hu | | | RBF: For the doc of the class RouterAdminProtocolTranslatorPB, it describes the function of the class ClientNamenodeProtocolTranslatorPB | Minor | rbf | Jian Zhang | Jian Zhang | | | BootstrapStandby needs to handle RollingUpgrade | Minor | namenode | Danny Becker | Danny Becker | | | Revert HADOOP-18649 | Major | mrv2 | Chenyu Zheng | Chenyu Zheng | | | HttpFS Add Support getTrashRoots API | Major | webhdfs | Hualong Zhang | Hualong Zhang | | | Add bock info when constructing remote block reader meets IOException | Trivial | hdfs-client | farmmamba | farmmamba | | | Support Concurrent Writes With S3A Magic Committer | Major | fs/s3 | Emanuel Velzi | Syed Shameerur Rahman | | | Improve BlockReceiver to throws DiskOutOfSpaceException when initialize | Major | datanode | Haiyang Hu | Haiyang Hu | | | Aggregate container launch debug artifacts automatically in case of error | Minor | yarn | Bence Kosztolnik | Bence Kosztolnik | | | Show file replication when listing corrupt files. | Major | fs, namanode | Shuyan Zhang | Shuyan Zhang | | | EC: Reduce unnecessary log when processing excess redundancy. | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | Zookeeper SSL/TLS support | Critical | resourcemanager | Ferenc Erdelyi | Ferenc Erdelyi | | | Fix comments in the RemoteParam class | Minor | rbf | xiaojunxiang | xiaojunxiang | | | Enhance the log message for striped block recovery | Major | logging | Haiyang Hu | Haiyang Hu | | | upgrade to commons-io 2.14.0 | Major | build | PJ Fanning | PJ Fanning | | | HdfsServerConstants.MIN\BLOCKS\FOR\_WRITE should be configurable | Major | hdfs | Haiyang Hu | Haiyang Hu | | | Add some datanode related metrics to Metrics.md | Minor | datanode, metrics | Zhaobo Huang | Zhaobo Huang | | | CONGESTION\_RATIO should be configurable | Minor | datanode | farmmamba | farmmamba | | | Mini cluster test flakiness | Major | hdfs, test | Steve Vaughan | Steve Vaughan | | | Add the metrics PendingAsyncDiskOperations in datanode | Major | datanode | Haiyang Hu | Haiyang Hu | | | Add lifeline RPC start up log when NameNode#startCommonServices | Major | namenode | Haiyang Hu | Haiyang Hu | | | remove okhttp usage | Major | build, common | PJ Fanning | PJ Fanning | | | Add documentation related to NodeFencer | Minor | documentation, ha | JiangHua Zhu | JiangHua Zhu | | | Improve Node Link for YARN Federation Web Page | Major | federation | Shilun Fan | Shilun Fan | | | Refactor FederationStateStoreFacade Cache Code | Major | federation | Shilun Fan | Shilun Fan | | | module-info classes from external dependencies appearing in uber jars | Major | build | PJ Fanning | PJ Fanning | | | Optimize AvailableSpaceBlockPlacementPolicy | Minor | hdfs | Fei Guo | guophilipse | | | Improve documentation related to BlockManager | Minor | block placement, documentation | JiangHua Zhu | JiangHua Zhu | | | Upgrade ZooKeeper to 3.6.4 | Minor | build | Masatake Iwasaki | Masatake Iwasaki | | | Upgrade ZooKeeper to

| Major | common | Masatake Iwasaki | Masatake Iwasaki | | | Fix javadoc errors in BlockManager | Major | documentation | Haiyang Hu | Haiyang Hu | | | Zookeeper SSL/TLS support in HDFS ZKFC | Major | common | Zita Dombi | Zita Dombi | | | Optimize the configuration and use of callqueue overflow trigger failover | Major | common | Haiyang Hu | Haiyang Hu | | | upgrade maven dependency plugin due to security issue | Major | build | PJ Fanning | PJ Fanning | | | RPC Metrics : Optimize logic for log slow RPCs | Major | metrics | Haiyang Hu | Haiyang Hu | | | upgrade netty to 4.1.100 due to CVE | Major | build | PJ Fanning | PJ Fanning | | | CacheReplicationMonitor hold lock for long time and lead to NN out of service | Major | caching, namenode | Xiaoqiao He | Xiaoqiao He | | | Add timeout to GPGUtils#invokeRMWebService. | Major | federation | Shilun Fan | Shilun Fan | | | Upgrade to jetty 9.4.53 | Major | build | PJ Fanning | PJ Fanning | | | Limit number of spill files getting created | Critical | mrv2 | Mudit Sharma | Mudit Sharma | | | Fix typos in .gitignore | Minor | common | | | | | Add the parameter storage type for getBlocks method | Major | balancer | Haiyang Hu | Haiyang Hu | | | Add getEnclosingRoot() API to filesystem interface and implementations | Major | fs | Tom McCormick | Tom McCormick | | | Filter NaN values from JMX json interface | Major | common | Bence Kosztolnik | Bence Kosztolnik | | | Update plugin for SBOM generation to 2.7.10 | Major | common | Vinod Anandan | Vinod Anandan | | | Support FSNamesystemLock Parameters reconfigurable | Major | namanode | Haiyang Hu | Haiyang Hu | | | Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider | Major | common | Zita Dombi | Istvn Fajth | | | Use StandardCharsets.UTF\_8 constant | Major | common | PJ Fanning | PJ Fanning | | | Remove json-io to 4.14.1 due to CVE-2023-34610 | Major | yarn | Benjamin Teke | Benjamin Teke | | | RBF: Fix client ls trash path cannot get except default nameservices trash path | Major | rbf | liuguanghua | liuguanghua | | | Upgrade grpc jars to v1.53.0 due to CVEs | Major | build | PJ Fanning | PJ Fanning | | | Fix typo in TestFsDatasetImpl Class. | Trivial | test | Zhaobo Huang | Zhaobo Huang | | | NameNode should process time out excess redundancy blocks | Major | namanode | Haiyang Hu | Haiyang Hu | | | EditLogTailer#triggerActiveLogRoll should handle thread Interrupted | Major | hdfs | Haiyang Hu | Haiyang Hu | | | Stabilize TestNMClient | Major | yarn | Bence Kosztolnik | Susheel Gupta | | | Fix doc about loading native libraries | Major | documentation | Shuyan Zhang | Shuyan Zhang | | | [Federation] Router Supports CLI Commands | Major | federation | Shilun Fan | Shilun Fan | | | S3A: add option "fs.s3a.optimized.copy.from.local.enabled" to enable/disable CopyFromLocalOperation | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Use thread pool to improve the speed of creating control files in TestDFSIO | Major | benchmarks, common | farmmamba | farmmamba | | | RBF: Fix link to Fedbalance document | Major | rbf | Haiyang Hu | Haiyang Hu | | | NNThroughputBenchmark should support specifying the base directory for multi-client test | Major | namenode | caozhiqiang | caozhiqiang | | | Fix the documentation of count command in FileSystemShell.md | Trivial | documentation | farmmamba | farmmamba | | | Make congestion backoff time configurable | Minor | hdfs-client | farmmamba | farmmamba | | | Reconfig 'SlowIoWarningThreshold' parameters for

| Minor | datanode | Zhaobo Huang | Zhaobo Huang | | | Passing admin Java options to container localizers | Major | yarn | Peter Szucs | Peter Szucs | | | Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment | Trivial | capacityscheduler | wangzhongwei | wangzhongwei | | | Upgrade ZooKeeper to version 3.8.3 | Major | common | Tamas Penzes | Bilwa S T | | | Speed-up TestTimelineClient | Minor | yarn | Bence Kosztolnik | Bence Kosztolnik | | | RBF: Add a safe mode check period configuration | Minor | rbf | liuguanghua | liuguanghua | | | RBF: Fix some method annotations about @throws | Minor | rbf | xiaojunxiang | xiaojunxiang | | | Judge whether the block has been deleted in the block report | Minor | hdfs | lei w | lei w | | | The NameNode should remove block from the BlocksMap if the block is marked as deleted. | Major | namanode | Haiyang Hu | Haiyang Hu | | | Delete invalid code logic in namenode format | Minor | namenode | zhangzhanchang | zhangzhanchang | | | Upgrade Bouncy Castle to 1.70 | Major | build | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | DiskBalancer: Enhance the log message for submitPlan | Major | datanode | Haiyang Hu | Haiyang Hu | | | RBF: Record proxy time when call invokeConcurrent method. | Minor | rbf | farmmamba | farmmamba | | | Add metrics for ContainerMonitorImpl. | Minor | nodemanager | Xianming Lei | Xianming Lei | | | RBF:Router should not return nameservices that does not enable observer nodes in RpcResponseHeaderProto | Major | rdf, router | liuguanghua | liuguanghua | | | RetryCache#MAX\CAPACITY seems to be MIN\CAPACITY | Trivial | ipc | farmmamba | farmmamba | | | Doc: Fix the documentation of fs expunge command in FileSystemShell.md | Minor | documentation, fs | liuguanghua | liuguanghua | | | Optimize the namenode format code logic. | Major | namenode | Zhaobo Huang | Zhaobo Huang | | | Fix Flaky Test TestTimelineAuthFilterForV2#testPutTimelineEntities | Major | timelineservice | Shilun Fan | Shilun Fan | | | DebugAdmin metaOut not need multiple close | Major | hdfs | xy | xy | | | packetsReceived metric should ignore heartbeat packet | Major | datanode | farmmamba | farmmamba | | | RBF: SQLDelegationTokenSecretManager should use version of tokens updated by other routers | Major | rbf | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Fix Download Maven Url Not Found | Major | common | Shilun Fan | Shilun Fan | | | Change add-opens flag's default value from true to false | Major | mrv2 | Benjamin Teke | Benjamin Teke | | | upgrade to commons-compress 1.24.0 due to CVE | Major | build | PJ Fanning | PJ Fanning | | | mvn site commands fails due to MetricsSystem And MetricsSystemImpl changes. | Major | build | Shilun Fan | Shilun Fan | | | Enhance access control for RunJar | Major | security | Xiaoqiao He | Xiaoqiao He | | | Improve create-release RUN script | Major | build | Shilun Fan | Shilun Fan | | | Revert HDFS-16016. BPServiceActor to provide new thread to handle IBR | Major | namenode | Shilun Fan | Shilun Fan | | | Hadoop 3.4.0 Highlight big features and

| Major | common | Shilun Fan | Shilun Fan | | | [Umbrella] New capacity modes for CS | Major | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | Hadoop 3.4.0 Big feature/improvement highlight addendum | Major | common | Benjamin Teke | Benjamin Teke | | | [Umbrella] Queue Creation in Capacity Scheduler - Tech debts | Major | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | EC: recheck failed streamers should only after flushing all packets. | Minor | ec | farmmamba | farmmamba | | | Corrections to Hadoop FileSystem API Definition | Minor | documentation | Dieter De Paepe | Dieter De Paepe | | | S3A: Add option fs.s3a.classloader.isolation (#6301) | Minor | fs/s3 | Antonio Murgia | Antonio Murgia | | | S3A: update AWS SDK to 2.23.19 to support S3 Access Grants | Minor | build, fs/s3 | Jason Han | Jason Han | | | S3A: make fs.s3a.create.performance an option you can set for the entire bucket | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Update Protocol Buffers installation to 3.21.12 | Major | build | Zhaobo Huang | Zhaobo Huang | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: RouterRpcServer getListing cannot list large dirs correctly | Critical | rbf | Fengnan Li | Fengnan Li | | | HttpFS: setWorkingDirectory should not accept invalid paths | Major | httpfs | Hemanth Boyina | Hemanth Boyina | | | Fix typo in DataXceiverServer#run() | Trivial | datanode | Lisheng Sun | Lisheng Sun | | | ThrottledAsyncChecker is not thread-safe. | Major | federation | Toshihiro Suzuki | Toshihiro Suzuki | | | Fix the logic of scope and excluded scope in Network Topology | Major | net | Ayush Saxena | Ayush Saxena | | | CLOSE\_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI | Major | yarn | Siddharth Ahuja | Siddharth Ahuja | | | NPE in Capacity Scheduler while using %primary\_group queue mapping | Critical | capacity scheduler | Peter Bacsko | Peter Bacsko | | | NameNode should check the authorization API version only once during initialization | Blocker | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Making \`getBoolean\` log warning message for unrecognized value | Major | conf | Ctest | Ctest | | | TestSequenceFile#testRecursiveSeqFileCreate fails in subsequent run | Minor | common, test | Ctest | Ctest | | | TaskAttemptListenerImpl excessive log messages | Major | test | Ahmed Hussein | Ahmed Hussein | | | NPE when hadoop.security.authorization is enabled but the input PolicyProvider for ZKFCRpcServer is NULL | Critical | common, ha | Ctest | Ctest | | | YARN service placement constraints is broken | Blocker | yarn | Eric Yang | Eric Yang | | | [YARN UI2] No Logs were found in "YARN Daemon Logs" page | Blocker | yarn-ui-v2 | Akhil PB | Akhil PB | | | JHS: make sure that Kerberos relogin is performed when KDC becomes offline then online again | Major | jobhistoryserver | Peter Bacsko | Peter Bacsko | | | Add missing DFSOps Statistics in WebHDFS | Major | webhdfs | Ayush Saxena | Ayush Saxena | | | RBF: MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. | Major | rbf | Surendra Singh Lilhore | Surendra Singh Lilhore | |

TestFileContextResolveAfs#testFileContextResolveAfs creates dangling link and fails for subsequent runs | Minor | common, fs, test | Ctest | Ctest | | | HttpFS: Response of Create was not correct with noredirect and data are true | Major | httpfs | Hemanth Boyina | Hemanth Boyina | | | Concat on INodeRefernce fails with illegal state exception | Critical | namanode | Hemanth Boyina | Hemanth Boyina | | | ZKFC ignores dfs.namenode.rpc-bind-host and uses dfs.namenode.rpc-address to bind to host address | Major | ha, namenode | Dhiraj Hegde | Dhiraj Hegde | | | TestNNHandlesBlockReportPerStorage::blockReport\_02 fails intermittently in trunk | Major | datanode, test | Mingliang Liu | Ayush Saxena | | | Fix the findbugs warnings introduced in HDFS-15217 | Major | namanode | Toshihiro Suzuki | Toshihiro Suzuki | | | statfs function in hdfs-fuse is not working | Major | fuse-dfs, libhdfs | Aryan Gupta | Aryan Gupta | | | EC : File write hanged when DN is shutdown by admin command. | Major | ec | Surendra Singh Lilhore | Surendra Singh Lilhore | | | The same distance and load nodes don't shuffle when consider DataNode load | Major | datanode | Lisheng Sun | Lisheng Sun | | | HttpFS: validate content-type in HttpFSUtils | Major | httpfs | Hemanth Boyina | Hemanth Boyina | | | Remove redundant String.valueOf method on ExtendedBlockId.java | Trivial | hdfs-client | bianqi | bianqi | | | NodeBase.normalize doesn't removing all trailing slashes. | Major | net | Ayush Saxena | Ayush Saxena | | | Tolerate leading and trailing spaces in fs.defaultFS | Major | common | Ctest | Ctest | | | StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter | Major | webhdfs | Akira Ajisaka | Akira Ajisaka | | | TestRefreshCallQueue is failing due to changed CallQueue constructor | Major | test | Konstantin Shvachko | Fengnan Li | | | Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic | Major | test | Ahmed Hussein | Ahmed Hussein | | | Account for \*env == NULL in hdfsThreadDestructor | Major | libhdfs | Babneet Singh | Babneet Singh | | | Remove invalid exclusions that minicluster dependency on HDFS | Major | build | Wanqiang Ji | Wanqiang Ji | | | TestContainerResizing fails randomly | Minor | test | Bibin Chundatt | Ahmed Hussein | | | Quota Space consumed was wrong in truncate with Snapshots | Major | qouta, snapshots | Hemanth Boyina | Hemanth Boyina | | | PlacementRule order is not maintained in CS | Major | capacity scheduler | Bibin Chundatt | Bilwa S T | | | StandbyNode fails transition to active due to insufficient transaction tailing | Major | namenode, qjm | Konstantin Shvachko | Konstantin Shvachko | | | Fix invalid metastore configuration in S3GuardTool tests | Minor | fs/s3, test | Masatake Iwasaki | Masatake Iwasaki | | | TestHDFSCLI fails for user names with the dot/dash character | Major | test | Yan Xiaole | Yan Xiaole | | | Setting

to true can crash the system because of unhandled UnresolvedAddressException | Major | hdfs-client | Ctest | Ctest | | | SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data | Major | io, test | zhao bo | Akira Ajisaka | | | Add an option to prevent sub-directories of protected directories from deletion | Major | namenode | liuyanyu | liuyanyu | | | EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads | Major | ec | Guo Lei | Guo Lei | | | Too many InvalidStateTransitionException with SLS | Major | scheduler-load-simulator | Bibin Chundatt | Bilwa S T | | | Trivial typo(s) which are 'timout', 'interruped' in comment, LOG and documents | Trivial | documentation | Sungpeo Kook | Sungpeo Kook | | | RBF: updateActiveNamenode() is invalid when RPC address is IP | Major | rbf | ZanderXu | ZanderXu | | | BytesWritable causes OOME when array size reaches Integer.MAX\_VALUE | Major | io | Joseph Smith | Joseph Smith | | | CS Dynamic Queues cannot be configured with absolute resources | Major | capacity scheduler | Sunil G | Manikandan R | | | Deletion failure should not remove directory from snapshottables | Major | namanode | Hemanth Boyina | Hemanth Boyina | | | Dependency netty-all-4.1.27.Final doesn't support ARM platform | Major | buid | liusheng | liusheng | | | Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue | Major | buid | liusheng | liusheng | | | YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource | Minor | api | Gergely Pollk | Gergely Pollk | | | Relax the condition for accepting a fsimage when receiving a checkpoint | Critical | namenode | Chen Liang | Chen Liang | | | ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). | Major | fs, viewfs | Uma Maheswara Rao G | Abhishek Das | | | Job fails with InvalidStateTransitonException: Invalid event: JOB\TASK\COMPLETED at SUCCEEDED/COMMITTING | Major | mrv2 | Varun Saxena | Bilwa S T | | | ITestS3GuardFsck, others fails when run using a local metastore | Major | fs/s3 | Siddharth Seth | Masatake Iwasaki | | | Very large files can be truncated when written through S3AFileSystem | Major | fs/s3 | Andrew Olson | Mukund Thakur | | | Yarn Service fails if am java opts contains ZK authentication file path | Major | yarn | Bilwa S T | Bilwa S T | | | javax.activation-api and jakarta.activation-api define overlapping classes | Major | build | Akira Ajisaka | Akira Ajisaka | | | Fix intermittent failure of ITestBlockingThreadPoolExecutorService | Minor | fs/s3, test | Masatake Iwasaki | Masatake Iwasaki | | | BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup | Major | block placement | Hemanth Boyina | Hemanth Boyina | | | RENAME.TO\_TRASH is ignored When RENAME.OVERWRITE is specified | Major | hdfs-client | Harshakiran Reddy | Ayush Saxena | | | Fix DataNode's xceiver count calculation | Major | datanode, hdfs | Lukas Majercak | Lisheng Sun | | | Fix number of threads in IPCLoggerChannel#createParallelExecutor | Major | journal-node | Ayush Saxena | Ayush Saxena | | | FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks | Major | snapshots | Hemanth Boyina | Hemanth Boyina | | | Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false | Major | task | Tarun Parimi | Tarun Parimi | | | Wrong description of copyFromLocal and copyToLocal in documentation | Minor | documentation | Jingguo Yao | Andras Bokor | | |

throws unchecked exception (UnresolvedAddressException) causing clients to abort | Major | net | Dhiraj Hegde | Dhiraj Hegde | | | CapacityScheduler incorrect User Group Mapping after leaf queue change | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | Fix shelldocs path in Jenkinsfile | Major | build | Akira Ajisaka | Akira Ajisaka | | | shelldoc fails in hadoop-common | Major | build | Akira Ajisaka | Akira Ajisaka | | | PendingContainers bugs in the scheduler outputs | Critical | resourcemanager | Adam Antal | Andras Gyori | | | Fix TestViewFileSystemOverloadSchemeHdfsFileSystemContract#testListStatusRootDir | Major | test | Ayush Saxena | Ayush Saxena | | | ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories | Major | datanode | Toshihiro Suzuki | Toshihiro Suzuki | | | EC: hdfs client hangs due to exception during addBlock | Critical | ec, hdfs-client | Hongbing Wang | Hongbing Wang | | | appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat | Major | am | Eric Badger | Eric Badger | | | ArrayIndexOfboundsException in ViewFileSystem#listStatus | Major | viewfs | Hemanth Boyina | Hemanth Boyina | | | Make ContainerPBImpl#getId/setId synchronized | Minor | yarn-common | Benjamin Teke | Benjamin Teke | | | listStatus and getFileStatus behave inconsistent in the case of ViewFs implementation for isDirectory | Major | viewfs | Srinivasu Majeti | Uma Maheswara Rao G | | | Add support for yarn logs -logFile to retain backward compatibility | Major | client | Jim Brennan | Jim Brennan | | | Blocks scheduled count was wrong on truncate | Major | block placement | Hemanth Boyina | Hemanth Boyina | | | NPE in FileIoProvider#transferToSocketFully | Major | datanode | Hemanth Boyina | Hemanth Boyina | | | Files in snapshots no longer see attribute provider permissions | Major | snapshots | Stephen O'Donnell | Stephen O'Donnell | | | dfs -chown does not like "+" plus sign in user name | Minor | fs | Marc Villacorta | Andras Bokor | | | Update javadoc and variable names for keytab in yarn services as it supports filesystems other than hdfs and local file system | Minor | documentation | Bilwa S T | Bilwa S T | | | Fix NoClassDefFoundError on 'mapred minicluster' | Major | scripts | Masatake Iwasaki | Masatake Iwasaki | | | ViewFS does not return correct user/group and ACL | Major | fs, viewfs | Abhishek Das | Abhishek Das | | | Document block placement policies | Major | documentation | igo Goiri | Amithsha | | | client fails forever when namenode ipaddr changed | Major | hdfs-client | Sean Chow | Sean Chow | | | TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk | Major | test | Hemanth Boyina | Hemanth Boyina | | | Too many ZK Curator NodeExists exception logs in YARN Service AM logs | Major | yarn | Bilwa S T | Bilwa S T | | | Support reservations continue looking for Node Labels | Major | capacity scheduler | Tarun Parimi | Jim Brennan | | | Upgrade node.js to 10.21.0 | Critical | build, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | Handle an internal dir in viewfs having multiple children mount points pointing to different filesystems | Major | fs, viewfs | Abhishek Das | Abhishek Das | | | ApplicationHistory Web UI incorrect column indexing | Minor | yarn | Andras Gyori | Andras Gyori | | | Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule | Major | capacity scheduler, capacityscheduler, test | Peter Bacsko | Peter Bacsko | | | CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with error java.io.FileNotFoundException: Directory does not exist:

| Major | hdfs | Srinivasu Majeti | Stephen O'Donnell | | | Restarting name node stuck in safe mode when using provided storage | Major | namenode | shanyu zhao | shanyu zhao | | | Fix typos in hadoop-aws documentation | Trivial | documentation, fs/s3 | Sebastian Nagel | Sebastian Nagel | | | Sync netty versions in hadoop-yarn-csi | Major | build | Akira Ajisaka | Akira Ajisaka | | | Yarn Service Container Completed event doesn't get processed | Critical | service-scheduler | Bilwa S T | Bilwa S T | | | Skip Retry INFO logging on first failover from a proxy | Major | ha | Hanisha Koneru | Hanisha Koneru | | | WASB : NativeAzureFsOutputStream#close() throwing IllegalArgumentException | Major | fs/azure | Anoop Sam John | Anoop Sam John | | | Allow RM to always cancel tokens after app completes | Major | yarn | Jim Brennan | Jim Brennan | | | TestCombineFileInputFormat#testMissingBlocks fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | TestUserGroupMappingPlacementRule fails | Major | test | Akira Ajisaka | Bilwa S T | | | Junit class missing from hadoop-mapreduce-client-jobclient-\*-tests jar | Major | test | Eric Badger | Masatake Iwasaki | | | LeaseManager can loop forever on the file for which create has failed | Major | namenode | Sergey Shelukhin | Stephen O'Donnell | | | Timeline Client in Nodemanager gets 403 errors when simple auth is used in kerberos environments | Major | timelineclient | Tarun Parimi | Tarun Parimi | | | RBF: Add test for MountTableRefresherService failed to refresh other router MountTableEntries in secure mode | Major | rbf | Chenyu Zheng | Chenyu Zheng | | | Jetty upgrade to 9.4.x causes MR app fail with IOException | Major | common | Bilwa S T | Bilwa S T | | | ArrayIndexOfboundsException in BlockManager CreateLocatedBlock | Major | namanode | Hemanth Boyina | Hemanth Boyina | | | Fix spotbugs warnings surfaced after upgrade to 4.0.6 | Minor | build | Masatake Iwasaki | Masatake Iwasaki | | | Some containers can be skipped during log aggregation after NM restart | Major | nodemanager | Jason Darrell Lowe | Jim Brennan | | | Add boost installation steps to build instruction on CentOS 8 | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Failed to get nodejs 10.21.0 when building docker image | Blocker | build, webapp | Akira Ajisaka | Akira Ajisaka | | | Javadoc for TimelineReaderAuthenticationFilterInitializer is broken | Minor | documentation | Xieming Li | Xieming Li | | | Yarn rmadmin help message shows two labels for one node for --replaceLabelsOnNode | Major | yarn | Tanu Ajmera | Tanu Ajmera | | | Fix typo in MultipleOutputFormat | Trivial | mrv1 | ywheel | ywheel | | | Ensure inodes in active filesystem are not deleted during snapshot delete | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | missing libhdfspp libs in dist-package | Major | build, libhdfs++ | Yuan Zhou | Yuan Zhou | | | Log container report only if list is not empty | Minor | nodemanager | Bilwa S T | Bilwa S T | | | Truncate info should be logged at INFO level | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | File and directory permissions are not able to be modified from WebUI | Major | ui | Hemanth Boyina | Hemanth Boyina | | |

is inconsistent with the source code | Minor | documentation | zhaoshengjie | zhaoshengjie | | | Clicking on queue in Capacity Scheduler legacy ui does not show any applications | Major | ui | Tarun Parimi | Tarun Parimi | | | Add --mvn-custom-repos parameter to yetus calls | Major | build | Mingliang Liu | Mingliang Liu | | | Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion | Major | httpfs | Mingliang Liu | Mingliang Liu | | | Fixing javadoc in ListingOperationCallbacks | Major | build, documentation | Akira Ajisaka | Mukund Thakur | | | UGI loginUserFromKeytab doesn't set the last login time | Major | security | Sandeep Guggilam | Sandeep Guggilam | | | ApplicationResourceUsageReport should return ALL reserved resource | Major | scheduler | Bibin Chundatt | Bibin Chundatt | | | RMNode updatedCapability flag not set while RecommissionNodeTransition | Major | resourcemanager | Pranjal Protim Borah | Pranjal Protim Borah | | | Dead links in breadcrumbs | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. | Major | datanode | AMC-team | AMC-team | | | Absolute Resource [memory=0] is considered as Percentage config type | Major | capacity scheduler | Prabhu Joseph | Bilwa S T | | | [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | [JDK 11] Fix javadoc errors in hadoop-hdfs module | Major | documentation | Akira Ajisaka | Xieming Li | | | [JDK 11] Fix javadoc errors in hadoop-hdfs-client module | Major | documentation | Akira Ajisaka | Xieming Li | | | Fix C/C++ standard warnings | Major | build | Gautham Banasandra | Gautham Banasandra | | | Fix the findbugs warnings from HDFS-15520 | Major | namenode | Tsz-wo Sze | Tsz-wo Sze | | | Fix typo in Hadoop KMS document | Trivial | documentation, kms | Akira Ajisaka | Xieming Li | | | RM page should throw exception when command injected in RM REST API to get applications | Major | webapp | Rajshree Mishra | Bilwa S T | | | Setting dfs.mover.retry.max.attempts to negative value will retry forever. | Major | balancer & mover | AMC-team | AMC-team | | | --module-gpu functionality is broken in container-executor | Major | nodemanager | Eric Badger | Eric Badger | | | RBF: Fix Namespace path to snapshot path resolution for snapshot API | Major | rbf | Ayush Saxena | Ayush Saxena | | | Rename with Snapshots does not honor quota limit | Major | snapshots | Shashikant Banerjee | Hemanth Boyina | | | Erasure Coding: Native library memory leak | Major | native | Sean Chow | Sean Chow | | | Upgrade slf4j to

( To Address: CVE-2018-8088) | Major | build, common | Brahma Reddy Battula | Brahma Reddy Battula | | | Removing from LowRedundancyBlocks does not remove the block from all queues | Major | namenode | Hui Fei | Hui Fei | | | RBF: Clear Quota in Router was not consistent | Critical | rbf | Hemanth Boyina | Hemanth Boyina | | | RBF: Quota and Content Summary was not correct in Multiple Destinations | Critical | rbf | Hemanth Boyina | Hemanth Boyina | | | Directories protected from delete can still be moved to the trash | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | TestHDFSContractMultipartUploader fails on trunk | Major | test | Ahmed Hussein | Steve Loughran | | | NPE in HttpServer during NameNode startup | Major | namenode | Konstantin Shvachko | Simbarashe Dzinamarira | | | Fix wrong command line for setting up CentOS 8 | Minor | documentation | Masatake Iwasaki | Takeru Kuramoto | | | Javadoc error in hadoop-yarn-server-common module | Major | build, documentation | Akira Ajisaka | Masatake Iwasaki | | | Typos in YarnScheduler#allocate method's doc comment | Minor | docs | Wanqiang Ji | Siddharth Ahuja | | | Add RootedOzFS AbstractFileSystem to core-default.xml | Major | fs | Bharat Viswanadham | Bharat Viswanadham | | | Test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints | Minor | capacity scheduler | Bilwa S T | Bilwa S T | | | Only log warning if considerLoad and considerStorageType are both true | Major | hdfs | Stephen O'Donnell | Stephen O'Donnell | | | Log improvements in NodeStatusUpdaterImpl | Minor | nodemanager | Bilwa S T | Bilwa S T | | | Switch to Yetus main branch | Major | build | Akira Ajisaka | Akira Ajisaka | | | Fix build the hadoop-build Docker image failed | Major | build | Wanqiang Ji | Wanqiang Ji | | | Setting dfs.disk.balancer.max.disk.errors = 0 will fail the block copy | Major | balancer & mover | AMC-team | AMC-team | | | Test failures in ITestAzureBlobFileSystemCheckAccess in ABFS | Major | fs/azure | Mehakmeet Singh | Thomas Marqardt | | | Only application master should upload resource to Yarn Shared Cache | Major | mrv2 | zhenzhao wang | zhenzhao wang | | | Correct spelling errors for separator | Trivial | common | Hui Fei | Hui Fei | | | Document options of logs CLI | Major | yarn | Adam Antal | Ankit Kumar | | | Handle null containerId in ClientRMService#getContainerReport() | Major | resourcemanager | Raghvendra Singh | Shubham Gupta | | | Upgrade to jQuery 3.5.1 in hadoop-yarn-common | Major | build, common | Wei-Chiu Chuang | Aryan Gupta | | | RBF: Fix webHdfs file display error | Major | rbf, webhdfs | Zhaohui Wang | Zhaohui Wang | | | Fix wrong comment in LongLong.java | Trivial | documentation, examples | Akira Ajisaka | Wanqiang Ji | | | TestRouterQuota fails in trunk | Major | rbf | Ayush Saxena | huangtianhua | | | NMs should supply a health status when registering with RM | Major | nodemanager, resourcemanager | Eric Badger | Eric Badger | | | TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing | Major | test | Peter Bacsko | Peter Bacsko | | | Use Yetus before YETUS-994 to enable adding comments to GitHub | Major | build | Akira Ajisaka | Akira Ajisaka | | | TestNameNodeRetryCacheMetrics fails intermittently | Major | hdfs, namenode | Ahmed Hussein | Hui Fei | | | Fix typos existance to existence | Trivial | common | Ikko Ashimine | Ikko Ashimine | | | RBF: Write Should allow, when a subcluster is unavailable for RANDOM mount points with fault Tolerance enabled. | Major | rbf | Harshakiran Reddy | Hemanth Boyina | | | MR job live lock caused by completed state container leak in heartbeat between node manager and RM | Major | nodemanager, yarn | zhenzhao wang | Jim Brennan | | | Set default throttle value on dfs.image.transfer.bandwidthPerSec | Major | namenode | Karthik Palanisamy | Karthik Palanisamy | | | Reduce datanode upgrade/hardlink thread | Major | datanode | Karthik Palanisamy | Karthik Palanisamy | | |

is not consistent | Major | test | Ahmed Hussein | Ahmed Hussein | | | TestExternalStoragePolicySatisfier fails intermittently | Major | test | Ahmed Hussein | Leon Gao | | | update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 | Blocker | common | Pranav Bheda | Pranav Bheda | | | SLS should set default user to handle SYNTH format | Major | scheduler-load-simulator | Qi Zhu | Qi Zhu | | | HttpFS server throws NPE if a file is a symlink | Major | fs, httpfs | Ahmed Hussein | Ahmed Hussein | | | Audit log deletes before collecting blocks | Major | logging, namenode | Ahmed Hussein | Ahmed Hussein | | | Javadoc warnings and errors are ignored in the precommit jobs | Major | build, documentation | Akira Ajisaka | Akira Ajisaka | | | Compute datanode load based on StoragePolicy | Major | hdfs, namenode | Karthik Palanisamy | Ayush Saxena | | | Touch command with -c option is broken | Major | command | Ayush Saxena | Ayush Saxena | | | Backslash in username causes build failure in the environment started by start-build-env.sh. | Minor | build | Takeru Kuramoto | Takeru Kuramoto | | | [JDK 11] Fix Javadoc errors in hadoop-hdfs-client | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Use shaded guava in ClientCache.java | Minor | build | Akira Ajisaka | Akira Ajisaka | | | Add partition resource info to get-node-labels and label-mappings api responses | Major | yarn | Akhil PB | Akhil PB | | | Deleted blocks linger in the replications queue | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | DataNode could meet deadlock if invoke refreshNameNode | Critical | datanode | Hongbing Wang | Hongbing Wang | | | LazyPersist Overwrite fails in direct write mode | Major | command | Ayush Saxena | Ayush Saxena | | | [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail | Major | test | Peter Bacsko | Peter Bacsko | | | Failed volumes can cause DNs to stop block reporting | Major | block placement, datanode | Ahmed Hussein | Ahmed Hussein | | | Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640 | Major | build, command | Brahma Reddy Battula | Brahma Reddy Battula | | | ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers | Major | resourcemanager | Haibo Chen | Haibo Chen | | | mvn site commands fails due to MetricsSystemImpl changes | Major | build, common | Xiaoqiao He | Xiaoqiao He | | | RM should make sure node label file highly available | Major | resourcemanager | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Client could not obtain block when DN CommandProcessingThread exit | Major | datanode | Yiqun Lin | Mingxiang Li | | | Upgrade commons-codec to

| Minor | build, common | Dongjoon Hyun | Dongjoon Hyun | | | TestLdapGroupsMapping failing -string mismatch in exception validation | Major | test | Steve Loughran | Steve Loughran | | | Audit log record the unexpected allowed result when delete called | Major | hdfs | Baolong Mao | Baolong Mao | | | Update PATCH\NAMING\RULE in the personality file | Minor | build | Akira Ajisaka | Akira Ajisaka | | | Hive On Tez queries fails upon submission to dynamically created pools | Major | resourcemanager | Anand Srinivasan | Peter Bacsko | | | Fix outdated properties of JournalNode when performing rollback | Minor | journal-node | Deegue | Deegue | | | Fix ZStandardCompressor input buffer offset | Major | io | Stephen Jung (Stripe) | Stephen Jung (Stripe) | | | Don't relocate org.bouncycastle in shaded client jars | Critical | build | Chao Sun | Chao Sun | | | hadoop-client-integration-tests doesn't work when building with skipShade | Major | build | Chao Sun | Chao Sun | | | Improve excessive reloading of Configurations | Major | conf | Ahmed Hussein | Ahmed Hussein | | | (S)Webhdfs will not use updated delegation tokens available in the ugi after the old ones expire | Major | webhdfs | Issac Buenrostro | Issac Buenrostro | | | Fix the documentation for dfs.namenode.replication.max-streams in hdfs-default.xml | Major | documentation | Xieming Li | Xieming Li | | | Doing hadoop ls on Har file triggers too many RPC calls | Major | fs | Ahmed Hussein | Ahmed Hussein | | | TimelineConnector swallows InterruptedException | Major | yarn-client | Ahmed Hussein | Ahmed Hussein | | | Log the remote address for authentication success | Minor | ipc | Ahmed Hussein | Ahmed Hussein | | | [JDK 14] TestConfiguredFailoverProxyProvider#testResolveDomainNameUsingDNS fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | [JDK 11] TestMRJobsWithProfiler fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Max applications calculation per queue disregards queue level settings in absolute mode | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Skip license check on lz4 code files | Major | build | Zhihua Deng | Zhihua Deng | | | Fair call queue is defeated by abusive service principals | Major | common, ipc | Ahmed Hussein | Ahmed Hussein | | | When building new web ui with root user, the bower install should support it. | Major | build, yarn-ui-v2 | Qi Zhu | Qi Zhu | | | TestNodeStatusUpdater does not handle early failure in threads | Major | nodemanager | Ahmed Hussein | Ahmed Hussein | | | Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair | Minor | fairscheduler | Szilard Nemeth | Ankit Kumar | | | Fix the typo of dfshealth.html after HDFS-15358 | Trivial | namenode | Hui Fei | Hui Fei | | | Fix Yarn CapacityScheduler Markdown document | Trivial | documentation | zhaoshengjie | zhaoshengjie | | | lz4 sources missing for native Visual Studio project | Major | native | Gautham Banasandra | Gautham Banasandra | | | NN should not let the balancer run in safemode | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Reshape SchedulerHealth class | Minor | resourcemanager, yarn | Adam Antal | D M Murali Krishna Reddy | | | Update yarn.nodemanager.env-whitelist value in docs | Minor | documentation | Andrea Scarpino | Andrea Scarpino | | | KerberosAuthentication does not disconnect HttpURLConnection leading to CLOSE\_WAIT cnxns | Major | auth, security | Prabhu Joseph | Attila Magyar | | | hadoop-aws does not work with hadoop-client-api | Major | fs/s3 | Keith Turner | Chao Sun | | | StorageTypeProto is not compatiable between 3.x and

| Major | build | Ryan Wu | Ryan Wu | | | NNTop counts don't add up as expected | Major | hdfs, metrics, namenode | Ahmed Hussein | Ahmed Hussein | | | EC: Socket file descriptor leak in StripedBlockChecksumReconstructor | Major | datanode, ec, erasure-coding | Yushi Hayasaka | Yushi Hayasaka | | | make the rpath of container-executor configurable | Major | yarn | angerszhu | angerszhu | | | Erasure Coding: dirty buffer causes reconstruction block error | Blocker | datanode, erasure-coding | HuangTao | HuangTao | | | Fix deprecation warnings in SLSWebApp.java | Minor | build | Akira Ajisaka | Ankit Kumar | | | ServerSocketUtil.getPort() should use loopback address, not 0.0.0.0 | Major | net | Eric Badger | Eric Badger | | | Lease Recovery never completes for a committed block which the DNs never finalize | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | EC: Block gets marked as CORRUPT in case of failover and pipeline recovery | Critical | erasure-coding | Ayush Saxena | Ayush Saxena | | | TestRouterWebServicesREST fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Client in distributedShell swallows interrupt exceptions | Major | client, distributed-shell | Ahmed Hussein | Ahmed Hussein | | | Correct spelling of comments for NNStorage.setRestoreFailedStorage | Trivial | namanode | Xudong Cao | Xudong Cao | | | Fix -Pdist build failure of hadoop-hdfs-native-client | Major | build | Masatake Iwasaki | Masatake Iwasaki | | | Missing Javadoc for a param in DFSNetworkTopology | Minor | hdfs | zhanghuazong | zhanghuazong | | | TestDistributedShell leaks resources on timeout/failure | Major | distributed-shell, test, yarn | Ahmed Hussein | Ahmed Hussein | | | Fix failure of TestDistributedShell#testDSShellWithOpportunisticContainers | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout | Critical | hdfs, journal-node, namanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2 | Major | webapp, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | ADLFS: Update SDK version from 2.3.6 to 2.3.9 | Minor | fs/adl | Bilahari T H | Bilahari T H | | | maxAMShare should only be accepted for leaf queues, not parent queues | Major | fairscheduler | Siddharth Ahuja | Siddharth Ahuja | | | Refactor TestDistributedShell | Major | distributed-shell, test | Ahmed Hussein | Ahmed Hussein | | | Increase docker memory limit in Jenkins | Major | build, scripts, test, yetus | Ahmed Hussein | Ahmed Hussein | | | Clear the fileMap in JHEventHandlerForSigtermTest | Minor | test | Zhengxi Li | Zhengxi Li | | | SLS generates a realtimetrack.json file but that file is missing the closing ']' | Minor | scheduler-load-simulator | Grant Sohn | Agshin Kazimli | | | Stale record should be remove when MutableRollingAverages generating aggregate data. | Major | metrics | Haibin Huang | Haibin Huang | | | Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags | Major | test | Peter Bacsko | Peter Bacsko | | | Install Intel ISA-L library in Dockerfile | Blocker | build | Takanobu Asanuma | Takanobu Asanuma | | | AbstractContractDeleteTest should set recursive parameter to true for recursive test cases. | Major | test | Konstantin Shvachko | Anton Kutuzov | | | Build failure due to python2.7 deprecation by pip | Blocker | build | Gautham Banasandra | Gautham Banasandra | | | The DeadNodeDetector shouldn't be shared by different DFSClients. | Major | datanode | Jinglun | Jinglun | | | Intermittent test failure org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength.testSnapshotfileLength | Major | hdfs, snapshots | Hanisha Koneru | Jim Brennan | | | Fix typo in

| Trivial | documentation | Gautham Banasandra | Gautham Banasandra | | | Add build instructions for installing GCC 9 and CMake 3.19 | Trivial | documentation | Gautham Banasandra | Gautham Banasandra | | | Possible Resource Leak in FSImageFormatProtobuf | Major | namenode | Narges Shadab | Narges Shadab | | | EC: Wrong checksum when reconstruction was failed by exception | Major | datanode, ec, erasure-coding | Yushi Hayasaka | Yushi Hayasaka | | | EC: fix NPE caused by StripedWriter.clearBuffers during reconstruct block | Major | erasure-coding | Hongbing Wang | Hongbing Wang | | | Fix that shaded should be used for google guava imports in YARN-10352. | Major | test | Qi Zhu | Qi Zhu | | | EC: Reconstruct task failed, and It would be XmitsInProgress of DN has negative number | Major | erasure-coding | Haiyang Hu | Haiyang Hu | | | Checkstyle IllegalImport does not catch guava imports | Major | build, common | Ahmed Hussein | Ahmed Hussein | | | Zombie applications in the YARN queue using FAIR + sizebasedweight | Critical | capacityscheduler | Guang Yang | Andras Gyori | | | User environment is unable to prepend PATH when mapreduce.admin.user.env also sets PATH | Major | container, nodeattibute | Eric Badger | Eric Badger | | | ClasscastException while loading FSImage | Major | nn | Renukaprasad C | Renukaprasad C | | | Fix incorrect string comparison in GpuDiscoverer | Major | resourcemanager | Peter Bacsko | Peter Bacsko | | | Upgrade ant to 1.10.9 | Major | common | Akira Ajisaka | Akira Ajisaka | | | RM UI2 Application page shows the AM preempted containers instead of the nonAM ones | Minor | yarn-ui-v2 | Benjamin Teke | Benjamin Teke | | | TestDelegationTokenRenewer fails intermittently | Major | test | Akira Ajisaka | Masatake Iwasaki | | | RBF: Cannot get method setBalancerBandwidth on Router Client | Major | rbf | Yang Yun | Yang Yun | | | DeadNodeDetector should close all the threads when it is closed. | Major | datanode | Jinglun | Jinglun | | | Upgrade Jackson databind to 2.10.5.1 | Major | build | Adam Roberts | Akira Ajisaka | | | Remove job\history\summary.py | Major | examples | Akira Ajisaka | Akira Ajisaka | | | Fix TestRMNodeLabelsManager failed after YARN-10501. | Major | test | Qi Zhu | Qi Zhu | | | Hadoop prints sensitive Cookie information. | Major | security | Renukaprasad C | Renukaprasad C | | | Reported IBR is partially replaced with stored info when queuing. | Critical | namenode | Kihwal Lee | Stephen O'Donnell | | | CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource() | Major | capacity scheduler | Haibo Chen | Haibo Chen | | | ClusterMapReduceTestCase does not clean directories | Major | test | Ahmed Hussein | Ahmed Hussein | | | Parsing error in CapacityScheduler.md | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Skip any credentials stored in HDFS when starting ZKFC | Major | hdfs | Krzysztof Adamski | Stephen O'Donnell | | | ExpiredHeartbeats metric should be of Type.COUNTER | Major | metrics | Konstantin Shvachko | Qi Zhu | | | Fix some spelling errors | Trivial | documentation | jiaguodong | jiaguodong | | | Fix

leak | Major | resourcemanager | Max Xie | Max Xie | | | All testcases in TestReservations are flaky | Major | reservation system, test | Szilard Nemeth | Szilard Nemeth | | | skip-dir option is not processed by Yetus | Major | build, precommit, yetus | Ahmed Hussein | Ahmed Hussein | | | Improve code quality in TestTimelineAuthenticationFilterForV1 | Minor | test, timelineservice | Szilard Nemeth | Szilard Nemeth | | | Consolidate YARN-10672 and YARN-10447 | Major | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Try blocks without catch blocks in SLS scheduler classes can swallow other exceptions | Major | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Logger of SLSFairScheduler is provided with the wrong class | Major | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Fix assertion failure message in BaseSLSRunnerTest | Trivial | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Better logging of uncaught exceptions throughout SLS | Major | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Fix Typo in TestSchedulingRequestContainerAllocation | Minor | capacity scheduler, test | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Check whether file is being truncated before truncate | Major | datanode, fs, namanode | Hui Fei | Hui Fei | | | Fix compilation error of OBSFileSystem in trunk | Major | fs/huawei | Masatake Iwasaki | Masatake Iwasaki | | | Replace GitHub App Token with GitHub OAuth token | Major | build | Akira Ajisaka | Akira Ajisaka | | | Add option to disable/enable free disk space checking and percentage checking for full and not-full disks | Major | nodemanager | Qi Zhu | Qi Zhu | | | Fix reference to LOG is ambiguous after HADOOP-17482 | Major | test | Xiaoyu Yao | Xiaoyu Yao | | | Upgrade org.codehaus.woodstox:stax2-api to 4.2.1 | Major | build, common | Ayush Saxena | Ayush Saxena | | | Fix shouldAvoidStaleDataNodesForWrite returns when no stale node in cluster. | Minor | block placement | Yang Yun | Yang Yun | | | Correct timestamp format in the docs for the touch command | Major | common | Stephen O'Donnell | Stephen O'Donnell | | | DeadNodeDetector doesn't remove live nodes from dead node set. | Major | datanode | Jinglun | Jinglun | | | Yarn Job execution get failed when LZ4 Compression Codec is used | Major | common | Bhavik Patel | Bhavik Patel | | | Percentage of queue and cluster is zero in WebUI | Major | resourcemanager | Bilwa S T | Bilwa S T | | | The scheduler monitor policies conf should trim values separated by comma | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | revisiting TestMRIntermediateDataEncryption | Major | job submission, security, test | Ahmed Hussein | Ahmed Hussein | | | Capacity Scheduler fails to handle user weights for a user that has a "." (dot) in it | Major | capacity scheduler | Siddharth Ahuja | Siddharth Ahuja | | | Improve UGI debug log to help troubleshooting TokenCache related issues | Major | security | Xiaoyu Yao | Xiaoyu Yao | | | Fix typos in AbstractCSQueue | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Fix potential null pointer error of gpuNodeResourceUpdateHandler in

| Major | nodemanager | Qi Zhu | Qi Zhu | | | Possible Resource Leak in EditLogFileOutputStream | Major | namanode | Narges Shadab | Narges Shadab | | | Fix the wrong CIDR range example in Proxy User documentation | Minor | documentation | Kwangsun Noh | Kwangsun Noh | | | Upgrade com.github.eirslett:frontend-maven-plugin to 1.11.2 | Major | buid | Mingliang Liu | Mingliang Liu | | | Intermediate data encryption is broken in LocalJobRunner | Major | job submission, security | Ahmed Hussein | Ahmed Hussein | | | Fix java doc issue introduced by HADOOP-17578 | Minor | documentation | Xiaoyu Yao | Xiaoyu Yao | | | Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal | Major | journal-node | Narges Shadab | Narges Shadab | | | Replace bzero with explicit\_bzero for better safety | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Resources are displayed in bytes in UI for schedulers other than capacity | Major | ui, webapp | Bilwa S T | Bilwa S T | | | Replace RAND\pseudo\bytes in sasl\digest\md5.cc | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Upgrade JUnit to 4.13.1 | Major | build, security, test | Ahmed Hussein | Ahmed Hussein | | | Use memcpy for copying non-null terminated string in jni\_helper.c | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode | Major | rbf | Harunobu Daikoku | Harunobu Daikoku | | | Use memcpy for copying non-null terminated string | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Can't remove all node labels after add node label without nodemanager port | Critical | yarn | caozhiqiang | caozhiqiang | | | Destroy yarn service if any YarnException occurs during submitApp | Minor | yarn-native-services | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Yarn Service AM listens on all IP's on the machine | Minor | security, yarn-native-services | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Add support for hadoop.http.rmwebapp.scheduler.page.class | Major | scheduler | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Fix NullPointerException in yarn-services Component.java | Minor | yarn-service | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Fix typo in ContainerRuntime | Trivial | documentation | Wanqiang Ji | xishuhai | | | Replace RAND\pseudo\bytes in rpc\_engine.cc | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Replace RAND\pseudo\bytes in util.cc | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Catch polymorphic type by reference | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix CapacityScheduler#initScheduler log error. | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | TestReplicaCachingGetSpaceUsed#testReplicaCachingGetSpaceUsedByRBWReplica Fails on Windows | Major | datanode, test | Ravuri Sushma sree | Ravuri Sushma sree | | | Correct the "hdfs fsck -list-corruptfileblocks" command output | Minor | hdfs, tools | Souryakanta Dwivedy | Ravuri Sushma sree | | | DelegationTokenAuthenticator prints token information | Major | security | Ravuri Sushma sree | Ravuri Sushma sree | | | Kinit with keytab should not display the keytab file's full path in any logs | Major | security | Ravuri Sushma sree | Ravuri Sushma sree | | | Prevent truncation by snprintf | Critical | fuse-dfs, libhdfs | Gautham Banasandra | Gautham Banasandra | | | Fix some @param errors in DirectoryScanner. | Minor | datanode | Qi Zhu | Qi Zhu | | | Fix DelegationTokenRenewer#updateRenewalTime java doc error. | Minor | documentation | Qi Zhu | Qi Zhu | | | Remove unused

import | Major | hdfs-client | Gautham Banasandra | Gautham Banasandra | | | Replace deprecated protobuf APIs | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix integer overflow | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | CryptoInputStream#close() should be synchronized | Major | crypto | Renukaprasad C | Renukaprasad C | | | hadoop-auth to remove jetty-server dependency | Major | auth | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix test4tests for libhdfspp | Critical | build, libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Incorrect representation of RESPONSE for Get Key Version in KMS index.md.vm file | Major | documentation | Ravuri Sushma sree | Ravuri Sushma sree | | | TestHistoryViewerPrinter could be failed when the locale isn't English. | Minor | test | Sungpeo Kook | Sungpeo Kook | | | DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff | Major | distcp | Srinivasu Majeti | Ayush Saxena | | | HadoopPipes task may fail when linux kernel version change from 3.x to 4.x | Major | pipes | chaoli | chaoli | | | TestJobEndNotifier fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Fix TestKMS failure | Major | kms | Akira Ajisaka | Akira Ajisaka | | | Fix GetApplicationsRequest JavaDoc | Major | documentation | Miklos Gergely | Miklos Gergely | | | RBF: WebHDFS create shouldn't choose DN from all sub-clusters | Major | rbf, webhdfs | Chao Sun | Fengnan Li | | | Call explicit\_bzero only if it is available | Major | libhdfs++ | Akira Ajisaka | Akira Ajisaka | | | Unreleased volume references cause an infinite loop | Critical | datanode | Shuyan Zhang | Shuyan Zhang | | | Remove appender EventCounter to avoid instantiation | Major | common | Viraj Jasani | Viraj Jasani | | | Update the ubuntu version in the build instruction | Major | build, documentation | Akira Ajisaka | Masatake Iwasaki | | | Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail | Major | nodemanager, test | Peter Bacsko | Peter Bacsko | | | public interface GroupMappingServiceProvider needs default impl for getGroupsSet() | Major | security | Vinayakumar B | Vinayakumar B | | | RBF: Unable to display the datanode UI of the router | Major | rbf, ui | Xiangyi Zhu | Xiangyi Zhu | | | Upgrade Jetty to 9.4.40 | Blocker | common | Akira Ajisaka | Akira Ajisaka | | | Misleading DEBUG log for container assignment needs to be removed when the container is actually reserved, not assigned in FairScheduler | Minor | yarn | Siddharth Ahuja | Siddharth Ahuja | | | Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647 | Major | nodelabel | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0 | Blocker | namanode | Brahma Reddy Battula | Brahma Reddy Battula | | | Fails to build using Maven

| Major | build | Wei-Chiu Chuang | Viraj Jasani | | | Datanode DirectoryScanner uses excessive memory | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Use spotbugs-maven-plugin in hadoop-huaweicloud | Major | build | Akira Ajisaka | Akira Ajisaka | | | Fix the SetQuotaByStorageTypeOp problem after updating hadoop | Major | hdfs | YaYun Wang | huangtianhua | | | RBF: Fix NullPointException when start dfsrouter | Major | rbf | Xie Lei | Fengnan Li | | | Interrupt DataStreamer thread | Minor | datanode | Karthik Palanisamy | Karthik Palanisamy | | | RBF: RBFMetrics's TotalCapacity out of bounds | Major | rbf | Weison Wei | Fengnan Li | | | SequenceFile.Writer should implement StreamCapabilities | Major | io | Kishen Das | Kishen Das | | | Remove additional junit 4.11 dependency from javadoc | Major | build, test, timelineservice | ANANDA G B | Akira Ajisaka | | | Fix the error of TestDynamometerInfra | Major | test | Akira Ajisaka | Takanobu Asanuma | | | TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes | Blocker | hdfs | Konstantin Shvachko | Akira Ajisaka | | | Avoid potential NPE by using Path#getParentPath API in hadoop-huaweicloud | Major | fs | Error Reporter | Viraj Jasani | | | Avoid Potential NPE in org.apache.hadoop.fs | Major | fs | Error Reporter | Viraj Jasani | | | Stabilise HDFS Pre-Commit | Major | build, test | Ayush Saxena | Ayush Saxena | | | Fix outdated properties of journal node when perform rollback | Minor | . | Deegue | Deegue | | | hadoop-azure parallel tests not working on recent JDKs | Major | build, fs/azure | Steve Loughran | Steve Loughran | | | Missing access check before getAppAttempts | Critical | webapp | lujie | lujie | | | checkcompatibility.py errors out when specifying annotations | Major | scripts | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove hardcoded SunX509 usage from SSLFactory | Major | common | Xiaoyu Yao | Xiaoyu Yao | | | Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp | Major | yarn-ui-v2 | Wei-Chiu Chuang | Wei-Chiu Chuang | | | The yarn.resource-types should support multi types without trimmed. | Major | resourcemanager | Qi Zhu | Qi Zhu | | | Build of Mapreduce Native Task module fails with unknown opcode "bswap" | Major | common | Anup Halarnkar | Anup Halarnkar | | | [UI2] Bump moment-timezone to 0.5.33 | Major | yarn, yarn-ui-v2 | Andras Gyori | Andras Gyori | | | Explicitly set locale in the Dockerfile | Blocker | build | Wei-Chiu Chuang | Wei-Chiu Chuang | | | ExitUtil#halt info log should log HaltException | Major | common | Viraj Jasani | Viraj Jasani | | | FS QueueManager should not create default queue at init | Major | fairscheduler | Wilfred Spiegelenburg | Benjamin Teke | | | container-executor permission is wrong in SecureContainer.md | Major | documentation | Akira Ajisaka | Siddharth Ahuja | | | DominantResourceCalculator isInvalidDivisor should consider only countable resource types | Major | yarn-common | Bilwa S T | Bilwa S T | | | Possible Resource Leak in org.apache.hadoop.hdfs.server.aliasmap#InMemoryAliasMap | Major | server | Narges Shadab | Narges Shadab | | | TestFrameworkUploader#testNativeIO fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Race condition with async edits logging due to updating txId outside of the namesystem log | Major | hdfs, namenode | Konstantin Shvachko | Konstantin Shvachko | | | TestBalanceProcedureScheduler and TestDistCpProcedure timeout | Major | rbf, test | Akira Ajisaka | Akira Ajisaka | | | [build] fix the Dockerfile for ARM | Blocker | build | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Misspelt words in

line 881 and line 885 | Trivial | datanode | Ning Sheng | Ning Sheng | | | Fix NullPointException In listOpenFiles | Major | namanode | Haiyang Hu | Haiyang Hu | | | Logging parameter issues in scheduler package | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0% | Major | capacity scheduler, capacityscheduler | Peter Bacsko | Peter Bacsko | | | Some dynamometer tests fail | Major | test | Akira Ajisaka | Akira Ajisaka | | | Configuration ${env.VAR:-FALLBACK} should eval FALLBACK when restrictSystemProps=true | Minor | common | Steve Loughran | Steve Loughran | | | testWithHbaseConfAtHdfsFileSystem consistently failing | Major | test | Viraj Jasani | Viraj Jasani | | | Fix asf license errors in newly added files by HADOOP-17727 | Major | build | Takanobu Asanuma | Takanobu Asanuma | | | [JDK 11] TestRMFailoverProxyProvider and TestNoHaRMFailoverProxyProvider fails by ClassCastException | Major | test | Akira Ajisaka | Akira Ajisaka | | | Make sure the order for location in ENTERING\_MAINTENANCE state | Minor | block placement | Tao Li | Tao Li | | | Avoid doing delegation token ops when yarn.timeline-service.http-authentication.type=simple | Major | timelineclient | Tarun Parimi | Tarun Parimi | | | Fix test failures in org.apache.hadoop.fs.azure.ITestOutputStreamSemantics | Minor | fs/azure | Anoop Sam John | Anoop Sam John | | | Quota is not preserved in snapshot INode | Major | hdfs | Siyao Meng | Siyao Meng | | | WebHdfsFileSystem has a possible connection leak in connection with HttpFS | Major | webhdfs | Takanobu Asanuma | Takanobu Asanuma | | | Yarn Logs Command retrying on Standby RM for 30 times | Major | yarn-common | D M Murali Krishna Reddy | D M Murali Krishna Reddy | | | Improve datanode shutdown latency | Major | datanode | Ahmed Hussein | Ahmed Hussein | | | Delete hadoop.ssl.enabled and dfs.https.enable from docs and core-default.xml | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet | Major | namnode | Yiqun Lin | Haibin Huang | | | DFTestUtil.waitReplication can produce false positives | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | LeaseRenewer#daemon threads leak in DFSClient | Major | dfsclient | Tao Yang | Renukaprasad C | | | [UI2] Upgrade Node.js to at least v12.22.1 | Major | yarn-ui-v2 | Akira Ajisaka | Masatake Iwasaki | | | Upgrade JUnit to

| Major | build, test | Ahmed Hussein | Ahmed Hussein | | | Title not set for JHS and NM webpages | Major | nodemanager | Rajshree Mishra | Bilwa S T | | | Avoid creating LayoutFlags redundant objects | Major | hdfs | Viraj Jasani | Viraj Jasani | | | Make bpServiceToActive to be volatile | Major | datanode | Shuyan Zhang | Shuyan Zhang | | | Fix flaky some unit tests since they offen timeout | Minor | test | Tao Li | Tao Li | | | Incorrect log placeholders used in JournalNodeSyncer | Minor | journal-node | Viraj Jasani | Viraj Jasani | | | Mapreduce job fails when NM is stopped | Major | task | Bilwa S T | Bilwa S T | | | Iterative snapshot diff report can generate duplicate records for creates, deletes and Renames | Major | snapshots | Srinivasu Majeti | Shashikant Banerjee | | | ConcurrentModificationException error happens on NameNode occasionally | Critical | hdfs | Daniel Ma | Daniel Ma | | | Better token validation | Major | security | Artem Smotrakov | Artem Smotrakov | | | DatanodeAdminMonitor scan should be delay based | Major | datanode | Ahmed Hussein | Ahmed Hussein | | | Improper pipeline close recovery causes a permanent write failure or data loss. | Major | hdfs | Kihwal Lee | Kihwal Lee | | | yarn logs cli fails to retrieve logs if any TFile is corrupt or empty | Major | yarn | Jim Brennan | Jim Brennan | | | RBF balance process is stuck at DisableWrite stage | Major | rbf | Eric Yin | Eric Yin | | | ViewFS should initialize target filesystems lazily | Major | client-mounts, fs, viewfs | Uma Maheswara Rao G | Abhishek Das | | | [UI2] Ambiguous queue name resolution | Major | yarn-ui-v2 | Andras Gyori | Andras Gyori | | | Upgrade jetty version to 9.4.43 | Major | build, common | Wei-Chiu Chuang | Renukaprasad C | | | RM logs endpoint vulnerable to clickjacking | Major | webapp | Benjamin Teke | Benjamin Teke | | | [JDK 11] Upgrade dnsjava to remove illegal access warnings | Major | common | Akira Ajisaka | Akira Ajisaka | | | HDFS default value change (with adding time unit) breaks old version MR tarball work with Hadoop 3.x | Critical | configuration, hdfs | Junping Du | Akira Ajisaka | | | Use separate source dir for platform builds | Critical | build | Gautham Banasandra | Gautham Banasandra | | | Add a configuration to RoundRobinVolumeChoosingPolicy to avoid failed volumes at datanodes. | Major | datanode | Zhihai Xu | Zhihai Xu | | | CopyListing fails with FNF exception with snapshot diff | Major | distcp | Shashikant Banerjee | Shashikant Banerjee | | | Set default capacity of root for node labels | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Revert HDFS-15372 (Files in snapshots no longer see attribute provider permissions) | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | TestTimelineClientV2Impl.testSyncCall fails intermittently | Minor | ATSv2, test | Prabhu Joseph | Andras Gyori | | | Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog | Critical | namnode | Yicong Cai | Wan Chang | | | CS considers only the default maximum-allocation-mb/vcore property as a maximum when it creates dynamic queues | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | RM HA startup can fail due to race conditions in ZKConfigurationStore | Major | capacity scheduler | Tarun Parimi | Tarun Parimi | | | SetTimes to throw FileNotFoundException if inode is not found | Major | hdfs | Harshakiran Reddy | Wei-Chiu Chuang | | | NPE in S3AInputStream read() after failure to reconnect to store | Major | fs/s3 | Bobby Wang | Bobby Wang | | | Entities missing from ATS when summary log file info got returned to the ATS before the domain log | Critical | yarn | Sushmitha Sreenivasan | Xiaomin Zhang | | | HistoryServerRest.html#Task\Counters\API, modify the jobTaskCounters's itemName from "taskcounterGroup" to "taskCounterGroup". | Minor | documentation | jenny | jenny | | | TestNMSimulator imports com.google.common.base.Supplier; | Major | buid | Steve Loughran | Steve Loughran | | | Run optional CI for changes in C | Critical | build | Gautham Banasandra | Gautham Banasandra | | | Upgrade commons-compress to

| Major | common | Dongjoon Hyun | Akira Ajisaka | | | Improve the parameter comments related to ProtobufRpcEngine2#Server() | Minor | documentation | JiangHua Zhu | JiangHua Zhu | | | Upgrade JSON smart to 2.4.7 | Major | build, common | Renukaprasad C | Renukaprasad C | | | Graceful Decommission ignores launched containers and gets deactivated before timeout | Major | RM | Prabhu Joseph | Srinivas S T | | | Refactor TempFile and TempDir in libhdfs++ | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Bug fix for Util#receiveFile | Minor | hdfs-common | Tao Li | Tao Li | | | Improve logging on ABFS error reporting | Minor | fs/azure | Steve Loughran | Steve Loughran | | | Make recursive rmdir in libhdfs++ cross platform | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | YARN shouldn't start with empty hadoop.http.authentication.signature.secret.file | Major | security | Benjamin Teke | Tamas Domok | | | Avoid possible class loading deadlock with VerifierNone initialization | Major | common | Viraj Jasani | Viraj Jasani | | | Run junit in Jenkins only if surefire reports exist | Major | build | Gautham Banasandra | Gautham Banasandra | | | Upgrade ant to 1.10.11 | Major | build, common | Ahmed Hussein | Ahmed Hussein | | | ExceptionsHandler to add terse/suppressed Exceptions in thread-safe manner | Major | common | Viraj Jasani | Viraj Jasani | | | Datanode caches namenode DNS lookup failure and cannot startup | Minor | ipc | Karthik Palaniappan | Chris Nauroth | | | Resolve log placeholders in NamenodeBeanMetrics | Minor | metrics | Viraj Jasani | Viraj Jasani | | | HTTP Filesystem to qualify paths in open()/getFileStatus() | Minor | fs | VinothKumar Raman | VinothKumar Raman | | | Avoid using implicit dependency on junit-jupiter-api | Major | test | Masatake Iwasaki | Masatake Iwasaki | | | Permission checking error on an existing directory in LogAggregationFileController#verifyAndCreateRemoteLogDir | Major | nodemanager | Tamas Domok | Tamas Domok | | | BuiltInGzipCompressor header and trailer should not be static variables | Critical | compress, io | L. C. Hsieh | L. C. Hsieh | | | Prometheus metrics only include the last set of labels | Major | common | Adam Binford | Adam Binford | | | Remove NN logs stack trace for non-existent xattr query | Major | namenode | Ahmed Hussein | Ahmed Hussein | | | Performance degradation in Text.append() after HADOOP-16951 | Critical | common | Peter Bacsko | Peter Bacsko | | | Test Result Not Working In Jenkins Result | Major | build, test | Ayush Saxena | Ayush Saxena | | | Too many "Failed to accept allocation proposal" because of wrong Headroom check for DRF | Major | capacityscheduler | jackwangcs | jackwangcs | | | SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing | Major | snapshots | Srinivasu Majeti | Shashikant Banerjee | | | Short circuit read leaks Slot objects when InvalidToken exception is thrown | Major | block placement | Eungsop Yoo | Eungsop Yoo | | | Missing user filtering check -\> yarn.webapp.filter-entity-list-by-user for RM Scheduler page | Major | yarn | Siddharth Ahuja | Gergely Pollk | | | lz4-java and snappy-java should be excluded from relocation in shaded Hadoop libraries | Major | build | L. C. Hsieh | L. C. Hsieh | |

FileUtil#fullyDelete deletes contents of sym-linked directory when symlink cannot be deleted because of local fs fault | Major | fs | Weihao Zheng | Weihao Zheng | | | Fix typo in LogAggregationFileController | Trivial | log-aggregation | Tamas Domok | Tibor Kovcs | | | Fix command line example in Hadoop Cluster Setup documentation | Minor | documentation | Rintaro Ikeda | Rintaro Ikeda | | | Fix Hadoop build on Debian 10 | Blocker | build | Gautham Banasandra | Gautham Banasandra | | | Fix log message arguments in LogAggregationFileController | Trivial | . | Tamas Domok | Tibor Kovcs | | | Remove irrelevant trim() call in TestStorageRestore | Trivial | test | Thomas Leplus | Thomas Leplus | | | HttpFS signature secret file misusage | Major | httpfs | Tamas Domok | Tamas Domok | | | Make hdfs\_allowSnapshot tool cross platform | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient | Major | webservice | Bilwa S T | Bilwa S T | | | Do not use exception handler to implement copy-on-write for EnumCounters | Major | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Deadlock in LeaseRenewer for static remove method | Major | hdfs | angerszhu | angerszhu | | | Example command for daemonlog is not correct | Minor | documentation | Renukaprasad C | Renukaprasad C | | | Fix typos in usage message in winutils.exe | Minor | winutils | igo Goiri | Gautham Banasandra | | | Replace unshaded guava in HttpFSServerWebServer | Major | httpfs | Masatake Iwasaki | Masatake Iwasaki | | | Upgrade Kafka to 2.8.1 | Major | build, common | Takanobu Asanuma | Takanobu Asanuma | | | Standby RM should expose prom endpoint | Major | resourcemanager | Max Xie | Max Xie | | | Expose all node labels for root without explicit configurations | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | [SBN Read] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN | Critical | journal-node, metrics | Zhaohui Wang | Zhaohui Wang | | | Cleanup protobuf on exit of hdfs\_allowSnapshot | Major | hdfs-client, libhdfs++, tools | Gautham Banasandra | Gautham Banasandra | | | BUILDING.txt should not encourage to activate docs profile on building binary artifacts | Minor | documentation | Rintaro Ikeda | Masatake Iwasaki | | | Add the necessary write lock in Checkpointer#doCheckpoint() | Major | namenode | JiangHua Zhu | JiangHua Zhu | | | Fix TestViewFsTrash to use the correct homeDir. | Minor | test, viewfs | Steve Loughran | Xing Lin | | | Balancer stuck when moving striped blocks due to NPE | Major | balancer & mover, erasure-coding | Leon Gao | Leon Gao | | | RBF: NullPointerException when setQuota through routers with quota disabled | Major | rbf | Chengwei Wang | Chengwei Wang | | | Fix resource leak due to Files.walk | Minor | nodemanager | lujie | lujie | | | Distcp file length comparison have no effect | Major | common, tools, tools/distcp | yinan zhan | yinan zhan | | | Int overflow in computing safe length during EC block recovery | Critical | ec, erasure-coding | daimin | daimin | | | Add missing RELEASENOTES and CHANGELOG to upstream | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Exclude IBM Java security classes from being shaded/relocated | Major | build | Nicholas Marion | Nicholas Marion | | |

uses incorrect default value for cacheDir | Major | test | Konstantin Shvachko | Michael Kuchenbecker | | | Disable JIRA plugin for YETUS on Hadoop | Critical | build | Gautham Banasandra | Gautham Banasandra | | | [Fix] Improve NNThroughputBenchmark#blockReport operation | Major | benchmarks, namenode | JiangHua Zhu | JiangHua Zhu | | | Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger) | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Use libcrypto in Windows for libhdfspp | Blocker | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Locate OpenSSL libs for libhdfspp | Major | build, hdfs-client, native | Sangjin Lee | Gautham Banasandra | | | Metric metadataOperationRate calculation error in DataNodeVolumeMetrics | Major | datanode, metrics | Tao Li | Tao Li | | | Fix race condition of User object acquisitions | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms | Major | build | Viraj Jasani | Viraj Jasani | | | numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage | Major | namanode | Max Xie | Max Xie | | | No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem | Major | viewfs | Abhishek Das | Abhishek Das | | | Fix log format for BlockManager | Minor | block placement | Tao Li | Tao Li | | | Fix incorrect placeholder for Exception logs in DiskBalancer | Major | datanode | Viraj Jasani | Viraj Jasani | | | Correct disk balancer param desc | Minor | documentation, hdfs | guophilipse | guophilipse | | | Correct NameNode ACL description | Minor | documentation | guophilipse | guophilipse | | | Add some debug logs when the dfsUsed are not used during Datanode startup | Major | datanode | Mukul Kumar Singh | Mukul Kumar Singh | | | Number of allocated OPPORTUNISTIC containers can dip below 0 | Minor | resourcemanager | Andrew Chung | Andrew Chung | | | Make certain methods LimitedPrivate in S3AUtils.java | Major | fs/s3 | Mehakmeet Singh | Mehakmeet Singh | | | Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method | Minor | distributed-shell | Ashutosh Gupta | Ashutosh Gupta | | | Fallback to simple auth does not work for a secondary DistributedFileSystem instance | Major | ipc | Istvn Fajth | Istvn Fajth | | | Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson | Major | common | Haiyang Hu | Haiyang Hu | | | Datanode start time should be set after RPC server starts successfully | Minor | datanode | Viraj Jasani | Viraj Jasani | | | Correct words in YARN documents | Minor | documentation | guophilipse | guophilipse | | | EntityGroupFSTimelineStore#ActiveLogParser parses already processed files | Major | timelineserver | Prabhu Joseph | Ravuri Sushma sree | | | Fix log format for QueryCommand | Minor | command, diskbalancer | Tao Li | Tao Li | | | Include static imports in the maven plugin rules | Major | build | Viraj Jasani | Viraj Jasani | | | RBF: RouterRpcServer#invokeAtAvailableNs does not take effect when retrying | Major | rbf | Tao Li | Tao Li | | | Expired block token causes slow read due to missing handling in sasl handshake | Major | datanode, dfs, dfsclient | Shinya Yoshida | Shinya Yoshida | | | Provide a public wrapper

Configuration#substituteVars | Major | conf | Andras Gyori | Andras Gyori | | | RBF: Fix the retry logic of RouterRpcServer#invokeAtAvailableNs | Major | rbf | Ayush Saxena | Ayush Saxena | | | Fix assert message for BlockInfo | Minor | block placement | Tao Li | Tao Li | | | Client sleeps and holds 'dataQueue' when DataNodes are congested | Major | hdfs-client | Yuanxin Zhu | Yuanxin Zhu | | | ATS 1.5 fails to start if RollingLevelDb files are corrupt or missing | Major | timelineserver, timelineservice | Tarun Parimi | Ashutosh Gupta | | | jsonschema2pojo-maven-plugin version is not defined | Major | build | Akira Ajisaka | Tamas Domok | | | Extend the root QueueInfo with max-parallel-apps in CapacityScheduler | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | DBOutputFormat.DBRecordWriter#write must throw exception when it fails | Major | mrv2 | Stamatis Zampetakis | Stamatis Zampetakis | | | Queue weight is incorrectly reset to zero | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Fix error log in BlockManagerSafeMode | Minor | hdfs | guophilipse | guophilipse | | | Correct the statement for pmem cache to reflect cache persistence support | Minor | documentation | Feilong He | Feilong He | | | Allow overriding user limit factor and maxAMResourcePercent with AQCv2 templates | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | fix balancer bug when transfer an EC block | Major | balancer & mover, erasure-coding | qinyuren | qinyuren | | | [UI2] No container is found for an application attempt with a single AM container | Major | yarn-ui-v2 | Andras Gyori | Andras Gyori | | | Use mina-core 2.0.22 to fix LDAP unit test failures | Major | test | Akira Ajisaka | Akira Ajisaka | | | Fix MiniDFSCluster restart in case of multiple namenodes | Major | test | Ayush Saxena | Ayush Saxena | | | Fix an issue in checking native pmdk lib by 'hadoop checknative' command | Major | native | Feilong He | Feilong He | | | ATSv2 storage monitor fails to read from hbase cluster | Major | timelineservice | Viraj Jasani | Viraj Jasani | | | TestApplicationLimits.testLimitsComputation() has some ineffective asserts | Major | capacity scheduler, test | Benjamin Teke | Benjamin Teke | | | Should CheckNotNull before access FsDatasetSpi | Major | datanode | Tao Li | Tao Li | | | Duplicate Job IDs in SLS output | Major | scheduler-load-simulator | Drew Merrill | Szilard Nemeth | | | Nodemanager resource usage metrics sometimes are negative | Major | nodemanager | YunFan Zhou | Benjamin Teke | | | Synchronizing iteration of Configuration properties object | Major | conf | Jason Darrell Lowe | Dhananjay Badaya | | | ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled | Major | timelineservice | Minni Mittal | Viraj Jasani | | | Fix Datanode retrieve slownode information bug. | Major | datanode | Jackson Wang | Jackson Wang | | | Global Scheduler async thread crash caused by 'Comparison method violates its general contract | Major | capacity scheduler | tuyu | Andras Gyori | | | TestWebHdfsFileSystemContract#testResponseCode fails | Major | test | secfree | secfree | | | AuxService should not use class name as default system classes | Major | auxservices | Cheng Pan | Cheng Pan | | | Remove

NNThroughputBenchmark#dummyActionNoSynch() | Major | benchmarks, namenode | JiangHua Zhu | JiangHua Zhu | | | Fix typo: validateEncrytionSecrets -\> validateEncryptionSecrets | Major | fs/s3 | Ashutosh Gupta | Ashutosh Gupta | | | Disable TestDynamometerInfra | Major | test | Akira Ajisaka | Akira Ajisaka | | | Unknown frame descriptor when decompressing multiple frames in ZStandardDecompressor | Major | compress, io | ZanderXu | ZanderXu | | | Remove unused import AbstractJavaKeyStoreProvider in Shell class | Minor | command | JiangHua Zhu | JiangHua Zhu | | | RBF: Fix TestRouterRPCMultipleDestinationMountTableResolver | Major | rbf | Ayush Saxena | Ayush Saxena | | | Fix typo: testHasExeceptionsReturnsCorrectValue -\> testHasExceptionsReturnsCorrectValue | Trivial | test | Ashutosh Gupta | Ashutosh Gupta | | | Ensure LeaseRecheckIntervalMs is greater than zero | Major | namenode | ECFuzz | ECFuzz | | | Insecure Xml parsing in OfflineEditsXmlLoader | Minor | tools | Ashutosh Gupta | Ashutosh Gupta | | | RBF: StaticRouterRpcFairnessPolicyController init fails with division by 0 if concurrent ns handler count is configured | Minor | rbf | Felix N | Felix N | | | ProfileOutputServlet unable to proceed due to NPE | Major | common | Viraj Jasani | Viraj Jasani | | | Avoid deleting unique data blocks when deleting redundancy striped blocks | Critical | ec, erasure-coding | qinyuren | Jackson Wang | | | In cgroups-operations.c some fprintf format strings don't end with "\\n" | Minor | nodemanager | Gera Shegalov | Gera Shegalov | | | Bump follow-redirects from 1.13.3 to 1.14.7 in hadoop-yarn-ui | Major | yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | Improve HeartbeatManager logic to avoid incorrect stats | Major | datanode | Tao Li | Tao Li | | | hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency | Major | build | Steve Loughran | lixianwei | | | Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp | Critical | webapp | Akira Ajisaka | Akira Ajisaka | | | Losing over 100 datanodes in state decommissioning results in full blockage of all datanode decommissioning | Major | block placement, datanode | Kevin Wikant | Kevin Wikant | | | Fix edge case where DatanodeAdminDefaultMonitor doubly enqueues a DatanodeDescriptor on exception | Major | hdfs | Kevin Wikant | Kevin Wikant | | | Containers going from New to Scheduled transition for killed container on recovery | Major | container, nodemanager | Minni Mittal | Minni Mittal | | | Bump aliyun-sdk-oss to 3.13.2 and jdom2 to 2.0.6.1 | Major | build, common | Aswin Shakil | Aswin Shakil | | | RBF: RouterId is NULL when set dfs.federation.router.rpc.enable=false | Major | rbf | YulongZ | YulongZ | | | DataNode metric ReadsFromLocalClient does not count short-circuit reads | Minor | datanode, metrics | secfree | secfree | | | Distcp: Sync moves filtered file to home directory rather than deleting | Critical | tools/distcp | Ayush Saxena | Ayush Saxena | | | Fix hadoop web site release notes and changelog not available | Minor | documentation | guophilipse | guophilipse | | | TestCsiClient fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Update junit 5 version due to build issues | Major | build | PJ Fanning | PJ Fanning | | | Update transitive log4j2 dependency to

| Major | buid | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Improve DirectoryScanner: add regular file check related block | Major | datanode | JiangHua Zhu | JiangHua Zhu | | | AutoCreatedQueueTemplate incorrect wildcard level | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | Minimum resource ratio is overridden by subsequent labels | Major | yarn | Andras Gyori | Andras Gyori | | | isAbsoluteResource is not correct for dynamically created queues | Minor | yarn | Tamas Domok | Tamas Domok | | | Follow up YARN-10237: fix the new test case in TestRMWebServicesCapacitySched | Major | webapp | Tamas Domok | Tamas Domok | | | Fix the documentation for max-parallel-apps in CS | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | Fix testQueueSubmitWithACLsEnabledWithQueueMapping in TestAppManager | Major | yarn | Tamas Domok | Tamas Domok | | | Switch the baseurl for Centos 8 | Blocker | build | Gautham Banasandra | Gautham Banasandra | | | Snapshot diff on snapshotable directory fails with not snapshottable error | Major | namanode | Stephen O'Donnell | Stephen O'Donnell | | | Resource overcommitment due to incorrect resource normalisation logical order | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Change URI[] in INodeLink to String[] to reduce memory footprint of ViewFileSystem | Major | viewfs | Abhishek Das | Abhishek Das | | | Should verify whether the path name is valid in the WebHDFS | Major | webhdfs | Tao Li | Tao Li | | | Fix typo in RM audit log | Major | resourcemanager | Junfan Zhang | Junfan Zhang | | | Introduce the config to control the refresh interval in RMDelegatedNodeLabelsUpdater | Major | nodelabel | Junfan Zhang | Junfan Zhang | | | Fix StackOverflowError in SLS scheduler event handling | Major | scheduler-load-simulator | Szilard Nemeth | Szilard Nemeth | | | Fix dependency error in hadoop-hdfs on M1 Mac | Major | build | Akira Ajisaka | Akira Ajisaka | | | Fix NPE for checkBlockReportLease | Major | datanode, namanode | Tao Li | Tao Li | | | Fix the test failure due to missing conf of

| Major | test | Junfan Zhang | Junfan Zhang | | | KeyProviderCache close cached KeyProvider with Hadoop ShutdownHookManager | Major | hdfs | Lei Yang | Lei Yang | | | getDelegationTokens in ViewFs should also fetch the token from the fallback FS | Major | fs | Xing Lin | Xing Lin | | | Fix spotbugs error in hadoop-sls module | Major | scheduler-load-simulator | Akira Ajisaka | Szilard Nemeth | | | EC: NameNode should not send a reconstruction work when the source datanodes are insufficient | Critical | ec, erasure-coding | Yuanbo Liu | Takanobu Asanuma | | | Fix decommission UnsupportedOperationException: Remove unsupported | Major | namenode | daimin | daimin | | | When NodeLabel is enabled for a YARN cluster, AM blacklist program does not work properly | Major | resourcemanager | Xiping Zhang | Xiping Zhang | | | EC: Decommission a rack with only on dn will fail when the rack number is equal with replication | Critical | ec, namenode | caozhiqiang | caozhiqiang | | | SlotReleaser should reuse the domain socket based on socket paths | Major | hdfs-client | Quanlong Huang | Quanlong Huang | | | Ensure that default permissions of directories under internal ViewFS directories are the same as directories on target filesystems | Major | viewfs | Chentao Yu | Chentao Yu | | | Fix thread safety of EC decoding during concurrent preads | Critical | dfsclient, ec, erasure-coding | daimin | daimin | | | ReverseXML processor doesn't accept XML files without the SnapshotDiffSection. | Critical | hdfs | yanbin.zhang | yanbin.zhang | | | [SBN read] Avoid purging edit log which is in progress | Critical | namanode | Tao Li | Tao Li | | | YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging | Critical | webproxy | Qi Zhu | Qi Zhu | | | Source path with storagePolicy cause wrong typeConsumed while rename | Major | hdfs, namenode | lei w | lei w | | | YARN incorrectly validates maximum capacity resources on the validation API | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Explicitly declare serialVersionUID in LogMutation class | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Unable to unregister FsDatasetState MBean if DataNode is shutdown twice | Trivial | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | \`org.wildfly.openssl\` should not be shaded by Hadoop build | Major | build | Andr F. | Andr F. | | | create-release fails fatal: unsafe repository ('/build/source' is owned by someone else) | Major | build | Steve Loughran | Steve Loughran | | | EC decoding failed due to not enough valid inputs | Major | erasure-coding | qinyuren | qinyuren | | | EC decoding failed due to invalid buffer | Major | erasure-coding | qinyuren | qinyuren | | | Recovery failure when node-label configure-type transit from delegated-centralized to centralized | Major | yarn | Junfan Zhang | Junfan Zhang | | | Fix NPE for TestBlockManager | Major | test | Tao Li | Tao Li | | | In MapredAppMasterRest#Mapreduce\Application\Master\Info\API, the datatype of appId should be "string". | Major | documentation | jenny | Ashutosh Gupta | | | Document "io.file.buffer.size" must be greater than zero | Minor | io | ECFuzz | ECFuzz | | | Removing hadoop-yarn-project/hadoop-yarn/README as it is no longer maintained. | Minor | documentation | N Sanketh Reddy | Ashutosh Gupta | | | Use uint32\t for hdfs\find | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Update the link to compatibility guide | Minor | documentation | Akira Ajisaka | Ashutosh Gupta | | | Fix comment in LowRedundancyBlocks.java | Minor | documentation | Akira Ajisaka | Ashutosh Gupta | | | Fix the examples of hadoop config prefix | Minor | documentation | Akira Ajisaka | Ashutosh Gupta | | | AggregateWordCount fetches wrong results | Major | aggregate | Ayush Saxena | Ayush Saxena | | | Fix typo in readme of hadoop-project-dist | Trivial | documentation | Gautham Banasandra | Gautham Banasandra | | | Prevent DelegationTokenSecretManagerMetrics from registering multiple times | Major | common | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | System.err should be used when error occurs in multiple methods in DFSAdmin class | Major | dfsadmin | yanbin.zhang | yanbin.zhang | | | ResourceManager webapps test failures due to org.apache.hadoop.metrics2.MetricsException and subsequent

Address already in use | Major | resourcemanager | Szilard Nemeth | Szilard Nemeth | | | Avoid unnecessary preemption for tiny queues under certain corner cases | Major | capacity scheduler, scheduler preemption | Jian Chen | Jian Chen | | | Remove unused imports in MapReduce project | Minor | build | Ashutosh Gupta | Ashutosh Gupta | | | Data locality is lost when DataNode pod restarts in kubernetes | Major | namenode | Huaxiang Sun | Huaxiang Sun | | | Upgrade jquery ui to 1.13.1 | Major | buid | D M Murali Krishna Reddy | Ashutosh Gupta | | | YarnClient gets the wrong EffectiveMinCapacity value | Major | api | Zilong Zhu | Zilong Zhu | | | TimelineService v2 lists containers for all attempts when filtering for one | Major | timelinereader | Benjamin Teke | Benjamin Teke | | | ZKConfigurationStore Java deserialisation vulnerability | Major | yarn | Tamas Domok | Tamas Domok | | | Capacity Scheduler does not support ambiguous queue names when moving application across queues | Major | capacity scheduler | Andrs Gyri | Andrs Gyri | | | ResourceUsage and QueueCapacities classes provide node label iterators that are not thread safe | Major | capacity scheduler | Andrs Gyri | Andrs Gyri | | | QueueMetrics is leaking memory when creating a new queue during reinitialisation | Major | capacity scheduler | Andrs Gyri | Andrs Gyri | | | Extend KMS related exceptions that get mapped to ConnectException | Major | kms | Ritesh Shukla | Ritesh Shukla | | | Hadoop auth does not handle HTTP Headers in a case-insensitive way | Critical | auth | Daniel Fritsi | Jnos Makai | | | Upgrade okhttp from 2.7.5 to 4.9.3 | Major | hdfs-client | Ivan Viaznikov | Ashutosh Gupta | | | Set the zk acl for nodes created by ZKConfigurationStore. | Major | resourcemanager | Owen O'Malley | Owen O'Malley | | | Purge FsDatasetAsyncDiskService threadgroup; it causes BPServiceActor$CommandProcessingThread IllegalThreadStateException 'fatal exception and exit' | Major | datanode | Michael Stack | Michael Stack | | | Fix failure of extracting JIRA id from commit message in git\jira\fix\version\check.py | Minor | build | Masatake Iwasaki | Masatake Iwasaki | | | Handle error returned by strtol | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix comments in TestProportionalCapacityPreemptionPolicy\* | Minor | capacityscheduler, documentation | Ashutosh Gupta | Ashutosh Gupta | | | RBF: Add snapshot counts to content summary in router | Major | rbf | Quan Li | Ayush Saxena | | | DatanodeAdminDefaultMonitor can get stuck in an infinite loop | Major | datanode | Stephen O'Donnell | Stephen O'Donnell | | | Install gtest via FetchContent\_Declare in CMake | Blocker | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Install Maven from Apache archives | Blocker | build | Gautham Banasandra | Gautham Banasandra | | | Use CMake 3.19.0 in Debian 10 | Blocker | build | Gautham Banasandra | Gautham Banasandra | | | Use "defined" directive along with #if | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Fix the link in TestClientProtocolForPipelineRecovery | Minor | documentation | Samrat Deb | Samrat Deb | | | Namenode WebUI prints sensitive information on Token Expiry | Major | namanode, security, webhdfs | Renukaprasad C | Renukaprasad C | | | IllegalArgumentException in LifelineSender | Major | datanode | ZanderXu | ZanderXu | | | RBF: Correct target directory when move to trash for kerberos login user. | Major | rbf | Xiping Zhang | Xiping Zhang | | | Determine when to invalidate corrupt replicas based on number of usable replicas | Major | datanode, namenode | Kevin Wikant | Kevin Wikant | | | Fix Http Response code in GenericExceptionHandler. | Major | webapp | Abhishek Modi | Ashutosh Gupta | | | Fix javadoc error in Java 11 | Major | build, documentation | Akira Ajisaka | Ashutosh Gupta | | |

TestJHSSecurity#testDelegationToken AssertionError due to HDFS-16563 | Major | test | Shilun Fan | Shilun Fan | | | MapReduce tasks timing out when spends more time on MultipleOutputs#close | Major | mrv1, mrv2 | Prabhu Joseph | Ashutosh Gupta | | | Pending app metrics are increased doubly when a queue reaches its max-parallel-apps limit | Major | capacity scheduler | Andrs Gyri | Andrs Gyri | | | Typo in description of "mapreduce.application.classpath" in mapred-default.xml | Trivial | mrv2 | Christian Bartolomus | Christian Bartolomus | | | Certificate doesn't match any of the subject alternative names: [\*.s3.amazonaws.com, s3.amazonaws.com] | Major | fs/s3 | Andr F. | Andr F. | | | YARN Native Service HttpProbe logs THIS\_HOST in error messages | Minor | yarn-native-services | Prabhu Joseph | Ashutosh Gupta | | | TestLocalDistributedCacheManager failing after HADOOP-16202 | Major | test | Steve Loughran | Steve Loughran | | | Only files belong to the first file controller are removed even if multiple log aggregation file controllers are configured | Major | log-aggregation | Szilard Nemeth | Szilard Nemeth | | | CS UI: queue filter and openQueues param do not work as expected | Major | capacity scheduler | Chengbing Liu | Chengbing Liu | | | Upgrade Apache Xerces Java to 2.12.2 | Major | build | Ashutosh Gupta | Ashutosh Gupta | | | Replace FSDataInputStream#read with readFully in Log Aggregation | Major | log-aggregation | Prabhu Joseph | Ashutosh Gupta | | | One yarn rest api example of yarn document is error | Minor | documentation | bright.zhou | Ashutosh Gupta | | | Reserved Space For Replicas is not released on some cases | Major | hdfs | Prabhu Joseph | Ashutosh Gupta | | | StateStoreZooKeeper fails to initialize | Major | rbf | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Various MapReduce tests fail with NPE in AggregatedLogDeletionService.stopRMClient | Major | log-aggregation | Szilard Nemeth | Szilard Nemeth | | | Fix when to read an additional record from a BZip2 text file split | Critical | io | Ashutosh Gupta | Ashutosh Gupta | | | DistCp does not clean staging folder if class extends DistCp | Minor | tools/distcp | Lawrence Andrews | Lawrence Andrews | | | Change scope of inner classes in InodeTree to make them accessible outside package | Major | viewfs | Abhishek Das | Abhishek Das | | | shutdownhookmanager should not be multithreaded (deadlock possible) | Minor | util | Catherinot Remi | Catherinot Remi | | | Deletion of assigned resources

GPU's, NUMA, FPGA's) from State Store | Major | nodemanager | Prabhu Joseph | Samrat Deb | | | Partial/Incomplete groups list can be returned in LDAP groups lookup | Major | security | Philippe Lanoe | Larry McCay | | | Erasure Coding: Recovery may cause excess replicas when busy DN exsits | Major | ec, erasure-coding | Ruinan Gu | Ruinan Gu | | | Link OpenSSL lib for CMake deps check | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | TestRouterWebServicesREST failing after YARN-9827 | Major | federation | Shilun Fan | Shilun Fan | | | Fix duplicate sources for hdfspp\test\shim\_static | Critical | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Use malloc for buffer allocation in uriparser2 | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | QueueMetrics leaks Configuration objects when validation API is called multiple times | Major | capacity scheduler | Andrs Gyri | Andrs Gyri | | | Skip libhdfspp Valgrind tests on Windows | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | Do not pass GCC flags for MSVC in libhdfspp | Major | libhdfs++ | Gautham Banasandra | Gautham Banasandra | | | MapReduce set permission too late in copyJar method | Major | mrv2 | Zhang Dongsheng | Zhang Dongsheng | | | COMPOSITE\_CRC failed between replicated file and striped file due to invalid requested length | Major | hdfs, hdfs-client | ZanderXu | ZanderXu | | | Fix YARN RMAdminCLI retry logic for non-retryable kerberos configuration exception | Major | client | Kevin Wikant | Kevin Wikant | | | Upgrade Netty to 4.1.77.Final | Major | build | Renukaprasad C | Wei-Chiu Chuang | | | All method metrics related to the RPC protocol should be initialized | Major | metrics | Shuyan Zhang | Shuyan Zhang | | | Fix bug preventing hadoop-metrics2 from emitting metrics to \> 1 Ganglia servers. | Major | metrics | Ashutosh Gupta | Ashutosh Gupta | | | Fix out of sync import for HADOOP-18321 | Minor | common | Ashutosh Gupta | Ashutosh Gupta | | | Fix incorrect placeholder in hadoop-common | Minor | common | Shilun Fan | Shilun Fan | | | Bug while disabling proxy failover with Federation | Major | federation | Ashutosh Gupta | Ashutosh Gupta | | | Codecs with @DoNotPool annotation are not closed causing memory leak | Major | common | Kevin Sewell | Kevin Sewell | | | Fix broken link to wiki help page in org.apache.hadoop.util.Shell | Major | documentation | Paul King | Paul King | | | DatanodeAdminManager$Monitor reports a node as invalid continuously | Major | namenode | Prabhu Joseph | Ashutosh Gupta | | | hadoop-minikdc dependency duplicated in hadoop-yarn-server-nodemanager | Minor | nodemanager | Clara Fang | Clara Fang | | | RBF: fix some unreasonably annotated docs | Major | documentation, rbf | JiangHua Zhu | JiangHua Zhu | | | Make initialDelay configurable for FederationStateStoreService#scheduledExecutorService | Major | federation | Shen Yinjie | Ashutosh Gupta | | | Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name. | Major | security | Ahad Rana | Steve Vaughan | | | RBF should support get the information about ancestor mount points | Major | rbf | ZanderXu | ZanderXu | | | Support spark dynamic partitioning in the Manifest Committer | Major | mrv2 | Steve Loughran | Steve Loughran | | | RBF throw IndexOutOfBoundsException with disableNameServices | Major | rbf | ZanderXu | ZanderXu | | | Invalid CallerContext caused NullPointerException | Critical | namanode | ZanderXu | ZanderXu | | | [SBN READ] Avoid get location from observer when the block report is delayed. | Critical | hdfs | Chenyu Zheng | Chenyu Zheng | | | RBF: fix some bugs when handling getContentSummary RPC | Major | rbf | ZanderXu | ZanderXu | | | Fix failure of shelltest for hadoop\add\ldlibpath | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory after YARN-10793 | Major | build, test | Shilun Fan | Shilun Fan | | | Parameterize platform toolset version | Major | common | Gautham Banasandra | Gautham Banasandra | | | TestQJMWithFaults.testUnresolvableHostName() can fail due to unexpected host resolution | Minor | test | Steve Vaughan | Steve Vaughan | | | NameNode should use

to log audit event to avoid possible NPE | Major | namanode | ZanderXu | ZanderXu | | | Correct inaccurate BlocksRemoved metric on DataNode side | Minor | datanode, metrics | ZanderXu | ZanderXu | | | RBF: DFSClient should uniquely identify writing files by namespace id and iNodeId | Critical | rbf | ZanderXu | ZanderXu | | | JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId | Critical | journal-node | ZanderXu | ZanderXu | | | Improve the accuracy of MutableStat mean | Major | common | Shuyan Zhang | Shuyan Zhang | | | RBF proxies the client's user by the login user to enable CacheEntry | Major | rbf | ZanderXu | ZanderXu | | | Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory after YARN-11269 | Major | test, timelineserver | Shilun Fan | Shilun Fan | | | Fix TestKMS#testKMSHAZooKeeperDelegationToken Failed By Hadoop-18427 | Major | common, test | Shilun Fan | Shilun Fan | | | Fix file split duplicating records from a succeeding split when reading BZip2 text files | Critical | common | Ashutosh Gupta | Ashutosh Gupta | | | refreshHostsReader should use the new configuration | Major | datanode, namanode | ZanderXu | ZanderXu | | | Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703) | Major | log-aggregation | Shilun Fan | Shilun Fan | | | TestDNS.testRDNS can fail with ServiceUnavailableException | Minor | common, net | Steve Loughran | Ashutosh Gupta | | | ViewFS doc points to wrong mount table name | Minor | documentation | Prabhu Joseph | Samrat Deb | | | Fix SQLFederationStateStore#Sql script bug | Major | federation | Shilun Fan | Shilun Fan | | | NullPointerException in ObjectListingIterator's constructor | Blocker | fs/s3 | Quanlong Huang | Steve Loughran | | | Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash | Major | fs | Xing Lin | Xing Lin | | | Upgrade snakeyaml to 1.32 | Major | security | Ashutosh Gupta | Ashutosh Gupta | | | hdfs ec command loads (administrator provided) erasure code policy files without disabling xml entity expansion | Major | erasure-coding, security | Jing | Ashutosh Gupta | | | SerialNumberMap should decrease current counter if the item exist | Major | namanode | ZanderXu | ZanderXu | | | datatables@1.10.17 sonatype-2020-0988 vulnerability | Major | ui | Eugene Shinn (Truveta) | Ashutosh Gupta | | | Revisit try blocks without catch blocks but having finally blocks | Minor | scheduler-load-simulator | Szilard Nemeth | Susheel Gupta | | | LogAggregationFileControllerFactory::getFileControllerForRead can leak threads | Blocker | log-aggregation | Rajesh Balamohan | Steve Loughran | | | An unhandled ArrayIndexOutOfBoundsException in DefaultStringifier.storeArray() if provided with an empty input | Minor | common, io | ConfX | ConfX | | | NumAllSinks metrics shows lower value than NumActiveSinks | Major | metrics | Akira Ajisaka | Ashutosh Gupta | | | Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId | Major | federation | Shilun Fan | Shilun Fan | | | S3A to support HTTPS web proxies | Major | fs/s3 | Mehakmeet Singh | Mehakmeet Singh | | | Fix typo in class StartEndTImesBase | Trivial | mrv2 | Samrat Deb | Samrat Deb | | | Fix NM class not found on Windows | Blocker | yarn | Gautham Banasandra | Gautham Banasandra | | | Initialization race condition with TemporaryAWSCredentialsProvider | Major | auth, fs/s3 | Jason Sleight | Jimmy Wong | | | Document Fix for

| Major | yarn | teng wang | teng wang | | | Maven parallel builds (skipping tests) fail | Critical | build | Steve Vaughan | Steve Vaughan | | | [Federation] Fix DefaultRequestInterceptorREST Client NPE | Major | federation, router | Shilun Fan | Shilun Fan | | | An unhandled NullPointerException in class KeyProvider | Major | common | ConfX | ConfX | | | PoolAlignmentContext should not max poolLocalStateId with sharedGlobalStateId when sending requests to the namenode. | Major | namnode | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Disable abfs prefetching by default | Major | fs/azure | Mehakmeet Singh | Mehakmeet Singh | | | StandbyCheckpointer can still trigger rollback fs image after RU is finalized | Major | hdfs | Lei Yang | Lei Yang | | | MutableGaugeFloat#incr(float) get stuck in an infinite loop | Major | metrics | asdfgh19 | Ashutosh Gupta | | | Interrupting RPC Client calls can lead to thread exhaustion | Critical | ipc | Owen O'Malley | Owen O'Malley | | | Display (fs -text) shouldn't hard-depend on Writable serialized sequence files. | Minor | fs | Harsh J | Ashutosh Gupta | | | Update command usage in FileSystemShell.md | Trivial | documentation | guophilipse | guophilipse | | | [SBN READ] Fix NPE when check the block location of empty directory | Major | namanode | Chenyu Zheng | Chenyu Zheng | | | [SBN read] Namenode in safe mode should not be transfered to observer state | Major | namanode | Tao Li | Tao Li | | | get\_executable in container-executor should provide meaningful error codes | Minor | container-executor | Miklos Szegedi | Susheel Gupta | | | Fix NPE in testAddRplicaProcessorForAddingReplicaInMap | Major | test | ZanderXu | ZanderXu | | | [RBF SBN] ConnectionManager should advance the client stateId for every request | Major | namanode, rbf | ZanderXu | ZanderXu | | | [ABFS]: Error introduced when SAS Token containing '?' prefix is passed | Minor | fs/azure | Sree Bhattacharyya | Sree Bhattacharyya | | | Add configuration flag to enable observer reads on routers without using ObserverReadProxyProvider | Critical | configuration | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | RBF: StateStore writer should not commit tmp fail if there was an error in writing the file. | Critical | hdfs, rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | [ABFS]: ITestAbfsManifestCommitProtocol fails on nonHNS configuration | Minor | fs/azure, test | Pranav Saxena | Sree Bhattacharyya | | | [SBN read] Improper cache-size for journal node may cause cluster crash | Major | journal-node | Tao Li | Tao Li | | | EC striped block is not sufficient when doing in maintenance | Major | ec, erasure-coding | dingshun | dingshun | | | [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace | Major | rbf | ZanderXu | ZanderXu | | | S3A committer NPE in spark job abort | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | YARN-10978. Fix ApplicationClassLoader to Correctly Expand Glob for Windows Path | Major | utils | Akshat Bordia | Akshat Bordia | | | Fix issue with classpath resolution | Critical | nodemanager | Gautham Banasandra | Gautham Banasandra | | | TestResourceTrackerService.testNodeRemovalNormally: Shutdown nodes should be 0 now expected: \<1\> but was: \<0\> | Major | yarn | Bence Kosztolnik | Bence Kosztolnik | | | Fix audit log duplicate issue when an ACE occurs in

| Major | fs | Beibei Zhao | Beibei Zhao | | | NFS Gateway may release buffer too early | Blocker | nfs | Attila Doroszlai | Attila Doroszlai | | | Changing log level of IOStatistics increment to make the DEBUG logs less noisy | Major | fs/s3 | Mehakmeet Singh | Mehakmeet Singh | | | Register the shutdown hook only when not in shutdown for KeyProviderCache constructor | Minor | hdfs | Xing Lin | Xing Lin | | | LogThrottlingHelper: the dependent recorder is not triggered correctly | Major | common | Chengbing Liu | Chengbing Liu | | | DiskBalancer process may throws IllegalArgumentException when the target DataNode has capital letter in hostname | Major | datanode | Daniel Ma | Daniel Ma | | | Standby NameNode crashes when transitioning to Active with in-progress tailer | Critical | namanode | ZanderXu | ZanderXu | | | [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time | Major | namanode, rbf | ZanderXu | ZanderXu | | | Resource Manager UI, cluster/appattempt/\*, can not present FINAL\_SAVING state | Critical | yarn | Bence Kosztolnik | Bence Kosztolnik | | | Incorrect WARN text in ACL check for application tag based placement | Minor | resourcemanager | Brian Goerlitz | Susheel Gupta | | | RBF. Truncate API always fails when dirs use AllResolver oder on Router | Major | rbf | Max Xie | Max Xie | | | ClientRMService implemented getCallerUgi and verifyUserAccessForRMApp methods but forget to use sometimes, caused audit log missing. | Major | yarn | Beibei Zhao | Beibei Zhao | | | Warn if AccessControlEnforcer runs for a long time to check permission | Major | namanode | Tsz-wo Sze | Tsz-wo Sze | | | Namenode doesn't use alignment context in TestObserverWithRouter | Major | hdfs, rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Handle Server KDC re-login when Server and Client run in same JVM. | Major | common | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Fix TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed | Major | configuration, namanode | Haiyang Hu | Haiyang Hu | | | Fix log throttling by declaring LogThrottlingHelper as static members | Major | namanode | Chengbing Liu | Chengbing Liu | | | Fix TestFsDatasetImpl#testConcurrentWriteAndDeleteBlock failed | Major | test | Haiyang Hu | Haiyang Hu | | | CS Mapping rule bug: User matcher does not work correctly for usernames with dot | Major | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Fix Junit Test ERROR Introduced By YARN-6412 | Major | api | Shilun Fan | Shilun Fan | | | Fix a typo in Trash | Minor | documentation | xiaoping.huang | xiaoping.huang | | | JobSubmissionFiles don't set right permission after mkdirs | Major | mrv2 | Zhang Dongsheng | Zhang Dongsheng | | | ObserverNamenode handles addBlock rpc and throws a FileNotFoundException | Critical | namanode | ZanderXu | ZanderXu | | | Garbage collect map entries in shared RouterStateIdContext using information from namenodeResolver instead of the map of active connectionPools. | Critical | rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Update wildfly openssl to 1.1.3.Final | Major | build, common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix build failure with docs profile | Major | build | Masatake Iwasaki | Masatake Iwasaki | | | maven site generation doesn't include javadocs | Blocker | site | Steve Loughran | Steve Loughran | | | Fix regression in HDFS-13522 that enables observer reads by

| Major | hdfs | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | [NFS GW] Fix regression after netty4 migration | Major | common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Cancel fileMonitoringTimer even if trustManager isn't defined | Major | common, test | Steve Vaughan | Steve Vaughan | | | Java 11 JavaDoc fails due to missing package comments | Major | build, common | Steve Loughran | Steve Vaughan | | | Avoid mixing canonical and non-canonical when performing comparisons | Minor | common, test | Steve Vaughan | Steve Vaughan | | | NamenodeHeartbeatService should use credentials of logged in user | Major | rbf | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN | Major | namanode | Haiyang Hu | Haiyang Hu | | | Namenode UI for Datanodes page not loading if any data node is down | Major | namenode, ui | Krishna Reddy | Zita Dombi | | | Namenode audit log to only include IP address of client | Major | namanode | Viraj Jasani | Viraj Jasani | | | Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method | Major | yarn | Susheel Gupta | Susheel Gupta | | | No need to clean tmp files in distcp direct mode | Major | tools/distcp | 10000kang | 10000kang | | | cyclonedx maven plugin breaks builds on recent maven releases (3.9.0) | Major | build | Steve Loughran | Steve Loughran | | | Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp | Critical | test | Ashutosh Gupta | Akira Ajisaka | | | LocalDirAllocator cannot recover from directory tree deletion during the life of a filesystem client | Minor | fs, fs/azure, fs/s3 | Steve Loughran | Steve Loughran | | | Fix ShuffleHandler tests | Major | tets | Tamas Domok | Tamas Domok | | | TestFsDatasetImpl.testReportBadBlocks brittle | Minor | test | Steve Loughran | Viraj Jasani | | | The getListing RPC will throw NPE if the path does not exist | Critical | namenode | ZanderXu | ZanderXu | | | HDFS Client hedged read has increased failure rate than without hedged read | Major | hdfs-client | Tom McCormick | Tom McCormick | | | Workflow priority mappings is case sensitive | Major | yarn | Aparajita Choudhary | Aparajita Choudhary | | | Fix the thread safety bug in LowRedundancyBlocks | Major | namanode | Shuyan Zhang | Shuyan Zhang | | | org.apache.hadoop.hdfs.tools.TestDFSAdmin#testAllDatanodesReconfig regression | Minor | dfsadmin, test | Steve Loughran | Shilun Fan | | | Send error to datanode if FBR is rejected due to bad lease | Major | datanode, namenode | Stephen O'Donnell | Stephen O'Donnell | | | Path capability probe for truncate is only honored by RawLocalFileSystem | Major | fs, httpfs, viewfs | Viraj Jasani | Viraj Jasani | | | A whitelist of endpoints to skip Kerberos authentication doesn't work for ResourceManager and Job History Server | Major | security | YUBI LEE | YUBI LEE | | | Add support for IBM Semeru OE JRE

and greater | Major | auth, common | Jack | Jack | | | ListFiles with recursive fails with FNF | Major | common | Ayush Saxena | Ayush Saxena | | | NPE in determineMissingParents when the queue is invalid | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems | Major | tools | Wei-Chiu Chuang | Szabolcs Gl | | | Insufficient heap during full test runs in Docker container. | Minor | build | Chris Nauroth | Chris Nauroth | | | Update ReadTransferRate to ReadLatencyPerGB for effective percentile metrics | Minor | datanode | Ravindra Dingankar | Ravindra Dingankar | | | Distcp with snapshot diff to support Ozone filesystem. | Major | distcp | Sadanand Shenoy | Sadanand Shenoy | | | [Federation] Add RM FederationStateStoreService Metrics | Major | federation, resourcemanager | Shilun Fan | Shilun Fan | | | Use the right Quantiles Array for Inverse Quantiles snapshot | Minor | datanode, metrics | Ravindra Dingankar | Ravindra Dingankar | | | RBF: The operation of renaming a multi-subcluster directory to a single-cluster directory should throw ioexception | Minor | rbf | Max Xie | Max Xie | | | EC: Fix locationBudget in getListing() | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | Wrong StringUtils.join() called in AbstractContractRootDirectoryTest | Trivial | test | Attila Doroszlai | Attila Doroszlai | | | ABFS should exclude incompatible credential providers | Major | fs/azure | Tamas Domok | Tamas Domok | | | FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage. | Major | namanode | Tsz-wo Sze | Tsz-wo Sze | | | Filesystem Spelling Mistake | Trivial | fs | Sebastian Baunsgaard | Sebastian Baunsgaard | | | MR Fetcher class to use an AtomicInteger to generate IDs. | Major | build, client | Steve Loughran | Steve Loughran | | | Fix lease interval comparison in BlockReportLeaseManager | Trivial | namenode | dzcxzl | dzcxzl | | | Consider changing label called "max resource" on UIv1 and UIv2 | Major | yarn, yarn-ui-v2 | Riya Khandelwal | Riya Khandelwal | | | Delete a snapshot may deleteCurrentFile | Major | snapshots | Tsz-wo Sze | Tsz-wo Sze | | | Fix bug of DRF comparison DominantResourceFairnessComparator2 in fair scheduler | Major | fairscheduler | Xiaoqiao He | Xiaoqiao He | | | Fix abundant Broken pipe exception in BlockSender | Minor | hdfs | fanluo | fanluo | | | Fix mvnsite on Windows 10 | Critical | build, site | Gautham Banasandra | Gautham Banasandra | | | RBF: The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. | Major | rbf | ZanderXu | ZanderXu | | | Create qbt.sh symlink on Windows | Critical | build | Gautham Banasandra | Gautham Banasandra | | | Fix wrong use of processFirstBlockReport() | Major | block placement | Shuyan Zhang | Shuyan Zhang | | | RM failover may fail when the nodes.exclude-path file does not exist | Minor | resourcemanager | dzcxzl | dzcxzl | | | Fix data missing issue when delete local block file. | Major | datanode | Chengwei Wang | Chengwei Wang | | | Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker | Major | resourcemanager | Chun Chen | Chun Chen | | | [UI2] Refresh buttons don't work after EmberJS upgrade | Minor | yarn-ui-v2 | Brian Goerlitz | Susheel Gupta | | |

raises NullPointerException | Minor | hdfs-client | Patrick Grandjean | Patrick Grandjean | | | Improve dfsclient log format | Minor | dfsclient | Xianming Lei | Xianming Lei | | | Add logs if resources are not available in NameNodeResourcePolicy | Minor | namenode | ECFuzz | ECFuzz | | | Usage of incorrect regex range A-z | Minor | httpfs | Marcono1234 | Nishtha Shah | | | Fix the exception message to print the Identifier pattern | Minor | httpfs | Nishtha Shah | Nishtha Shah | | | Fix the issue of arguments number limit in report command in DFSAdmin. | Major | dfsadmin | Haiyang Hu | Haiyang Hu | | | Install Python 3 for Windows 10 docker image | Major | build | Gautham Banasandra | Gautham Banasandra | | | JMX QueueMetrics breaks after mutable config validation in CS | Major | capacityscheduler | Tamas Domok | Tamas Domok | | | Potential infinite loop in TestDFSStripedOutputStreamUpdatePipeline.testDFSStripedOutputStreamUpdatePipeline | Major | test | Marcono1234 | Marcono1234 | | | RBF: Add supports for observer.auto-msync-period when using routers | Major | rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Fix flaky testFsCloseAfterClusterShutdown in TestFileCreation | Major | test | Uma Maheswara Rao G | Nishtha Shah | | | Fix concat operation doesn't honor dfs.permissions.enabled | Major | namenode | caozhiqiang | caozhiqiang | | | Fix the metric of "HttpPort" at DataNodeInfo | Minor | datanode | Zhaohui Wang | Zhaohui Wang | | | Optimize the logic for reconfigure slow peer enable for Namenode | Major | namanode | Haiyang Hu | Haiyang Hu | | | Erasure Coding: invalidate wrong block after reporting bad blocks from datanode | Critical | namenode | farmmamba | farmmamba | | | Fix several maven build warnings | Minor | build | Dongjoon Hyun | Dongjoon Hyun | | | ManifestCommitter OOM on azure job | Major | client | Steve Loughran | Steve Loughran | | | RBF: top real owners metrics can't been parsed json string | Minor | rbf | Max Xie | Nishtha Shah | | | RBF: Fix putAll impl for mysql and file based state stores | Major | rbf | Viraj Jasani | Viraj Jasani | | | File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted. | Major | namenode, snapshots | Tsz-wo Sze | Tsz-wo Sze | | | Applications submitted to ambiguous queue fail during recovery if "Specified" Placement Rule is used | Major | yarn | Susheel Gupta | Susheel Gupta | | | Race condition in closing FadvisedFileRegion | Major | yarn | Benjamin Teke | Benjamin Teke | | | Fix incorrect output path in javadoc build phase | Critical | build | Gautham Banasandra | Gautham Banasandra | | | Improve BlockPlacementPolicyRackFaultTolerant to avoid choose nodes failed when no enough Rack. | Major | namanode | Hualong Zhang | Hualong Zhang | | | Lock triple-beam to the version compatible with node.js 12 to avoid compilation error | Major | build | Ayush Saxena | Masatake Iwasaki | | | TestFSQueueConverter#testAutoCreateV2FlagsInWeightMode has a missing dot before

for method call assertNoValueForQueues | Major | yarn | Susheel Gupta | Susheel Gupta | | | EC: Add logic for striped blocks in isSufficientlyReplicated | Major | erasure-coding | Haiyang Hu | Haiyang Hu | | | S3A Committer only finalizes the commits in a single thread | Major | fs/s3 | Moditha Hewasinghage | Moditha Hewasinghage | | | EC: Fix bug in block recovery when there are stale datanodes | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | Intermittent TIME\_OUT of LogAggregationReport | Major | log-aggregation, resourcemanager, yarn | Adam Antal | Adam Antal | | | Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy | Major | hdfs | Xing Lin | Xing Lin | | | Incorrect exception handling during container recovery | Major | yarn | Peter Szucs | Peter Szucs | | | Close child file systems in ViewFileSystem when cache is disabled. | Major | fs | Shuyan Zhang | Shuyan Zhang | | | exception message is not intusive when accessing the job configuration web UI | Major | applicationmaster | Jiandan Yang | Jiandan Yang | | | Add Labeler Github Action. | Major | build | Ayush Saxena | Ayush Saxena | | | RBF: Optimize msync to only call nameservices that have observer reads enabled. | Major | rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Flexible AQC: setting capacity with leaf-template doesn't work | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | CS UI: queue filter do not work as expected when submitting apps with leaf queues name | Major | resourcemanager | Jiandan Yang | Jiandan Yang | | | RBF: Fix duplicate results of getListing through Router. | Major | rbf | Shuyan Zhang | Shuyan Zhang | | | NegativeArraySizeException when running MR jobs with large data size | Major | mrv1 | Peter Szucs | Peter Szucs | | | FS2CS not converts ACLs when all users are allowed | Major | yarn | Peter Szucs | Peter Szucs | | | Rectify the table length discrepancy in the DataNode UI. | Major | ui | Hualong Zhang | Hualong Zhang | | | abfs getFileStatus(/) fails with "Value for one of the query parameters specified in the request URI is invalid.", 400 | Major | fs/azure | Sergey Shabalov | Anuj Modi | | | EC: Fix the bug of failed lease recovery. | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | EC: Fix bug in updateBlockForPipeline after failover | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | Client may receive old state ID which will lead to inconsistent reads | Minor | rbf | Chunyi Yang | Chunyi Yang | | | RM format-conf-store should delete all the content of ZKConfigStore | Major | resourcemanager | Benjamin Teke | Benjamin Teke | | | EC: Fix wrong metadata in BlockInfoStriped after recovery | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | Fix block report lease issue to avoid missing some storages report. | Minor | namenode | Yanlei Yu | Yanlei Yu | | | timeline endpoint: /clusters/{clusterid}/apps/{appid}/entity-types Error when using hdfs store | Major | timelineservice | Jiandan Yang | Jiandan Yang | | | Fix TestRMFailover#testWebAppProxyInStandAloneMode Failed | Major | resourcemanager | Shilun Fan | Shilun Fan | | | RBF: Throwing NoNamenodesAvailableException for a long time, when failover | Major | rbf | Jian Zhang | Jian Zhang | | | A race in SerialNumberMap will cause wrong owner, group and XATTR | Major | namanode | ZanderXu | ZanderXu | | | Observer NameNode -observer startup option conflicts with -rollingUpgrade startup option | Minor | namenode | Danny Becker | Danny Becker | | | CURATOR-599 change broke functionality introduced in HADOOP-18139 and HADOOP-18709 | Major | common | Ferenc Erdelyi | Ferenc Erdelyi | | | ZKDelegationTokenSecretManager causes ArithmeticException due to improper numRetries value checking | Critical | common | ConfX | ConfX | | | EC: Fix bug of OIV processing XAttr. | Major | erasure-coding | Shuyan Zhang | Shuyan Zhang | | | RBF: We changed the

configuration of one router, the other routers stopped working | Major | rbf | Xiping Zhang | Xiping Zhang | | | mistakenly purge editLogs even after it is empty in NNStorageRetentionManager | Minor | namanode | ConfX | ConfX | | | RBF: fix bug of getRepresentativeQuorum when records have same dateModified | Major | rbf | Jian Zhang | Jian Zhang | | | Add config option to make container allocation prefer nodes without reserved containers | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Fix dependency convergence error on hbase2 profile | Major | buid, yarn | Masatake Iwasaki | Masatake Iwasaki | | | upgrade snappy-java to 1.1.10.4 due to CVE | Major | build | PJ Fanning | PJ Fanning | | | TestFsDatasetImpl missing null check when cleaning up | Critical | test | ConfX | ConfX | | | Correct comments to align with the code | Trivial | datanode | Yu Wang | Yu Wang | | | Fix performance issue of permission check in verifyAndCreateRemoteLogDir | Major | log-aggregation | Tamas Domok | Tamas Domok | | | Race condition in ZKDelegationTokenSecretManager creating znode | Major | common | Kevin Risden | Kevin Risden | | | Build failure while trying to create apache 3.3.7 release locally. | Critical | build | Mukund Thakur | PJ Fanning | | | RM process stuck after calling confStore.format() when ZK SSL/TLS is enabled, as netty thread waits indefinitely | Major | resourcemanager | Ferenc Erdelyi | Ferenc Erdelyi | | | fix same available space policy in AvailableSpaceVolumeChoosingPolicy | Major | hdfs | Fei Guo | Fei Guo | | | Modify HBase version in BUILDING.txt | Minor | common | Zepeng Zhang | Zepeng Zhang | | | Fix hadoop-yarn-client#java.lang.NoClassDefFoundError | Major | yarn-client | Shilun Fan | Shilun Fan | | | Remove IPCLoggerChannel Metrics when the logger is closed | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | HA: Safemode should exit when resources are from low to available | Major | ha | kuper | kuper | | | Potential data race introduced by HDFS-15865 | Major | dfsclient | Wei-Chiu Chuang | Segawa Hiroaki | | | NPE when getting the static files in SLSWebApp | Major | scheduler-load-simulator | Junfan Zhang | Junfan Zhang | | | Negative timeout in ZKFailovercontroller due to overflow | Major | common | ConfX | ConfX | | | [CS] Attempting to create Leaf Queue with empty shortname should fail without crashing RM | Major | capacity scheduler | Brian Goerlitz | Brian Goerlitz | | | Fix shaded client for building Hadoop on Windows | Major | hdfs-client | Gautham Banasandra | Gautham Banasandra | | | Fixed TestHistoryViewerPrinter flakiness during string comparison | Minor | test | Rajiv Ramachandran | Rajiv Ramachandran | | | Incorrect log4j properties file in SLS sample conf | Major | scheduler-load-simulator | Junfan Zhang | Junfan Zhang | | | QueueCapacityVectorInfo NPE when accesible labels config is used | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Fix TestDFSUtil.testIsValidName() unit test failure | Minor | test | liuguanghua | liuguanghua | | | S3A: AbstractS3ACostTest to clear bucket

flag | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | Fast fail when multiple attribute kvs are specified | Major | nodeattibute | Junfan Zhang | Junfan Zhang | | | RBF: Fix getFileInfo return wrong path when get mountTable path which multi-level | Minor | rbf | liuguanghua | liuguanghua | | | Web UI DN report shows random order when sorting with dead DNs | Minor | namenode, rbf, ui | Felix N | Felix N | | | The conf dfs.datanode.lifeline.interval.seconds is not considering time unit seconds | Major | datanode | Hemanth Boyina | Palakur Eshwitha Sai | | | Fix the logic for reconfigure slow peer enable for Namenode. | Major | namanode | Zhaobo Huang | Zhaobo Huang | | | RBF: Fix NoNamenodesAvailableException for a long time, when use observer | Major | rbf | Jian Zhang | Jian Zhang | | | RBF: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper client to get token in some case | Major | rbf | lei w | lei w | | | Fixed the verbose log.warn in DFSUtil.addTransferRateMetric() | Major | logging | Bryan Beaudreault | Ravindra Dingankar | | | RBF: Throwing an exception prevents the permit from being released when using FairnessPolicyController | Major | rbf | Jian Zhang | Jian Zhang | | | Detect order dependent flakiness in TestViewfsWithNfs3.java under hadoop-hdfs-nfs module | Minor | nfs, test | Ruby | Ruby | | | Possible ConcurrentModificationException if Exec command fails | Major | common | Attila Doroszlai | Attila Doroszlai | | | Fix missing comma in HistoryServerRest.html response body | Minor | documentation | wangzhongwei | wangzhongwei | | | Fix a typo in DataStorage.java | Trivial | datanode | Yu Wang | Yu Wang | | | EC: Fix verifyClusterSetup output in case of an invalid param. | Major | erasure-coding | Ayush Saxena | Zhaobo Huang | | | Fix NPE in DataNode.handleBadBlock and BlockSender | Major | datanode | Haiyang Hu | Haiyang Hu | | | EC: Fix int overflow in calculating numEcReplicatedTasks and numReplicationTasks during block recovery | Major | ec, namenode | Hualong Zhang | Hualong Zhang | | | NullPointerException in Hadoop Credential Check CLI Command | Major | common | Anika Kelhanka | Anika Kelhanka | | | DataSetLockManager.lockLeakCheck() is not thread-safe. | Minor | datanode | liuguanghua | liuguanghua | | | RBF: Fix Router Safemode check contidition error | Major | rbf | liuguanghua | liuguanghua | | | QueueCapacityConfigParser shouldn't ignore capacity config with 0 memory | Major | capacityscheduler | Tamas Domok | Tamas Domok | | | HDFS: add client rpc backoff metrics due to disconnection from lowest priority queue | Major | metrics | Lei Yang | Lei Yang | | | upgrade sshd-core due to CVEs | Major | build, common | PJ Fanning | PJ Fanning | | | ConcurrentModificationException and NPE in PriorityUtilizationQueueOrderingPolicy | Major | capacity scheduler | Ferenc Erdelyi | Ferenc Erdelyi | | | Class loader leak caused by StatisticsDataReferenceCleaner thread | Major | common | Jia Fan | Jia Fan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Remove duplication of checking for invalid application ID in TestLogsCLI | Trivial | test | Hudky Mrton Gyula | Hudky Mrton Gyula | | | MiniMRYarnCluster has hard-coded timeout waiting to start history server, with no way to disable | Major | test | Nick Dimiduk | Masatake Iwasaki | | | Fix TestLongLong#testRightShift | Minor | test | Wanqiang Ji | Wanqiang Ji | | | Remove useless dfs.webhdfs.enabled | Minor | test | Hui Fei | Hui Fei | | | Move personality file from Yetus to Hadoop repository | Major | test, yetus | Chao Sun | Chao Sun | | | Add Test annotation

TestPersistBlocks#testRestartDfsWithSync | Minor | hdfs | Hui Fei | Hui Fei | | | Complement initialize member variables in TestHdfsConfigFields#initializeMemberVariables | Minor | test | Lisheng Sun | Lisheng Sun | | | Erasure Coding: Add rs and rs-legacy codec test for addPolicies | Minor | erasure-coding, test | Hui Fei | Hui Fei | | | TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently | Major | yarn | Prabhu Joseph | Peter Bacsko | | | Add lz4-java as hadoop-hdfs test dependency | Major | test | L. C. Hsieh | L. C. Hsieh | | | Deprecated the residual nested class for the LCEResourceHandler | Major | nodemanager | Wanqiang Ji | Wanqiang Ji | | | Test case TestOfflineImageViewer fails | Minor | test | Hui Fei | Hui Fei | | | Flaky test TestBalancer#testBalancerWithSortTopNodes() | Major | balancer & mover, test | Viraj Jasani | Viraj Jasani | | | TestErasureCodingCLI fails | Minor | test | Hui Fei | Hui Fei | | | Stop RMService in TestClientRedirect.testRedirect() | Minor | test | Zhengxi Li | Zhengxi Li | | | Fix non-idempotent test in TestTaskProgressReporter | Minor | test | Zhengxi Li | Zhengxi Li | | | testBalancerWithObserverWithFailedNode times out | Trivial | test | Leon Gao | Leon Gao | | | Add more test for the BuiltInGzipCompressor | Major | test | L. C. Hsieh | L. C. Hsieh | | | TestLocalFSCopyFromLocal.testDestinationFileIsToParentDirectory failure after reverting HADOOP-16878 | Major | test | Chao Sun | Chao Sun | | | Make TestViewfsWithNfs3.testNfsRenameSingleNN() idempotent | Minor | nfs | Zhengxi Li | Zhengxi Li | | | TestAMRMClient#testAMRMClientWithContainerResourceChange fails intermittently | Major | yarn | Ray Chiang | Andras Gyori | | | Test coverage for Async profiler servlets | Minor | common | Viraj Jasani | Viraj Jasani | | | TestYarnConfigurationFields consistently keeps failing | Minor | test | Viraj Jasani | Viraj Jasani | | | Fix test TestDFSStripedInputStreamWithRandomECPolicy | Minor | test | daimin | daimin | | | TestHDFSCLI#testAll consistently failing | Major | test | Viraj Jasani | Viraj Jasani | | | Add unit test for FINISHED\CONTAINERS\PULLED\BY\AM event on DECOMMISSIONING | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | Unit tests aren't checking for PMDK availability | Major | test | Steve Vaughan | Steve Vaughan | | | Prevent resource leaks in TestClientRMService. | Minor | test | Chris Nauroth | Chris Nauroth | | | Document TestContainerResourceUsage#waitForContainerCompletion | Major | resourcemanager, test | Karthik Kambatla | Susheel Gupta | | | Add a subtree test to TestSnapshotDiffReport | Minor | test | Tsz-wo Sze | Tsz-wo Sze | | | Add a unit test | Minor | client | Lu Yuan | Lu Yuan | | | Fix intermittently failing unit test: TestAMRMProxy.testAMRMProxyTokenRenewal | Major | yarn | Susheel Gupta | Susheel Gupta | | | Close webhdfs during the teardown | Major | hdfs | Steve Vaughan | Steve Vaughan | | | Fix junit dependency for running parameterized tests in hadoop-hdfs-rbf | Major | . | Takanobu Asanuma | Takanobu Asanuma | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | ABFS: Bug fix for getPathProperties | Major | fs/azure | Da Zhou | Da Zhou | | |

testBalancerRPCDelayQpsDefault fails intermittently | Minor | balancer, test | Ahmed Hussein | Ahmed Hussein | | | RBF: Impose directory level permissions for Mount entries | Major | rbf | Xiaoqiao He | Xiaoqiao He | | | FS-CS converter: don't enable auto-create queue property for root | Critical | fairscheduler | Peter Bacsko | Peter Bacsko | | | Prevent Fatal CancelledException in TimelineV2Client when stopping | Major | ATSv2 | Tarun Parimi | Tarun Parimi | | | ABFS: Avoid storage calls to check if the account is HNS enabled or not | Minor | fs/azure | Bilahari T H | Bilahari T H | | | TimelineConnector does not destroy the jersey client | Major | ATSv2 | Prabhu Joseph | Tanu Ajmera | | | YARN RMWebServices /scheduler-conf/validate leaks ZK Connections | Blocker | capacityscheduler | Akhil PB | Prabhu Joseph | | | Endpoint for obtaining direct URL for the logs | Major | yarn | Adam Antal | Andras Gyori | | | Adding RM Cluster Id in ApplicationReport | Major | applications, federation | Giovanni Matteo Fumarola | Bilwa S T | | | Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests | Major | federation, router, test | Giovanni Matteo Fumarola | Bilwa S T | | | Erasure Coding: metrics xmitsInProgress become to negative. | Major | datanode, erasure-coding | Baolong Mao | Baolong Mao | | | Extend ViewFS and provide ViewFSOverloadScheme implementation with scheme configurable. | Major | fs, hadoop-client, hdfs-client, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: skip increment properties for mem/vcores and fix DRF check | Major | fairscheduler, fs-cs | Peter Bacsko | Peter Bacsko | | | Add tests for reading fair call queue capacity weight configs | Major | ipc | Fengnan Li | Fengnan Li | | | PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value | Minor | federation | Akshay Agarwal | Bilwa S T | | | Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement | Major | capacityscheduler | Prabhu Joseph | Prabhu Joseph | | | Make mount-table to read from central place ( Let's say from HDFS) | Major | configuration, hadoop-client | Uma Maheswara Rao G | Uma Maheswara Rao G | | | RBF: Check each component length of destination path when add/update mount entry | Major | rbf | Xiaoqiao He | Xiaoqiao He | | | RBF: Implement BalanceProcedureScheduler basic framework | Major | rbf | Jinglun | Jinglun | | | Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. | Major | fs, nflyFs, viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping | Major | capacity scheduler, fs-cs | Prabhu Joseph | Gergely Pollk | | | ABFS: FS initialize fails for incompatible account-agnostic Token Provider setting | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Make DFSAdmin tool to work with ViewFSOverloadScheme | Major | dfsadmin, fs, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Add lazy initialization of LogAggregationFileControllerFactory in LogServlet | Major | log-aggregation, yarn | Adam Antal | Adam Antal | | | Document the ViewFSOverloadScheme details in ViewFS guide | Major | viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme | Major | dfsadmin, viewfsOverloadScheme | Ayush Saxena | Ayush Saxena | | | Add all available fs.viewfs.overload.scheme.target.\<scheme\>.impl classes in core-default.xml

| Major | configuration, viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Reserved Containers not allocated from available space of other nodes in CandidateNodeSet in MultiNodePlacement (YARN-10259) | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | FSUsage$DF should consider ViewFSOverloadScheme in processPath | Minor | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler | Major | fairscheduler | Benjamin Teke | Benjamin Teke | | | FedBalance tool implementation | Major | rbf | Jinglun | Jinglun | | | [JDK11] Support JDK11 in the precommit job | Major | build | Akira Ajisaka | Akira Ajisaka | | | ABFS: Improve the ABFS driver documentation | Minor | fs/azure | Bilahari T H | Bilahari T H | | | ViewFileSystemOverloadScheme should represent mount links as non symlinks | Major | hdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | ABFS: Change in User-Agent header | Minor | fs/azure | Bilahari T H | Bilahari T H | | | Support max running app logic for CapacityScheduler | Major | capacity scheduler, capacityscheduler | zhoukang | Peter Bacsko | | | Javadocs fails for hadoop-federation-balance | Minor | documentation | Xieming Li | Xieming Li | | | Merged ListStatus with Fallback target filesystem and InternalDirViewFS. | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | FS-CS converter: convert maxAppsDefault, maxRunningApps settings | Major | fairscheduler, fs-cs | Peter Bacsko | Peter Bacsko | | | ABFS: Fix idempotency test failures when SharedKey is set as AuthType | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | ABFS: Make PUT and POST operations idempotent | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | mkdirs should work when parent dir is internalDir and fallback configured. | Major | hdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call | Major | federation | Giovanni Matteo Fumarola | Bilwa S T | | | Default mount table name used by ViewFileSystem should be configurable | Major | viewfs, viewfsOverloadScheme | Virajith Jalaparti | Virajith Jalaparti | | | Add separated config file hdfs-fedbalance-default.xml for fedbalance tool | Major | rbf | Jinglun | Jinglun | | | Add documentation for fedbalance tool | Major | documentation, rbf | Jinglun | Jinglun | | | Document max-parallel-apps for Capacity Scheduler | Major | capacity scheduler, capacityscheduler | Peter Bacsko | Peter Bacsko | | | ABFS: Adding metrics to AbfsInputStream (AbfsInputStreamStatistics) | Major | fs/azure | Gabor Bota | Mehakmeet Singh | | | create should work when parent dir is internalDir and fallback

| Major | hdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Fix NN trash emptier to work if ViewFSOveroadScheme enabled | Major | namenode, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Replace Guava Optional with Java8+ Optional | Major | build, common | Ahmed Hussein | Ahmed Hussein | | | RBF: Get the datanode report from cache for federation WebHDFS operations | Major | federation, rbf, webhdfs | Ye Ni | Ye Ni | | | Optionally ignore port number in mount-table name when picking from initialized uri | Major | hdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | TestRMHATimelineCollectors fails on hadoop trunk | Major | test, yarn | Ahmed Hussein | Bilwa S T | | | Add fs.viewfs.overload.scheme.target.ofs.impl to core-default.xml | Major | configuration, viewfs, viewfsOverloadScheme | Siyao Meng | Siyao Meng | | | ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links | Major | viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Replace Guava Function with Java8+ Function | Major | build | Ahmed Hussein | Ahmed Hussein | | | Replace Guava Predicate with Java8+ Predicate | Minor | build | Ahmed Hussein | Ahmed Hussein | | | Ordered snapshot deletion: make it a configurable feature | Major | snapshots | Tsz-wo Sze | Tsz-wo Sze | | | When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target

| Major | hdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Replace Guava Supplier with Java8+ Supplier in Hadoop | Major | build | Ahmed Hussein | Ahmed Hussein | | | Ordered snapshot deletion: record snapshot deletion in XAttr | Major | snapshots | Tsz-wo Sze | Shashikant Banerjee | | | ABFS: Fix For Idempotency code | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Avoid sending RMNodeResourceupdate event if resource is same | Major | graceful | Bibin Chundatt | Sushil Ks | | | s3a create() doesn't check for an ancestor path being a file | Major | fs/s3 | Steve Loughran | Sean Mackrory | | | Add a command to list all snapshots for a snaphottable root with snapshot Ids | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Ordered snapshot deletion: garbage collect deleted snapshots | Major | snapshots | Tsz-wo Sze | Tsz-wo Sze | | | Show snapshots deletion status in snapList cmd | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | [JDK11] Fix Javadoc errors | Major | build | Uma Maheswara Rao G | Akira Ajisaka | | | [Federation] Client should be able to submit application to RM directly using normal client conf | Major | amrmproxy, federation | JohnsonGuo | Bilwa S T | | | Make snapshot limit on global as well per snapshot root directory configurable | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Refactor S3A Listing code for better isolation | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | [JDK 11] Fix javadoc error in Java API link detection | Major | build | Akira Ajisaka | Akira Ajisaka | | | ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic | Minor | fs/azure, test | Sneha Vijayarajan | Bilahari T H | | | ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey | Minor | fs/azure | Sneha Vijayarajan | Bilahari T H | | | ABFS: Add debug log for rename failures | Major | fs/azure | Bilahari T H | Bilahari T H | | | Make trash root inside each snapshottable directory | Major | hdfs, hdfs-client | Siyao Meng | Siyao Meng | | | Wrong operation name in FsNamesystem for listSnapshots | Major | snapshots | Mukul Kumar Singh | Aryan Gupta | | | Add UI for deleted snapshots | Major | snapshots | Mukul Kumar Singh | Vivek Ratnavel Subramanian | | | Add edit log entry for Snapshot deletion GC thread snapshot deletion | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Ordered snapshot deletion: Disallow rename between two snapshottable directories | Major | snapshots | Tsz-wo Sze | Shashikant Banerjee | | | Make trash root inside each snapshottable directory for WebHDFS | Major | webhdfs | Siyao Meng | Siyao Meng | | | Provide DFS API compatible class(ViewDistributedFileSystem), but use ViewFileSystemOverloadScheme inside | Major | dfs, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Support Multi Node Placement in SingleConstraintAppPlacementAllocator | Major | capacityscheduler, multi-node-placement | Prabhu Joseph | Prabhu Joseph | | | Yarn logs CLI filtering by application attempt | Trivial | yarn | Adam Antal | Hudky Mrton Gyula | | | Create an endpoint for remote application log directory path query | Minor | yarn | Andras Gyori | Andras Gyori | | | webUI update to allow end users to request thread dump | Major | nodemanager | Ming Ma | Siddharth Ahuja | | | In-order deletion of snapshots: Diff lists must be update only in the last snapshot | Major | snapshots | Mukul Kumar Singh | Tsz-wo Sze | | | Namenode UI: List snapshots in separate table for each snapshottable directory | Major | ui | Vivek Ratnavel Subramanian | Vivek Ratnavel Subramanian | | | Extract MockQueueHierarchyBuilder to a separate class | Major | resourcemanager, test | Gergely Pollk | Gergely Pollk | | | Improve MockQueueHierarchyBuilder to detect queue ambiguity | Major | resourcemanager, test | Gergely Pollk | Gergely Pollk | | | Create variable context class for CS queue mapping rules | Major | yarn | Gergely Pollk | Gergely Pollk | | | Create Matchers for CS mapping rules | Major | yarn | Gergely Pollk | Gergely Pollk | | | Add identified snapshot corruption tests for ordered snapshot deletion | Major | snapshots, test | Shashikant Banerjee | Shashikant Banerjee | | | Create new JSON schema for Placement Rules | Major | capacity scheduler, capacityscheduler | Peter Bacsko | Peter Bacsko | | | Create Actions for CS mapping rules | Major | yarn | Gergely Pollk | Gergely Pollk | | | Create MappingRule class to represent each CS mapping rule | Major | yarn | Gergely Pollk | Gergely Pollk | | | CS Mapping rule config parser should return MappingRule objects | Major | yarn | Gergely Pollk | Gergely Pollk | | | getChildFilesystems should include fallback fs as well | Critical | viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules | Major | yarn | Gergely Pollk | Gergely Pollk | | | RESOURCE\_UPDATE event was repeatedly registered in DECOMMISSIONING state | Minor | resourcemanager | yehuanhuan | yehuanhuan | | | Create an allowCreate flag for MappingRuleAction | Major | resourcemanager, scheduler | Gergely Pollk | Gergely Pollk | | | ViewDistributedFileSystem#recoverLease should call

when there are no mounts configured | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Create a group matcher which checks ALL groups of the user | Major | resourcemanager, scheduler | Gergely Pollk | Gergely Pollk | | | Handle transient stream read failures in FileSystem contract tests | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Implement logic which returns MappingRule objects based on mapping rules | Major | capacity scheduler, resourcemanager | Peter Bacsko | Peter Bacsko | | | Incorrect getTrashRoot return value when a non-snapshottable dir prefix matches the path of a snapshottable dir | Major | snapshots | Nilotpal Nandi | Siyao Meng | | | Tiny Improve for DeadNode detector | Minor | hdfs-client | dark\_num | imbajin | | | RBF: Refresh cacheNS when SocketException occurs | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | listFiles on root/InternalDir will fail if fallback root has file | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | When disallowing snapshot on a dir, throw exception if its trash root is not empty | Major | hdfs | Siyao Meng | Siyao Meng | | | namenode start failed to start when dfs.namenode.snapshot.max.limit set | Major | snapshots | Nilotpal Nandi | Shashikant Banerjee | | | Fix the rename issues with fallback fs enabled | Major | viewfs, viewfsOverloadScheme | Uma Maheswara Rao G | Uma Maheswara Rao G | | | ViewDFS#getDelegationToken should not throw UnsupportedOperationException. | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility | Major | resourcemanager, test | Benjamin Teke | Benjamin Teke | | | ABFS: Support for conditional overwrite | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | RBF: TestRouterRpc#testErasureCoding is flaky | Major | rbf | Chen Zhang | Chen Zhang | | | ABFS: Test testNegativeScenariosForCreateOverwriteDisabled fails for non-HNS account | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | namenode fails to start when ordered snapshot deletion feature is disabled | Major | snapshots | Nilotpal Nandi | Shashikant Banerjee | | | ViewHDFS#create(f, permission, cflags, bufferSize, replication, blockSize, progress, checksumOpt) should not be restricted to DFS only. | Major | hdfs-client | Uma Maheswara Rao G | Uma Maheswara Rao G | | | ViewHDFS#canonicalizeUri should not be restricted to DFS only API. | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Change fs2cs to generate mapping rules in the new format | Major | fs-cs, scheduler | Peter Bacsko | Peter Bacsko | | | Create trash dir when allowing snapshottable dir | Major | hdfs | Siyao Meng | Siyao Meng | | | RBF: Router FSCK fails after HDFS-14442 | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | Add list Snapshot command in WebHDFS | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Implement FileSystem.listStatusIterator() in S3AFileSystem | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer | Major | rbf | Baolong Mao | Hui Fei | | | Namenode trashEmptier should not init ViewFs on startup | Major | namenode, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Add applicationName policy | Major | capacity scheduler, resourcemanager | Peter Bacsko | Peter Bacsko | | | RBF: Fix test failures after HADOOP-17281 | Major | rbf, test | Akira Ajisaka | Akira Ajisaka | | | Initialize snapshot trash root during NameNode startup if enabled | Major | namanode, snapshots | Siyao Meng | Siyao Meng | | | ABFS: Test failure

| Major | fs/azure, test | Bilahari T H | Bilahari T H | | | ABFS: read-ahead error reporting breaks buffer management | Critical | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Use shaded guava from thirdparty | Major | common, hadoop-thirdparty | Ayush Saxena | Ayush Saxena | | | [JDK11] Fix javadoc errors in hadoop-common module | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Update the checkstyle config to ban some guava functions | Major | build | Akira Ajisaka | Akira Ajisaka | | | RBF: Fix wrong client IP info in CallerContext when requests mount points with multi-destinations. | Major | rbf | Chengwei Wang | Chengwei Wang | | | TestBlockTokenWithDFSStriped fails intermittently | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | Add diff threshold to FedBalance | Major | rbf | Jinglun | Jinglun | | | TestDFSClientRetries#testGetFileChecksum fails intermittently | Major | dfsclient, test | Ahmed Hussein | Ahmed Hussein | | | TestHAAppend#testMultipleAppendsDuringCatchupTailing is flaky | Major | test | Vinayakumar B | Ahmed Hussein | | | TestFsDatasetImpl fails intermittently | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | TestFileCreation#testServerDefaultsWithMinimalCaching fails intermittently | Major | hdfs, test | Ahmed Hussein | Ahmed Hussein | | | RBF: TestRouter#testNamenodeHeartBeatEnableDefault fails by BindException | Major | rbf, test | Akira Ajisaka | Akira Ajisaka | | | TestBPOfferService#testMissBlocksWhenReregister fails intermittently | Major | datanode | Ahmed Hussein | Ahmed Hussein | | | Update CS MappingRule documentation with the new format and features | Major | capacity scheduler, documentation | Gergely Pollk | Peter Bacsko | | | EC: Fix checksum computation in case of native encoders | Blocker | erasure-coding | Ahmed Hussein | Ayush Saxena | | | Allow configuring DISK/ARCHIVE storage types on same device mount | Major | datanode | Leon Gao | Leon Gao | | | Harmonize guava version and shade guava in yarn-csi | Major | common | Wei-Chiu Chuang | Akira Ajisaka | | | Replace the legacy placement engine in CS with the new one | Major | capacity scheduler, resourcemanager | Gergely Pollk | Gergely Pollk | | | TestBPOfferService#testMissBlocksWhenReregister fails on trunk | Major | datanode, test | Ahmed Hussein | Masatake Iwasaki | | | FS-CS converter: handle case when weight=0 and allow more lenient capacity checks in Capacity Scheduler | Major | yarn | Peter Bacsko | Peter Bacsko | | | Add a configuration switch to change between legacy and JSON placement rule format | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | ViewFileSystemOverloadScheme support specifying mount table loader imp through conf | Major | viewfsOverloadScheme | Junfan Zhang | Junfan Zhang | | | [JDK 11] mvn package -Pdocs fails | Major | build, documentation | Akira Ajisaka | Akira Ajisaka | |

TestRouterRpcMultiDestination#testGetCachedDatanodeReport fails on trunk | Major | rbf, test | Ahmed Hussein | Masatake Iwasaki | | | allow/disallowSnapshot on EZ roots shouldn't fail due to trash provisioning/emptiness check | Major | hdfs | Siyao Meng | Siyao Meng | | | TestUpgradeDomainBlockPlacementPolicy flaky | Major | namenode, test | Ahmed Hussein | Ahmed Hussein | | | Import logic of multi-node allocation in CapacityScheduler | Critical | capacity scheduler | Wangda Tan | Qi Zhu | | | Create a general purpose log request with additional query parameters | Major | yarn | Adam Antal | Andras Gyori | | | RMAppManager CS Placement ignores parent path | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | For Federation, we should support getApplicationAttemptReport. | Major | federation, router | Qi Zhu | Qi Zhu | | | TestReconstructStripedFile#testNNSendsErasureCodingTasks fails intermittently | Major | erasure-coding | Toshihiko Uchida | Hemanth Boyina | | | TestFileChecksum should be parameterized | Major | test | Ahmed Hussein | Masatake Iwasaki | | | RBF: Move the router related part from hadoop-federation-balance module to hadoop-hdfs-rbf. | Major | rbf | Jinglun | Jinglun | | | RBF: MockResolver.getMountPoints() breaks the semantic of FileSubclusterResolver. | Major | rbf | Jinglun | Jinglun | | | Add the capability to fs2cs to write the converted placement rules inside capacity-scheduler.xml | Major | capacity scheduler | Peter Bacsko | Peter Bacsko | | | S3A Input Stream bytes read counter isn't getting through to StorageStatistics/instrumentation properly | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Fix intermittent falilure of TestDecommission#testAllocAndIBRWhileDecommission | Minor | hdfs, test | Masatake Iwasaki | Masatake Iwasaki | | | Implement weight mode in Capacity Scheduler | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Remove "experimental" warning message from fs2cs | Major | scheduler | Peter Bacsko | Peter Bacsko | | | Fix dependency exclusion problem in poms | Critical | buid, resourcemanager | Peter Bacsko | Peter Bacsko | | | RBF: Isolation/Fairness documentation | Major | rbf | CR Hota | Fengnan Li | | | TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStream fails intermittently | Minor | hdfs, test | Toshihiko Uchida | Toshihiko Uchida | | | Add weight mode conversion to fs2cs | Major | capacity scheduler, fs-cs | Qi Zhu | Peter Bacsko | | | TestBalancerWithMultipleNameNodes#testBalancingBlockpoolsWithBlockPoolPolicy fails on trunk | Major | balancer, test | Ahmed Hussein | Masatake Iwasaki | | | Update queue creation logic to use weight mode and allow the flexible static/dynamic creation | Major | capacity scheduler, resourcemanager | Benjamin Teke | Andras Gyori | | | Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount | Major | datanode | Leon Gao | Leon Gao | | | Fix the FindBugs warning introduced in YARN-10506 | Major | capacity scheduler, resourcemanager | Gergely Pollk | Gergely Pollk | | | Make queue placement in CapacityScheduler compliant with auto-queue-placement | Major | capacity scheduler | Wangda Tan | Gergely Pollk | | | Enhance placement rule conversion in fs2cs in weight mode and enable it by default | Major | capacity scheduler | Peter Bacsko | Peter Bacsko | | | CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include mode of operation for CS | Major | capacity scheduler | Benjamin Teke | Szilard Nemeth | | | Fix Auto Queue Creation parent handling | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include weight values for queues | Major | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | RBF: Router federation rename of directory. | Major | rbf | Jinglun | Jinglun | | | Allow static definition of childless ParentQueues with auto-queue-creation-v2 enabled | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Be able to disable user limit factor for CapacityScheduler Leaf Queue | Major | capacity scheduler | Wangda Tan | Qi Zhu | | | Fix AutoCreateLeafQueueCreation cap related caculation when in absolute

| Major | capacity scheduler | Qi Zhu | Qi Zhu | | | CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | fs2cs should generate new "auto-queue-creation-v2.enabled" properties for all parents | Major | resourcemanager | Peter Bacsko | Peter Bacsko | | | Convert root queue in fs2cs weight mode conversion | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Replace HTrace with No-Op tracer | Major | common | Siyao Meng | Siyao Meng | | | Support auto queue creation without mapping rules | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Add queue-mappings-override.enable property in FS2CS conversions | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Create a class which can convert from legacy mapping rule format to the new JSON format | Major | resourcemanager, scheduler | Gergely Pollk | Gergely Pollk | | | Skip schedule on not heartbeated nodes in Multi Node Placement | Major | scheduler | Prabhu Joseph | Prabhu Joseph | | | Fix findbugs issue introduced in YARN-10585 | Major | scheduler | Gergely Pollk | Gergely Pollk | | | [JDK 16] KerberosUtil#getOidInstance is broken by JEP 396 | Major | auth | Akira Ajisaka | Akira Ajisaka | | | Fix Auto Queue Creation hierarchy construction to use queue path instead of short queue name | Critical | yarn | Andras Gyori | Andras Gyori | | | Ensure snapshot root trash provisioning happens only post safe mode exit | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Allow configuring DISK/ARCHIVE capacity for individual volumes | Major | datanode | Leon Gao | Leon Gao | | | Rename snapshots while marking them deleted | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | Fix TestFsDatasetImpl.testReadLockCanBeDisabledByConfig | Minor | test | Leon Gao | Leon Gao | | | CS Mapping Rule %specified rule catches default submissions | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | fs2cs: parentQueue for certain placement rules are not set during conversion | Major | capacity scheduler | Peter Bacsko | Peter Bacsko | | | Add OutputStream + Syncable to the Filesystem Specification | Major | fs | Steve Loughran | Steve Loughran | | | Support max queues limit configuration in new auto created queue, consistent with old auto created. | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Fix preemption policy to exclude childless ParentQueues | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | RBF: Fix contract tests after HADOOP-13327 | Major | rbf | Akira Ajisaka | Akira Ajisaka | | | Support disabling buffered reads in ABFS positional reads | Major | fs/azure | Anoop Sam John | Anoop Sam John | | | add guava BaseEncoding to illegalClasses | Major | build, common | Ahmed Hussein | Ahmed Hussein | | | Remove the usage of

| Major | hdfs-common | Akira Ajisaka | Akira Ajisaka | | | CSMapping rule can return paths with empty parts | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | CS Auto Queue creation should reject submissions with empty path parts | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | CS Flexible Auto Queue Creation RM UIv2 modifications | Major | capacity scheduler, resourcemanager, ui | Benjamin Teke | Andras Gyori | | | RBF: Router fails to start due to NoClassDefFoundError for hadoop-federation-balance | Major | rbf | Takanobu Asanuma | Takanobu Asanuma | | | create client protocol: add ecPolicyName & storagePolicy param to debug statement string | Minor | erasure-coding, namanode | Bhavik Patel | Bhavik Patel | | | Migrate to Python 3 and upgrade Yetus to 0.13.0 | Major | common | Akira Ajisaka | Akira Ajisaka | | | Add metrics for how blocks are moved in replaceBlock | Minor | datanode | Leon Gao | Leon Gao | | | Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue) | Major | documentation | Qi Zhu | Qi Zhu | | | Extend logging to give more information about weight mode | Major | yarn | Benjamin Teke | Benjamin Teke | | | Limit queue creation depth relative to its first static parent | Major | yarn | Andras Gyori | Andras Gyori | | | Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used | Major | capacity scheduler | Wangda Tan | Qi Zhu | | | Queueinfo related capacity, should adjusted to weight mode. | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Adjust the queue Configured capacity to Configured weight number for weight mode in UI. | Major | capacity scheduler, ui | Qi Zhu | Qi Zhu | | | Snapshot Operations: Add debug logs at the entry point | Minor | snapshots | Bhavik Patel | Bhavik Patel | | | Move CS placement rule related changes to a separate package | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | ABFS: Toggle Store Mkdirs request overwrite parameter | Major | fs/azure | Sumangala Patki | Sumangala Patki | | | Fix the findbugs issues in extractFloatValueFromWeightConfig. | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Fix TestCapacitySchedulerAutoQueueCreation#testAutoQueueCreationFailsForEmptyPathWithAQCAndWeightMode | Major | capacity scheduler | Qi Zhu | Qi Zhu | | | Improve the Logs for File Concat Operation | Minor | namenode | Bhavik Patel | Bhavik Patel | | | TestBalancer#testMaxIterationTime fails sporadically | Major | balancer, test | Jason Darrell Lowe | Toshihiko Uchida | | | ClusterMetrics should support GPU capacity related metrics. | Major | metrics, resourcemanager | Qi Zhu | Qi Zhu | | | Improve CS MappingRule %secondary\_group evaluation | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | Add Node GPU Utilization and apply to NodeMetrics. | Major | gpu | Qi Zhu | Qi Zhu | | | Refactor the max app related update, and fix maxApplications update error when add new queues. | Critical | capacity scheduler | Qi Zhu | Qi Zhu | | | fs2cs should generate auto-created queue deletion properties | Major | scheduler | Qi Zhu | Qi Zhu | | | Improve the log for HTTPFS server operation | Minor | httpfs | Bhavik Patel | Bhavik Patel | | | ClusterMetrics should support custom resource capacity related

| Major | metrics | Qi Zhu | Qi Zhu | | | In Federation Router Nodes/Applications/About pages throws 500 exception when https is enabled | Critical | federation | Sushanta Sen | Bilwa S T | | | CSMappingPlacementRule should not create new instance of Groups | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | Improve the log for the Storage Policy Operations | Minor | namenode | Bhavik Patel | Bhavik Patel | | | NodesListManager event improvement | Critical | resourcemanager | Bibin Chundatt | Qi Zhu | | | Some tests in TestBlockRecovery are consistently failing | Major | test | Viraj Jasani | Viraj Jasani | | | Remove dangling dynamic queues on reinitialization | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Support Auto Queue Creation template configurations | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Add cluster metric for amount of CPU used by RM Event Processor | Minor | yarn | Jim Brennan | Jim Brennan | | | Support queue capacity in terms of absolute resources with custom resourceType. | Critical | gpu | Qi Zhu | Qi Zhu | | | [JDK 15] TestPrintableString fails due to Unicode 13.0 support | Major | test | Akira Ajisaka | Akira Ajisaka | | | Remove EventCounter and Log counters from JVM Metrics | Major | common | Akira Ajisaka | Viraj Jasani | | | ABFS: Disable throttling update for auth failures | Major | fs/azure | Sumangala Patki | Sumangala Patki | | | Change CS nodes page in UI to support custom resource. | Major | resourcemanager | Qi Zhu | Qi Zhu | | | ABFS: Support infinite lease dirs | Minor | common | Billie Rinaldi | Billie Rinaldi | | | Dots '.' in CSMappingRule path variables should be replaced | Major | capacity scheduler | Gergely Pollk | Peter Bacsko | | | whitespace not allowed in paths when saving files to s3a via committer | Blocker | fs/s3 | Krzysztof Adamski | Krzysztof Adamski | | | fs2cs: add queue autorefresh policy during conversion | Major | fairscheduler, fs-cs | Qi Zhu | Qi Zhu | | | mvn versions:set fails to parse pom.xml | Blocker | build | Wei-Chiu Chuang | Wei-Chiu Chuang | | | standby namenode failed to start ordered snapshot deletion is enabled while having snapshottable directories | Major | snapshots | Nilotpal Nandi | Shashikant Banerjee | | | GenericEventHandler.printEventQueueDetails causes RM recovery to take too much time | Critical | resourcemanager | Zhanqi Cai | Qi Zhu | | | Update NFS gateway to use Netty4 | Major | nfs | Brandon Li | Wei-Chiu Chuang | | | Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. | Major | gpu, yarn | Qi Zhu | Qi Zhu | | | Do not use guava's Files.createTempDir() | Major | common | Wei-Chiu Chuang | Wei-Chiu Chuang | |

TestRouterRpcMultiDestination#testProxyGetTransactionID and testProxyVersionRequest are flaky | Major | rbf | Harunobu Daikoku | Akira Ajisaka | | | RBF: Authentication failed when rename accross sub clusters | Major | rbf | zhuobin zheng | zhuobin zheng | | | Add back the exceptions removed by HADOOP-17432 for compatibility | Blocker | build | Akira Ajisaka | Quan Li | | | Implement dfsadmin -provisionSnapshotTrash -all | Major | dfsadmin | Siyao Meng | Siyao Meng | | | Race condition: AsyncDispatcher can get stuck by the changes introduced in YARN-8995 | Critical | resourcemanager | Chenyu Zheng | Chenyu Zheng | | | Add dispatcher metrics to RM | Major | metrics, resourcemanager | Jonathan Hung | Qi Zhu | | | Refactor dynamic queue handling logic | Minor | capacity scheduler | Andras Gyori | Andras Gyori | | | Fix junit deprecation warnings in hadoop-common module | Major | test | Akira Ajisaka | Akira Ajisaka | | | Add more event type to RM Dispatcher event metrics. | Major | resourcemanager | Qi Zhu | Qi Zhu | | | Ignore missing keystore configuration in reloading mechanism | Major | common | Borislav Iordanov | Borislav Iordanov | | | Remove useless property hadoop.assemblies.version in pom file | Trivial | build | Wei-Chiu Chuang | Akira Ajisaka | | | Replace Guava Sets usage by Hadoop's own Sets in hadoop-common and hadoop-tools | Major | common | Ahmed Hussein | Viraj Jasani | | | Update LICENSE for 3.3.1 | Blocker | common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add cluster metric for size of SchedulerEventQueue and RMEventQueue | Major | metrics, resourcemanager | chaosju | chaosju | | | Replace Guava Sets usage by Hadoop's own Sets in hadoop-mapreduce-project | Major | common | Viraj Jasani | Viraj Jasani | | | Replace Guava Sets usage by Hadoop's own Sets in hadoop-hdfs-project | Major | common | Viraj Jasani | Viraj Jasani | | | Replace Guava Sets usage by Hadoop's own Sets in hadoop-yarn-project | Major | common | Viraj Jasani | Viraj Jasani | | | Allow definition of auto queue template properties in root | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Extend /scheduler endpoint with template properties | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | RBF: Add permission check before doing router federation rename. | Major | rbf | Jinglun | Jinglun | | | Implement wrapper for guava newArrayList and newLinkedList | Major | common | Ahmed Hussein | Viraj Jasani | | | Parents node labels are incorrectly added to child queues in weight mode | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Fix Auto Queue template to properly set all configuration properties | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Optimise retrieval of configured node labels in CS queues | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Set dfs.namenode.redundancy.considerLoad to false in MiniDFSCluster | Major | test | Akira Ajisaka | Ahmed Hussein | | | [JDK 15] TestDNS fails by UncheckedIOException | Major | test | Akira Ajisaka | Akira Ajisaka | | | TestBalancerRPCDelay#testBalancerRPCDelayQpsDefault fails on Trunk | Major | balancer, test | Ahmed Hussein | Ahmed Hussein | | | ABFS: Change default Readahead Queue Depth from num(processors) to const | Major | fs/azure | Sumangala Patki | Sumangala Patki | | | ABFS: Append blob tests with non HNS accounts fail | Minor | fs/azure | Sneha Varma | Sneha Varma | | | ABFS: testBlobBackCompatibility, testRandomRead & WasbAbfsCompatibility tests fail when triggered with default configs | Minor | test | Sneha Varma | Sneha Varma | | | Provide fallbacks for callqueue.impl and scheduler.impl | Major | ipc | Viraj Jasani | Viraj Jasani | | | Update HikariCP to 4.0.3 | Major | build, common | Yuming Wang | Viraj Jasani | | | Support Append API in NNThroughputBenchmark | Minor | namanode | Renukaprasad C | Renukaprasad C | | | We should make max application per queue to support node

| Major | capacity scheduler | Qi Zhu | Andras Gyori | | | Support getApplications API in FederationClientInterceptor | Major | federation, router | Akshat Bordia | Akshat Bordia | | | TestBootstrapAliasmap fails by BindException | Major | test | Akira Ajisaka | Akira Ajisaka | | | ParentQueue does not validate the queue on removal | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | CS Flexible AQC: Add separate parent and leaf template property. | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Provide fallbacks for identity/cost providers and backoff enable | Major | ipc | Viraj Jasani | Viraj Jasani | | | Fix token reset synchronization for UAM response token | Minor | federation | Minni Mittal | Minni Mittal | | | Implement an optimised version of Configuration getPropsWithPrefix | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | De-flake TestBlockScanner#testSkipRecentAccessFile | Major | test | Viraj Jasani | Viraj Jasani | | | TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits is flaky | Major | test | Akira Ajisaka | Viraj Jasani | | | ViewDistributedFileSystem#rename wrongly using src in the place of dst. | Major | viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Clear abfs readahead requests on stream close | Major | fs/azure | Rajesh Balamohan | Mukund Thakur | | | Update Capacity Scheduler documentation with JSON-based placement mapping | Major | capacity scheduler, documentation | Peter Bacsko | Benjamin Teke | | | Document for Flexible Auto Queue Creation in Capacity Scheduler | Major | capacity scheduler | Qi Zhu | Benjamin Teke | | | TestCapacitySchedulerWeightMode test descriptor comments doesn't reflect the correct scenario | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Remove LeafQueue#scheduler field | Minor | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | Add metrics for getClusterMetrics and getApplications APIs in FederationClientInterceptor | Major | federation, metrics, router | Akshat Bordia | Akshat Bordia | | | Simplify duplicated code for tracking ResourceUsage in AbstractCSQueue | Minor | capacity scheduler | Szilard Nemeth | Tamas Domok | | | AbstractCSQueue#setupQueueConfigs: Separate validation logic from initialization logic | Minor | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | Optimise CSConfiguration getAllUserWeightsForQueue | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate | Major | capacity scheduler | Andras Gyori | Benjamin Teke | | | AbstractCSQueue#updateConfigurableResourceRequirement: Separate validation logic from initialization logic | Minor | capacity scheduler | Szilard Nemeth | Tamas Domok | | | Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy | Minor | capacity scheduler | Szilard Nemeth | Tamas Domok | | | AbstractCSQueue: Simplify complex logic in methods: deriveCapacityFromAbsoluteConfigurations and updateEffectiveResources | Minor | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | RBF: Use HdfsConfiguration for passing in Router principal | Major | rbf | Akira Ajisaka | Fengnan Li | | | RBF: Set default value of hdfs.fedbalance.procedure.scheduler.journal.uri by adding appropriate config resources | Major | rbf | Akira Ajisaka | Viraj Jasani | | | testMoverWithStripedFile fails intermittently | Major | test | Viraj Jasani | Viraj Jasani | | | AbstractCSQueue: Group preemption methods and fields into a separate class | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Code cleanup in QueueCapacities | Minor | capacity scheduler | Szilard Nemeth | Adam Antal | | | AbstractCSQueue: Create a separate class for usernames and weights that are travelling in a Map | Minor | capacity scheduler, test | Szilard Nemeth | Szilard Nemeth | | | Flaky

TestFsDatasetImpl#testDnRestartWithHardLink | Major | test | Viraj Jasani | Viraj Jasani | | | Introduce QueuePath class | Major | resourcemanager, yarn | Andras Gyori | Andras Gyori | | | TestCapacityScheduler: reuse appHelper where feasible | Major | capacity scheduler, test | Tamas Domok | Tamas Domok | | | RBF: Set default map tasks and bandwidth in RouterFederationRename | Major | rbf | Akira Ajisaka | Viraj Jasani | | | [JDK 17] TestNetUtils fails | Major | common | Akira Ajisaka | Viraj Jasani | | | Fix TestDataNodeMetrics#testReceivePacketSlowMetrics | Major | datanode, metrics | Haiyang Hu | Haiyang Hu | | | Use invokeConcurrent Overload with Collection in getClusterMetrics | Major | federation, router | Akshat Bordia | Akshat Bordia | | | Extract test queues and related methods from TestCapacityScheduler | Major | capacity scheduler, test | Tamas Domok | Tamas Domok | | | implement non-guava Precondition checkArgument | Major | command | Ahmed Hussein | Ahmed Hussein | | | Fix ViewDFS with mount points for HDFS only API | Major | viewfs | Ayush Saxena | Ayush Saxena | | | Support S3 Access Points | Major | fs/s3 | Steve Loughran | Bogdan Stolojan | | | AccessPoint verifyBucketExistsV2 always returns false | Trivial | fs/s3 | Bogdan Stolojan | Bogdan Stolojan | | | Provide alternative to Guava VisibleForTesting | Major | common | Viraj Jasani | Viraj Jasani | | | implement non-guava Precondition checkState | Major | common | Ahmed Hussein | Ahmed Hussein | | | Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-common-project modules | Major | common | Viraj Jasani | Viraj Jasani | | | Do not extend from CapacitySchedulerTestBase when not needed | Major | capacity scheduler, test | Tamas Domok | Tamas Domok | | | Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-hdfs-project modules | Major | build, common | Viraj Jasani | Viraj Jasani | | | Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-cloud-storage-project and hadoop-mapreduce-project modules | Major | build, common | Viraj Jasani | Viraj Jasani | | | Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools modules | Major | build, common | Viraj Jasani | Viraj Jasani | | | Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules | Major | build, common | Viraj Jasani | Viraj Jasani | | | remove guava Preconditions from Hadoop-common-project modules | Major | common | Ahmed Hussein | Ahmed Hussein | | | RBF: Remove the useless configuration of rpc isolation in md | Minor | documentation, rbf | Xiangyi Zhu | Xiangyi Zhu | | | Move AbstractCSQueue fields to separate objects that are tracking usage | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Remove commented code block from CSQueueUtils#loadCapacitiesByLabelsFromConf | Trivial | capacity scheduler | Szilard Nemeth | Andras Gyori | | | Simplify AbstractCSQueue#updateMaxAppRelatedField and find a more meaningful name for this method | Minor | capacity scheduler | Szilard Nemeth | Andras Gyori | | | Use correct configuration for Group service init in CSMappingPlacementRule | Major | capacity scheduler | Peter Bacsko | Szilard Nemeth | | | Simplify GuaranteedOrZeroCapacityOverTimePolicy#computeQueueManagementChanges | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | |

SchedulerQueue#activeQueue to activateQueue | Minor | capacity scheduler | Szilard Nemeth | Adam Antal | | | Introduce universal configured capacity vector | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | AbstractCSQueue: Annotate all methods with VisibleForTesting that are only used by test code | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | unguava: remove Preconditions from hdfs-projects module | Major | common | Ahmed Hussein | Ahmed Hussein | | | Clean up CapacityScheduler#initScheduler | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Add some tests to verify ACL behaviour in CapacitySchedulerConfiguration | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | AliyunOSS: support ListObjectsV2 | Major | fs/oss | wujinhu | wujinhu | | | Add YARN\ROUTER\HEAPSIZE to yarn-env for routers | Minor | federation, router | Minni Mittal | Minni Mittal | | | unguava: remove Preconditions from hadoop-tools modules | Major | common | Viraj Jasani | Viraj Jasani | | | unguava: remove Preconditions from hadoop-yarn-project modules | Major | common | Viraj Jasani | Viraj Jasani | | | Make RMNode aware of all (OContainer inclusive) allocated resources | Minor | container, resourcemanager | Andrew Chung | Andrew Chung | | | De-flake TestRollingUpgrade#testRollback | Minor | hdfs, test | Kevin Wikant | Viraj Jasani | | | De-flake testDecommissionStatus | Major | test | Viraj Jasani | Viraj Jasani | | | Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom | Major | common | Viraj Jasani | Viraj Jasani | | | Upgrade HBase version to 1.7.1 for hbase1 profile | Major | build | Viraj Jasani | Viraj Jasani | | | Improve the maintainability of RM webapp tests like TestRMWebServicesCapacitySched | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | Fix testQueueSubmitWithACL\* tests in TestAppManager | Major | yarn | Tamas Domok | Tamas Domok | | | Implement the core QUEUE\LENGTH\THEN\_RESOURCES OContainer allocation policy | Minor | resourcemanager | Andrew Chung | Andrew Chung | | | Replace all occurences of queuePath with the new QueuePath class | Major | capacity scheduler | Andras Gyori | Tibor Kovcs | | | Upgrade hbase2 version and fix TestTimelineWriterHBaseDown | Major | build | Viraj Jasani | Viraj Jasani | | | Create an AbstractLeafQueue to store the common LeafQueue + AutoCreatedLeafQueue functionality | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Minimize usages of AbstractCSQueue#csContext | Major | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | Do not use a separate config in legacy CS AQC | Minor | capacity scheduler | Szilard Nemeth | Benjamin Teke | | | Clean up checkstyle warnings from YARN-11024/10907/10929 | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Split TestCapacityScheduler by test categories | Major | capacity scheduler | Tamas Domok | Tamas Domok | | | CapacityScheduler: Move all fields and initializer code that belongs to async scheduling to a new class | Minor | capacity scheduler | Szilard Nemeth | Szilard Nemeth | | | Prune Jackson 1 from the codebase and restrict it's usage for future | Major | common | Wei-Chiu Chuang | Viraj Jasani | |

TestHDFSFileSystemContract#testAppend fails | Major | test | Hui Fei | secfree | | | Add audit logs for router service | Major | router | Bibin Chundatt | Minni Mittal | | | Make auto queue creation maximum allowed depth configurable | Major | capacity scheduler | Qi Zhu | Andras Gyori | | | Add enhanced headroom in AllocateResponse | Major | federation | Minni Mittal | Minni Mittal | | | Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl | Major | hdfs | Mingxiang Li | Mingxiang Li | | | Fix TestBlockTokenWithDFSStriped#testEnd2End failure | Major | test | Hui Fei | secfree | | | Move PendingApplicationComparator from GuaranteedOrZeroCapacityOverTimePolicy | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | Simplify AbstractCSQueue#initializeQueueState | Minor | capacity scheduler | Szilard Nemeth | Andras Gyori | | | AbstractCSQueue: Eliminate code duplication in overloaded versions of setMaxCapacity | Minor | capacity scheduler | Szilard Nemeth | Andras Gyori | | | Consider legacy auto queue creation absolute resource template to avoid rounding errors | Major | capacity scheduler | Qi Zhu | Andras Gyori | | | [SPS]: Fix bug for unit test of reconfiguring SPS mode | Major | sps, test | Tao Li | Tao Li | | | Follow-up changes for YARN-10904 | Minor | capacityscheduler | Szilard Nemeth | Benjamin Teke | | | Add javadoc to all methods of AbstractCSQueue | Major | capacity scheduler, documentation | Szilard Nemeth | Andrs Gyri | | | Simplify method: CapacitySchedulerQueueManager#parseQueue | Minor | capacity scheduler | Szilard Nemeth | Andras Gyori | | | Use Slf4jRequestLog for HttpRequestLog | Major | common | Akira Ajisaka | Duo Zhang | | | Do not inherit from TestRMWebServicesCapacitySched | Major | capacity scheduler, test | Tamas Domok | Tamas Domok | | | FIFOOrderingPolicy Improvements | Major | scheduler | Manikandan R | Benjamin Teke | | | [SPS]: Should not start indefinitely while another SPS process is running | Major | sps | Tao Li | Tao Li | | | Follow-up to YARN-10504 | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | RBF: Namenode need to choose block location for the client | Major | rbf | Wu Weiwei | Owen O'Malley | | | Improve oiv tool to parse fsimage file in parallel with delimited format | Major | tools | Hongbing Wang | Hongbing Wang | | | Upgrade Avro to

| Major | build | Java Developer | PJ Fanning | | | [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS | Major | sps | Tao Li | Tao Li | | | Remove use of scala jar twitter util-core with java futures in S3A prefetching stream | Major | fs/s3 | PJ Fanning | PJ Fanning | | | [SPS]: Handle failure retries for moving tasks | Major | sps | Tao Li | Tao Li | | | [SPS]: Fix an infinite loop bug in SPSPathIdProcessor thread | Major | sps | qinyuren | qinyuren | | | Add metrics for slow DataNode | Major | datanode, metrics | Renukaprasad C | Renukaprasad C | | | RBF: Fix dead link to fedbalance document | Minor | documentation | Akira Ajisaka | Ashutosh Gupta | | | [SPS]: Expose metrics to JMX for external SPS | Major | metrics, sps | Tao Li | Tao Li | | | document use and architecture design of prefetching s3a input stream | Major | documentation, fs/s3 | Steve Loughran | Ahmar Suhail | | | ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters | Major | fs/azure | Sumangala Patki | Sumangala Patki | | | Use jersey-json that is built to use jackson2 | Major | build | Akira Ajisaka | PJ Fanning | | | RouterClientRMService Has Unused import | Minor | federation, router | Shilun Fan | Shilun Fan | | | Support getClusterNodes API in FederationClientInterceptor | Major | federation | Shilun Fan | Shilun Fan | | | Fix Hadoop Common Java Doc Errors | Major | build, common | Shilun Fan | Shilun Fan | | | Support getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation | Major | federation | D M Murali Krishna Reddy | Shilun Fan | | | Improve log message in FederationClientInterceptor | Minor | federation | Shilun Fan | Shilun Fan | | | RBF: Fix TestRouterWebHDFSContractCreate#testSyncable | Major | hdfs, rbf | Renukaprasad C | Hanley Yang | | | Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation | Major | federation, router | D M Murali Krishna Reddy | Shilun Fan | | | Support failApplicationAttempt, updateApplicationPriority, updateApplicationTimeouts API's for Federation | Major | federation | Shilun Fan | Shilun Fan | | | Fix DataNode FsDatasetImpl lock issue without GS checks. | Major | datanode | ZanderXu | ZanderXu | | | Remove WhiteBox in hadoop-kms module. | Minor | common | Shilun Fan | Shilun Fan | | | Fix deadlock of fine-grain lock for FsDatastImpl of DataNode. | Major | datanode | ZanderXu | ZanderXu | | | Support signalToContainer API for Federation | Major | federation, yarn | Sushanta Sen | Shilun Fan | | | Replace with HashSet/TreeSet constructor in Hadoop-common-project | Trivial | common | Samrat Deb | Samrat Deb | | | Remove unnecessary LevelDb write call in LeveldbConfigurationStore#confirmMutation | Minor | capacityscheduler | Prabhu Joseph | Ashutosh Gupta | | | Minor fixes in HDFS Fedbalance document | Minor | documentation | Akira Ajisaka | Ashutosh Gupta | | | TimelineCollectorWebService#putEntities blocked when ATSV2 HBase is down. | Major | ATSv2 | Prabhu Joseph | Ashutosh Gupta | | | Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping | Major | capacity scheduler | Akhil PB | Ashutosh Gupta | | | GET /apps/{appid}/entities/YARN\_APPLICATION accesses application table instead of entity table | Major | ATSv2 | Prabhu Joseph | Ashutosh Gupta | | | RBF: improve renewLease() to call only a specific NameNode rather than make fan-out calls | Major | rbf | Aihua Xu | ZanderXu | | | tests in ITestS3AInputStreamPerformance are failing | Minor | fs/s3 | Ahmar Suhail | Ahmar Suhail | | | Add in configuration option to enable prefetching | Minor | fs/s3 | Ahmar Suhail | Ahmar Suhail | | | Support getResourceProfiles, getResourceProfile API's for Federation | Major | federation | Shilun Fan | Shilun Fan | | | [Router] Federation: routing getContainers REST invocations transparently to multiple RMs | Major | federation, router | Giovanni Matteo Fumarola | Shilun Fan | | | RBF: Client maybe get an unexpected result with network anomaly | Critical | rbf | Hui Fei | ZanderXu | | | Fix typo in hadoop-yarn-server-router module | Minor | federation | Shilun Fan | Shilun Fan | | | Support getAttributesToNodes, getClusterNodeAttributes, getNodesToAttributes API's for Federation | Major | federation | Shilun Fan | Shilun Fan | | | [Router] Router Audit Log Add Client IP Address. | Major | federation, router | chaosju | Shilun Fan | | | Collect IOStatistics during S3A prefetching | Major | fs/s3 | Steve Loughran | Ahmar Suhail | | | AWS SDK update to

to address jackson CVE-2018-7489 and AWS CVE-2022-31159 | Major | fs/s3 | Steve Loughran | Steve Loughran | | | [Federation] Add getNodeToLabels REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Refactor some code of getNewApplication, submitApplication, forceKillApplication, getApplicationReport | Minor | federation | Shilun Fan | Shilun Fan | | | [Federation] Add getLabelsToNodes, getClusterNodeLabels, getLabelsOnNode REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [Router] Add missing methods in RMWebProtocol | Major | federation, router | Giovanni Matteo Fumarola | Shilun Fan | | | Adding RM ClusterId in AppInfo | Major | federation | Giovanni Matteo Fumarola | Tanuj Nayak | | | [Federation] Add getContainer, signalToContainer REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [RESERVATION] Refactor Policy Code and Define getReservationHomeSubcluster | Major | federation | Shilun Fan | Shilun Fan | | | Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice | Major | test | ANANDA G B | Ashutosh Gupta | | | [Federation] Add getAppTimeout, getAppTimeouts, updateApplicationTimeout REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | RBF: Extend RouterRpcClient to use multiple sockets | Major | rbf | igo Goiri | igo Goiri | | | Create SecureLogin inside Router | Minor | federation, router | Giovanni Matteo Fumarola | Xie YiFan | | | In federation and security mode, nm recover may fail. | Major | nodemanager | Chenyu Zheng | Chenyu Zheng | | | [RESERVATION] Implement FederationReservationHomeSubClusterStore With MemoryStore | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Add getAppPriority, updateApplicationPriority REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Add getAppQueue, updateAppQueue REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [RESERVATION] Yarn Federation Router Supports Update / Delete Reservation in MemoryStore | Major | federation | Shilun Fan | Shilun Fan | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timeline-pluginstorage | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | Capture the Performance Metrics of ZookeeperFederationStateStore | Major | federation | Shilun Fan | Shilun Fan | | |

to be read through longBytesOption | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | [Federation] Add getAppActivities, getAppStatistics REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [Router] Add cache for fast answers to getApps | Major | federation, router | Giovanni Matteo Fumarola | Shilun Fan | | | [Federation] Add batchFinishApplicationMaster in UAMPoolManager | Major | federation, nodemanager | Shilun Fan | Shilun Fan | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-csi | Major | yarn-csi | Ashutosh Gupta | Ashutosh Gupta | | | [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk | Major | federation, reservation system | Shilun Fan | Shilun Fan | | | S3A storage class option only picked up when buffering writes to disk | Major | fs/s3 | Steve Loughran | Monthon Klongklaew | | | Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation | Major | federation | Shilun Fan | Shilun Fan | | | Handle containerId duplicate without failing the heartbeat in Federation Interceptor | Minor | federation, router | Botong Huang | Shilun Fan | | | [Federation] Improve UnmanagedAMPoolManager WithoutBlock ServiceStop | Major | federation | Shilun Fan | Shilun Fan | | | [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL | Major | federation | Shilun Fan | Shilun Fan | | | S3AInputStream.unbuffer() async drain not releasing http connections | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Improve Yarn Router Junit Test Close MockRM | Major | federation | Shilun Fan | Shilun Fan | | | Improve Yarn Router Reservation Submission Code | Major | federation | Shilun Fan | Shilun Fan | | | HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients. | Major | federation, namenode | Erik Krogen | Simbarashe Dzinamarira | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-sharedcachemanager | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | Remove WhiteBox in hadoop-common module. | Minor | common | Shilun Fan | Shilun Fan | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | RBF: Support observer node from Router-Based Federation | Major | rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | s3a prefetching to use SemaphoredDelegatingExecutor for submitting work | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | [Federation] Router Support DelegationToken storeNewMasterKey/removeStoredMasterKey With MemoryStateStore | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Fix Typo of NodeManager AMRMProxy. | Minor | federation, nodemanager | Shilun Fan | Shilun Fan | | | hadoop-aws maven build to add a prefetch profile to run all tests with prefetching | Major | fs/s3, test | Steve Loughran | Viraj Jasani | | | Fix Yarn Router Broken Link | Minor | federation, router | Shilun Fan | Shilun Fan | | | s3a prefetching Executor should be closed | Major | fs/s3 | Viraj Jasani | Viraj Jasani | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-client | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-common | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | [Federation] Fix Yarn federation.md table format | Major | federation | Shilun Fan | Shilun Fan | | | Router Page display the db username and password in mask mode | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Refactoring Router's Federation Web Page | Major | federation | Shilun Fan | Shilun Fan | | | Optimizing FederationClientInterceptor Call with Parallelism | Major | federation | Shilun Fan | Shilun Fan | | | Improve FederationInterceptorREST#createInterceptorForSubCluster Use WebAppUtils | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Fix some PBImpl classes to avoid NPE. | Major | federation, router, yarn | Shilun Fan | Shilun Fan | | | Upgrade AWS SDK to V2 - Prerequisites | Minor | fs/s3 | Ahmar Suhail | Ahmar Suhail | | | [Federation] Add SQLServer Script and Supported DB Version in Federation.md | Major | federation | Shilun Fan | Shilun Fan | | | Implement readFully(long position, byte[] buffer, int offset, int length) | Minor | fs/s3 | Ahmar Suhail | Alessandro Passaro | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | Remove the redundant lock in deepCopyReplica and getFinalizedBlocks | Major | datanode | ZanderXu | ZanderXu | | | Remove the redundant lock in DataSetLockManager#removeLock. | Major | datanode | ZanderXu | ZanderXu | | | upgrade AWS SDK to

| Major | build, fs/s3 | Steve Loughran | Steve Loughran | | | [Federation] YARN Federation Router Supports Cross-Origin. | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Refactoring Yarn Router's About Web Page. | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Improve SubClusterState#fromString parameter and LogMessage | Trivial | federation | Shilun Fan | Shilun Fan | | | [Federation] Refactoring Yarn Router's Node Web Page. | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Router Support DelegationToken storeNewToken/updateStoredToken/removeStoredToken With MemoryStateStore | Major | federation | Shilun Fan | Shilun Fan | | | [Router] Federation: Improve Router REST API Metrics | Minor | federation, router | YR | Shilun Fan | | | Remove unused classes introduced by YARN-9615 | Minor | resourcemanager | Shilun Fan | Shilun Fan | | | Improve S3A committers documentation clarity | Trivial | documentation | Daniel Carl Jones | Daniel Carl Jones | | | S3PrefetchingInputStream to support status probes when closed | Minor | fs/s3 | Steve Loughran | Viraj Jasani | | | S3A server-side encryption tests fail before checking encryption tests should skip | Minor | fs/s3, test | Daniel Carl Jones | Daniel Carl Jones | | | [Federation] Refactor FederationClientInterceptor#submitApplication Use FederationActionRetry | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Router Support DelegationToken in MemoryStore mode | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Refactoring Yarn Router's Application Web Page. | Major | federation, router | Shilun Fan | Shilun Fan | | | Upgrade Junit 4 to 5 in hadoop-yarn-applications-catalog-webapp | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | Upgrade Junit 4 to 5 in hadoop-yarn-applications-unmanaged-am-launcher | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | [Federation] Add checkUserAccessToQueue REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration. | Major | federation, router | Shilun Fan | Shilun Fan | | | Upgrade Junit 4 to 5 in hadoop-yarn-applications-mawo | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | Upgrade Junit 4 to 5 in hadoop-yarn-services-api | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests | Major | test, yarn | Ashutosh Gupta | Ashutosh Gupta | | | ITestS3APrefetchingInputStream does not skip if no CSV test file available | Minor | fs/s3 | Daniel Carl Jones | Daniel Carl Jones | | | Improve equals, hashCode(), toString() methods of the Federation Base Object | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Add Yarn Router's NodeLabel Web Page. | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Improve Yarn Router's Federation Page style. | Major | federation, router | Shilun Fan | Shilun Fan | | | Upgrade MR ShuffleHandler to use Netty4 | Major | common | Xiaoyu Yao | Szilard Nemeth | | | Avoid to hold write lock to improve performance when add volume. | Major | datanode | ZanderXu | ZanderXu | | | [Federation] Routing admin invocations transparently to multiple

| Major | federation, router | Shilun Fan | Shilun Fan | | | Upgrade Junit 4 to 5 in hadoop-mapreduce-examples | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | Upgrade JUnit from 4 to 5 in hadoop-yarn-common | Major | test | Akira Ajisaka | Ashutosh Gupta | | | [Federation] Refactor FederationInterceptorREST#createNewApplication\\submitApplication Use FederationActionRetry | Major | federation | Shilun Fan | Shilun Fan | | | assertion failure in ITestS3APrefetchingInputStream | Major | fs/s3, test | Steve Loughran | Ashutosh Gupta | | | ABFS: Support for account level throttling | Major | fs/azure | Anmol Asrani | Anmol Asrani | | | AbstractCSQueue: Create separate class for constructing Queue API objects | Minor | capacity scheduler | Szilard Nemeth | Peter Szucs | | | Support getDelegationToken, renewDelegationToken, cancelDelegationToken API's for Federation | Major | federation | Shilun Fan | Shilun Fan | | | AvroFSInput opens a stream twice and discards the second one without closing | Blocker | fs | Steve Loughran | Steve Loughran | | | [Federation] Support refreshQueuesrefreshNodes API's for Federation. | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Router Support DelegationToken With ZK | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Add FederationInterceptor#allow-partial-result config. | Major | federation, router | Shilun Fan | Shilun Fan | | | Leak of S3AInstrumentation instances via hadoop Metrics references | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | disable purging list of in progress reads in abfs stream closed | Blocker | fs/azure | Steve Loughran | Pranav Saxena | | | ABFS: add probes of readahead fix | Major | fs/azure | Steve Loughran | Steve Loughran | | | [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Add postDelegationToken, postDelegationTokenExpiration, cancelDelegationToken REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Improve S3A delegations token documentation | Minor | fs/s3 | Ahmar Suhail | Ahmar Suhail | | | [Federation] Support refreshSuperUserGroupsConfigurationrefreshUserToGroupsMappings API's for Federation | Major | federation, router | Shilun Fan | Shilun Fan | | | Upgrade Junit 4 to 5 in hadoop-mapreduce-client-uploader | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | Upgrade Junit 4 to 5 in hadoop-mapreduce-client-hs-plugins | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | [Federation] Add getSchedulerInfo REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Clean up different ways to create resources | Minor | resourcemanager, scheduler | Yufei Gu | Riya Khandelwal | | | Centralize queue resource calculation based on CapacityVectors | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | [Federation] Add getActivities, getBulkActivities REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Remove lower limit on s3a prefetching/caching block size | Minor | fs/s3 | Daniel Carl Jones | Ankit Saurabh | | | [Federation] Add dumpSchedulerLogs REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Avoid using grizzly-http-\* APIs | Major | common | Viraj Jasani | Viraj Jasani | | | Cleanup the commons-logging references in the code base | Major | common | Duo Zhang | Viraj Jasani | | | Add gh-pages in

to deploy the current trunk doc | Major | common | Ayush Saxena | Simhadri Govindappa | | | Federation maintenance mechanisms (simple CLI and command propagation) | Major | nodemanager, resourcemanager | Carlo Curino | Shilun Fan | | | Refactoring Router services to use common util classes for pipeline creations | Major | federation | Giovanni Matteo Fumarola | Shilun Fan | | | S3A prefetching: Error logging during reads | Minor | fs/s3 | Ahmar Suhail | Ankit Saurabh | | | [Federation] Router Support DelegationToken With SQL | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Add getAppAttempts, getAppAttempt REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Add versioning for FederationStateStore | Major | federation, router | Subramaniam Krishnan | Shilun Fan | | | [Federation] Add addToClusterNodeLabels, removeFromClusterNodeLabels REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Improve NM FederationInterceptor removeAppFromRegistry | Major | federation, nodemanager | Shilun Fan | Shilun Fan | | | [Federation] Add replaceLabelsOnNodes, replaceLabelsOnNode REST APIs for Router | Major | federation | Shilun Fan | Shilun Fan | | | Cut excess dependencies from hadoop-azure, hadoop-aliyun transitive imports; fix LICENSE-binary | Blocker | build, fs/azure, fs/oss | Steve Loughran | Steve Loughran | | | [Federation] Support refreshAdminAclsrefreshServiceAcls API's for Federation | Major | federation, router | Shilun Fan | Shilun Fan | | | Avoid loading kms log4j properties dynamically by KMSWebServer | Major | kms | Viraj Jasani | Viraj Jasani | | | [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size | Major | federation, router | Giovanni Matteo Fumarola | Shilun Fan | | | LogLevel servlet to determine log impl before using setLevel | Major | common | Viraj Jasani | Viraj Jasani | | | CLA and CRLA appenders to be replaced with RFA | Major | common | Viraj Jasani | Viraj Jasani | | | Remove unused custom appender TaskLogAppender | Major | common | Viraj Jasani | Viraj Jasani | | | [Federation] Add getClusterInfo, getClusterUserInfo REST APIs for Router. | Major | federation | Shilun Fan | Shilun Fan | | | Migrate Async appenders to log4j properties | Major | common | Viraj Jasani | Viraj Jasani | | | Remove Log4Json Layout | Major | common | Viraj Jasani | Viraj Jasani | | | [Federation] Support updateNodeResourcerefreshNodesResources API's for Federation. | Major | federation, router | Shilun Fan | Shilun Fan | | | Add reason in in x-ms-client-request-id on a retry API call. | Minor | fs/azure | Pranav Saxena | Pranav Saxena | | | ABFS: Add changes for expect hundred continue header with append requests | Major | fs/azure | Anmol Asrani | Anmol Asrani | | | [Federation] Add updateSchedulerConfiguration, getSchedulerConfiguration REST APIs for Router. | Major | federation | Shilun Fan | Shilun Fan | | | Refactor FederationInterceptorREST Code | Major | federation, router | Shilun Fan | Shilun Fan | | | x-ms-client-request-id to have some way that identifies retry of an

| Minor | fs/azure | Pranav Saxena | Pranav Saxena | | | ABFS: Enable config controlled ETag check for Rename idempotency | Major | fs/azure | Sneha Vijayarajan | Sree Bhattacharyya | | | [Federation] Support addToClusterNodeLabelsremoveFromClusterNodeLabelsreplaceLabelsOnNode API's for Federation | Major | federation, router | Shilun Fan | Shilun Fan | | | Add dispatcher metrics to NM | Major | nodemanager | chaosju | Shilun Fan | | | Optimize FederationClientInterceptor audit log | Major | federation | Shilun Fan | Shilun Fan | | | S3A Prefetch - SingleFilePerBlockCache to use LocalDirAllocator | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | [Federation] Support checkForDecommissioningNodesrefreshClusterMaxPriority API's for Federation | Major | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Support mapAttributesToNodesgetGroupsForUser API's for Federation | Major | federation, router | Shilun Fan | Shilun Fan | | | Add application submit data to state store | Major | federation | Bibin Chundatt | Shilun Fan | | | Make an AbstractParentQueue to store common ParentQueue and ManagedParentQueue functionality | Major | capacity scheduler | Benjamin Teke | Susheel Gupta | | | [Federation] Improve SQLFederationStateStore DataSource Config | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Router AdminCLI Supports DeregisterSubCluster. | Major | federation | Shilun Fan | Shilun Fan | | | Federation Router (hiding multiple RMs for ApplicationClientProtocol) phase 2 | Major | federation, router | Giovanni Matteo Fumarola | Shilun Fan | | | S3A audit header to include count of items in delete ops | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | [Federation] ConfiguredRMFailoverProxyProvider Supports Randomly Select an Router. | Major | federation | Shilun Fan | Shilun Fan | | | Fix FederationInterceptor#allocate to set application priority in allocateResponse | Major | federation | Bibin Chundatt | Shilun Fan | | | Upgrade Junit 4 to 5 in hadoop-mapreduce-client-common | Major | test | Ashutosh Gupta | Ashutosh Gupta | | | [Federation] SQLFederationStateStore Support Store ApplicationSubmitData | Major | federation | Shilun Fan | Shilun Fan | | | Race condition between second app attempt and UAM timeout when first attempt node is down | Major | federation | Botong Huang | Shilun Fan | | | Improve createJerseyClient#setConnectTimeout Code | Major | federation | Shilun Fan | Shilun Fan | | | Fix typos in hadoop-yarn-server-common#federation | Minor | federation | Shilun Fan | Shilun Fan | | | Change fs.s3a.directory.marker.retention to "keep" | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Refactor AMRMProxy#FederationInterceptor#registerApplicationMaster | Major | amrmproxy, federation, nodemanager | Shilun Fan | Shilun Fan | | | [Federation] Add Steps To Set up a Test Cluster. | Major | federation | Shilun Fan | Shilun Fan | | | Improve existsApplicationHomeSubCluster/existsReservationHomeSubCluster Log Level | Minor | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Fix NodeManager#TestFederationInterceptor Flaky Unit Test | Major | federation, nodemanager | Shilun Fan | Shilun Fan | | | Improve Federation#RouterCLI deregisterSubCluster Code | Major | federation, router | Shilun Fan | Shilun Fan | | | CachingBlockManager to use AtomicBoolean for closed flag | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | [Federation] Add RouterAuditLog to log4j.properties | Major | router | Shilun Fan | Shilun Fan | | | [GPG] Support Secure Mode | Major | gpg | tuyu | Shilun Fan | | | Replace queue resource calculation logic in updateClusterResource | Major | capacity scheduler | Andras Gyori | Andras Gyori | | | Improve the Policy Description in Federation.md | Major | federation | Shilun Fan | Shilun Fan | | | The FederationInterceptor#launchUAM Added retry logic. | Minor | amrmproxy | Shilun Fan | Shilun Fan | | | [Federation] Improve DefaultRequestInterceptor#init Code | Minor | federation, router | Shilun Fan | Shilun Fan | | | [Federation] Code cleanup for NodeManager#amrmproxy | Minor | federation | Shilun Fan | Shilun Fan | | | [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of

| Major | federation, router | Shilun Fan | Shilun Fan | | | CapacityScheduler CapacityConfigType changed in legacy queue allocation mode | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | s3a DelegationToken plugin to expand return type of deploy/binding | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Create a test set that runs with both Legacy/Uniform queue calculation | Major | capacityscheduler | Tamas Domok | Tamas Domok | | | [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code | Minor | federation | Shilun Fan | Shilun Fan | | | Support capacity vector for AQCv2 dynamic templates | Major | capacityscheduler | Tamas Domok | Benjamin Teke | | | Fix checkstyle issues after YARN-11520 | Major | capacity scheduler | Benjamin Teke | Benjamin Teke | | | s3a audit logs to publish range start/end of GET requests in audit header | Minor | fs/s3 | Steve Loughran | Ankit Saurabh | | | [Federation] Router CLI Supports Batch Save the SubClusterPolicyConfiguration Of Queues. | Major | federation | Shilun Fan | Shilun Fan | | | Make proxy server support YARN federation. | Major | yarn | Chenyu Zheng | Chenyu Zheng | | | Make AMRMProxyPolicy aware of SC load | Major | amrmproxy | Young Chen | Shilun Fan | | | Upgrade aws-java-sdk to 1.12.499+ | Major | fs/s3 | Viraj Jasani | Viraj Jasani | | | Make router support proxy server. | Major | yarn | Chenyu Zheng | Chenyu Zheng | | | [GPG] Support HTTPS in GPG | Major | federation | Bilwa S T | Shilun Fan | | | AWS SDK v2: make the v1 bridging support optional | Major | fs/s3 | Steve Loughran | Steve Loughran | | | AWS SDK V2 - Upgrade SDK to 2.20.28 and restores multipart copy | Major | fs/s3 | Ahmar Suhail | Ahmar Suhail | | | Running RM tests against the Router | Minor | federation, resourcemanager | Giovanni Matteo Fumarola | Shilun Fan | | | [Router] FederationStateStoreFacade is not reinitialized with Router conf | Major | federation, router, yarn | Aparajita Choudhary | Shilun Fan | | | [Federation] Router CLI Supports List SubClusterPolicyConfiguration Of Queues. | Major | federation | Shilun Fan | Shilun Fan | | | [Router] UGI conf doesn't read user overridden configurations on Router startup | Major | federation, router, yarn | Aparajita Choudhary | Shilun Fan | | | Mapreduce application container start fail after AM restart. | Major | federation | Bibin Chundatt | Chenyu Zheng | | | Advanced Federation UI based on YARN UI v2 | Major | yarn, yarn-ui-v2 | Carlo Curino | Shilun Fan | | | AWS SDK V2 - AuditFailureExceptions aren't being translated properly | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Merge aws v2 upgrade feature branch into trunk | Major | fs/s3 | Steve Loughran | Steve Loughran | | | [Federation] GPG Support Query Policies In Web. | Major | federation | Shilun Fan | Shilun Fan | | | Router's main() should support generic options | Major | federation, router, yarn | Aparajita Choudhary | Aparajita Choudhary | | | S3A. createS3AsyncClient() always enables multipart | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde

| Major | security | Xiaoqiao He | Xiaoqiao He | | | Add YARN\GLOBALPOLICYGENERATOR\HEAPSIZE to yarn-env for GPG | Minor | federation | Shilun Fan | Shilun Fan | | | [Federation] Router Supports Remove individual application records from FederationStateStore. | Major | federation | Shilun Fan | Shilun Fan | | | YARN Router Web supports displaying information for Non-Federation. | Major | federation, router | Shilun Fan | Shilun Fan | | | Fix 'Physical Mem Used' and 'Physical VCores Used' are not displaying data | Major | federation, router | Shilun Fan | Shilun Fan | | | ABFS: Change default from disk to bytebuffer for fs.azure.data.blocks.buffer | Major | build | Anmol Asrani | Anmol Asrani | | | ABFS: Fix failing CPK tests on trunk | Minor | build | Anmol Asrani | Anmol Asrani | | | Add znode hierarchy in Federation ZK State Store | Major | federation | Bibin Chundatt | Shilun Fan | | | Fix uncleaned threads in YARN Federation interceptor threadpool | Major | federation, router | Jeffrey Chang | Jeffrey Chang | | | S3A: V2 SDK client does not work with third-party store | Critical | fs/s3 | Steve Loughran | Steve Loughran | | | AWS v2 SDK: fail meaningfully if legacy v2 signing requested | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3ARetryHandler to treat SocketExceptions as connectivity failures | Major | fs/s3 | Steve Loughran | Steve Loughran | | | [GPG] Add Information About YARN GPG in Federation.md | Major | federation | Shilun Fan | Shilun Fan | | | Improve s3a region handling, including determining from endpoint | Major | fs/s3 | Steve Loughran | Ahmar Suhail | | | s3a prefetch LRU cache eviction metric | Major | fs/s3 | Viraj Jasani | Viraj Jasani | | | S3A: testMultiObjectExceptionFilledIn() assertion error | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() | Critical | fs/s3 | Steve Loughran | Steve Loughran | | | Improve FederationInterceptorREST AuditLog | Major | federation, router | Shilun Fan | Shilun Fan | | | Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565 | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | [Federation] Improve command line help information. | Major | federation | Shilun Fan | Shilun Fan | | | ITestS3GuardTool fails if SSE/DSSE encryption is used | Minor | fs/s3, test | Viraj Jasani | Viraj Jasani | | | Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS) | Major | fs/s3, security | Akira Ajisaka | Viraj Jasani | | | [Federation] Improve Yarn Federation documentation | Major | federation | Shilun Fan | Shilun Fan | | | Improve the time unit for FederationRMAdminInterceptor#heartbeatExpirationMillis | Minor | federation | WangYuanben | WangYuanben | | | [Federation] Router Supports Format FederationStateStore. | Major | federation | Shilun Fan | Shilun Fan | | | Make YARN Router throw Exception to client clearly | Major | federation, router, yarn | Yuan Luo | Shilun Fan | | | [Federation] Router Supports Yarn Client CLI Cmds. | Major | federation | Shilun Fan | Shilun Fan | | | ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations | Major | build | Anmol Asrani | Anuj Modi | | | [Federation] Router AdminCLI Supports Clean Finish

| Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Add WeightedHomePolicyManager | Major | federation | Shilun Fan | Shilun Fan | | | Improve FederationInterceptorREST Method Result | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Router Supports Yarn Admin CLI Cmds. | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Add Federation PolicyManager Validation Rules | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Improve FederationClientInterceptor To Return Partial Results of subClusters. | Major | federation | Shilun Fan | Shilun Fan | | | S3A: Upgrade AWS SDK version to 2.21.33 for Amazon S3 Express One Zone support | Major | fs/s3 | Ahmar Suhail | Ahmar Suhail | | | Tune/extend S3A http connection and thread pool settings | Major | fs/s3 | Ahmar Suhail | Steve Loughran | | | S3A to provide full support for S3 Express One Zone | Major | fs/s3 | Ahmar Suhail | Steve Loughran | | | [Federation] GPG Supports Format PolicyStateStore. | Major | federation | Shilun Fan | Shilun Fan | | | [Federation] Router CLI Supports Delete SubClusterPolicyConfiguration Of Queues. | Major | federation | Shilun Fan | Shilun Fan | | | S3A: Add option fs.s3a.s3express.create.session to enable/disable CreateSession | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | [Federation] Router CLI Supports List SubClusters. | Major | federation | Shilun Fan | Shilun Fan | | | S3A: Upgrade AWS SDK to 2.21.41 | Major | fs/s3 | Steve Loughran | Steve Loughran | | | [GPG] Improve GPGPolicyFacade#getPolicyManager | Major | federation | Shilun Fan | Shilun Fan | | | [GPG] Improve GPGOverviewBlock Infomation | Major | federation | Shilun Fan | Shilun Fan | | | ABFS: Support for Encryption Context | Major | fs/azure | Sumangala Patki | Pranav Saxena | | | [Doc] Add allow-partial-result description to Yarn Federation documentation | Major | federation | Shilun Fan | Shilun Fan | | | ABFS: Enable Footer Read Optimizations with Appropriate Footer Read Buffer Size | Major | fs/azure | Anuj Modi | Anuj Modi | | | Let Federation.md more standardized | Minor | documentation | WangYuanben | WangYuanben | | | Change the time unit of scCleanerIntervalMs in Router | Minor | router | WangYuanben | WangYuanben | | | S3A: Support Authentication through HttpSigner API | Major | fs/s3 | Steve Loughran | Harshit Gupta | | | ABFS: Adding 100 continue in userAgent String and dynamically removing it if retry is without the header enabled. | Minor | build | Anmol Asrani | Anmol Asrani | | | S3A: S3AInputStream doesn't recover from HTTP/channel exceptions | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A: disable checksum validation | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Use builder for prefetch CachingBlockManager | Major | fs/s3 | Viraj Jasani | Viraj Jasani | | | Expect-100 JDK bug resolution: prevent multiple server calls | Major | fs/azure | Pranav Saxena | Pranav Saxena | | | Increase fs.s3a.connection.maximum to 500 to minimize risk of Timeout waiting for connection from pool | Major | fs/s3 | Mukund Thakur | Mukund Thakur | | | AWS SDK v2: extend support for FIPS endpoints | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A: update AWS sdk versions to 2.23.5 and

| Major | build, fs/s3 | Steve Loughran | Steve Loughran | | | S3A: Cut S3 Select | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A credential provider remapping: make extensible | Minor | fs/s3 | Steve Loughran | Viraj Jasani | | | AWS SDK V2 - Update S3A region logic | Major | fs/s3 | Ahmar Suhail | Viraj Jasani | | | HADOOP-19045. S3A: CreateSession Timeout after 10 seconds | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Use hadoop-thirdparty 1.2.0 | Major | hadoop-thirdparty | Shilun Fan | Shilun Fan | | | prune dependency exports of hadoop-\* modules | Blocker | build | Steve Loughran | Steve Loughran | | | Add Protobuf Compatibility Notes | Major | documentation | Shilun Fan | Shilun Fan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode | Minor | federation, webhdfs | Toshihiko Uchida | Toshihiko Uchida | | | Make some parameters configurable for SlowDiskTracker and SlowPeerTracker | Major | block placement | Tao Li | Tao Li | | | Remove unused configuration dfs.namenode.stripe.min | Minor | configuration | Tao Li | Tao Li | | | Add metrics for FSNamesystem read/write lock hold long time | Major | hdfs | Tao Li | Tao Li | | | Add namenode address in logs for block report | Minor | datanode, hdfs | Tao Li | Tao Li | | | Close FSImage and FSNamesystem after formatting is complete | Minor | namanode | Tao Li | Tao Li | | | Add metric for editPendingQ in FSEditLogAsync | Minor | metrics | Tao Li | Tao Li | | | Fix java doc in FSEditLog | Minor | documentation | Tao Li | Tao Li | | | Remove unused parameters in NameNodeProxiesClient | Minor | hdfs-client | Tao Li | Tao Li | | | Use LongAdder instead of AtomicLong | Major | metrics | Tao Li | Tao Li | | | Add location into datanode info for NameNodeMXBean | Minor | metrics, namanode | Tao Li | Tao Li | | | Remove unused parameters for DatanodeManager.handleLifeline() | Minor | namanode | Tao Li | Tao Li | | | Improve the block state change log | Minor | block placement | Tao Li | Tao Li | | | EC: Add metric EcReconstructionValidateTimeMillis for StripedBlockReconstructor | Minor | erasure-coding, metrics | Tao Li | Tao Li | | | Remove unused parameter and fix java doc for DiskBalancerCLI | Minor | diskbalancer, documentation | Tao Li | Tao Li | | | Fix flaky unit test TestDFSShell | Minor | test | Tao Li | Tao Li | | | Remove unused method reportChecksumFailure in DFSClient | Minor | dfsclient | Tao Li | Tao Li | | | Show storage type for failed volumes on namenode web | Minor | namanode, ui | Tao Li | Tao Li | | | Simplify the code with DatanodeID#getXferAddrWithHostname | Minor | datanode, metrics, namanode | Tao Li | Tao Li | | | Fix typo

ShortCircuitReplica#isStale | Minor | hdfs-client | Tao Li | Tao Li | | | Fix flaky unit tests failed due to timeout | Minor | test | Tao Li | Tao Li | | | Improve error msg for BlockMissingException | Minor | hdfs-client | Tao Li | Tao Li | | | Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents | Minor | datanode, metrics | Tao Li | Tao Li | | | Correct log format for LdapGroupsMapping | Minor | security | Tao Li | Tao Li | | | Add metrics doc for ReadLockLongHoldCount and WriteLockLongHoldCount | Minor | metrics | Tao Li | Tao Li | | | Simplify the code for DiskBalancer | Minor | diskbalancer | Tao Li | Tao Li | | | Fix HDFSCommands.md | Minor | documentation | Tao Li | Tao Li | | | Show the threshold when mover threads quota is exceeded | Minor | datanode | Tao Li | Tao Li | | | Remove no need TODO comment for ObserverReadProxyProvider | Minor | namanode | Tao Li | Tao Li | | | Fix a typo in NameNodeLayoutVersion. | Minor | namanode | ZhiWei Shi | ZhiWei Shi | | | Allow configuring Handler number for the JournalNodeRpcServer | Major | journal-node | ZanderXu | ZanderXu | | | Fix a typo in Dispatcher. | Minor | balancer | ZhiWei Shi | ZhiWei Shi | | | BlockManager#addStoredBlock should log storage id when AddBlockResult is REPLACED | Minor | hdfs | farmmamba | farmmamba | | | [QA] Need more doc for yarn.federation.policy-manager-params when LoadBasedRouterPolicy is used | Major | federation | Shen Yinjie | Shilun Fan | | | Application priority queue ACLs are not respected | Blocker | capacity scheduler | Sunil G | Sunil G | | | Update commons-codec from 1.11 to 1.14 | Major | common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove residual code of Ozone | Major | common, ozone | Wanqiang Ji | Wanqiang Ji | | | Merge QueueMapping and QueueMappingEntity | Major | yarn | Gergely Pollk | Gergely Pollk | | | Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule | Major | capacity scheduler | Gergely Pollk | Gergely Pollk | | | Avoid unnecessary QueueMappingEntity creations | Minor | resourcemanager | Gergely Pollk | Hudky Mrton Gyula | | | CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy | Major | capacity scheduler | Gergely Pollk | Szilard Nemeth | | | Upgrade spotbugs to 4.0.6 | Minor | build, command | Tsuyoshi Ozawa | Masatake Iwasaki | | | Add .asf.yaml to allow github and jira integration | Major | build, common | Ayush Saxena | Ayush Saxena | | | Distcp doesn't close the job after the job is completed | Major | distcp | Aasha Medhi | Aasha Medhi | | | Update Mockserver | Major | hdfs-client | Wei-Chiu Chuang | Attila Doroszlai | | | Remove unused joda-time | Major | build, common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review | Major | capacity scheduler, test | Gergely Pollk | Szilard Nemeth | | | Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes | Critical | webapp | Sunil G | Jim Brennan | | | Update the year to 2021 | Major | common | Xiaoqiao He | Xiaoqiao He | | | Reduce threadCount for unit tests to reduce the memory usage | Major | build, test | Akira Ajisaka | Akira Ajisaka | | | Upgrade com.fasterxml.woodstox:woodstox-core for security reasons | Major | common | Viraj Jasani | Viraj Jasani | | | DFSAdmin#printOpenFiles has

String#format usage | Minor | dfsadmin | Viraj Jasani | Viraj Jasani | | | Removed duplicate dependency of hadoop-annotations | Minor | hdfs | Viraj Jasani | Viraj Jasani | | | Bump netty to the latest 4.1.61 | Blocker | build, common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Avoid usage of deprecated IOUtils#cleanup API | Minor | common | Viraj Jasani | Viraj Jasani | | | Remove any rocksdb exclusion code | Major | common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Update to Jetty 9.4.39 | Major | build, common | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Split TestBalancer into two classes | Major | balancer, test | Viraj Jasani | Viraj Jasani | | | Superuser actions should be reported to external enforcers | Major | security | Vivek Ratnavel Subramanian | Vivek Ratnavel Subramanian | | | RmWebApp add default-node-label-expression to the queue info | Major | resourcemanager, webapp | Gergely Pollk | Gergely Pollk | | | TestMetricsInvariantChecker.testManyRuns is broken since HADOOP-17524 | Major | test | Gergely Pollk | Gergely Pollk | | | Bump YARN CSI protobuf version to 3.7.1 | Major | yarn | Siyao Meng | Siyao Meng | | | Restrict imports from org.apache.curator.shaded | Major | common | Viraj Jasani | Viraj Jasani | | | Update commons-io to 2.8.0 | Major | build, common | Wei-Chiu Chuang | Akira Ajisaka | | | Upgrade to hadoop-thirdparty-1.1.0 | Major | hadoop-thirdparty | Ayush Saxena | Wei-Chiu Chuang | | | Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl | Major | resourcemanager | Peter Bacsko | Peter Bacsko | | | Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom | Major | common | Viraj Jasani | Viraj Jasani | | | Use hadoop-thirdparty 1.1.1 | Major | hadoop-thirdparty | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Replace Guava Lists usage by Hadoop's own Lists in hadoop-mapreduce-project | Major | common | Viraj Jasani | Viraj Jasani | | | Replace Guava Lists usage by Hadoop's own Lists in hadoop-common, hadoop-tools and cloud-storage projects | Major | common | Viraj Jasani | Viraj Jasani | | | Replace Guava Lists usage by Hadoop's own Lists in hadoop-hdfs-project | Major | hdfs-common | Viraj Jasani | Viraj Jasani | | | Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project | Major | yarn-common | Viraj Jasani | Viraj Jasani | | | Keep restrict-imports-enforcer-rule for Guava Lists in hadoop-main pom | Minor | common | Viraj Jasani | Viraj Jasani | | | Make GetClusterNodesRequestPBImpl thread safe | Major | client | Prabhu Joseph | SwathiChandrashekar | | | Replace IOUtils#closeQuietly usages | Major | common | Viraj Jasani | Viraj Jasani | | | Remove few duplicate dependencies from mapreduce-client's child poms | Minor | client | Viraj Jasani | Viraj Jasani | | | Update BPServiceActor Scheduler's nextBlockReportTime atomically | Major | datanode | Viraj Jasani | Viraj Jasani | | | ipc.Client not setting interrupt flag after catching InterruptedException | Minor | ipc | Viraj Jasani | Viraj Jasani | | | Use CuratorCache implementation instead of PathChildrenCache / TreeCache | Major | common | Viraj Jasani | Viraj Jasani | | | Remove ListenerHandle from Hadoop registry | Minor | common | Viraj Jasani | Viraj Jasani | | | Improve the GitHub pull request template | Major | build, documentation | Akira Ajisaka | Akira Ajisaka | | | Bump aliyun-sdk-oss to

| Major | build, common | Siyao Meng | Siyao Meng | | | Add Hadoop code formatter in dev-support | Major | common | Viraj Jasani | Viraj Jasani | | | Rename JobClientUnitTest to TestJobClient | Major | mrv2 | Dongjoon Hyun | Dongjoon Hyun | | | Provide replacement for deprecated APIs of commons-io IOUtils | Major | common | Viraj Jasani | Viraj Jasani | | | Bump netty to the latest 4.1.68 | Major | build, common | Takanobu Asanuma | Takanobu Asanuma | | | Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom | Major | build, common | Viraj Jasani | Viraj Jasani | | | Update commons-lang to 3.12.0 | Minor | build, common | Sean Busbey | Renukaprasad C | | | Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports | Major | build | Viraj Jasani | Viraj Jasani | | | DatanodeHttpServer doesn't require handler state map while retrieving filter handlers | Minor | datanode | Viraj Jasani | Viraj Jasani | | | CallerContext should not include some characters | Major | ipc | Takanobu Asanuma | Takanobu Asanuma | | | update GSON to 2.7+ | Minor | build | Sean Busbey | Igor Dvorzhak | | | Update the year to 2022 | Major | common | Ayush Saxena | Ayush Saxena | | | DistributedCache alternative APIs should not use DistributedCache APIs internally | Major | mrv1, mrv2 | Viraj Jasani | Viraj Jasani | | | Make AppPlacementAllocator configurable in AppSchedulingInfo | Major | scheduler | Minni Mittal | Minni Mittal | | | Basic verification for the release candidate vote | Major | build | Viraj Jasani | Viraj Jasani | | | Provide support to set Http and Rpc ports in MiniJournalCluster | Major | test | Viraj Jasani | Viraj Jasani | | | Upgrade maven enforcer plugin and relevant dependencies | Major | build | Viraj Jasani | Viraj Jasani | | | Reconfigure Block Invalidate limit | Major | block placement | Viraj Jasani | Viraj Jasani | | | Set Http and Ipc ports for Datanodes in MiniDFSCluster | Major | tets | Viraj Jasani | Viraj Jasani | | | Log retry count while handling exceptions in RetryInvocationHandler | Minor | common, io | Viraj Jasani | Viraj Jasani | | | Remove replace-guava from replacer plugin | Major | build | Viraj Jasani | Viraj Jasani | | | Utility to identify git commit / Jira fixVersion discrepancies for RC preparation | Major | build | Viraj Jasani | Viraj Jasani | | | Remove DummyGroupMapping as it is not longer used anywhere | Minor | httpfs, test | Viraj Jasani | Ashutosh Gupta | | | Update hadoop-vote to use HADOOP\RC\VERSION dir | Minor | build | Viraj Jasani | Viraj Jasani | | | Upgrade maven compiler plugin to 3.10.1 | Major | build | Viraj Jasani | Viraj Jasani | | | sync\file\range error should include more volume and file info | Minor | datanode | Viraj Jasani | Viraj Jasani | | | Remove the use if Sets#newHashSet and Sets#newTreeSet | Major | hdfs-common | Samrat Deb | Samrat Deb | | | Update google-gson to

| Minor | build | Igor Dvorzhak | Igor Dvorzhak | | | Shutdown AWSSecurityTokenService when its resources are no longer in use | Major | fs/s3 | Viraj Jasani | Viraj Jasani | | | Update the doc that append to EC files is supported | Major | documentation, erasure-coding | Wei-Chiu Chuang | Ashutosh Gupta | | | HostRestrictingAuthorizationFilter should pass through requests if they don't access WebHDFS API | Major | webhdfs | Takanobu Asanuma | Takanobu Asanuma | | | NameNode should log internal EC blocks instead of the EC block group when it receives block reports | Major | erasure-coding | Takanobu Asanuma | Takanobu Asanuma | | | Make XML transformer factory more lenient | Major | common | PJ Fanning | PJ Fanning | | | Update the year to 2023 | Major | common | Ayush Saxena | Ayush Saxena | | | upgrade to jettison 1.5.3 to fix CVE-2022-40150 | Major | common | PJ Fanning | PJ Fanning | | | Fix documentation for StateStoreRecordOperations#get(Class ..., Query ...) | Trivial | documentation | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Remove netty3 dependency | Major | build | Tamas Domok | Tamas Domok | | | Add Namenode status to BPServiceActor metrics and improve logging in offerservice | Major | namanode | Viraj Jasani | Viraj Jasani | | | Remove unused mapred/LoggingHttpResponseEncoder.java | Major | mrv1 | Tamas Domok | Tamas Domok | | | Deploy Hadoop trunk version website | Major | documentation | Ayush Saxena | Ayush Saxena | | | RBF: Routers should propagate the real user in the UGI via the caller context | Major | rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | RBF: Add period state refresh to keep router state near active namenode's | Major | rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Add transfer rate quantile metrics for DataNode reads | Minor | datanode | Ravindra Dingankar | Ravindra Dingankar | | | snakeyaml dependency: upgrade to v2.0 | Major | common | PJ Fanning | PJ Fanning | | | Include jettison as direct dependency of hadoop-common | Major | common | Andras Katona | Andras Katona | | | RBF: Implement MySQL based StateStoreDriver | Major | hdfs, rbf | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code | Major | common | PJ Fanning | PJ Fanning | | | RBF: Add ops metrics for getSlowDatanodeReport in RouterClientActivity | Major | rbf | Viraj Jasani | Viraj Jasani | | | Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 | Major | common | Viraj Jasani | Viraj Jasani | | | Fix RBF JDK 11 javadoc warnings | Major | rbf | Viraj Jasani | Viraj Jasani | | | Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml | Major | build | Devaspati Krishnatri | Devaspati Krishnatri | | | upgrade to snappy-java 1.1.10.1 due to CVEs | Major | common | PJ Fanning | PJ Fanning | | | Upgrade maven-shade-plugin to 3.4.1 | Minor | build | Rohit Kumar | Rohit Kumar | | | upgrade netty to 4.1.94 due to CVE | Major | common | PJ Fanning | PJ Fanning | | | Upgrade Okio to 3.4.0 due to CVE-2023-3635 | Major | common | Rohit Kumar | Rohit Kumar | | | Add configurable logic to split resource request to least loaded SC | Major | federation | Minni Mittal | Minni