Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.1.4.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Allow overriding application submissions based on server side configs | Major | . | Jonathan Hung | pralabhkumar | | | Support configuring application priorities on a workflow level | Major | . | Jonathan Hung | Varun Saxena | | | Backport HDFS persistent memory read cache support to branch-3.1 | Major | . | Feilong He | Feilong He | | | Consistent Reads from Standby Node | Major | hdfs | Konstantin Shvachko | Konstantin Shvachko | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Change App Name Placement Rule to use App Name instead of App Id for configuration | Major | yarn | Zian Chen | Zian Chen | | | Refactor TestQueueMetrics | Minor | resourcemanager | Szilard Nemeth | Szilard Nemeth | | | Upgrade netty version to 3.10.6 | Major | . | Xiao Chen | Xiao Chen | | | WEBHDFS: Support Enable/Disable EC Policy | Major | erasure-coding, webhdfs | Ayush Saxena | Ayush Saxena | | | EC : Add Configuration to restrict UserDefined Policies | Major | erasure-coding | Ayush Saxena | Ayush Saxena | | | EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs | Major | erasure-coding, httpfs, webhdfs | Souryakanta Dwivedy | Ayush Saxena | | | Refactor name node to allow different token verification implementations | Major | . | CR Hota | CR Hota | | | KeyProvider class should implement Closeable | Major | kms | Kuhu Shukla | Kuhu Shukla | | | Make warning message more clear when there are not enough data nodes for EC write | Major | erasure-coding | Kitti Nanasi | Kitti Nanasi | | | Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon | Major | . | Surendra Singh Lilhore | Ranith Sardar | | | ipc.Client.stop() may sleep too long to wait for all connections | Major | ipc | Tsz-wo Sze | Tsz-wo Sze | | | hadoop fs expunge to add -immediate option to purge trash immediately | Major | fs | Stephen O'Donnell | Stephen O'Donnell | | | KMS should log the IP address of the clients | Major | kms | Zsombor Gegesy | Zsombor Gegesy | | | DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured | Major | . | CR Hota | CR Hota | | | When decommissioning a node, log remaining blocks to replicate periodically | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Remove unnecessary search in INodeDirectory.addChild during image loading | Major | namenode | zhouyingchao | Lisheng Sun | | | Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du | Major | datanode, performance | Lisheng Sun | Lisheng Sun | | | Allow triggerBlockReport to a specific namenode | Major | datanode | Leon Gao | Leon Gao | | | Remove excess read lock for NetworkToplogy | Major |

| Wu Weiwei | Wu Weiwei | | | Write lock held by metasave impact following RPC processing | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Create metric that sums total memory/vcores preempted per round | Major | capacity scheduler | Eric Payne | Manikandan R | | | Add queue capacity/maxcapacity percentage metrics | Major | . | Jonathan Hung | Shubham Gupta | | | Log more detail for slow RPC | Major | . | Chen Zhang | Chen Zhang | | | Print application tags in application summary | Major | . | Jonathan Hung | Manoj Kumar | | | Print application submission context label in application summary | Major | . | Jonathan Hung | Manoj Kumar | | | Fall back to configured queue ordering policy class name | Major | . | Jonathan Hung | Jonathan Hung | | | Support configure ZK\DTSM\ZK\KERBEROS\PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /\_HOST | Minor | common | luhuachao | luhuachao | | | Add submission context label to audit logs | Major | . | Jonathan Hung | Manoj Kumar | | | Optimize FileSystemAccessService#getFileSystemConfiguration | Major | httpfs, performance | Lisheng Sun | Lisheng Sun | | | Track missing DFS operations in Statistics and StorageStatistics | Major | . | Ayush Saxena | Ayush Saxena | | | Backport HADOOP-14624 to branch-3.1 | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add more tests to ratio method in TestResourceCalculator | Major | . | Szilard Nemeth | Zoltan Siegl | | | Move Superuser Check Before Taking Lock For Encryption API | Major | . | Ayush Saxena | Ayush Saxena | | | Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing | Major | . | Ayush Saxena | Ayush Saxena | | | Remove dead code from HealthMonitor | Minor | . | Fei Hui | Fei Hui | | | Use separate configs for free disk space checking for full and not-full disks | Minor | yarn | Jim Brennan | Jim Brennan | | | Tuning TaskRuntimeEstimator | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex | Minor | . | Lisheng Sun | Lisheng Sun | | | [Observer Node] Balancer should submit getBlocks to Observer Node when possible | Major | balancer & mover, hdfs | Erik Krogen | Erik Krogen | | | MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log | Major | metrics | Lukas Majercak | Lukas Majercak | | | Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml | Trivial | . | Wei-Chiu Chuang | Xieming Li | | | Backport HADOOP-16152 to branch-3.1 | Major | . | Siyao Meng | Siyao Meng | | | Skip safemode if blockTotal is 0 in new NN | Trivial | namenode | Rajesh Balamohan | Xiaoqiao He | | | Expose metrics for custom resource types in QueueMetrics | Major | . | Szilard Nemeth | Szilard Nemeth | | | Code duplication in UserGroupMappingPlacementRule | Major | . | Szilard Nemeth | Kevin Su | | | Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Allow disabling Server Name Indication (SNI) for Jetty | Major |

| Siyao Meng | Aravindan Vijayan | | | Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS\SESSION\TOKEN | Minor | documentation, fs/s3 | Mingliang Liu | Mingliang Liu | | | Guaranteed and max capacity queue metrics for custom resources | Major | . | Jonathan Hung | Manikandan R | | | Optimize log information when DFSInputStream meet CannotObtainBlockLengthException | Major | dfsclient | Xiaoqiao He | Xiaoqiao He | | | TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources | Major | capacity scheduler, test | Eric Payne | Eric Payne | | | Allow disabling app submission from REST endpoints | Major | . | Jonathan Hung | Jonathan Hung | | | CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. | Major | capacity scheduler, test | Eric Payne | Eric Payne | | | Update checkstyle to 8.26 and maven-checkstyle-plugin to 3.1.0 | Major | build | Andras Bokor | Andras Bokor | | | In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined | Critical | capacity scheduler | Eric Payne | Eric Payne | | | When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. | Major | erasure-coding, hdfs-client | lufei | lufei | | | DataNode.DataTransfer thread should catch all the expception and log it. | Major | datanode | Surendra Singh Lilhore | Hemanth Boyina | | | Recover data blocks from persistent memory read cache during datanode restarts | Major | caching, datanode | Feilong He | Feilong He | | | Purge log in KMS and HttpFS | Minor | httpfs, kms | Doris Gu | Doris Gu | | | Add ability to know datanode staleness | Minor | datanode, logging, namenode | Ahmed Hussein | Ahmed Hussein | | | Improve error handling when application recovery fails with exception | Major | resourcemanager | Gergo Repas | Wilfred Spiegelenburg | | | Allow expiration of cached locations in DFSInputStream | Minor | dfsclient | Ahmed Hussein | Ahmed Hussein | | | MRApp helpers block for long intervals (500ms) | Minor | mr-am | Ahmed Hussein | Ahmed Hussein | | | Allow inheritance of max app lifetime / default app lifetime | Major | capacity scheduler | Eric Payne | Eric Payne | | | Support wildcard in CLASSPATH for libhdfs | Major | libhdfs | John Zhuge | Muhammad Samir Khan | | | Expose diagnostics in RMAppManager summary | Major | . | Jonathan Hung | Jonathan Hung | | | Decrease lease hard limit | Minor | . | Eric Payne | Hemanth Boyina | | | Block scheduled counter never get decremet if the block got deleted before replication. | Major | 3.1.1 | Surendra Singh Lilhore | Hemanth Boyina | | | Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations | Major | . | Lisheng Sun | Lisheng Sun | | | Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path | Major | . | Kuhu Shukla | Kuhu Shukla | | | Reset LowRedundancyBlocks Iterator periodically | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Update jackson-databind to 2.9.10.2 in branch-3.1,

| Blocker | . | Wei-Chiu Chuang | Lisheng Sun | | | backport HADOOP-16775: distcp copies to s3 are randomly corrupted | Blocker | tools/distcp | Amir Shenavandeh | Amir Shenavandeh | | | [SBN read] Change ObserverRetryOnActiveException log to debug | Minor | hdfs | Chen Liang | Chen Liang | | | Add number of containers to RMAppManager summary | Major | . | Jonathan Hung | Jonathan Hung | | | Add .diff to gitignore | Minor | . | Ayush Saxena | Ayush Saxena | | | Create separate configuration for max global AM attempts | Major | . | Jonathan Hung | Bilwa S T | | | Configurable max application tags and max tag length | Major | . | Jonathan Hung | Bilwa S T | | | The suffix name of the unified compression class | Major | io | bianqi | bianqi | | | AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. | Minor | . | Jinglun | Jinglun | | | Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime | Minor | common | Jim Brennan | Jim Brennan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory | Minor | erasure-coding | Souryakanta Dwivedy | Ayush Saxena | | | Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API | Major | webhdfs | Weiwei Yang | Weiwei Yang | | | Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException | Major | common | Suma Shivaprasad | Suma Shivaprasad | | | TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk | Major | . | Ayush Saxena | Ayush Saxena | | | TestUpgradeDomainBlockPlacementPolicy is flaky | Major | . | Ayush Saxena | Ayush Saxena | | | PlacementRule interface should be for all YarnSchedulers | Major | . | Bibin Chundatt | Bibin Chundatt | | | DecayRpcScheduler decay thread should run as a daemon | Major | ipc | Erik Krogen | Erik Krogen | | | EC: Native XOR decoder should reset the output buffer before using it. | Major | ec, hdfs | Surendra Singh Lilhore | Ayush Saxena | | | "dfs.disk.balancer.max.disk.throughputInMBperSec" property is not working as per set value. | Major | diskbalancer | Ranith Sardar | Ranith Sardar | | | Distcp It should clear sub directory ACL before applying new ACL on it. | Major | tools/distcp | Ranith Sardar | Ranith Sardar | | | In ipc.Client, put a new connection could happen after stop | Major | ipc | Tsz-wo Sze | Tsz-wo Sze | | | QueueMetrics needs to be cleared before MockRM is initialized | Major | scheduler | Daniel Templeton | Peter Bacsko | | | NetworkTopology#getWeightUsingNetworkLocation return unexpected result | Major | net | Xiaoqiao He | Xiaoqiao He | | | webhdfs that connect secure hdfs should not use

parameter | Minor | webhdfs | KWON BYUNGCHANG | KWON BYUNGCHANG | | | Stop all DataNodes may result in NN terminate | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Move Server logging of StatedId inside receiveRequestState() | Major | . | Konstantin Shvachko | Shweta | | | Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). | Critical | . | Paul Ward | Paul Ward | | | HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. | Critical | . | Paul Ward | Paul Ward | | | Misleading REM\_QUOTA value with snapshot and trash feature enabled for a directory | Major | snapshots | Shashikant Banerjee | Shashikant Banerjee | | | NPE during secure namenode startup | Major | hdfs | Fengnan Li | Fengnan Li | | | Regression: FileSystem cache lock parses XML within the lock | Major | fs | Gopal Vijayaraghavan | Gopal Vijayaraghavan | | | [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider | Major | . | Chao Sun | Chao Sun | | | Result of crypto -listZones is not formatted properly | Major | . | Hemanth Boyina | Hemanth Boyina | | | Connection thread's name should be updated after address changing is detected | Major | ipc | zhouyingchao | Lisheng Sun | | | HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null | Major | httpfs | Siyao Meng | Siyao Meng | | | ConcurrentModificationException in Configuration.overlay() method | Major | . | Oleksandr Shevchenko | Oleksandr Shevchenko | | | HDFS cat logs an info message | Major | . | Eric Badger | Eric Badger | | | NN should automatically set permissions on dfs.namenode.\*.dir | Major | namenode | Aaron Myers | Siddharth Wagle | | | Yarn REST API, services endpoint remote command ejection | Major | . | Eric Yang | Eric Yang | | | RM does not start on JDK11 when UIv2 is enabled | Critical | resourcemanager, yarn | Adam Antal | Adam Antal | | | RM logs InvalidStateTransitionException when app is submitted | Critical | . | Rohith Sharma K S | Prabhu Joseph | | | RBF: Display RPC (instead of HTTP) Port Number in RBF web UI | Minor | rbf, ui | Xieming Li | Xieming Li | | | Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold | Critical | ec | Zhao Yi Ming | Zhao Yi Ming | | | Race condition when DirectoryCollection.checkDirs() runs during container launch | Major | . | Peter Bacsko | Peter Bacsko | | | YARN Service fails to fetch status for Stopped apps with bigger spec files | Major | yarn-native-services | Tarun Parimi | Tarun Parimi | | | YARN Audit logging not added to log4j.properties | Major | . | Varun Saxena | Aihua Xu | | | FileIoProvider should not increase FileIoErrors metric in datanode volume metric | Minor | . | Aiphago | Aiphago | | | ValueQueue does not trigger an async refill when number of values falls below watermark | Major | common, kms | Yuval Degani | Yuval Degani | | | NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present | Major |

| Ranith Sardar | Ranith Sardar | | | DistCp job fails when new data is appended in the file while the distCp copy job is running | Critical | distcp | Mukund Thakur | Mukund Thakur | | | EC: Improper size values for corrupt ec block in LOG | Major | ec | Harshakiran Reddy | Ayush Saxena | | | Erasure Coding: the internal block is replicated many times when datanode is decommissioning | Major | ec, erasure-coding | HuangTao | HuangTao | | | Optimize RMContext getExclusiveEnforcedPartitions | Major | . | Jonathan Hung | Jonathan Hung | | | Snapshot memory leak | Major | snapshots | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove redundant super user priveledge checks from namenode. | Major | . | Ayush Saxena | Ayush Saxena | | | NullPointerException happens in NamenodeWebHdfs | Critical | . | lujie | lujie | | | Namenode may not replicate blocks to meet the policy after enabling upgradeDomain | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Header was wrong in Snapshot web UI | Major | . | Hemanth Boyina | Hemanth Boyina | | | [SBN Read] Namenode crashes if one of The JN is down | Critical | . | Harshakiran Reddy | Ayush Saxena | | | Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero | Major | hdfs | Srinivasu Majeti | Srinivasu Majeti | | | Fix documentation about NodeHealthScriptRunner | Major | documentation, nodemanager | Peter Bacsko | Peter Bacsko | | | FairScheduler: NODE\_UPDATE can cause NoSuchElementException | Major | fairscheduler | Peter Bacsko | Peter Bacsko | | | Erasure Coding : The number of Under-Replicated Blocks never reduced | Critical | ec | Hemanth Boyina | Hemanth Boyina | | | Class cast error in GetGroups with ObserverReadProxyProvider | Major | . | Shen Yinjie | Erik Krogen | | | EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe | Critical | ec, hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x | Blocker | . | Yuxuan Wang | Yuxuan Wang | | | In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec | Major | . | Hemanth Boyina | Hemanth Boyina | | | Additional Unit tests to verify queue limit and max-limit with multiple resource types | Major | capacity scheduler | Sunil G | Adam Antal | | | Setting permissions on name directory fails on non posix compliant filesystems | Blocker | . | hirik | Siddharth Wagle | | | Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException | Major | common | Adam Antal | Adam Antal | | | DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | StagingTestBase uses methods not available in Mockito 1.8.5 in branch-3.1 | Major | . | Lisheng Sun | Duo Zhang | | | Improve temporary directory name generation in LocalDistributedCacheManager for concurrent processes | Major | . | William Watson | Haibo Chen | | | Remove unnecessary InnerNode check in NetworkTopology#add() | Minor |

| Lisheng Sun | Lisheng Sun | | | Erasure Coding: Blocks are over-replicated while EC decommissioning | Critical | ec | Fei Hui | Fei Hui | | | Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType() | Major | . | Ayush Saxena | Ayush Saxena | | | Fix FindBug issue in QueueMetrics | Minor | . | Prabhu Joseph | Prabhu Joseph | | | DN may not send block report to NN after NN restart | Major | datanode | TanYuxin | Xiaoqiao He | | | INode access time is ignored because of race between open and rename | Major | . | Jinglun | Jinglun | | | Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. | Major | . | Tarun Parimi | Tarun Parimi | | | Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. | Blocker | . | igo Goiri | Wei-Chiu Chuang | | | DFSStripedInputStream curStripeBuf is not freed by unbuffer() | Major | ec | Joe McDonnell | Zhao Yi Ming | | | hdfs crypto commands limit column width | Major | . | Eric Badger | Eric Badger | | | TestRawLocalFileSystemContract.testPermission fails if no native library | Minor | common, test | Steve Loughran | Steve Loughran | | | Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission | Major | ec | Fei Hui | Fei Hui | | | BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false | Major | . | hu xiaodong | hu xiaodong | | | rename operation should check nest snapshot | Major | namenode | Junwang Zhao | Junwang Zhao | | | Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo | Minor | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Revise PacketResponder's log. | Minor | datanode | Xudong Cao | Xudong Cao | | | Erasure Coding: Block recovery failed during decommissioning | Major | . | Fei Hui | Fei Hui | | | When lastLocatedBlock token expire, it will take 1~3s second to refetch it. | Major | hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Bootstrap standby may fail if used in-progress tailing | Major | namenode | Chen Liang | Chen Liang | | | TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup | Minor | hdfs | Jim Brennan | Jim Brennan | | | DataNode shouldn't report block as bad block if the block length is Long.MAX\_VALUE. | Major | datanode | Surendra Singh Lilhore | Hemanth Boyina | | | Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. | Minor | common | Xudong Cao | Xudong Cao | | | Add sanity check that zone key equals feinfo key while setting Xattrs | Major | encryption, hdfs | Mukul Kumar Singh | Mukul Kumar Singh | | | AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path | Minor | fs, test |

| Xieming Li | | | FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Typo in YARN Service overview documentation | Trivial | documentation | Denes Gerencser | Denes Gerencser | | | Remove the disallowed element config within maven-checkstyle-plugin | Major | . | Wanqiang Ji | Wanqiang Ji | | | RpcQueueTime may be negative when the response has to be sent later | Minor | . | xuzq | xuzq | | | Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr | Major | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB | Minor | balancer & mover | Souryakanta Dwivedy | Hemanth Boyina | | | Fix resource inconsistency for queues when moving app with reserved container to another queue | Critical | capacity scheduler | jiulongzhu | jiulongzhu | | | Public Localizer is exiting in NodeManager due to NullPointerException | Major | nodemanager | Tarun Parimi | Tarun Parimi | | | Race condition during decommissioning | Major | nodemanager | Peter Bacsko | Peter Bacsko | | | Balancer getBlocks RPC dispersal does not function properly | Major | balancer & mover | Erik Krogen | Erik Krogen | | | Exception ' Invalid event: TA\TOO\MANY\FETCH\FAILURE at SUCCESS\FINISHING\CONTAINER' cause job error | Critical | . | luhuachao | luhuachao | | | ReplicaCachingGetSpaceUsed throws ConcurrentModificationException | Major | datanode, performance | Ryan Wu | Aiphago | | | Invalid event TA\TOO\MANY\FETCH\FAILURE at SUCCESS\CONTAINER\CLEANUP causes job failure | Critical | applicationmaster, mrv2 | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method | Major | datanode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Fix building instruction to enable zstd | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Data loss in case of distcp using snapshot diff. Replication should include rename records if file was skipped in the previous iteration | Major | distcp | Aasha Medhi | Aasha Medhi | | | Fix docker failed to build yetus/hadoop | Blocker | build | Kevin Su | Kevin Su | | | Balancer crashes when it fails to contact an unavailable NN via ObserverReadProxyProvider | Major | balancer & mover | Erik Krogen | Erik Krogen | | | Active NameNode should not silently fail the image transfer | Major | namenode | Konstantin Shvachko | Chen Liang | | | NameQuota is not update after concat operation, so namequota is wrong | Major | . | Ranith Sardar | Ranith Sardar | | | bower install fails | Blocker | build, yarn-ui-v2 | Akira Ajisaka | Akira Ajisaka | | | Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. | Major | test | Konstantin Shvachko | Konstantin Shvachko | | | Replace curator-shaded guava import with the standard one | Minor | hdfs-client | Akira Ajisaka | Chandra Sanivarapu | | | Update the link to HadoopJavaVersion | Minor | documentation | Akira Ajisaka | Chandra Sanivarapu | | | [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0 | Major |

| Takanobu Asanuma | Ayush Saxena | | | Placement rules do not use correct group service init | Major | yarn | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | DataNode could meet deadlock if invoke refreshVolumes when register | Major | datanode | Xiaoqiao He | Aiphago | | | Fix typo in MapReduce documentaion example | Trivial | documentation | Sergey Pogorelov | Sergey Pogorelov | | | HDFS MiniCluster fails to start when run in directory path with a % | Minor | . | Geoffrey Jacoby | Masatake Iwasaki | | | Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Fix the issue in reading persistent memory cached data with an offset | Major | caching, datanode | Feilong He | Feilong He | | | org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently | Major | . | Miklos Szegedi | Jim Brennan | | | TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out | Major | nodemanager | Jason Darrell Lowe | Chandni Singh | | | INodeReference Space Consumed was not same in QuotaUsage and ContentSummary | Major | namenode | Hemanth Boyina | Hemanth Boyina | | | Handling 0 progress in SimpleExponential task runtime estimator | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Configuration parsing of CDATA values are blank | Major | conf | Jonathan Turner Eagles | Daryn Sharp | | | Fix accidental comment in flaky test TestDecommissioningStatus | Major | hdfs | Ahmed Hussein | Ahmed Hussein | | | [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException on ObserverNode | Major | namenode | Konstantin Shvachko | Chen Liang | | | AvailableSpaceBlockPlacementPolicy always prefers local node | Major | block placement | Wei-Chiu Chuang | Ayush Saxena | | | Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException | Major | common | Adam Antal | Adam Antal | | | Fix javadoc error in SimpleExponentialSmoothing | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | RM Received RMFatalEvent of type CRITICAL\THREAD\CRASH | Major | fairscheduler, resourcemanager | Girish Bhat | Wilfred Spiegelenburg | | | Modify HistoryServerRest.html content,change The job attempt ids datatype from string to int | Major | documentation | zhaoshengjie | | | | Update decimal values for queue capacities shown on queue status CLI | Major | client | Prabhu Joseph | Prabhu Joseph | | | Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin | Minor | build | Akira Ajisaka | Xieming Li | | | Remove WARN log when ipc connection interrupted in Client#handleSaslConnectionFailure() | Minor | . | Lisheng Sun | Lisheng Sun | | | Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero | Major | . | kyungwan nam | kyungwan nam | | | checkDiskError doesn't work during datanode startup | Major | datanode | Yang Yun | Yang Yun | | | TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently | Critical | fs | Gabor Bota | Ahmed Hussein | | | testSpeculateSuccessfulWithUpdateEvents fails Intermittently | Minor | . | Ahmed Hussein | Ahmed Hussein | | | JobHistory#ServiceStop implementation is incorrect | Major | . | Jason Darrell Lowe | Ahmed Hussein | | | [SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. | Major | hdfs-client | Konstantin Shvachko | Chen Liang | | | Client-side SocketTimeoutException during Fsck | Major | namenode | Carl Steinbach | Stephen O'Donnell | | |

should not apply to primary NN port | Major | . | Chen Liang | Chen Liang | | | Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug | Major | . | wangzhixiang | wangzhixiang | | | Fix compilation failure for branch-3.1 | Major | . | Ayush Saxena | Ayush Saxena | | | The number of failed volumes mismatch with volumeFailures of Datanode metrics | Minor | datanode | Yang Yun | Yang Yun | | | start-build-env.sh behaves incorrectly when username is numeric only | Minor | build | Jihyun Cho | Jihyun Cho | | | When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close() | Major | . | Lisheng Sun | Lisheng Sun | | | Fix TestDelegationTokensWithHA | Major | . | Ayush Saxena | Ayush Saxena | | | ipc.Server readAndProcess threw NullPointerException | Major | rpc-server | Tsz-wo Sze | Tsz-wo Sze | | | Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure | Major | build | Akira Ajisaka | Akira Ajisaka | | | WebHDFS getTrashRoot leads to OOM due to FileSystem object creation | Major | webhdfs | Wei-Chiu Chuang | Masatake Iwasaki | | | StartupProgress reports edits segments until the entire startup completes | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | | | Remove redundant field fStream in ByteStringLog | Major | . | Konstantin Shvachko | Xieming Li | | | The description of hadoop.http.authentication.signature.secret.file contains outdated information | Minor | documentation | Akira Ajisaka | Xieming Li | | | Fix typo 'complaint' which means quite different in Federation.md | Minor | documentation, federation | Sungpeo Kook | Sungpeo Kook | | | LazyPersistTestCase wait logic is error-prone | Minor | . | Ahmed Hussein | Ahmed Hussein | | | Support Fuse with Users from multiple Security Realms | Critical | fuse-dfs | Sailesh Patel | Istvan Fajth | | | stopStandbyServices() should log which service state it is transitioning from. | Major | hdfs, logging | Konstantin Shvachko | Xieming Li | | | NPE in BlockSender | Major | . | Ayush Saxena | Ayush Saxena | | | Upgrade jackson-databind to 2.9.10.3 | Blocker | . | Siyao Meng | Siyao Meng | | | AliyunOSS: getFileStatus throws FileNotFoundException in versioning bucket | Major | fs/oss | wujinhu | wujinhu | | | TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically | Major | scheduler, test | Prabhu Joseph | Ahmed Hussein | | | Wrong Use Case of -showprogress in fsck | Major | . | Ravuri Sushma sree | Ravuri Sushma sree | | | EC: File write hangs during close in case of Exception during updatePipeline | Critical | . | Ayush Saxena | Ayush Saxena | | | Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs | Trivial | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | DFS Client will stuck when ResponseProcessor.run throw Error | Major | hdfs-client | zhengchenyu | zhengchenyu | | | pylint fails in the build environment | Critical | build | Akira Ajisaka | Akira Ajisaka | | | Upgrade maven-clean-plugin to 3.1.0 | Major | build | Allen Wittenauer | Akira Ajisaka | | | TaskAttemptListenerImpl excessive log messages | Major |

| Ahmed Hussein | Ahmed Hussein | | | Cache pool MAXTTL is not persisted and restored on cluster restart | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Use Yetus 0.12.0 in GitHub PR | Major | build | Akira Ajisaka | Akira Ajisaka | | | Concat on INodeRefernce fails with illegal state exception | Critical | . | Hemanth Boyina | Hemanth Boyina | | | ZKFC ignores dfs.namenode.rpc-bind-host and uses dfs.namenode.rpc-address to bind to host address | Major | ha, namenode | Dhiraj Hegde | Dhiraj Hegde | | | TestNNHandlesBlockReportPerStorage::blockReport\_02 fails intermittently in trunk | Major | datanode, test | Mingliang Liu | Ayush Saxena | | | Upgrade jackson-databind to 2.9.10.4 | Blocker | . | Siyao Meng | Siyao Meng | | | Concat on a same files deleting the file | Critical | . | Hemanth Boyina | Hemanth Boyina | | | ViewFileSystem.close doesn't close child filesystems and causes FileSystem objects leak. | Major | . | Jinglun | Jinglun | | | NetUtils.connect() throws unchecked exception (UnresolvedAddressException) causing clients to abort | Major | net | Dhiraj Hegde | Dhiraj Hegde | | | Fix double locking in CapacityScheduler#reinitialize in branch-3.1 | Critical | capacity scheduler | Masatake Iwasaki | Masatake Iwasaki | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | TestCSAllocateCustomResource failures | Major | yarn | Jim Brennan | Jim Brennan | | | TestRouterWebServicesREST is corrupting STDOUT | Minor | yarn | Jim Brennan | Jim Brennan | | | TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature | Major | fs, test | John Zhuge | Jim Brennan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add ITestDynamoDBMetadataStore | Minor | fs/s3, test | Sean Mackrory | Gabor Bota | | | [JDK11] TestIPC.testRTEDuringConnectionSetup fails | Major | . | Akira Ajisaka | Zsolt Venczel | | | [SBN read] Unclear Log.WARN message in GlobalStateIdContext | Major | hdfs | Shweta | Shweta | | | [SBN Read] Add the document link to the top page | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Queue Mutation API does not allow to remove a config | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | [SBN read] Revisit GlobalStateIdContext locking when getting server state id | Major | hdfs | Chen Liang | Chen Liang | | | [SBN read] Change client logging to be less aggressive | Major | hdfs | Chen Liang | Chen Liang | | | Format CS Configuration present in Configuration Store | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | SchedConfCli does not work with https mode | Major | . | Prabhu Joseph | Prabhu Joseph | | | [SBN read] Allow configurably enable/disable AlignmentContext on NameNode | Major | hdfs | Chen Liang | Chen Liang | | | StandbyNode should upload FsImage to ObserverNode after checkpointing. | Major | hdfs | Konstantin Shvachko | Chen Liang | | | Mutation API Config Change need to update Version Number | Major |

| Prabhu Joseph | Prabhu Joseph | | | Balancer should work with ObserverNode | Major | . | Konstantin Shvachko | Erik Krogen | | | Add QueueMetrics for Custom Resources | Major | . | Manikandan R | Manikandan R | | | Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches | Major | common | Duo Zhang | Duo Zhang | | | Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | Upgrade to yetus 0.11.1 and use emoji vote on github pre commit | Major | build | Duo Zhang | Duo Zhang | | | Offline format of YarnConfigurationStore | Major | capacity scheduler | Prabhu Joseph | Prabhu Joseph | | | General usability improvements in showSimulationTrace.html | Minor | scheduler-load-simulator | Adam Antal | Adam Antal | | | Refine testing.md to tell user better how to use auth-keys.xml | Minor | fs/s3 | Mingliang Liu | Mingliang Liu | | | Add Jenkinsfile for all active branches | Major | build | Duo Zhang | Akira Ajisaka | | | Create RM Rest API to validate a CapacityScheduler Configuration | Major | . | Kinga Marton | Kinga Marton | | | RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' | Minor | documentation, rbf | panlijie | panlijie | | | ValidateAndGetSchedulerConfiguration API fails when cluster max allocation \> default 8GB | Major | . | Prabhu Joseph | Prabhu Joseph | | | [FGL] Remove redundant locking on NameNode. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | | | Remove the Local Dynamo DB test option | Major | fs/s3 | Steve Loughran | Gabor Bota | | | Extend ViewFS and provide ViewFSOverloadScheme implementation with scheme configurable. | Major | fs, hadoop-client, hdfs-client, viewfs | Uma Maheswara Rao G | Uma Maheswara Rao G | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Refine PlacementRule interface and add a app-name queue mapping rule as an example | Major | . | Zian Chen | Zian Chen | | | Upgrade jetty version to 9.3.27 | Major | . | Hrishikesh Gadre | Hrishikesh Gadre | | | Update commons-beanutils version to 1.9.4 | Major | . | Wei-Chiu Chuang | Kevin Su | | | The changelog\*.md seems not generated when create-release | Blocker | . | Zhankun Tang | | | | Support forcing configured partitions to be exclusive based on app node label | Major | . | Jonathan Hung | Jonathan Hung | | | [SBNN read] access time should be turned off | Major | documentation | Wei-Chiu Chuang | Chao Sun | | | Update the year to 2020 | Major | . | Ayush Saxena | Ayush Saxena | | | Upgrade Netty version to 4.1.45.Final to handle CVE-2019-20444,CVE-2019-16869 | Major | . | Aray Chenchu Sukesh | Aray Chenchu Sukesh | | | Update Netty to 4.1.48.Final | Blocker | . | Wei-Chiu Chuang | Lisheng Sun |