Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.2.10.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: Document Router and State Store metrics | Major | documentation | Yiqun Lin | Yiqun Lin | | | RBF: Add ACL support for mount table | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Use the ZooKeeper as the default State Store | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Upgrade AWS SDK to 1.11.271 in branch-2 | Blocker | fs/s3 | Akira Ajisaka | Akira Ajisaka | | | Backport HADOOP-14816 to branch-2 | Major | build | Akira Ajisaka | Akira Ajisaka | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: Fix doc error setting up client | Major | federation | tartarus | tartarus | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Percentage based Reserved Space Calculation for DataNode | Major | datanode, hdfs | Lukas Majercak | Lukas Majercak | | | RBF: Support global quota | Major | . | igo Goiri | Yiqun Lin | | | Failover for LdapGroupsMapping | Major | common, security | Lukas Majercak | Lukas Majercak | | | Allow overriding application submissions based on server side configs | Major | . | Jonathan Hung | pralabhkumar | | | Support configuring application priorities on a workflow level | Major | . | Jonathan Hung | Varun Saxena | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve KMSClientProvider log around delegation token checking | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | CryptoInputStream should implement unbuffer | Major | fs, security | John Zhuge | John Zhuge | | | Add GC time percentage monitor/alerter | Major | . | Misha Dmitriev | Misha Dmitriev | | | ValueQueue should also validate (lowWatermark \* numValues) \> 0 on construction | Minor | . | Xiao Chen | Xiao Chen | | | Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat | Minor | . | Wei Yan | Wei Yan | | | Improve robustness of the AggregatedLogDeletionService | Major | log-aggregation | Jonathan Turner Eagles | Jonathan Turner Eagles | | |

TestUnbuffer#testUnbufferException failure | Minor | test | Jack Bearden | Jack Bearden | | | Add readahead, dropbehind, and unbuffer to StreamCapabilities | Major | fs | John Zhuge | John Zhuge | | | AliyunOSS: change the default value of max error retry | Major | fs/oss | wujinhu | wujinhu | | | Ability to disable elasticity at leaf queue level | Major | capacityscheduler | Scott Brokaw | Zian Chen | | | Add test case to verify context update after container promotion or demotion with or without auto update | Minor | nodemanager | Weiwei Yang | Weiwei Yang | | | AliyunOSS: backport HADOOP-14993 to branch-2 | Major | fs/oss | Genmao Yu | Genmao Yu | | | Support multiple storages in DataNodeCluster / SimulatedFSDataset | Minor | datanode, test | Erik Krogen | Erik Krogen | | | When NN is not able to identify DN for replication, reason behind it can be logged | Critical | hdfs-client, namenode | Surendra Singh Lilhore | Xiao Chen | | | Ability to enable logging of container memory stats | Major | nodemanager | Jim Brennan | Jim Brennan | | | Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods | Minor | . | Wei Yan | Wei Yan | | | Allow fair-scheduler configuration on HDFS | Minor | fairscheduler, resourcemanager | Greg Phillips | Greg Phillips | | | Improve container-executor validation check | Major | security, yarn | Eric Yang | Eric Yang | | | FileOutputCommitter is slow on filesystems lacking recursive delete | Minor | . | Karthik Palaniappan | Karthik Palaniappan | | | MR AM to clean up temporary files from previous attempt in case of no recovery | Major | applicationmaster | Gergo Repas | Gergo Repas | | | backport HADOOP-15039 to branch-2 and branch-3 | Blocker | . | Genmao Yu | Genmao Yu | | | Add independent secret manager method for logging expired tokens | Major | security | Daryn Sharp | Daryn Sharp | | | Expose container preemptions related information in Capacity Scheduler queue metrics | Major | . | Eric Payne | Eric Payne | | | Uber AM can crash due to unknown task in statusUpdate | Major | mr-am | Peter Bacsko | Peter Bacsko | | | ADLS to support per-store configuration | Major | fs/adl | John Zhuge | Sharad Sonker | | | Capacity Scheduler Intra-queue Preemption should be configurable for each queue | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | Authentication Tokens should use HMAC instead of MAC | Major | security | Robert Kanter | Robert Kanter | | | Optimize disk access for last partial chunk checksum of Finalized replica | Major | datanode | Wei-Chiu Chuang | Gabor Bota | | | [branch-2] Upper/Lower case conversion support for group names in LdapGroupsMapping | Major | . | Nanda kumar | Nanda kumar | | | increase maven heap size recommendations | Minor | build, documentation, test | Allen Wittenauer | Allen Wittenauer | | | BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo | Major | namenode | Konstantin Shvachko | chencan | | | Upgrade Maven surefire plugin | Major | build | Arpit Agarwal | Arpit Agarwal | | | Undocumented KeyProvider configuration keys | Major | . | Wei-Chiu Chuang | LiXin Ge | | | Fix the CapacityScheduler Queue configuration documentation | Major | . | Arun Suresh | Jonathan Hung | | | NameNode should optionally exit if it detects FsImage corruption | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | NetworkTopology should be configurable when enable DFSNetworkTopology | Major | . | Tao Jie | Tao Jie | | | Backport PowerShell NodeFencer HADOOP-14309 to branch-2 | Minor | . | igo Goiri | igo Goiri | | | Add BIND\_HOST configuration for JournalNode's HTTP and RPC Servers | Major | hdfs, journal-node | Lukas Majercak | Lukas Majercak | | | Limit httpfs binds to certain IP addresses in branch-2 | Major | httpfs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | renameSnapshot a Non-Existent snapshot to itself should throw error | Minor | hdfs | Sailesh Patel | Kitti Nanasi | | | Log kms url and token service at debug level. | Minor |

| Wei-Chiu Chuang | Gabor Bota | | | Improve logging for JournalNode in federated cluster | Major | federation, hdfs | Hanisha Koneru | Hanisha Koneru | | | Few REST api's in RMWebServices are missing static user check | Critical | webapp, yarn | Sunil G | Sunil G | | | Make NetworkTopology#netLock fair | Major | net | Nanda kumar | Nanda kumar | | | Increase default timeout of ZK session to avoid frequent NameNode failover | Critical | common | Karthik Palanisamy | Karthik Palanisamy | | | Add checkOperation(WRITE) checks in FSNamesystem | Major | ha, namenode | Erik Krogen | Chao Sun | | | Backport HDFS-10376 to branch-2 | Major | . | Yiqun Lin | Zsolt Venczel | | | Make dfs.client.failover.random.order a per nameservice configuration | Major | federation | Ekanth Sethuramalingam | Ekanth Sethuramalingam | | | Add overall metrics for FSNamesystemLock | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | Fix TestNameNodePrunesMissingStorages test failures on Windows | Major | hdfs, namenode, test | Lukas Majercak | Lukas Majercak | | | Docs around dfs.image.transfer.timeout are misleading | Major | documentation | Kitti Nanasi | Kitti Nanasi | | | make DEFAULT\MAX\CORRUPT\FILEBLOCKS\RETURNED configurable | Minor | hdfs | Lantao Jin | Lantao Jin | | | Improve ATSv2 client logging in RM and NM publisher | Major | . | Rohith Sharma K S | Abhishek Modi | | | Remove super user privilege requirement for NameNode.getServiceStatus | Minor | namenode | Chao Sun | Chao Sun | | | Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint | Major | resourcemanager | Anna Savarin | Weiwei Yang | | | Exit NameNode if dangling child inode is detected when saving FsImage | Major | hdfs, namenode | Siyao Meng | Siyao Meng | | | RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache | Major | hdfs | Fei Hui | Fei Hui | | | Add "\*.patch" into .gitignore file of branch-2 | Major | . | Rui Gao | Rui Gao | | | Make block increment deletion number configurable | Major | . | Yiqun Lin | Ryan Wu | | | TestRMEmbeddedElector#testCallbackSynchronization is flakey | Major | test | Robert Kanter | Robert Kanter | | | WASB: improve listStatus performance | Major | fs/azure | Thomas Marqardt | Thomas Marqardt | | | RBF: Choose to enable the default nameservice to read/write files | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | Fix the inconsistent default refresh interval on Caching documentation | Trivial | documentation | David Mollitor | Hrishikesh Gadre | | | Registering MutableQuantiles via Metric annotation | Major | metrics | Sushil Ks | Sushil Ks | | | Add JMX, conf and stacks menus to the datanode page | Minor | datanode | fengchuang | fengchuang | | | Create utility to limit frequency of log statements | Major | common, util | Erik Krogen | Erik Krogen | | | Allow updating application tracking url after registration | Major | . | Jonathan Hung | Jonathan Hung | | | Refactor TestQueueMetrics | Minor | resourcemanager | Szilard Nemeth | Szilard Nemeth | | | Limit the maximum number of container assignments per heartbeat | Major | . | Weiwei Yang | Zhankun Tang | | | upgrade to commons-compress 1.18 | Major | . | PJ Fanning | Akira Ajisaka | | | Update the doc about the default value of "maximum-container-assignments" for capacity scheduler | Minor |

| Zhankun Tang | Zhankun Tang | | | Add queue and partition info to RM audit log | Major | resourcemanager | Jonathan Hung | Eric Payne | | | AliyunOSS: Enable Yarn to use OSS | Major | fs/oss | wujinhu | wujinhu | | | AliyunOSS: add missing owner & group attributes for oss FileStatus | Major | fs/oss | wujinhu | wujinhu | | | Escape newlines in health report in YARN UI | Major | . | Jonathan Hung | Keqiu Hu | | | Add Guaranteed and MaxCapacity to CSQueueMetrics | Major | . | Jonathan Hung | Jonathan Hung | | | Performance improvement in Tailing EditLog | Major | namenode | Kenneth Yang | Kenneth Yang | | | Emit FairCallQueue stats as metrics | Major | metrics, rpc-server | Erik Krogen | Christopher Gregorian | | | Node.js and npm package loading in the Dockerfile failing on branch-2 | Major | build | Allen Wittenauer | Akhil PB | | | AliyunOSS: update oss-sdk version to 3.4.1 | Major | fs/oss | wujinhu | wujinhu | | | RM UI should sort memory / cores numerically | Major | . | Eric Maynard | Ahmed Hussein | | | Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication' | Trivial | . | Szilard Nemeth | Charan Hebri | | | ipc.Client.stop() may sleep too long to wait for all connections | Major | ipc | Tsz-wo Sze | Tsz-wo Sze | | | Repeat adding node description into network topology | Minor | datanode | HuangTao | HuangTao | | | Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2 | Major | ATSv2 | Sushil Ks | Sushil Ks | | | Improve HDFS append performance | Major | hdfs | Chao Sun | Chao Sun | | | Backport HDFS-6440 to branch-2 | Major | . | Chen Liang | Chao Sun | | | Backport HDFS-9659 to branch-2 | Minor | . | Chao Sun | Chao Sun | | | Backport HDFS-13799 to branch-2 | Trivial | . | Chao Sun | Chao Sun | | | dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml | Minor | hdfs | puleya7 | puleya7 | | | Log correct cpu controller path on error while initializing CGroups. | Major | nodemanager | Jonathan Hung | Jonathan Hung | | | keepResults option in NNThroughputBenchmark should call saveNamespace() | Major | benchmarks, hdfs | Konstantin Shvachko | Konstantin Shvachko | | | https everywhere in Maven settings | Minor | build | Akira Ajisaka | Akira Ajisaka | | | Resource report REST API could return NaN or Inf | Minor | . | Ahmed Hussein | Ahmed Hussein | | | FSImage which is saving should be clean while NameNode shutdown | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | Fix zstandard shortname misspelled as zts | Major | . | Jonathan Turner Eagles | Jonathan Turner Eagles | | | Bundle ZSTD native in branch-2 | Major | native | Chao Sun | Chao Sun | | | Should update maxQueueSize when refresh call queue | Minor | . | Tao Jie | Tao Jie | | | Add more fine-grained processing time metrics to the RPC layer | Minor | ipc | Christopher Gregorian | Erik Krogen | | | Making md5 computing being in parallel with image loading | Major | . | zhouyingchao | Lisheng Sun | | | Reduce useless #getNumLiveDataNodes call in SafeModeMonitor | Major | namenode | Xiaoqiao He | Xiaoqiao He | | |

costs additional memory even the storage type quota is not set. | Major | . | Jinglun | Jinglun | | | Backport HDFS-14513 to branch-2 | Minor | namenode | Xiaoqiao He | Xiaoqiao He | | | Reduce log output from container status calls | Major | . | Shane Kumpf | Craig Condit | | | Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du | Major | datanode, performance | Lisheng Sun | Lisheng Sun | | | Backport HDFS-11273 to branch-2 (Move TransferFsImage#doGetUrl function to a Util class) | Major | . | Siyao Meng | Siyao Meng | | | Edit log tailing fast-path should allow for backoff | Major | namenode, qjm | Erik Krogen | Erik Krogen | | | container working directory has group read permissions | Minor | yarn | Jim Brennan | Jim Brennan | | | Backport [HADOOP-16266] "Add more fine-grained processing time metrics to the RPC layer" to branch-2 | Major | . | Erik Krogen | Erik Krogen | | | Add JAVA\LIBRARY\PATH to HTTPFS startup options in branch-2 | Major | httpfs | Masatake Iwasaki | Masatake Iwasaki | | | Add helper method FSNamesystem#setBlockManagerForTesting() in branch-2 | Blocker | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | [SBN read] Reduce tailing overhead | Major | ha, namenode | Wei-Chiu Chuang | Ayush Saxena | | | Improve fsimage load time by writing sub-sections to the fsimage index | Major | namenode | Stephen O'Donnell | Stephen O'Donnell | | | Create metric that sums total memory/vcores preempted per round | Major | capacity scheduler | Eric Payne | Manikandan R | | | The StorageType quota and consume in QuotaFeature is not handled for rename | Major | . | Jinglun | Jinglun | | | Upgrade bundled Tomcat in branch-2 | Major | httpfs, kms | Masatake Iwasaki | Masatake Iwasaki | | | Add queue capacity/maxcapacity percentage metrics | Major | . | Jonathan Hung | Shubham Gupta | | | Print application tags in application summary | Major | . | Jonathan Hung | Manoj Kumar | | | Print application submission context label in application summary | Major | . | Jonathan Hung | Manoj Kumar | | | Update xercesImpl in branch-2 | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fall back to configured queue ordering policy class name | Major | . | Jonathan Hung | Jonathan Hung | | | Changes for initializing placement rules with ResourceScheduler in branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Add submission context label to audit logs | Major | . | Jonathan Hung | Manoj Kumar | | | Backport [HDFS-14403] "Cost-based FairCallQueue" to branch-2 | Major | . | Erik Krogen | Erik Krogen | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Document wrong spelling in usage of MapredTestDriver tools. | Major | documentation | LiXin Ge | LiXin Ge | | | Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS | Major | httpfs, webhdfs | Zoran Dimitrijevic | Zoran Dimitrijevic | | | Fix IOException condition for openInfo in DFSInputStream | Major | . | legend | legend | | | FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured | Major |

| Manoj Govindassamy | Manoj Govindassamy | | | NPE when accessing container logs due to null dirsHandler | Major | . | Jonathan Hung | Jonathan Hung | | | Preemption properties should be refreshable | Major | capacity scheduler, scheduler preemption | Eric Payne | Gergely Novk | | | Add containerId to Localizer failed logs | Minor | nodemanager | Prabhu Joseph | Prabhu Joseph | | | Cleanup FixedValueResource to avoid dependency to ResourceUtils | Major | resourcemanager | Sunil G | Wangda Tan | | | [branch-2] "dfsrouter" should use hdfsScript | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Reset the upload button when file upload fails | Critical | ui, webhdfs | Brahma Reddy Battula | Brahma Reddy Battula | | | Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env | Major | common | Peter Bacsko | Peter Bacsko | | | ConcurrentModificationException in RMAppImpl#getRMAppMetrics | Major | capacityscheduler | Tao Yang | Tao Yang | | | Incorrect query parameters in cluster nodes REST API document | Minor | documentation | Tao Yang | Tao Yang | | | RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1 | Major | ha | Mukul Kumar Singh | Mukul Kumar Singh | | | ISA-L build options are documented in branch-2 | Major | build, documentation | Akira Ajisaka | Akira Ajisaka | | | GC time percentage reported in JvmMetrics should be a gauge, not counter | Major | . | Misha Dmitriev | Misha Dmitriev | | | ContainerLocalizer doesn't have a valid log4j config when using LinuxContainerExecutor | Major | nodemanager | Yufei Gu | Yufei Gu | | | Lease renewal can hit a deadlock | Major | . | Kuhu Shukla | Kuhu Shukla | | | INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit | Critical | namenode | DENG FEI | Konstantin Shvachko | | | Make FsServerDefaults cache configurable. | Minor | . | Rushabh Shah | Mikhail Erofeev | | | Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0 | Minor | fs/azure | Rajesh Balamohan | Rajesh Balamohan | | | Delete copy-on-truncate block along with the original block, when deleting a file being truncated | Blocker | hdfs | Jiandan Yang | Konstantin Shvachko | | | TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently | Major | resourcemanager | Daniel Templeton | Gergo Repas | | | AM lacks flow control for task events | Major | mr-am | Jason Darrell Lowe | Peter Bacsko | | | quote\and\append\_arg can overflow buffer | Major | nodemanager | Jason Darrell Lowe | Jim Brennan | | | TestNMWebServices#testGetNMResourceInfo fails on trunk | Major | nodemanager, webapp | Gergely Novk | Gergely Novk | | | Handle old RMDelegationToken format when recovering RM | Major | resourcemanager | Tatyana But | Robert Kanter | | | Remove subversion related code from VersionInfoMojo.java | Minor | build | Akira Ajisaka | Ajay Kumar | | | DFSZKFailoverController daemon exits with wrong status code | Major | auto-failover | Doris Gu | Bharat Viswanadham | | | Router UI is missing robots.txt file | Major |

| Bharat Viswanadham | Bharat Viswanadham | | | Remove the trailing duplicated timestamp in container diagnostics message | Minor | nodemanager | Weiwei Yang | Weiwei Yang | | | Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x "json-lib" | Blocker | fs/oss | Christopher Douglas | Sammi Chen | | | Incorrect sTarget column causing DataTable warning on RM application and scheduler web page | Major | resourcemanager, webapp | Weiwei Yang | Gergely Novk | | | Distcp : Update the usage of delete option for dependency with update and overwrite option | Minor | distcp, hdfs | Harshakiran Reddy | usharani | | | NM print inappropriate error log when node-labels is enabled | Minor | . | Yang Wang | Yang Wang | | | Allow read-only access to reserved raw for non-superusers | Major | namenode | Daryn Sharp | Rushabh Shah | | | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Darrell Lowe | Ajay Kumar | | | Container launching code suppresses close exceptions after writes | Major | nodemanager | Jason Darrell Lowe | Jim Brennan | | | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Darrell Lowe | Jim Brennan | | | KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5\_CONFIG | Minor | security | Vipin Rathor | Vipin Rathor | | | NodeManager metrics return wrong value after update node resource | Major | . | Yang Wang | Yang Wang | | | TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently | Critical | test | Xiao Chen | Bharat Viswanadham | | | [Atsv2] Define new set of configurations for reader and collectors to bind. | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | Update Timeline Reader web app address in UI2 | Major | . | Rohith Sharma K S | Sunil G | | | Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED | Major | . | Arun Suresh | Sampada Dehankar | | | NPE due to Invalid KerberosTicket in UGI | Major | . | Jitendra Nath Pandey | Mukul Kumar Singh | | | Skip validating priority acls while recovering applications | Blocker | resourcemanager | Charan Hebri | Sunil G | | | Concurrent task progress updates causing NPE in Application Master | Blocker | mr-am | Gergo Repas | Gergo Repas | | | Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | queueUsagePercentage is coming as INF for getApp REST api call | Major | webapp | Sunil G | Sunil G | | | NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | Ignore expired containers from removed nodes in FairScheduler | Critical | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode | Major | capacityscheduler | Tao Yang | Tao Yang | | | RMAppImpl:Invalid event: START at KILLED | Major | resourcemanager | lujie | lujie | | | Invalid event: ATTEMPT\ADDED at FINAL\SAVING | Major | yarn | lujie | lujie | | | TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky | Major |

| Miklos Szegedi | Miklos Szegedi | | | Fix typo in YARN documentation | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector | Minor | yarn | Prabhu Joseph | Prabhu Joseph | | | Sync rbw dir on the first hsync() to avoid file lost on power failure | Critical | . | Kanaka Kumar Avvaru | Vinayakumar B | | | Create the container log directory with correct sticky bit in C code | Major | nodemanager | Yufei Gu | Yufei Gu | | | Make Datanode Netty reverse proxy port to be configurable | Major | datanode | Vinayakumar B | Vinayakumar B | | | Add an additional check to the validity of container and application ids passed to container-executor | Major | nodemanager | Miklos Szegedi | Yufei Gu | | | in FsShell, UGI params should be overidden through env vars(-D arg) | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Typo in SecureMode.md | Trivial | documentation | Masahiro Tanaka | Masahiro Tanaka | | | Distributed Shell should use timeline async api's | Major | distributed-shell | Sumana Sathish | Rohith Sharma K S | | | Encounter NullPointerException when using DecayRpcScheduler | Major | . | Tao Jie | Tao Jie | | | Possible race condition in JHS if the job is not loaded | Major | jobhistoryserver | Peter Bacsko | Peter Bacsko | | | prelaunch.err file not found exception on container failure | Major | . | Jonathan Hung | Keqiu Hu | | | Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md | Trivial | documentation | Jianfei Jiang | Jianfei Jiang | | | NM heartbeat stuck when responseId overflows MAX\_INT | Critical | . | Botong Huang | Botong Huang | | | MR should not try to clean up at first job attempt | Major | . | Takanobu Asanuma | Gergo Repas | | | Task timeout in uber mode can crash AM | Major | mr-am | Akira Ajisaka | Peter Bacsko | | | [Atsv2] GSSException: No valid credentials provided - Failed to find any Kerberos tgt thrown by Timelinev2Client & HBaseClient in NM | Blocker | . | Sumana Sathish | Rohith Sharma K S | | | Exception message is not printed when creating an encryption zone fails with AuthorizationException | Minor | encryption | fang zhenyi | fang zhenyi | | | A misleading variable's name in ApplicationAttemptEventDispatcher | Minor | resourcemanager | Jinjiang Ling | Jinjiang Ling | | | Add an option to not disable short-circuit reads on failures | Major | hdfs-client, performance | Andre Araujo | Xiao Chen | | | Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. | Critical | hdfs, webhdfs | Yongjun Zhang | Yongjun Zhang | | | TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error | Major | test | Jason Darrell Lowe | Botong Huang | | | AmFilterInitializer should addFilter after fill all parameters | Critical | . | Sumana Sathish | Wangda Tan | | | NPE during container relaunch | Major | . | Billie Rinaldi | Jason Darrell Lowe | | | In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted | Major |

| Yongjun Zhang | Yongjun Zhang | | | Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up | Major | tools | Jianfei Jiang | Jianfei Jiang | | | Snapshot diff could be corrupted after concat | Major | namenode, snapshots | Xiaoyu Yao | Xiaoyu Yao | | | ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster. | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | refreshNamenodes does not support adding a new standby to a running DN | Critical | datanode, ha | Jian Fang | Ajith S | | | TestFixedLengthInputFormat#testFormatCompressedIn is flaky | Major | client, test | Peter Bacsko | Peter Bacsko | | | Token expiration edits may cause log corruption or deadlock | Critical | namenode | Daryn Sharp | Daryn Sharp | | | Timed out tasks can fail to produce thread dump | Major | . | Jason Darrell Lowe | Jason Darrell Lowe | | | BZip2 drops and duplicates records when input split size is small | Major | . | Aki Tanaka | Aki Tanaka | | | Fix http method name in Cluster Application Timeout Update API example request | Minor | docs, documentation | Charan Hebri | Charan Hebri | | | Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | Java Doc error in UnmanagedAMPoolManager for branch-2 | Major | . | Rohith Sharma K S | Botong Huang | | | Add json-smart explicitly to pom.xml | Major | common | Ray Chiang | Ray Chiang | | | Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider | Trivial | documentation | Akira Ajisaka | Gabor Bota | | | HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails | Critical | harchive | Gergely Novk | Gergely Novk | | | After Datanode down, In Namenode UI Datanode tab is throwing warning message. | Major | datanode | Harshakiran Reddy | Brahma Reddy Battula | | | Failed block recovery leaves files open indefinitely and at risk for data loss | Major | . | Daryn Sharp | Kihwal Lee | | | Backport HADOOP-13514 (surefire upgrade) to branch-2 | Major | test | Christopher Douglas | Christopher Douglas | | | CachePool permissions incorrectly checked | Major | . | Yiqun Lin | Jianfei Jiang | | | branch-2 site not building after ADL troubleshooting doc added | Major | documentation | Steve Loughran | Steve Loughran | | | [Atsv2] Race condition in NM while publishing events if second attempt is launched on the same node | Critical | . | Rohith Sharma K S | Rohith Sharma K S | | | Incorrect javadoc for return type of RetryPolicy#shouldRetry | Minor | documentation | Nanda kumar | Nanda kumar | | | NPE in ContainerLocalizer when localization failed for running container | Major | nodemanager | Tao Yang | Tao Yang | | | TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun | Minor | test | Gergely Novk | Gergely Novk | | | Upgrade from findbugs 3.0.1 to spotbugs 3.1.2 in branch-2 to fix docker image build | Major | . | Xiao Chen | Akira Ajisaka | | | Remove unused imports from

| Minor | test | Akira Ajisaka | Ajay Kumar | | | Kerberized inotify client fails despite kinit properly | Major | namenode | Wei-Chiu Chuang | Xiao Chen | | | Fix itemization in YARN federation document | Minor | documentation | Akira Ajisaka | Sen Zhao | | | File not closed if streamer fail with DSQuotaExceededException | Major | hdfs-client | Xiao Chen | Xiao Chen | | | Support fully qualified hdfs path in EZ commands | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml | Major | mrv2 | Daniel Templeton | Sen Zhao | | | TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition | Major | . | Eric Badger | Eric Badger | | | TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available | Major | . | Yiqun Lin | Yiqun Lin | | | TestFSImage fails without -Pnative | Major | test | Akira Ajisaka | Akira Ajisaka | | | Fix incorrect null value check | Minor | hdfs | Jianfei Jiang | Jianfei Jiang | | | Ignore minReplication for block recovery | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer | Minor | . | Miklos Szegedi | Miklos Szegedi | | | GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases | Major | . | Xiao Liang | Xiao Liang | | | TestWebHdfsFileContextMainOperations fails on Windows | Major | . | igo Goiri | Xiao Liang | | | Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread | Major | . | Jonathan Turner Eagles | Jonathan Turner Eagles | | | Revert YARN-6078 | Blocker | . | Billie Rinaldi | Billie Rinaldi | | | DataNode conf page cannot display the current value after reconfig | Minor | datanode | maobaolong | maobaolong | | | DistributedShellTimelinePlugin wrongly check for entityId instead of entityType | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ | Blocker | yarn | Sunil G | Sunil G | | | LDAP Group Mapping over SSL can not specify trust store | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Improve NetworkTopology chooseRandom's loop | Major | . | Xiao Chen | Xiao Chen | | | TestCommonConfigurationFields is broken by HADOOP-15312 | Major | test | Konstantin Shvachko | LiXin Ge | | | WebHdfs file path gets truncated when having semicolon (;) inside | Major | webhdfs | Zsolt Venczel | Zsolt Venczel | | | Branch-2 pre-commit failed to build docker image | Major | . | Xiao Chen | Xiao Chen | | | Configuration.getPropsWithPrefix no longer does variable substitution | Major | . | Jim Brennan | Jim Brennan | | | Update mapreduce.job.tags description for making use for ATSv2 purpose. | Major | . | Charan Hebri | Charan Hebri | | | TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | NPE occurred when container allocation proposal is applied but its resource requests are removed before | Critical | . | Tao Yang | Tao Yang | | | Fix the section titles of transparent encryption document | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Over-allocate node resource in async-scheduling mode of CapacityScheduler | Major | capacityscheduler | Tao Yang | Tao Yang | | | Potential null dereference in DFSck#doWork() | Minor | . | Ted Yu | skrho | | | JVM can crash with SIGSEGV when exiting due to custom leveldb logger | Major | nodemanager, resourcemanager | Jason Darrell Lowe | Jason Darrell Lowe | |

TestClientRMService#testGetApplications sporadically fails | Major | test | Jason Darrell Lowe | Jason Darrell Lowe | | | MiniHadoopClusterManager doesn't respect lack of format option | Minor | . | Erik Krogen | Erik Krogen | | | Increase the default value of yarn.timeline-service.app-collector.linger-period.ms | Major | . | Rohith Sharma K S | Charan Hebri | | | Incorrect queue name logging in AbstractContainerAllocator | Trivial | capacityscheduler | Weiwei Yang | Weiwei Yang | | | Fix a potential NPE in AbstractSchedulerPlanFollower | Major | . | lujie | lujie | | | OIV ReverseXML Processor fails with escaped characters | Critical | hdfs | Erik Krogen | Erik Krogen | | | branch-2 : daemon processes' sysout overwrites 'ulimit -a' in daemon's out file | Minor | scripts | Ranith Sardar | Ranith Sardar | | | Some java source files are executable | Minor | . | Akira Ajisaka | Shashikant Banerjee | | | [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | NullPointerException while launching ApplicationMaster | Major | . | lujie | lujie | | | MiniDFSCluster to support being built on randomized base directory | Major | test | Xiao Liang | Xiao Liang | | | Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens | Critical | . | Xiao Chen | Xiao Chen | | | Test cases of TestWriteToReplica failed in windows | Major | . | Xiao Liang | Xiao Liang | | | Document how to use classpath isolation for aux-services in YARN | Major | . | Xuan Gong | Xuan Gong | | | Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap | Critical | yarn | Sumana Sathish | Suma Shivaprasad | | | Many tests are failing in hadoop-distcp project in branch-2 | Critical | tools/distcp | Rushabh Shah | Jason Darrell Lowe | | | Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled | Major | . | Yesha Vora | Xuan Gong | | | Application State is not updated to ATS if AM launching is delayed. | Critical | . | Sumana Sathish | Rohith Sharma K S | | | mapred job -history prints duplicate counter in human output | Major | client | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps | Major | webapp | Sunil G | Sunil G | | | Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows | Major | . | Xiao Liang | Xiao Liang | | | Optimize TokenCache#obtainTokensForNamenodesInternal | Major | . | Bibin Chundatt | Bibin Chundatt | | | UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty | Major | yarn | Jiandan Yang | Tao Yang | | | RMContainer lost queue name when RM HA happens | Major | resourcemanager | Hu Ziqian | Hu Ziqian | | | TestHdfsHelper does not generate jceks path properly for relative path in Windows | Major | . | Xiao Liang | Xiao Liang | | | WASB: PageBlobInputStream.skip breaks HBASE replication | Major | fs/azure | Thomas Marqardt | Thomas Marqardt | | | TestContainerSchedulerQueuing.testStartMultipleContainers failed | Major |

| Miklos Szegedi | Jim Brennan | | | Fsync fails on directories on Windows | Critical | datanode, hdfs | Lukas Majercak | Lukas Majercak | | | Backport HDFS-12378 to branch-2 | Major | datanode, hdfs, test | Lukas Majercak | Lukas Majercak | | | WASB: hflush() and hsync() regression | Major | fs/azure | Thomas Marqardt | Thomas Marqardt | | | Avoid fsync storm triggered by DiskChecker and handle disk full situation | Blocker | . | Kihwal Lee | Arpit Agarwal | | | Optimize ByteString conversions in PBHelper | Major | . | Andrew Wang | Andrew Wang | | | Fix TestFsDatasetImpl test failures on Windows | Major | . | Xiao Liang | Xiao Liang | | | Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats | Major | . | Robert Kanter | Robert Kanter | | | Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows | Major | log-aggregation | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Upgrading to 3.1 kills running containers with error "Opportunistic container queue is full" | Blocker | . | Rohith Sharma K S | Jason Darrell Lowe | | | Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient | Major | . | Andrew Wang | Andrew Wang | | | Fix TestDataNodeFaultInjector test failures on Windows | Major | test | Xiao Liang | Xiao Liang | | | Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997 | Critical | kms | Gabor Bota | Gabor Bota | | | Fix incorrect username when deny the setOwner operation | Minor | namenode | luhuachao | Zsolt Venczel | | | Fix TestHistoryViewerPrinter on windows due to a mismatch line separator | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Exclude containermanager.linux test classes on Windows | Major | . | Giovanni Matteo Fumarola | Jason Darrell Lowe | | | Refactor ConfiguredFailoverProxyProvider to make inheritance easier | Minor | hdfs-client | Chao Sun | Chao Sun | | | RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773 | Major | . | Rohith Sharma K S | igo Goiri | | | TestFileOutputCommitter fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang | | | Add config parameter to allow JHS to alway scan user dir irrespective of modTime | Critical | . | Wangda Tan | Thomas Marqardt | | | Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down | Blocker | . | Rohith Sharma K S | Rohith Sharma K S | | | Speed up TestDFSAdminWithHA | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | TestNameNodeMetrics fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang | | | TestEditLogRace fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang | | | Shell command timeout given is in seconds whereas it is taken as millisec while scheduling | Major | . | Bilwa S T | Bilwa S T | | | NodeResourceMonitor crashes on bad swapFree value | Major | . | Jim Brennan | Jim Brennan | | | Compilation is broken with -Pyarn-ui | Major | webapp | Sunil G | Sunil G | | | [UI2] new ui is not accessible with out internet connection | Blocker | . | Bibin Chundatt | Bibin Chundatt | | | Multiple NM heartbeat thread created when a slow NM resync with RM | Major | nodemanager | Botong Huang | Botong Huang | | | Randomize local dirs | Minor |

| Jim Brennan | Jim Brennan | | | Containers being launched as app tears down can leave containers in NEW state | Major | nodemanager | Jason Darrell Lowe | Jason Darrell Lowe | | | Fix broken links to RBF documentation | Minor | documentation | jwhitter | Gabor Bota | | | container-executor can crash with SIGPIPE after nodemanager restart | Major | . | Jim Brennan | Jim Brennan | | | when moving app, activeUsers is increased, even though app does not have outstanding request | Major | . | kyungwan nam | | | | TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails | Major | . | Kihwal Lee | Weiwei Yang | | | CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps | Critical | capacity scheduler, capacityscheduler | Karam Singh | Manikandan R | | | LocalFs#listLocatedStatus does not filter out hidden .crc files | Minor | fs | Erik Krogen | Erik Krogen | | | Hadoop Docker Image Pip Install Fails on branch-2 | Critical | build | Haibo Chen | Haibo Chen | | | NodeManager metrics may return wrong negative values when NM restart | Major | . | Yang Wang | Szilard Nemeth | | | Race condition in NM container launched after done | Major | . | Yang Wang | Pradeep Ambati | | | DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction | Major | namenode | Wei-Chiu Chuang | chencan | | | Fix NM Collector Service Port issue in YarnConfiguration | Major | ATSv2 | Prabha Manepalli | Prabha Manepalli | | | Test failure TestSSLHttpServer.testExcludedCiphers with TLS\ECDHE\RSA\WITH\AES\128\CBC\_SHA256 cipher suite | Major | common | Gabor Bota | Szilard Nemeth | | | Restore previous state in container-executor after failure | Major | . | Jim Brennan | Jim Brennan | | | [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked | Major | . | Rohith Sharma K S | Wangda Tan | | | The OpenSolaris taxonomy link is dead in InterfaceClassification.md | Major | documentation | Daniel Templeton | Rui Gao | | | NPE in localizer hearbeat processing if a container is killed while localizing | Major | . | lujie | lujie | | | ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns | Major | ipc | Steve Loughran | Kihwal Lee | | | Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate | Major | . | Anthony Hsu | Anthony Hsu | | | RBF: Handle mount table znode with null value | Major | federation, hdfs | yanghuafeng | yanghuafeng | | |

to support minimum number of volumes to be available | Major | datanode | usharani | Ranith Sardar | | | CS preemption monitor always fails since one under-served queue was deleted | Major | capacityscheduler, scheduler preemption | Tao Yang | Tao Yang | | | Fix dead lock during async editlog rolling if edit queue is full | Major | namenode | zhangwei | Daryn Sharp | | | Fix DN UI logs link broken when https is enabled after HDFS-13902 | Minor | datanode | Jianfei Jiang | Jianfei Jiang | | | History Server task attempts REST API returns invalid data | Major | jobhistoryserver | Oleksandr Shevchenko | Oleksandr Shevchenko | | | CapacityScheduler does not enforce max resource allocation check at queue level | Major | capacity scheduler, capacityscheduler, resourcemanager | Tarun Parimi | Tarun Parimi | | | Fix the fmt\_bytes function in the dfs-dust.js | Minor | hdfs, ui | yanghuafeng | yanghuafeng | | | StringUtils#createStartupShutdownMessage throws NPE when args is null | Major | . | Lokesh Jain | Dinesh Chitlangia | | | org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently | Major | mrv2 | Robert Joseph Evans | Jason Darrell Lowe | | | MRAppBenchmark.benchmark1() fails with NullPointerException | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko | | | ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko | | | TestDataNodeMultipleRegistrations is flaky | Major | . | igo Goiri | Ayush Saxena | | | Remove the 'Path ... should be specified as a URI' warnings on startup | Major | conf | Arpit Agarwal | Ayush Saxena | | | resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Memory leak when CapacityScheduler allocates from reserved container with non-default label | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Add missing cleanupSSLConfig() call for TestTimelineClient test | Minor | test, timelineclient | Aki Tanaka | Aki Tanaka | | | Reuse Object Mapper in KMSJSONReader | Major | kms | Jonathan Turner Eagles | Jonathan Turner Eagles | | | ZStandardDecompressor native code sets an integer field as a long | Blocker | . | Jason Darrell Lowe | Jason Darrell Lowe | | | RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice | Major | . | Ayush Saxena | Ayush Saxena | | | Adding replicas to volume map makes DataNode start slowly | Major | . | Yiqun Lin | Surendra Singh Lilhore | | | Fix deprecated maven-surefire-plugin configuration in hadoop-kms module | Minor | kms | Akira Ajisaka | Vidura Bhathiya Mudalige | | | RBF: Remove FSCK from Router Web UI | Major | . | Fei Hui | Fei Hui | | | ZStandardDecompressor.c mistakes a class for an instance | Blocker | . | Ben Lau | Jason Darrell Lowe | | | CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0 | Critical | tools/distcp | Ted Yu | Ted Yu | | | Nodemanager restart docs should describe nodemanager supervised property | Major | documentation | Jason Darrell Lowe | Suma Shivaprasad | | | Renamed HADOOP\SECURITY\GROUP\SHELL\COMMAND\_TIMEOUT keys break compatibility | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix lingering timeline collector after serviceStop in TimelineCollectorManager | Trivial | ATSv2 | Prabha Manepalli | Prabha Manepalli | | | zstd compressor can fail with a small output buffer | Major | . | Jason Darrell Lowe | Jason Darrell Lowe | | | TestUpgradeDomainBlockPlacementPolicy is flaky | Major | . | Ayush Saxena | Ayush Saxena | | | Update AWS Java SDK versions in NOTICE.txt | Major | . | Akira Ajisaka | Akira Ajisaka | | | Update JSch versions in LICENSE.txt | Major | . | Akira Ajisaka | Akira Ajisaka | | | Tolerate corrupted seen\_txid file | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. | Major |

| Sumana Sathish | Wangda Tan | | | DFSOutputStream close() throws exception on subsequent call after DataNode restart | Major | hdfs-client | Erik Krogen | Erik Krogen | | | NullPointerException when reaching max shuffle connections | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null | Critical | capacityscheduler | Tao Yang | Tao Yang | | | create-release script should set max-cache-ttl as well as default-cache-ttl for gpg-agent | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | Exclude MD5 checksum files from release artifact | Critical | build | Akira Ajisaka | Akira Ajisaka | | | The config and log of gpg-agent are removed in create-release script | Major | build | Akira Ajisaka | Dinesh Chitlangia | | | Fix error messages in HDFS-12716 | Minor | hdfs | Adam Antal | Ayush Saxena | | | SLSRunner is not loading timeline service jars causing failure | Blocker | scheduler-load-simulator | Sunil G | Yufei Gu | | | Fix typo in CommandsManual.md | Minor | . | Akira Ajisaka | Shweta | | | Configuration: Included properties are not cached if resource is a stream | Critical | . | Eric Payne | Eric Payne | | | Null resources check in ResourceInfo for branch-3.0 | Major | . | Jonathan Hung | Jonathan Hung | | | JSON License is included in the transitive dependency of aliyun-sdk-oss 3.0.0 | Blocker | . | Akira Ajisaka | Akira Ajisaka | | | AliyunOSS: bring fixes back from HADOOP-15671 | Blocker | fs/oss | wujinhu | wujinhu | | | Fix TestRMAdminCLI#testHelp | Major | resourcemanager, test | Ayush Saxena | Ayush Saxena | | | Avoid potential integer overflow when computing fair shares | Major | fairscheduler | liyakun | liyakun | | | TestSSLFactory#testServerWeakCiphers sporadically fails in precommit builds | Major | security, test | Jason Darrell Lowe | Akira Ajisaka | | | DecayRpcScheduler decay thread should run as a daemon | Major | ipc | Erik Krogen | Erik Krogen | | | [UI2] YARN UI2 page loading failed due to js error under some time zone configuration | Critical | webapp | collinma | collinma | | | RM nodes web page can not display node info | Blocker | yarn | Jiandan Yang | Jiandan Yang | | | Add application launch time to Resource Manager REST API | Major | . | Abdullah Yousufi | Kanwaljeet Sachdev | | | Print launchTime in ApplicationSummary | Major | . | Jonathan Hung | Jonathan Hung | | | YARN AHS shows submitTime for jobs same as startTime | Minor | . | Prabhu Joseph | Prabhu Joseph | | | Disable speculative execution in TestDFSIO | Major | . | Kihwal Lee | Zhaohui Xin | | | RMServerUtils does not count SHUTDOWN as an accepted state | Major | . | Kuhu Shukla | Kuhu Shukla | | | Javadoc

LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference | Minor | documentation | Szilard Nemeth | Adam Antal | | | Enable more reliable SSL connection reuse | Major | common | Daryn Sharp | Daryn Sharp | | | fullBlockReportLeaseId should be reset after registering to NN | Critical | datanode | star | star | | | ContainerExecutor does not order environment map | Trivial | nodemanager | Remi Catherinot | Remi Catherinot | | | CallQueue backoff bug fixes: doesn't perform backoff when add() is used, and doesn't update backoff when refreshed | Major | ipc | Erik Krogen | Erik Krogen | | | Backport HDFS-10536 to branch-2 | Critical | . | Chao Sun | Chao Sun | | | Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes | Minor | . | Wanqiang Ji | Wanqiang Ji | | | Clean up findbugs warning in branch-2 | Major | . | Wei-Chiu Chuang | Dinesh Chitlangia | | | DistCp documentation for command line options is misaligned. | Minor | documentation | Chris Nauroth | Masatake Iwasaki | | | Stop decommission a rack of DataNodes caused NameNode fail over to standby | Major | namenode | yunjiong zhao | yunjiong zhao | | | Dynamic refresh Disk configuration results in overwriting VolumeMap | Blocker | . | xuzq | xuzq | | | RM UI progress column is of wrong type | Minor | yarn | Ahmed Hussein | Ahmed Hussein | | | NameNode StartupProgress continues to report edit log segments after the LOADING\_EDITS phase is finished | Major | namenode | Erik Krogen | Erik Krogen | | | Fix ASF License check in pom.xml | Major | . | Wanqiang Ji | Akira Ajisaka | | | Actual read size of open file in encryption zone still larger than listing size even after enabling HDFS-11402 in Hadoop 2 | Major | encryption, hdfs, snapshots | Siyao Meng | Siyao Meng | | | ONE\_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes) | Major | . | Shen Yinjie | Ayush Saxena | | | Suppress setReplication logging. | Major | . | Kihwal Lee | Kihwal Lee | | | Aggregate Resource Allocation for each job is not present in ATS | Major | ATSv2 | Prabhu Joseph | Prabhu Joseph | | | Potential NPE when instantiating FairCallQueue metrics | Major | ipc | Erik Krogen | Erik Krogen | | | Move Server logging of StatedId inside receiveRequestState() | Major | . | Konstantin Shvachko | Shweta | | | The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read | Major | hdfs-client | Zheng Hu | Zheng Hu | | | BlockReaderRemote.sendReadResult throws NPE | Major | hdfs-client | Wei-Chiu Chuang | Yuanbo Liu | | | dfsadmin -report does not always decrease "missing blocks (with replication factor 1)" metrics when file is deleted | Major | hdfs | Kitti Nanasi | Kitti Nanasi | | | Random failure of testListCorruptFilesCorruptedBlock | Major | test | Kihwal Lee | Zsolt Venczel | | | When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. | Major | . | Yicong Cai | Yicong Cai | | | When evictableMmapped or evictable size is zero, do not throw NoSuchElementException | Major | hdfs-client, performance | Zheng Hu | Lisheng Sun | | | Property value Hard Coded in DNConf.java | Trivial | . | hemanthboyina | hemanthboyina | | | Exceptions are fatal to decommissioning monitor | Critical | namenode | Daryn Sharp | Xiaoqiao He | | | NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY | Major | hdfs | Jiandan Yang | Weiwei Yang | | | FindBugs warning in branch-2: GlobalStorageStatistics defines non-transient non-serializable instance field map | Major | fs | Wei-Chiu Chuang | Masatake Iwasaki | | | MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test | Major | test | Masatake Iwasaki | Masatake Iwasaki | |

TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build | Minor | test | Rushabh Shah | kevin su | | | UGI conf doesn't read user overridden configurations on RM and NM startup | Major | . | Jonathan Hung | Jonathan Hung | | | NPE when trying to create an error message response of SASL RPC | Major | ipc | Steve Loughran | Steve Loughran | | | Enabling SSL within LdapGroupsMapping can break system SSL configs | Major | common, security | Erik Krogen | Erik Krogen | | | [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider | Major | . | Chao Sun | Chao Sun | | | Block remain in COMMITTED but not COMPLETE caused by Decommission | Major | . | Yicong Cai | Yicong Cai | | | Backport HDFS-12703 to branch-2 | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | RpcMetrics should not be retained forever | Critical | rpc-server | Zoltan Haindrich | Zoltan Haindrich | | | Remove unnecessary log message from DFSInputStream | Trivial | . | Kihwal Lee | Chao Sun | | | Result of crypto -listZones is not formatted properly | Major | . | hemanthboyina | hemanthboyina | | | QueueMetrics has incorrect metrics when labelled partitions are involved | Major | capacity scheduler | Muhammad Samir Khan | Muhammad Samir Khan | | | In KMS docs there should be one space between KMS\_LOG and NOTE | Minor | kms | Snigdhanjali Mishra | Snigdhanjali Mishra | | | WebHDFS throws "Error writing request body to server" instead of DSQuotaExceededException | Major | webhdfs | Erik Krogen | Simbarashe Dzinamarira | | | The DirectoryScanner doesn't fix the wrongly placed replica. | Major | . | Jinglun | Jinglun | | | Fix JDK7 compatibility in branch-2 | Blocker | . | Wei-Chiu Chuang | Chen Liang | | | Percent (%) and plus (+) characters no longer work in WebHDFS | Major | webhdfs | Jing Wang | Masatake Iwasaki | | | Yet another fsimage corruption related to snapshot | Major | snapshots | Yongjun Zhang | Shashikant Banerjee | | | Multi-threading conflict at layoutVersion when loading block pool storage | Major | rolling upgrades | Yicong Cai | Yicong Cai | | | Add SHA-256 or SHA-512 checksum to release artifacts to comply with the release distribution policy | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | NameNode can kill itself if it tries to send too many txns to a QJM simultaneously | Major | namenode, qjm | Erik Krogen | Erik Krogen | | | launchTime not written to state store for running applications | Major | . | Jonathan Hung | Jonathan Hung | | | NodeManager should go unhealthy when state store throws DBException | Major | nodemanager | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fix JN incompatibility issue in branch-2 due to backport of HDFS-10519 | Blocker | journal-node | Chen Liang | Chen Liang | | | TestNMSimulator#testNMSimulator fails in branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | RM logs InvalidStateTransitionException when app is submitted | Critical | . | Rohith Sharma K S | Prabhu Joseph | | | check block directory logic not correct when there is only meta file, print no meaning warn log | Minor | datanode, hdfs | qiang Liu | qiang Liu | | | LocalFileSystem's mkdirs() does not work as expected under viewfs. | Major |

| Kihwal Lee | Kihwal Lee | | | ValueQueue does not trigger an async refill when number of values falls below watermark | Major | common, kms | Yuval Degani | Yuval Degani | | | NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present | Major | . | Ranith Sardar | Ranith Sardar | | | Optimize RMContext getExclusiveEnforcedPartitions | Major | . | Jonathan Hung | Jonathan Hung | | | [SBN Read] Namenode crashes if one of The JN is down | Critical | . | Harshakiran Reddy | Ayush Saxena | | | Class cast error in GetGroups with ObserverReadProxyProvider | Major | . | Shen Yinjie | Erik Krogen | | | DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x | Blocker | . | Yuxuan Wang | Yuxuan Wang | | | Change cipher suite when fetching tomcat tarball for branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Backport HDFS-4275 to branch-2.9 | Minor | . | igo Goiri | Xiao Liang | | | Fix TestFsck test failures on Windows | Major | hdfs | Xiao Liang | Xiao Liang | | | TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu | | | TestHDFSServerPorts#testBackupNodePorts doesn't pass on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestResolveHdfsSymlink#testFcResolveAfs fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir | Minor | . | Anbang Hu | Anbang Hu | | | TestDFSAdmin#testListOpenFiles fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestDebugAdmin#testComputeMetaCommand fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestBlockScanner does not close TestContext properly | Minor | . | Anbang Hu | Anbang Hu | | | TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestDatanodeHttpXFrame does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu | | | TestDatanodeRegistration#testForcedRegistration does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu | | | TestNestedEncryptionZones does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu | | | Insufficient system resources exist to complete the requested service for some tests on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly | Minor | . | Anbang Hu | Anbang Hu | | | TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows | Minor | test | Anbang Hu | Anbang Hu | | | TestQuorumJournalManager fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | | Randomize the test directory path for TestHDFSFileSystemContract | Minor | . | Anbang Hu | Anbang Hu | | | TestDFSShell#testSetrepLow fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | | Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA | Minor | . | Anbang Hu | Anbang Hu | | | Fix TestJavaSerialization for Windows due a mismatch line separator | Minor |

| Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Randomize baseDir for MiniDFSCluster in TestBlockScanner | Minor | . | Anbang Hu | Anbang Hu | | | Some Node Manager tests fail on Windows due to improper path/file separator | Minor | . | Anbang Hu | Anbang Hu | | | TestAMSimulator failing with NPE | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | TestBasicDiskValidator fails with NoSuchFileException | Minor | . | igo Goiri | Giovanni Matteo Fumarola | | | TestDFSAdminWithHA times out on Windows | Minor | . | Anbang Hu | Lukas Majercak | | | Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows | Major | test | Xiao Liang | Xiao Liang | | | TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896 | Minor | capacity scheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Disable libhdfs zerocopy test on Mac | Minor | libhdfs | John Zhuge | Akira Ajisaka | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add support for multiple resource types in the Resource class | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Extend DominantResourceCalculator to account for all resources | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support to read resource types from a config file | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support for binary units | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add support for resource types in the nodemanager | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | Update DominantResourceCalculator to consider all resource types in calculations | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Update the Resources class to consider all resource types | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Fix build for YARN-3926 branch | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | ResourcePBImpl imports cleanup | Trivial | resourcemanager | Daniel Templeton | Yeliang Cang | | | Improve performance of resource profile branch | Blocker | nodemanager, resourcemanager | Sunil G | Sunil G | | | Remove last uses of Long from resource types code | Minor | resourcemanager | Daniel Templeton | Daniel Templeton | | | Improve API implementation in Resources and DominantResourceCalculator class | Major | nodemanager, resourcemanager | Sunil G | Sunil G | | | DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | Performance optimizations in Resource and ResourceUtils class | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Clean up unit tests after YARN-6610 | Major | test | Daniel Templeton | Daniel Templeton | | | Add Client API to get all supported resource types from RM | Major | nodemanager, resourcemanager | Sunil G | Sunil G | |

ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter | Minor | resourcemanager | Daniel Templeton | Yu-Tang Lin | | | Optimize ResourceType information display in UI | Critical | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Fix javac and javadoc errors in YARN-3926 branch | Major | nodemanager, resourcemanager | Sunil G | Sunil G | | | Improve log message in ResourceUtils | Trivial | nodemanager, resourcemanager | Sunil G | Sunil G | | | ResourceUtils.DISALLOWED\_NAMES check is duplicated | Major | resourcemanager | Daniel Templeton | Manikandan R | | | Move newly added APIs to unstable in YARN-3926 branch | Blocker | nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Update nimbus-jose-jwt to 4.41.1 | Major | . | Ray Chiang | Ray Chiang | | | GPU Isolation: Incorrect minor device numbers written to devices.deny file | Major | . | Jonathan Hung | Jonathan Hung | | | Add hadoop-aliyun as dependency of hadoop-cloud-storage | Minor | fs/oss | Genmao Yu | Genmao Yu | | | FileNotFound handling in ResourceUtils is inconsistent | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | Use LightWeightedResource when number of resource types more than two | Critical | resourcemanager | Daniel Templeton | Sunil G | | | RBF: Set MountTableResolver as default file resolver | Minor | . | igo Goiri | igo Goiri | | | Enable user re-mapping for Docker containers by default | Blocker | security, yarn | Eric Yang | Eric Yang | | | AliyunOSS: support user agent configuration and include that & Hadoop version information to oss server | Major | fs, fs/oss | Sammi Chen | Sammi Chen | | | RBF: Add router admin commands usage in HDFS commands reference doc | Minor | documentation | Yiqun Lin | Yiqun Lin | | | RBF: Fix Javadoc parameter errors | Minor | . | Wei Yan | Wei Yan | | | Gpu Information page could be empty for nodes without GPU | Major | webapp, yarn-ui-v2 | Sunil G | Sunil G | | | Webhdfs file system should get delegation token from kms provider. | Major | encryption, kms, webhdfs | Rushabh Shah | Rushabh Shah | | | Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService | Major | . | Botong Huang | Botong Huang | | | RBF: Complete logic for -readonly option of dfsrouteradmin add command | Major | . | Yiqun Lin | igo Goiri | | | Node resource is not parsed correctly for resource names containing dot | Major | nodemanager, resourcemanager | Jonathan Hung | Gergely Novk | | | Fix AMRMToken rollover handling in AMRMProxy | Minor | . | Botong Huang | Botong Huang | | | RBF: Add more unit tests for router admin commands | Major | test | Yiqun Lin | Yiqun Lin | | | [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled. | Critical | . | Rohith Sharma K S | Rohith Sharma K S | | | RBF: Mount table entries not properly updated in the local cache | Major | . | igo Goiri | igo Goiri | | | RBF: Control MountTableResolver cache size | Major | . | igo Goiri | igo Goiri | | | RBF: Federation supports global quota | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Display mount table quota info in Web UI and admin command | Major | . | Yiqun Lin | Yiqun Lin | | | Consider writing to both ats v1 & v2 from RM for smoother upgrades | Major | timelineserver | Vrushali C | Aaron Gresch | | | AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance | Major | fs/oss | wujinhu | wujinhu | | | RBF: Document global quota supporting in federation | Major |

| Yiqun Lin | Yiqun Lin | | | RBF: Fix spurious TestRouterRpc#testProxyGetStats | Minor | . | igo Goiri | igo Goiri | | | RBF: Federation Router State State Store internal API | Major | . | igo Goiri | igo Goiri | | | RBF: Heartbeat Router State | Major | . | igo Goiri | igo Goiri | | | RBF: Inconsistent Router OPTS config in branch-2 and branch-3 | Minor | . | Wei Yan | Wei Yan | | | Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. | Major | . | Sumana Sathish | Sunil G | | | Add CryptoInputStream to WebHdfsFileSystem read call. | Major | encryption, kms, webhdfs | Rushabh Shah | Rushabh Shah | | | RBF: Add a safe mode for the Router | Major | . | igo Goiri | igo Goiri | | | RBF: Expose the state of the Routers in the federation | Major | . | igo Goiri | igo Goiri | | | RBF: Add router admin option to manage safe mode | Major | . | igo Goiri | Yiqun Lin | | | Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning | Major | . | Weiwei Yang | Sunil G | | | RBF: Manage unavailable clusters | Major | . | igo Goiri | Yiqun Lin | | | Document GPU isolation feature | Blocker | . | Wangda Tan | Wangda Tan | | | RBF: Fix Routers information shown in the web UI | Minor | . | Wei Yan | Wei Yan | | | RBF: Improve the unit test TestRouterRPCClientRetries | Minor | test | Yiqun Lin | Yiqun Lin | | | RBF: Fix the hdfs router page missing label icon issue | Major | federation, hdfs | maobaolong | maobaolong | | | Add ADL troubleshooting doc | Major | documentation, fs/adl | Steve Loughran | Steve Loughran | | | Refactor timelineservice-hbase module into submodules | Major | timelineservice | Haibo Chen | Haibo Chen | | | Backport the code structure changes in YARN-7346 to branch-2 | Major | . | Haibo Chen | Haibo Chen | | | RBF: Complete document of Router configuration | Major | . | Tao Jie | Yiqun Lin | | | S3A multipart upload fails when SSE-C encryption is enabled | Critical | fs/s3 | Anis Elleuch | Anis Elleuch | | | RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns | Minor | . | Wei Yan | Chao Sun | | | RBF: MountTableResolver doesn't return the correct mount point of the given path | Major | hdfs | wangzhiyuan | wangzhiyuan | | | RBF: Fix router location cache issue | Major | federation, hdfs | Wu Weiwei | Wu Weiwei | | | RBF: ConnectionPool should return first usable connection | Minor | . | Wei Yan | Ekanth Sethuramalingam | | | RBF: Update some inaccurate document descriptions | Minor | . | Yiqun Lin | Yiqun Lin | | | Many tests fails in Windows due to injecting disk failures | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: TestRouterSafemode failed if the port 8888 is in use | Major | hdfs, test | maobaolong | maobaolong | | | RBF: Quota management incorrect parent-child relationship judgement | Major |

| Yiqun Lin | Yiqun Lin | | | RBF: Throw the exception if mount table entry validated failed | Major | hdfs | maobaolong | maobaolong | | | TestConfiguration fails on Windows because of paths | Major | test | igo Goiri | Xiao Liang | | | RBF: Improve State Store FS implementation | Major | . | igo Goiri | igo Goiri | | | RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router | Minor | . | Wei Yan | Wei Yan | | | RBF: Resolvers to support mount points across multiple subclusters | Major | . | igo Goiri | igo Goiri | | | RBF : Fix compilation error in branch-2 (TestMultipleDestinationResolver) | Blocker | . | Brahma Reddy Battula | Brahma Reddy Battula | | | AliyunOSS: move files under a directory in parallel when rename a directory | Major | fs/oss | wujinhu | wujinhu | | | RBF: Move Router to its own module | Major | . | igo Goiri | Wei Yan | | | RBF: Improve the use of setQuota command | Major | . | liuhongtong | liuhongtong | | | RBF: Router to manage requests across multiple subclusters | Major | . | igo Goiri | igo Goiri | | | RBF: Fix FindBugs in hadoop-hdfs-rbf | Minor | . | igo Goiri | Ekanth Sethuramalingam | | | RBF: Test Router-based federation using HDFSContract | Major | . | igo Goiri | igo Goiri | | | HBase filters are not constructed correctly in ATSv2 | Major | ATSv2 | Haibo Chen | Haibo Chen | | | ATSv2 REST API queries do not return results for uppercase application tags | Critical | . | Charan Hebri | Charan Hebri | | | RBF: Add WebHDFS | Major | fs | igo Goiri | Wei Yan | | | RBF: Implement available space based OrderResolver | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Optimize name service safe mode icon | Minor | . | liuhongtong | liuhongtong | | | RBF: Add xsl stylesheet for hdfs-rbf-default.xml | Major | documentation | Takanobu Asanuma | Takanobu Asanuma | | | Add config in FederationRMFailoverProxy to not bypass facade cache when failing over | Minor | . | Botong Huang | Botong Huang | | | RBF: Cache datanode reports | Minor | . | igo Goiri | igo Goiri | | | RBF: TestConnectionManager#testCleanup() test case need correction | Minor | . | Dibyendu Karmakar | Dibyendu Karmakar | | | RBF: Support NamenodeProtocol in the Router | Major | . | igo Goiri | igo Goiri | | | Update okhttp version to 2.7.5 | Major | fs/adl | Ray Chiang | Ray Chiang | | | [Atsv2] Retrospect storing entities into sub application table from client perspective | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | RBF: TestRouterWebHDFSContractCreate failed | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | Give an informative message when incorrect format is used in ATSv2 filter attributes | Major | ATSv2 | Charan Hebri | Rohith Sharma K S | | | AMRMProxy recover should catch for all throwable to avoid premature exit | Major | . | Botong Huang | Botong Huang | | | RBF: Fix java doc for StateStoreFileSystemImpl | Minor | hdfs | Yiran Wu | Yiran Wu | | | RBF: mv/rm fail after the directory exceeded the quota limit | Major |

| Wu Weiwei | Yiqun Lin | | | RBF: Support federation with no subclusters | Minor | . | igo Goiri | igo Goiri | | | RBF: Improve timeout RPC call mechanism | Minor | . | igo Goiri | igo Goiri | | | RBF: Improve error message returned from subcluster | Minor | . | Wei Yan | igo Goiri | | | RBF: Remove LinkedList From StateStoreFileImpl.java | Trivial | federation | David Mollitor | David Mollitor | | | AliyunOSS: provide one asynchronous multi-part based uploading mechanism | Major | fs/oss | Genmao Yu | Genmao Yu | | | TestDockerContainerRuntime test failures due to UID lookup of a non-existent user | Major | . | Shane Kumpf | Shane Kumpf | | | RBF: Improve the error loggings for printing the stack trace | Major | . | Yiqun Lin | Yiqun Lin | | | Container-executor doesn't remove Docker containers that error out early | Major | yarn | Eric Badger | Eric Badger | | | RBF: Add more router-related information to the UI | Minor | . | Wei Yan | Wei Yan | | | RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table | Major | . | Dibyendu Karmakar | Dibyendu Karmakar | | | RBF: Disabled Nameservice store API | Major | . | igo Goiri | igo Goiri | | | RBF: Fix setSafeMode in the Router | Major | . | igo Goiri | igo Goiri | | | RBF: Disable Nameservices from the federation | Major | . | igo Goiri | igo Goiri | | | RBF: Improve the interfaces to modify and view mount tables | Minor | . | Wei Yan | Gang Li | | | RBF: Show disabled name services in the UI | Minor | . | igo Goiri | igo Goiri | | | ATS v2 returns invalid YARN\CONTAINER\ALLOCATED\HOST\HTTP\_ADDRESS from NM | Critical | ATSv2 | Yesha Vora | Rohith Sharma K S | | | RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries | Minor | . | Ekanth Sethuramalingam | Ekanth Sethuramalingam | | | RBF: Fix dead links in RBF document | Major | documentation | Akira Ajisaka | Chetna Chaudhari | | | RBF: Reject requests when a Router is overloaded | Major | . | igo Goiri | igo Goiri | | | RBF: Add unit test TestStateStoreDisabledNameservice | Major | . | Yiqun Lin | Yiqun Lin | | | HTTPS Ats v2 api call fails with "bad HTTP parsed" | Critical | ATSv2 | Yesha Vora | Charan Hebri | | | TestRollingFileSystemSinkWithLocal fails on Windows | Major | test | Xiao Liang | Xiao Liang | | | RBF: Fix synchronization of router quota and nameservice quota | Major | . | liuhongtong | Yiqun Lin | | | Incorrect HTTP status code returned by ATSv2 for non-whitelisted users | Critical | ATSv2 | Charan Hebri | Rohith Sharma K S | | | Race condition when container events are published for KILLED applications | Major | ATSv2 | Charan Hebri | Rohith Sharma K S | | | [AMRMProxy] AMRMClientRelayer for stateful FederationInterceptor | Major | . | Botong Huang | Botong Huang | | | TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows | Minor | . | Anbang Hu | Anbang Hu | | | TestTrash should use proper test path to avoid failing on Windows | Minor |

| Anbang Hu | Anbang Hu | | | RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN\_VALUE | Critical | federation | CR Hota | CR Hota | | | Support intra-queue preemption for fairness ordering policy. | Major | scheduler | Wangda Tan | Eric Payne | | | Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks | Minor | fs/azure | Esfandiar Manii | Esfandiar Manii | | | ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Make WASB listStatus messages consistent | Trivial | fs/azure | Esfandiar Manii | Esfandiar Manii | | | TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows | Minor | test | Xiao Liang | Xiao Liang | | | AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters | Minor | amrmproxy, federation | Botong Huang | Botong Huang | | | RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries | Major | . | Dibyendu Karmakar | Dibyendu Karmakar | | | RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | RBF: Fix RBF configuration links | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Admin cannot enforce Router enter SafeMode | Major | . | Wei Yan | Chao Sun | | | RBF: Add Web UI configurations and descriptions to RBF document | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Router admin clrQuota is not synchronized with nameservice | Major | . | Dibyendu Karmakar | Dibyendu Karmakar | | | RBF: Router ID in RouterRpcClient is always null | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Improve error message for invalid value in fields attribute | Minor | ATSv2 | Charan Hebri | Abhishek Modi | | | [AMRMProxy] Add sub-cluster timeout in LocalityMulticastAMRMProxyPolicy | Major | amrmproxy, federation | Botong Huang | Botong Huang | | | [AMRMProxy] More robust responseId resync after an YarnRM master slave switch | Major | amrmproxy | Botong Huang | Botong Huang | | | Refactor NameNode failover proxy providers | Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko | | | RBF: Configurable value in xml for async connection request queue size. | Major | federation | CR Hota | CR Hota | | | TestDistributedShell fails on Windows | Major | . | Botong Huang | Botong Huang | | | AliyunOSS: update oss-sdk version to 3.0.0 | Major | fs/oss | wujinhu | wujinhu | | | S3 listing inconsistency can raise NPE in globber | Major | fs | Steve Loughran | Steve Loughran | | | [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async | Major | nodemanager | Botong Huang | Botong Huang | | | AliyunOSS: Support Assume Roles in AliyunOSS | Major | fs/oss | wujinhu | wujinhu | | | RBF: Move ClientProtocol APIs to its own module | Major |

| igo Goiri | Chao Sun | | | Handle AM register requests asynchronously in FederationInterceptor | Major | amrmproxy, federation | Subramaniam Krishnan | Botong Huang | | | Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way | Major | timelineserver | Vrushali C | Abhishek Modi | | | [Storage implementation] Create HDFS backing storage implementation for ATS reads | Major | timelineserver | Tsuyoshi Ozawa | Abhishek Modi | | | DynamoDB table Update can fail S3A FS init | Major | fs/s3 | Steve Loughran | Steve Loughran | | | AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream | Critical | . | wujinhu | wujinhu | | | AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy | Major | fs/oss | wujinhu | wujinhu | | | [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client | Major | amrmproxy, federation | Botong Huang | Botong Huang | | | [Router] Add JvmMetricsInfo and pause monitor | Minor | . | Bibin Chundatt | Bilwa S T | | | AliyunOSS: fix incorrect ReadOps and WriteOps in statistics | Major | fs/oss | wujinhu | wujinhu | | | Replace the url of the repository in Apache Hadoop source code | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore | Major | fs/oss | wujinhu | wujinhu | | | Backport YARN-6445 resource profile performance improvements to branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-6232 for generic resource type usage to branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Use resource map for app metrics in TestCombinedSystemMetricsPublisher for branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Port YARN-7136 to branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-6852 for GPU-specific native changes to branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-6620 to YARN-8200/branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-7224 for refactoring of GpuDevice class | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-7330 for GPU in UI to branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | [SBN read] Unclear Log.WARN message in GlobalStateIdContext | Major | hdfs | Shweta | Shweta | | | Adding a NoOp timeline writer and reader plugin classes for ATSv2 | Minor | ATSv2, timelineclient, timelineserver | Prabha Manepalli | Prabha Manepalli | | | Parquet reading S3AFileSystem causes EOF | Blocker | fs/s3 | Dave Christianson | Steve Loughran | | | Fix empty NMResourceInfo object test failures in branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | AliyunOSS: improvements for copyFile/copyDirectory and logging | Major | fs/oss | wujinhu | wujinhu | | | Backport YARN-6927 for resource type support in MapReduce | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-7637 to branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Port resource type changes from YARN-7237 to branch-3.0/branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Backport YARN-7738 for refreshing max allocation for multiple resource types | Major | . | Jonathan Hung | Jonathan Hung | | | Backporting ABFS driver from trunk to branch

| Major | fs/azure | Esfandiar Manii | Yuan Gao | | | ABFS: add listFileStatus with StartFrom | Major | fs/azure | Da Zhou | Da Zhou | | | AliyunOSS: Remove temporary files when upload small files to OSS | Major | fs/oss | wujinhu | wujinhu | | | Support getQuotaUsage API in WebHDFS | Major | fs, webhdfs | Erik Krogen | Chao Sun | | | RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore | Major | restapi | Prabhu Joseph | Prabhu Joseph | | | Backport HDFS-14617 to branch-2 (Improve fsimage load time by writing sub-sections to the fsimage index) | Major | namenode | Xiaoqiao He | Xiaoqiao He | | | [SBN read] Revisit GlobalStateIdContext locking when getting server state id | Major | hdfs | Chen Liang | Chen Liang | | | [SBN read] Change client logging to be less aggressive | Major | hdfs | Chen Liang | Chen Liang | | | [SBN read] Allow configurably enable/disable AlignmentContext on NameNode | Major | hdfs | Chen Liang | Chen Liang | | | StandbyNode should upload FsImage to ObserverNode after checkpointing. | Major | hdfs | Konstantin Shvachko | Chen Liang | | | Backport HADOOP-16548 - "ABFS: Config to enable/disable flush operation" to branch-2 | Minor | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Backport HADOOP-16578 - "ABFS: fileSystemExists() should not call container level apis" to Branch-2 | Major | fs/azure | Sneha Vijayarajan | Sneha Vijayarajan | | | Balancer should work with ObserverNode | Major | . | Konstantin Shvachko | Erik Krogen | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | CryptoOutputStream should implement StreamCapabilities | Major | fs | Mike Drob | Xiao Chen | | | Update the release year to 2018 | Blocker | build | Akira Ajisaka | Bharat Viswanadham | | | Move ResourceRequest.clone logic everywhere into a proper API | Minor | . | Botong Huang | Botong Huang | | | WebHDFS: Document ALLOWSNAPSHOT and DISALLOWSNAPSHOT API doc | Minor | documentation, webhdfs | Siyao Meng | Siyao Meng | | | Document special paths in HDFS | Major | documentation | Christopher Douglas | Masatake Iwasaki | | | Move branch-2 precommit/nightly test builds to java 8 | Critical | . | Jonathan Hung | Jonathan Hung | | | Backport HDFS-13087 to branch-2 (Snapshotted encryption zone information should be immutable) | Major | encryption, snapshots | Wei-Chiu Chuang | Siyao Meng | | | Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9 | Major | . | Lisheng Sun | Lisheng Sun | | | Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9 | Major | . | Zheng Hu | Lisheng Sun | | | Create AbstractContainersLauncher for pluggable ContainersLauncher logic | Major | . | Jonathan Hung | Jonathan Hung | | | Backport HDFS-12914 to branch-2 (Block report leases cause missing blocks until next report) | Major | namenode | Wei-Chiu Chuang | Xiaoqiao He | | | Backport resource types/GPU features to branch-3.0/branch-2 | Major | . | Jonathan Hung | Jonathan Hung | | | Update commons-compress to 1.19 | Major | . | Wei-Chiu Chuang | YiSheng Lien | | | Support forcing configured partitions to be exclusive based on app node label | Major | . | Jonathan Hung | Jonathan Hung | | | update io.netty in branch-2 | Major | . | Wei-Chiu Chuang | Masatake Iwasaki