Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.2.9.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO | Major | fairscheduler | Yufei Gu | Yufei Gu | | | ConfServlet should respect Accept request header | Major | conf | Weiwei Yang | Weiwei Yang | | | yarn.app.mapreduce.client.job.max-retries should have a more useful default | Major | client | Daniel Templeton | Miklos Szegedi | | | Deprecate and remove DockerContainerExecutor | Critical | nodemanager | Daniel Templeton | Daniel Templeton | | | Upgrade Jackson 2.2.3 to 2.7.8 | Major | build | Steve Loughran | Sean Mackrory | | | Upgrade to AWS SDK 1.11.45 | Blocker | build, fs/s3 | Steve Loughran | Steve Loughran | | | Add ability to secure log servlet using proxy users | Major | . | Jeffrey E Rodriguez | Yuanbo Liu | | | Set default ADLS access token provider type to ClientCredential | Major | fs/adl | John Zhuge | John Zhuge | | | HDFS Snapshots should capture point-in-time copies of OPEN files | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | ZKRMStateStore: Limit the number of znodes under a znode | Critical | resourcemanager | Karthik Kambatla | Varun Saxena | | | Remove findbugs report from docs profile | Minor | . | Andras Bokor | Andras Bokor | | | Add support for work preserving NM restart when AMRMProxy is enabled | Major | amrmproxy, nodemanager | Subru Krishnan | Botong Huang | | | Move commons logging APIs over to slf4j in hadoop-common | Major | . | Akira Ajisaka | Wenxin He | | | Configuration.dumpConfiguration should redact sensitive information | Major | conf, security | Vihang Karajgaonkar | John Zhuge | | | Add support to turn off launching privileged containers in the container-executor | Blocker | nodemanager | Varun Vasudev | Varun Vasudev | | | Extend NMStateStore to save queued container information | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Arun Suresh | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Make JvmPauseMonitor an AbstractService | Major | . | Steve Loughran | Sunil Govindan | | | hadoop utilities need to support provided delegation tokens | Blocker | security | Allen Wittenauer | Heesoo Kim | | | Add an HDFS metrics sink | Major | metrics | Daniel Templeton | Daniel Templeton | | | hadoop daemonlog should support https and SPNEGO for Kerberized cluster | Major | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | YARN Timeline Service v.2: alpha 1 | Critical | timelineserver | Sangjin Lee | Sangjin Lee | | | [Umbrella] Make MapReduce work with Timeline Service Nextgen (YARN-2928) | Major | . | Vinod Kumar Vavilapalli | Sangjin Lee | | | support wildcard in libjars argument | Major | util | Sangjin Lee | Sangjin Lee | | | Limit the number of resources a single map reduce job can submit for localization | Major |

| Chris Trezzo | Chris Trezzo | | | Allow pluggable audit loggers in KMS | Major | kms | Xiao Chen | Xiao Chen | | | Improve distcp to support efficient restore to an earlier snapshot | Major | distcp | Yongjun Zhang | Yongjun Zhang | | | Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle | Major | nodemanager | Chris Douglas | Arun Suresh | | | Add Codec for ZStandard Compression | Major | . | churro morales | churro morales | | | Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers | Major | tools | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Support for multi-cluster delegation tokens | Minor | security | Clay B. | Jian He | | | YARN Capacity Scheduler - Queue Priorities | Major | . | Wangda Tan | Wangda Tan | | | Maintain aggregated peer performance metrics on NameNode | Major | namenode | Xiaobing Zhou | Arpit Agarwal | | | Allow users to specify racks and nodes for strict locality for AMs | Major | client | Robert Kanter | Robert Kanter | | | Add datanode admin command to get the storage info. | Major | . | Surendra Singh Lilhore | Surendra Singh Lilhore | | | add an entry point that can start any Yarn service | Major | api | Steve Loughran | Steve Loughran | | | Add an admin command to list currently open files | Major | . | Kihwal Lee | Manoj Govindassamy | | | Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration | Major | capacity scheduler | Mayank Bansal | Wei Yan | | | YARN Timeline Service v.2: alpha 2 | Critical | timelineserver | Sangjin Lee | Vrushali C | | | S3Guard: Improved Consistency for S3A | Major | fs/s3 | Chris Nauroth | Chris Nauroth | | | [Umbrella] Support maintenance state for datanodes | Major | datanode, namenode | Ming Ma | Ming Ma | | | Enable YARN RM scale out via federation using multiple RM's | Major | nodemanager, resourcemanager | Sriram Rao | Subru Krishnan | | | truly shared cache for jars (jobjar/libjar) | Major | . | Sangjin Lee | Chris Trezzo | | | Router-based HDFS federation | Major | fs | igo Goiri | igo Goiri | | | OrgQueue for easy CapacityScheduler queue configuration management | Major | . | Min Shen | Min Shen | | | Add support for the YARN Shared Cache | Major | . | Chris Trezzo | Chris Trezzo | | | Add additional deSelects params in RMWebServices#getAppReport | Major | resourcemanager, router | Giovanni Matteo Fumarola | Tanuj Nayak | | | mapreduce tasks for YARN Timeline Service v.2: alpha 2 | Major | . | Sangjin Lee | Vrushali C | | | Tool to estimate resource requirements of an application pipeline based on prior executions | Major | tools | Subru Krishnan | Rui Li | | | Support Application timeout feature in YARN. | Major | scheduler | nijel | Rohith Sharma K S | | | Extend YARN to support distributed scheduling | Major | nodemanager, resourcemanager | Sriram Rao | Konstantinos Karanasos | | | Scheduling of OPPORTUNISTIC containers through YARN RM | Major | resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Resource manager web service fields are not accessible | Trivial | resourcemanager | Krisztian Horvath | Krisztian Horvath | | | TestDiskError should get stored replicas through

| Minor | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Tests should get the number of pending async delets via FsDatasetTestUtils | Minor | test | Tony Wu | Tony Wu | | | Add a config to disable the /logs endpoints | Major | security | Robert Kanter | Robert Kanter | | | add doc about timeline performance tool usage | Major | . | Chang Li | Chang Li | | | Change TestDeleteBlockPool to not explicitly use File to check block pool existence. | Minor | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | describe dfs.ha.zkfc.port in hdfs-default.xml | Minor | . | Kanaka Kumar Avvaru | Kanaka Kumar Avvaru | | | Avoid creating temprorary strings in Block.toString() and getBlockName() | Minor | performance | Staffan Friberg | Staffan Friberg | | | Add NullGroupMapping | Major | . | Daniel Templeton | Daniel Templeton | | | Improve handling of container's stderr | Critical | . | Gera Shegalov | Naganarasimha G R | | | Remove Hard-Coded Values From FileSystem.java | Trivial | fs | BELUGA BEHR | BELUGA BEHR | | | Add number of samples in last interval in snapshot of MutableStat | Minor | metrics | Vikram Srivastava | Vikram Srivastava | | | Use a Maven plugin to build the native code using CMake | Minor | build | Colin P. McCabe | Colin P. McCabe | | | Disable spurious checkstyle checks | Major | . | Andrew Wang | Andrew Wang | | | The build should fail if a -Dbundle option fails | Minor | . | Kai Zheng | Kai Zheng | | | HTrace: collect position/length information on read operations | Major | hdfs-client, tracing | Zhe Zhang | Zhe Zhang | | | FairScheduler should mention user requested queuename in error message when failed in queue ACL check | Trivial | fairscheduler | Tao Jie | Tao Jie | | | The HTrace span for OpWriteBlock should record the maxWriteToDisk time | Major | datanode, tracing | Colin P. McCabe | Colin P. McCabe | | | Improve HA ResourceManager Failover detection on the client | Major | client, resourcemanager | Arun Suresh | Jian He | | | Re-factor support for CPU as a resource using the new ResourceHandler mechanism | Critical | nodemanager | Sidharta Seethana | Varun Vasudev | | | Add hdfsStreamBuilder API to libhdfs to support defaultBlockSizes greater than 2 GB | Major | libhdfs | Colin P. McCabe | Colin P. McCabe | | | FairScheduler: Disallow preemption from a queue | Major | fairscheduler | Tao Jie | Tao Jie | | | Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp | Minor | namenode | Jing Zhao | Mingliang Liu | | | JobClient should be an AutoClosable | Major | . | Andr Kelpe | Haibo Chen | | | DirectoryScanner shouldn't abort the scan if one directory had an error | Major | datanode | Rakesh R | Rakesh R | | | Make RegisterNodeManagerRequestPBImpl thread-safe | Major | nodemanager | Karthik Kambatla | Karthik Kambatla | | | Add additional logging to some NM state store operations | Minor | . | Sidharta Seethana | Sidharta Seethana | | | RollingFileSystemSink should eagerly rotate directories | Critical | . | Daniel Templeton | Daniel Templeton | | | Display application priority in yarn top | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Log uncaught exceptions/errors in various thread pools in YARN | Major |

| Sidharta Seethana | Sidharta Seethana | | | Tests for RollingFileSystemSink | Major | test | Daniel Templeton | Daniel Templeton | | | Remove incorrect part of maxResources in FairScheduler documentation | Major | documentation | Ray Chiang | Ray Chiang | | | Log uncaught exceptions/errors in various thread pools in mapreduce | Major | . | Sidharta Seethana | Sidharta Seethana | | | Enable TLS v1.1 and 1.2 | Major | security | Robert Kanter | Robert Kanter | | | Add machine-readable output to mapred job -history command | Major | client | Robert Kanter | Robert Kanter | | | Missing default mapper type in TimelineServer performance test tool usage | Minor | timelineserver | Kai Sasaki | Kai Sasaki | | | StatisticsDataReferenceCleaner swallows interrupt exceptions | Minor | fs | Gregory Chanan | Gregory Chanan | | | mapred job -history command should be able to take Job ID | Major | client | Robert Kanter | Robert Kanter | | | NM aggregation thread pool is not bound by limits | Critical | nodemanager | Haibo Chen | Haibo Chen | | | Allow DefaultContainerExecutor container log directory permissions to be configurable | Major | yarn | Ray Chiang | Ray Chiang | | | Update s3-related properties in core-default.xml | Minor | fs/s3 | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Rename variables in SchedulerNode to reduce ambiguity post YARN-1011 | Major | scheduler | Karthik Kambatla | igo Goiri | | | TransferFsImage.receiveFile should account and log separate times for image download and fsync to disk | Minor | . | Wellington Chevreuil | Wellington Chevreuil | | | Clarify the "expected client Kerberos principal is null" authorization message | Minor | security | Stephen Chu | Stephen Chu | | | Add a helper library to maintain node state and allows common queries | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Make HDFS commands guide up to date | Major | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Make scheduler error checking message more user friendly | Trivial | yarn | Ray Chiang | Ray Chiang | | | Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level | Major | . | Ming Ma | Ming Ma | | | \*ProcessTree classes have too many whitespace issues | Trivial | . | Karthik Kambatla | Gabor Liptak | | | /BUILDING example of zero-docs dist should skip javadocs | Trivial | build, documentation | Steve Loughran | Steve Loughran | | | DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled | Trivial | applications/distributed-shell | Daniel Templeton | Matt LaMantia | | | Remove dead code in TestDelegationTokenRenewer added in YARN-3055 | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | Pagination support for AppAttempt page TotalOutstandingResource Requests table | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI | Major | . | Varun Vasudev | Varun Vasudev | | | Add GenericTestUtils.getTestDir method and use it for temporary directory in tests | Major | build, test | Steve Loughran | Steve Loughran | | | Change log message in LocalizedResource#handle() to DEBUG | Minor |

| Ray Chiang | Ray Chiang | | | FairScheduler: Move some logs to debug and check if debug logging is enabled | Major | fairscheduler | Sandy Ryza | Yufei Gu | | | Add container resource to RM audit log | Major | resourcemanager, scheduler | Nathan Roberts | Nathan Roberts | | | Clean up RunJar and related test class | Trivial | util | Andras Bokor | Andras Bokor | | | Enable aux services to have their own custom classpath/jar file | Major | . | Xuan Gong | Xuan Gong | | | More informative diagnostics when applications killed by a user | Major | resourcemanager | Jason Lowe | Eric Badger | | | Add a way to download the job config from the mapred CLI | Major | client | Robert Kanter | Robert Kanter | | | Faster LDAP group name resolution with ActiveDirectory | Major | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add a configuration to limit the number of map tasks allowed per job. | Major | job submission | zhihai xu | zhihai xu | | | Expose scheduling policy and max running apps over JMX for Yarn queues | Major | yarn | Yufei Gu | Yufei Gu | | | RollingFileSystemSink should have configurable roll interval | Major | . | Daniel Templeton | Daniel Templeton | | | Suppress checkstyle JavadocPackage check for test source | Minor | . | John Zhuge | John Zhuge | | | NM should not aggregate logs older than the retention policy | Major | log-aggregation, nodemanager | Haibo Chen | Haibo Chen | | | Add non-decayed call metrics for DecayRpcScheduler | Major | ipc, metrics | Xiaoyu Yao | Xiaoyu Yao | | | Add support for nested groups in LdapGroupsMapping | Major | security | Gautam Gopalakrishnan | Esther Kundin | | | Upgrade Domains in HDFS | Major | . | Ming Ma | Ming Ma | | | Use GenericTestUtils.getTestDir method in tests for temporary directories | Major | build, test | Vinayakumar B | Vinayakumar B | | | YARN CLI for AM logs does not give any error message if entered invalid am value | Major | yarn | Sumana Sathish | Jian He | | | Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit | Major | . | sandflee | sandflee | | | The file localization process should allow for wildcards to reduce the application footprint in the state store | Critical | nodemanager | Daniel Templeton | Daniel Templeton | | | The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store | Critical | distributed-cache | Daniel Templeton | Daniel Templeton | | | mapreduce.reduce.shuffle.memory.limit.percent=0.0 should be legal to enforce shuffle to disk | Major | mrv2, task | Gera Shegalov | Gera Shegalov | | | Log message about input options in distcp lacks some items | Minor | tools/distcp | Takashi Ohnishi | Takashi Ohnishi | | | Reuse ObjectMapper instance in CombinedHostsFileReader and CombinedHostsFileWriter | Major | hdfs-client | Yiqun Lin | Yiqun Lin | | | Update maven-enforcer-plugin version to 1.4.1 | Major | build | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Fix message when NodeManager runs into errors initializing the recovery directory | Major | yarn | Ray Chiang | Ray Chiang | | | Remove deprecated StringUtils.getFormattedTimeWithDiff | Minor | . | Akira Ajisaka | Shen Yinjie | |

DataTransferProtocol#writeBlock missing some javadocs | Minor | datanode, hdfs | Yongjun Zhang | John Zhuge | | | passing file to -out for YARN log CLI doesnt give warning or error code | Major | . | Sumana Sathish | Xuan Gong | | | Clean up ContainerExecutor JavaDoc | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Add configuration property to prevent JHS from loading jobs with a task count greater than X | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | ClusterNodeTracker: add method to get list of nodes matching a specific resourceName | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic, or better yet parameterized | Minor | resourcemanager, test | Daniel Templeton | Kai Sasaki | | | Refactor JobResourceUploader#uploadFilesInternal | Minor | . | Chris Trezzo | Chris Trezzo | | | Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients | Major | fs/azure | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Change container runtime type logging in DelegatingLinuxContainerRuntime to debug | Trivial | yarn | Shane Kumpf | Shane Kumpf | | | LoadBalancingKMSClientProvider#doOp should log IOException stacktrace | Trivial | kms | Wei-Chiu Chuang | Chen Liang | | | Enhance logging for Cluster.java around InetSocketAddress | Minor | . | sarun singla | Vrushali C | | | Fix incomplete log info in ResourceLocalizationService | Trivial | . | Jun Gong | Jun Gong | | | TestBasicDiskValidator should not write data to /tmp | Minor | . | Lei (Eddy) Xu | Yufei Gu | | | Add debug log message when splitting is not possible due to unsplittable compression | Minor | client, mrv1, mrv2 | Peter Vary | Peter Vary | | | Update Javadocs for LinuxContainerExecutor | Major | nodemanager | Daniel Templeton | Daniel Templeton | | | FairScheduler: Allow setting maxResources for ad hoc queues | Major | fairscheduler | Karthik Kambatla | Daniel Templeton | | | Make block report size as a metric and add this metric to datanode web ui | Major | datanode, ui | Yuanbo Liu | Yuanbo Liu | | | yarn list command to support filtering by tags | Minor | client | Steve Loughran | Varun Saxena | | | Implement WebHdfsFileSystem#listStatusIterator | Major | webhdfs | Andrew Wang | Andrew Wang | | | Log DataNodes in the write pipeline | Trivial | hdfs-client | John Zhuge | John Zhuge | | | Clean up WeightAdjuster | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer | Major | tools | Yiqun Lin | Yiqun Lin | | | Complete the document for FileDistribution processor in OfflineImageViewer | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Standardize serializiation of WebHDFS DirectoryListing | Major | webhdfs | Andrew Wang | Andrew Wang | | | Checkstyle suppression by annotation or comment | Minor | . | John Zhuge | John Zhuge | | | Move dev-support/checkstyle/suppressions.xml to hadoop-build-tools | Trivial | . | John Zhuge | John Zhuge | | | Implement HttpFSFileSystem#listStatusIterator | Major | httpfs | Andrew Wang | Andrew Wang | | |

should be updated to use YARN-4629 | Minor | applicationmaster | Daniel Templeton | Daniel Templeton | | | Light cleanup in ZKRMStateStore | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | add progress log to JHS during startup | Minor | jobhistoryserver | Haibo Chen | Haibo Chen | | | Do not attempt to recover progress from previous job attempts if spill encryption is enabled | Major | applicationmaster | Karthik Kambatla | Haibo Chen | | | Support to retrieve specific property from configuration via REST API | Major | conf | Weiwei Yang | Weiwei Yang | | | Make class Token$PrivateToken private | Minor | . | John Zhuge | John Zhuge | | | Stop bundling HTML source code in javadoc JARs | Major | build | Andrew Wang | Andrew Wang | | | Ignore file backed pages from memory computation when smaps is enabled | Minor | . | Rajesh Balamohan | Rajesh Balamohan | | | DefaultContainerExecutor needs JavaDocs | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Add %A and %a formats for fs -stat command to print permissions | Major | fs | Alex Garbarini | Alex Garbarini | | | Cleanup DiskChecker interface | Major | util | Arpit Agarwal | Arpit Agarwal | | | Add a new instrumented read-write lock | Major | common | Jingcheng Du | Jingcheng Du | | | Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir() | Major | client | Santhosh G Nayak | Santhosh G Nayak | | | Many classes use bare yarn. properties instead of the defined constants | Major | . | Daniel Templeton | Daniel Templeton | | | The description of dfs.block.replicator.classname is not clear | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Tighten the authorization of datanode RPC | Major | datanode, security | Kihwal Lee | Kihwal Lee | | | Clean up Lint warnings in nodemanager | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Reuse objectMapper instance in Yarn | Major | timelineclient | Yiqun Lin | Yiqun Lin | | | Make all MockRM#waitForState consistent. | Major | resourcemanager | Yufei Gu | Yufei Gu | | | Log the trace information on FSAppAttempt#assignContainer | Major | applications, fairscheduler | Yiqun Lin | Yiqun Lin | | | Quash unnecessary safemode WARN message during NameNode startup | Trivial | . | Andrew Wang | Yiqun Lin | | | Minor cleanup to UGI use in AdminService | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | Expose getTrashRoot to HTTPFS and WebHDFS | Major | encryption, httpfs, webhdfs | Xiao Chen | Yuanbo Liu | | | NodeManager should communicate physical resource capability to ResourceManager | Major | nodemanager, resourcemanager | Nathan Roberts | igo Goiri | | | Make generic options help more consistent, and aligned | Minor | . | Grant Sohn | Grant Sohn | | | Replace DatanodeInfo constructors with a builder pattern | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with Hadoop. | Major | build | Chris Nauroth | Chris Nauroth | | | Add Builder methods for common yarn API records | Major | . | Arun Suresh | Tao Jie | | | In FairScheduler, parent queues should also display queue status | Major | fairscheduler | Siqi Li | Siqi Li | | | Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()} | Major | fs | Steve Loughran | Mingliang Liu | | |

MockRM#drainEvents implicitly in MockRM methods to reduce test failures | Major | resourcemanager | sandflee | sandflee | | | FairScheduler hides group resolution exceptions when assigning queue | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Allow handoff on the server side for RPC requests | Major | ipc | Siddharth Seth | Siddharth Seth | | | Clean up FileSystem javadocs, logging; improve diagnostics on FS load | Major | fs | Steve Loughran | Steve Loughran | | | Make Kdiag check whether hadoop.token.files points to existent and valid files | Major | . | Ravi Prakash | Ravi Prakash | | | FairScheduler should log information about AM-resource-usage and max-AM-share for queues | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Improve the HDFS Web UI browser to allow chowning / chmoding, creating dirs and uploading files | Major | ui, webhdfs | Ravi Prakash | Ravi Prakash | | | Update fair scheduler to use pluggable auth provider | Major | fairscheduler | Daniel Templeton | Tao Jie | | | Allow job\_conf.xml to be downloadable on the job overview page in JHS | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | Add a time unit to the DataNode client trace format | Minor | datanode | Akira Ajisaka | Jagadesh Kiran N | | | Datanodes not sorted properly by distance when the reader isn't a datanode | Major | . | Ming Ma | Nanda kumar | | | Ability to clean up subprocesses spawned by Shell when the process exits | Major | . | Eric Badger | Eric Badger | | | Refactor: Wrap Datanode IO related operations | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Adding per disk IO statistics and metrics in DataNode. | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Introduce fault injectors to simulate slow mirrors | Major | datanode, test | Xiaobing Zhou | Xiaobing Zhou | | | Remove TRUSTSTORE\_PASSWORD related scripts from KMS | Minor | kms | Xiao Chen | John Zhuge | | | Azure: Add a new SAS key mode for WASB. | Major | fs/azure | Dushyanth | Dushyanth | | | Collect peer performance statistics on DataNode. | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | FairShareComparator: Cache value of getResourceUsage for better performance | Major | fairscheduler | zhangshilong | zhangshilong | | | Cleanup unused DataNode#checkDiskErrorAsync() | Minor | . | Xiaoyu Yao | Hanisha Koneru | | | AsyncDispatcher thread name can be set to improved debugging | Major | . | Ajith S | Ajith S | | | Make FTPFileSystem's data connection mode and transfer mode configurable | Major | fs | Xiao Chen | Xiao Chen | | | Support multiple Datanode File IO hooks | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Automatically create YARN control group for pre-mounted cgroups | Minor | . | Miklos Szegedi | Miklos Szegedi | | | Support File IO sampling for Datanode IO profiling hooks | Major | datanode | Hanisha Koneru | Hanisha Koneru | | | Datanode should only check the failed volume upon IO errors | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Print remaining edit logs from buffer if edit log can't be

| Major | ha, namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Include file lengths in Mismatch in length error for distcp | Minor | . | Ted Yu | Ted Yu | | | Add document for container metrics | Major | documentation, nodemanager | Weiwei Yang | Weiwei Yang | | | Give fetchers hint when ShuffleHandler rejects a shuffling connection | Major | mrv2 | Haibo Chen | Haibo Chen | | | NameNode WebUI should display DataNode usage histogram | Major | namenode, ui | Zhe Zhang | Kai Sasaki | | | Make additional KMS tomcat settings configurable | Major | kms | Andrew Wang | Andrew Wang | | | Skip FSync in Test util CreateEditsLog to speed up edit log generation | Minor | hdfs | Hanisha Koneru | Hanisha Koneru | | | Use doxia macro to generate in-page TOC of HDFS site documentation | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | FairScheduler: Lower update interval for faster tests | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Document FairScheduler 'allowPreemptionFrom' queue property | Minor | fairscheduler | Yufei Gu | Yufei Gu | | | Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment | Major | . | Daniel Templeton | Udai Kiran Potluri | | | Use doxia macro to generate in-page TOC of MapReduce site documentation | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Journal Nodes should refuse to format non-empty directories | Major | . | Arpit Agarwal | Yiqun Lin | | | Add process name to kms process | Minor | kms, scripts | Rushabh S Shah | Rushabh S Shah | | | Update the link to HTrace SpanReceivers | Minor | documentation | Akira Ajisaka | Yiqun Lin | | | Allow port range to be specified while starting webapp | Major | . | Varun Saxena | Varun Saxena | | | Allow AM to specify a port range for starting its webapp | Major | applicationmaster | Varun Saxena | Varun Saxena | | | Update the links in PiEstimator document | Minor | documentation | Akira Ajisaka | Jung Yoo | | | Add process name to httpfs process | Minor | httpfs, scripts | John Zhuge | Weiwei Yang | | | DatanodeInfo getNetworkLocation and setNetworkLocation shoud use volatile instead of synchronized | Minor | performance | Chen Liang | Chen Liang | | | Add an UncaughtExceptionHandler for critical threads in RM | Major | resourcemanager | Yufei Gu | Yufei Gu | | | Use doxia macro to generate in-page TOC of YARN site documentation | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Print a user friendly error message when plugins are not found | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Log files pattern should be same for both running and finished container | Major | yarn | Sumana Sathish | Xuan Gong | | | Display the volume storage type in datanode UI | Minor | datanode, ui | Surendra Singh Lilhore | Surendra Singh Lilhore | | | The application attempt's diagnostic message should have a maximum size | Critical | resourcemanager | Daniel Templeton | Andras Piros | | | Remove unused getStartInstance and getFinalizeInstance in FSEditLogOp | Trivial | . | Andrew Wang | Alison Yu | | | Fix typo in error message of StoragePolicyAdmin tool | Trivial |

| Alison Yu | Alison Yu | | | Cluster capacity in SchedulingPolicy is updated only on allocation file reload | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | Remove Java6 specific code from GzipCodec.java | Minor | . | Akira Ajisaka | Elek, Marton | | | Add a finite shell command timeout to ShellBasedUnixGroupsMapping | Minor | security | Harsh J | Harsh J | | | Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete() | Major | namenode | X. Liang | Elek, Marton | | | KMS should support old SSL clients | Minor | kms | John Zhuge | John Zhuge | | | Add log4j configuration to enable logging in hadoop-distcp's tests | Minor | test | Xiao Chen | Xiao Chen | | | Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms | Major | namenode | Andrew Wang | Andrew Wang | | | Improve application status log message when RM restarted when app is in NEW state | Major | . | Yesha Vora | Junping Du | | | Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands | Major | federation | Brahma Reddy Battula | Brahma Reddy Battula | | | HttpFS should support old SSL clients | Minor | httpfs | John Zhuge | John Zhuge | | | DataNode Disk Outlier Detection | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Azure: Add Authorization support to WASB | Major | fs/azure | Dushyanth | Sivaguru Sankaridurg | | | Log message when DN is not selected for block replication | Minor | . | Yiqun Lin | Yiqun Lin | | | Proper error msg to be printed when invalid operation type is given to WebHDFS operations | Minor | webhdfs | Archana T | Surendra Singh Lilhore | | | CLI MiniCluster: add an option to specify NameNode HTTP port | Minor | . | Takanobu Asanuma | Takanobu Asanuma | | | Last full block report received time for each DN should be easily discoverable | Major | ui | Arpit Agarwal | Surendra Singh Lilhore | | | Simplify file IO profiling configuration | Minor | . | Hanisha Koneru | Hanisha Koneru | | | RMCriticalThreadUncaughtExceptionHandler.rmContext should be final | Minor | resourcemanager | Daniel Templeton | Corey Barker | | | Implement getHomeDirectory() method in NativeAzureFileSystem | Critical | fs/azure | Namit Maheshwari | Santhosh G Nayak | | | NULL\UPDATE\REQUESTS is redundant in TestFairScheduler | Minor | . | Daniel Templeton | Yuanbo Liu | | | Dump scheduler and queue state information into FairScheduler DEBUG log | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Support Timeout when checking single disk | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Expose slow disks via DataNode JMX | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Implement listStatusIterator, listLocatedStatus for ViewFs | Minor | viewfs | Erik Krogen | Erik Krogen | | | Add logs for slow BlockReceiver while writing data to disk | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | Fix typo in javadoc for DistributedCache | Trivial | . | Attila Sasvari | Attila Sasvari | | | Fair scheduler docs should have the queue and queuePlacementPolicy elements listed in bold so that they're easier to see | Minor | fairscheduler | Daniel Templeton | esmaeil mirzaee | | | Azure: Add Kerberos and Delegation token support to WASB

| Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | Propagate DataNode's slow disks info to the NameNode via Heartbeat | Major | . | Hanisha Koneru | Hanisha Koneru | | | hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable | Major | resourcemanager | Daniel Templeton | Laura Adams | | | Move Configuration runtime check for hadoop-site.xml to initialization | Major | . | Jonathan Eagles | Jonathan Eagles | | | Add counters for number of blocks in pending IBR | Major | hdfs | Xiaobing Zhou | Xiaobing Zhou | | | Refactor ClientRMService to unify error handling across apis | Minor | resourcemanager | Kai Sasaki | Kai Sasaki | | | Remove unused argument in ClientRMService | Trivial | . | Kai Sasaki | Kai Sasaki | | | Delay construction of PreCondition.check failure message in Configuration#set | Major | . | Jonathan Eagles | Jonathan Eagles | | | Configuration#get return value optimization | Major | . | Jonathan Eagles | Jonathan Eagles | | | Improve performance for createAndGetApplicationReport | Major | . | yunjiong zhao | yunjiong zhao | | | Add builder-based create API to FileSystem | Major | . | Sammi Chen | Sammi Chen | | | Remove unnecessary TODO comment from AppLogAggregatorImpl.java | Minor | . | Akira Ajisaka | victor bertschinger | | | Refine the behaviour of getFileChecksum when length = 0 | Minor | . | Kai Zheng | Sammi Chen | | | Handle SlowDiskReport from DataNode at the NameNode | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Improve slow mirror/disk warnings in BlockReceiver | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Expose slow disks via NameNode JMX | Major | namenode | Hanisha Koneru | Hanisha Koneru | | | All web UIs should include a robots.txt file | Minor | . | Lars Francke | Lars Francke | | | Clarify the behavior of HDFS Mover in documentation | Major | documentation | Xiaobing Zhou | Xiaobing Zhou | | | FSAppAttempt has several variables that should be final | Major | fairscheduler | Daniel Templeton | Ameet Zaveri | | | Improve Logging for SSLHostnameVerifier | Major | security | Xiaoyu Yao | Chen Liang | | | Client should always ask namenode for kms provider path. | Major | kms | Rushabh S Shah | Rushabh S Shah | | | Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches | Major | . | Chris Trezzo | Chris Trezzo | | | Add a nanosecond API to Time/Timer/FakeTimer | Minor | util | Erik Krogen | Erik Krogen | | | Export UsedCapacity and AbsoluteUsedCapacity to JMX | Major | capacityscheduler, metrics, yarn | Benson Qiu | Benson Qiu | | | BPServiceActor thread name is too long | Minor | datanode | Tsz Wo Nicholas Sze | Xiaobing Zhou | | | Authentication Tokens should use SecureRandom instead of Random and 256 bit secrets | Major | security | Robert Kanter | Robert Kanter | | | Lazy construct the IIP pathname | Major |

| Daryn Sharp | Daryn Sharp | | | Azure: Simplify Ranger-WASB policy model | Major | fs/azure | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Add a test example job that grows in memory usage over time | Major | test | Karthik Kambatla | Karthik Kambatla | | | Enable distcp to copy blocks in parallel | Major | tools/distcp | dhruba borthakur | Yongjun Zhang | | | Remove SchedulerRequestKeys when no more pending ResourceRequest | Major | . | Arun Suresh | Arun Suresh | | | Optimize BlockIterator when iterating starts in the middle. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Store KMS SSL keystore password in catalina.properties | Minor | kms | John Zhuge | John Zhuge | | | Expose Queue Configurations per Node Label through YARN client api | Major | . | Benson Qiu | Benson Qiu | | | Add submit time to Application Summary log | Minor | resourcemanager | zhihai xu | zhihai xu | | | FileUtil.checkDest() and RawLocalFileSystem.mkdirs() to throw stricter IOEs; RawLocalFS contract tests to verify | Minor | fs | Steve Loughran | Steve Loughran | | | Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike | Major | balancer & mover | yunjiong zhao | Konstantin Shvachko | | | Allow the directoryScanner to be rate-limited | Major | datanode | Nathan Roberts | Daniel Templeton | | | Change Datanode file IO profiling sampling to percentage | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Add new public encryption APIs required by Hive | Major | encryption | Andrew Wang | Lei (Eddy) Xu | | | Implement FileSystem that reads from HTTP / HTTPS endpoints | Major | fs | Haohui Mai | Haohui Mai | | | Allow custom SSL configuration to be supplied in WebApps | Major | webapp, yarn | Sanjay M Pujare | Sanjay M Pujare | | | Improve Configuration XML Parsing Performance | Major | . | Jonathan Eagles | Jonathan Eagles | | | Document output of 'hdfs count -u' should contain PATHNAME | Minor | hdfs | Xiaobing Zhou | Xiaobing Zhou | | | Add Javadoc comment for jitter parameter on CachingGetSpaceUsed | Trivial | . | Erik Krogen | Erik Krogen | | | Update default SSL cipher list for KMS | Minor | kms, security | John Zhuge | John Zhuge | | | Update default SSL cipher list for HttpFS | Minor | httpfs, security | John Zhuge | John Zhuge | | | Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Make KMS Tomcat SSL property sslEnabledProtocols and clientAuth configurable | Major | kms | John Zhuge | John Zhuge | | | Make HttpFS Tomcat SSL property sslEnabledProtocols and clientAuth configurable | Major | httpfs | John Zhuge | John Zhuge | | | Print requested node partition in assignContainer logs | Major | . | Jonathan Hung | Jonathan Hung | | | DistCp - Introduce a configurable copy buffer size | Major | tools/distcp | Omkar Aradhya K S | Omkar Aradhya K S | | | FSAppAttempt demand can be updated atomically in updateDemand() | Major | . | Karthik Kambatla | Karthik Kambatla | | | Make WebHDFS' ACLs RegEx configurable | Major | webhdfs | Harsh J | Harsh J | | | DU#refresh should print the path of the directory when an exception is caught | Minor | . | Chen Liang | Chen Liang | | | Owner support for ranger-wasb integration | Major | fs, fs/azure | Varada Hemeswari | Varada Hemeswari | | | Method length

ResourceManager#serviceInit() is too long | Minor | resourcemanager | Yufei Gu | Gergely Novk | | | Intern strings in BlockLocation and ExtendedBlock | Major | . | Misha Dmitriev | Misha Dmitriev | | | Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Add metrics for connections dropped | Major | . | Eric Badger | Eric Badger | | | Azure has messed doc structure | Major | documentation, fs/azure | Mingliang Liu | Mingliang Liu | | | Add more diagnosis info for fsimage transfer failure. | Major | . | Yongjun Zhang | Yongjun Zhang | | | Log HDFS Mover exception message of exit to its own log | Minor | balancer & mover | LiXin Ge | LiXin Ge | | | ipc.Client.Connection#sendRpcRequest should log request name | Trivial | ipc | John Zhuge | John Zhuge | | | LdapGroupsMapping - support user and group search base | Major | common, security | Shwetha G S | Shwetha G S | | | RolloverSignerSecretProvider.LOG should be @VisibleForTesting | Minor | security | Daniel Templeton | Arun Shanmugam Kumar | | | Add metric for time taken by NameNode resource check | Major | . | Chen Liang | Chen Liang | | | Make RollingAverages a mutable metric | Major | common | Hanisha Koneru | Hanisha Koneru | | | Add create() contract test that verifies ancestor dir creation | Minor | fs | Aaron Fabbri | Sean Mackrory | | | OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory | Major | . | Misha Dmitriev | Misha Dmitriev | | | Make CryptoCodec Closeable so it can be cleaned up proactively | Major | . | Xiao Chen | Xiao Chen | | | Add CRC32C performance test. | Minor | common | LiXin Ge | LiXin Ge | | | Document the configuration key for FSNamesystem lock fairness | Minor | documentation, namenode | Zhe Zhang | Erik Krogen | | | Replace commons-logging APIs with slf4j in FsDatasetImpl | Major | . | Akira Ajisaka | hu xiaodong | | | Add log info when connect to datanode socket address failed | Major | hdfs-client | chencan | chencan | | | Update azure-storage sdk to version 5.3.0 | Major | fs/azure | Mingliang Liu | Georgi Chalakov | | | LevelDBCacheTimelineStore should reuse ObjectMapper instances | Major | timelineserver | Zoltan Haindrich | Zoltan Haindrich | | | Specifically configure zookeeper-related log levels in KMS log4j | Major | kms | Xiao Chen | Xiao Chen | | | NameNode consumes a lot of memory for snapshot diff report generation | Major | hdfs, snapshots | Manoj Govindassamy | Manoj Govindassamy | | | NetworkTopology.DEFAULT\HOST\LEVEL is unused | Trivial | . | Daniel Templeton | Chen Liang | | | Display reserved resources in web UI per queue | Major | fairscheduler, webapp | Abdullah Yousufi | Abdullah Yousufi | | | Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption | Major | . | Misha Dmitriev | Misha Dmitriev | | | Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml | Minor | documentation, hdfs | Weiwei Yang | Weiwei Yang | | | Display reserved resources in web UI per application | Major | fairscheduler | Abdullah Yousufi | Abdullah Yousufi | | | SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code | Minor | scheduler | Daniel Templeton | Deepti Sawhney | | |

should be final | Minor | fairscheduler | Daniel Templeton | Yeliang Cang | | | Simplify the logic in FairScheduler#attemptScheduling | Trivial | fairscheduler | Yufei Gu | Yufei Gu | | | Azure: Support retry and client side failover for authorization, SASKey and delegation token generation | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | wasb: implement high-performance random access and seek of block blobs | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | Improve exception checking in FileContext related JUnit tests | Major | fs, test | Andras Bokor | Andras Bokor | | | PlacementRule should be configurable | Major | . | Jonathan Hung | Jonathan Hung | | | Optimizing permission check for getContentSummary | Major | namenode | Chen Liang | Chen Liang | | | DN dataset lock should be fair | Critical | datanode | Daryn Sharp | Daryn Sharp | | | Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API | Major | resourcemanager, restapi | Lantao Jin | Lantao Jin | | | Remove redundant 'public' modifiers from BlockCollection | Trivial | namenode | Chen Liang | Chen Liang | | | Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | In ResourceWeights, weights and setWeights() should be final | Minor | scheduler | Daniel Templeton | Daniel Templeton | | | Correct dfsadmin commands usage message to reflects IPC port | Major | . | steven-wugang | steven-wugang | | | Tests use assertTrue(....equals(...)) instead of assertEquals() | Minor | test | Daniel Templeton | Daniel Templeton | | | UGI getShortUserName does not need to search the Subject | Major | common | Daryn Sharp | Daryn Sharp | | | Improve performance of yarn api record toString and fromString | Major | . | Jonathan Eagles | Jonathan Eagles | | | DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting | Trivial | fairscheduler | Daniel Templeton | Yeliang Cang | | | Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue. | Trivial | fairscheduler | Yufei Gu | Yufei Gu | | | Remove MockitoMaker class | Major | test | Andras Bokor | Andras Bokor | | | Improve performance of getting and removing inode features | Major | namenode | Daryn Sharp | Daryn Sharp | | | Reduce IIP object allocations for inode lookup | Major | namenode | Daryn Sharp | Daryn Sharp | | | Enable 'hdfs dfs -stat' to display access time | Major | hdfs, shell | Yongjun Zhang | Yongjun Zhang | | | FSPreemptionThread cleanup for readability | Minor | fairscheduler | Daniel Templeton | Daniel Templeton | | | Support user-specific minimum user limit percentage in Capacity Scheduler | Major | capacityscheduler | Eric Payne | Eric Payne | | | ViewFileSystem#rename should support be supported within same nameservice with different mountpoints | Major | viewfs | Brahma Reddy Battula | Brahma Reddy Battula | | | RetryInvocationHandler$RetryInfo should override toString() | Minor | . | Akira Ajisaka | Yeliang Cang | | | Fix checkstyle warnings in ContractTestUtils | Minor | test | Steve Loughran | Thomas Marquardt | | | Tests use assertTrue(....equals(...)) instead of assertEquals() | Minor | test | Daniel Templeton | Daniel Templeton | | | Tests use

instead of assertEquals() | Minor | test | Daniel Templeton | Daniel Templeton | | | Adding a helper method to determine whether a log is Log4j implement | Minor | util | Wenxin He | Wenxin He | | | Add document for StreamCapabilities | Major | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | [API] Refactor ResourceManager WebServices to make API explicit | Critical | resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Add Max AM Resource and AM Resource Usage to Leaf Queue View in FairScheduler WebUI | Major | fairscheduler | YunFan Zhou | YunFan Zhou | | | DataNode uses a deprecated method IoUtils#cleanup. | Major | . | Ajay Kumar | Ajay Kumar | | | Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path | Minor | nodemanager | Miklos Szegedi | Miklos Szegedi | | | [ATS1.5] All history logs should be kept under its own User Directory. | Major | timelineclient, timelineserver | Rohith Sharma K S | Rohith Sharma K S | | | TestLeafQueue.testDRFUserLimits() has commented out code | Trivial | capacity scheduler, test | Daniel Templeton | Angela Wang | | | Optimize MapReduce Shuffle I/O for small partitions | Major | . | Robert Schmidtke | Robert Schmidtke | | | AllocationFileLoaderService.loadQueue() has an if without braces | Trivial | fairscheduler | Daniel Templeton | weiyuan | | | Support MSI and DeviceCode token provider in ADLS | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | Refactor curator based ZooKeeper communication into common library | Major | . | Subru Krishnan | igo Goiri | | | Queue path is recomputed from scratch on every allocation | Minor | capacityscheduler | Jason Lowe | Eric Payne | | | Update azure-storage sdk to version 5.4.0 | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | HealthReport should include disk full information | Major | nodemanager | Akira Ajisaka | Suma Shivaprasad | | | HttpFS does not seem to support SNAPSHOT related methods for WebHDFS REST Interface | Major | httpfs | Wellington Chevreuil | Wellington Chevreuil | | | Move curator transaction support to ZKCuratorManager | Major | . | Jonathan Hung | Jonathan Hung | | | Credential provider should handle property key deprecation | Critical | security | John Zhuge | John Zhuge | | | FSAppAttempt preemption related fields have confusing names | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster | Minor | documentation, security | Linlin Zhou | Linlin Zhou | | | Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices | Major | nodemanager | Tao Yang | Tao Yang | | | Backport MAPREDUCE-6870 to branch-2 while preserving compatibility | Major | . | Zhe Zhang | Peter Bacsko | | | ResourceWeights.toString() cleanup | Minor | scheduler | Daniel Templeton | weiyuan | | | NMLeveldbStateStore should be more tolerant of unknown keys | Major | nodemanager | Jason Lowe | Ajith S | | |

does not distinguish between "under replicated" and "missing" blocks | Trivial | hdfs | Wellington Chevreuil | Wellington Chevreuil | | | Intern strings in KeyVersion and EncryptedKeyVersion | Major | kms | Xiao Chen | Xiao Chen | | | Allow StopWatch to accept a Timer parameter for tests | Minor | common, test | Erik Krogen | Erik Krogen | | | Add some of the FSNamesystem JMX values as metrics | Minor | hdfs, namenode | Erik Krogen | Erik Krogen | | | Remove requirement to specify TenantGuid for MSI Token Provider | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | Remove the duplicated code in CGroupsHandlerImp.java | Trivial | nodemanager | Yufei Gu | weiyuan | | | WASB: Block compaction for Azure Block Blobs | Major | fs/azure | Georgi Chalakov | Georgi Chalakov | | | DistCp log output should contain copied and deleted files and directories | Major | tools/distcp | Konstantin Shaposhnikov | Yiqun Lin | | | Improve FsPermission symbolic parsing unit test coverage | Minor | fs | Jason Lowe | Bharat Viswanadham | | | FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | FSDataInputStream#unbuffer UOE should include stream class name | Minor | fs | John Zhuge | Bharat Viswanadham | | | Suppress UnresolvedPathException in namenode log | Minor | . | Kihwal Lee | Kihwal Lee | | | Add JUNIT timeout to TestBlockStatsMXBean | Minor | . | Lei (Eddy) Xu | Bharat Viswanadham | | | Shuffle audit logger should log size of shuffle transfer | Minor | . | Jason Lowe | Jason Lowe | | | Remove unused TaskLogAppender configurations from log4j.properties | Major | conf | Todd Lipcon | Todd Lipcon | | | Make QuorumJournalManager timeout properties configurable | Major | . | Ajay Kumar | Ajay Kumar | | | Remove FSLeafQueue#addAppSchedulable | Major | fairscheduler | Yufei Gu | Sen Zhao | | | Processor argument in Offline Image Viewer should be case insensitive | Minor | tools | Hanisha Koneru | Hanisha Koneru | | | Add more states and transitions to stabilize the NM Container state machine | Major | . | Arun Suresh | kartheek muthyala | | | Reduce RM app memory footprint once app has completed | Major | resourcemanager | Jason Lowe | Manikandan R | | | Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold | Major | fairscheduler | Yufei Gu | Chetna Chaudhari | | | Document caveats about the default JavaKeyStoreProvider in KMS | Major | documentation, kms | Xiao Chen | Xiao Chen | | | Enhance Allocate Protocol to Identify Requests Explicitly | Major | applications, resourcemanager | Subru Krishnan | Subru Krishnan | | | Add an option to disallow 'namenode format -force' | Major | . | Ajay Kumar | Ajay Kumar | | | KMS client needs retry logic | Major | . | Rushabh S Shah | Rushabh S Shah | | | add ability in Fair Scheduler to optionally configure maxResources in terms of percentage | Major | fairscheduler, scheduler | Ashwin Shankar | Yufei Gu | | | Cache the RM proxy server address | Major | RM | Yufei Gu | Yufei Gu | | | KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. | Major | kms | Xiaoyu Yao | Xiaoyu Yao | | | Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager | Major | nodemanager | Vinod Kumar Vavilapalli | Shane Kumpf | | | Enable async edit logging by default | Major | namenode | Andrew Wang | Andrew Wang | | | Log block and datanode details in BlockRecoveryWorker | Major | datanode | Xiao Chen | Xiao Chen | | |

map should be initialized lazily | Major | . | Misha Dmitriev | Misha Dmitriev | | | Backport all SLS improvements from trunk to branch-2 | Major | . | Carlo Curino | Carlo Curino | | | [KMS] Document&test missing KMS client side configs | Minor | . | Wei-Chiu Chuang | Gabor Bota | | | Do not catch and throw unchecked exceptions if IBRs fail to process | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic | Minor | . | Haibo Chen | Haibo Chen | | | Add JvmMetrics to KMS | Major | kms | Xiao Chen | Xiao Chen | | | Add debug message for better download latency monitoring | Major | nodemanager | Yufei Gu | Yufei Gu | | | Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations | Trivial | resourcemanager | Daniel Templeton | Sen Zhao | | | Audit getQueueInfo and getApplications calls | Major | . | Chang Li | Chang Li | | | NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports | Minor | net | Steve Loughran | Varun Saxena | | | Make TestResourceManager Scheduler agnostic | Major | test | Robert Kanter | Robert Kanter | | | TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler | Minor | resourcemanager | Haibo Chen | Haibo Chen | | | Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow | Major | . | Robert Kanter | Robert Kanter | | | backport HADOOP-14553 parallel tests to branch-2 | Major | fs/azure, test | Steve Loughran | Steve Loughran | | | Upgrade Avro patch version | Major | build | Chris Douglas | Bharat Viswanadham | | | FileSystem based Yarn Registry implementation | Major | amrmproxy, api, resourcemanager | Ellen Hui | Ellen Hui | | | More logs when short-circuit read is failed and disabled | Major | datanode | Weiwei Yang | Weiwei Yang | | | Logging task counters | Major | task | Prabhu Joseph | Prabhu Joseph | | | ber-jira adl:// Azure Data Lake Phase I: Stabilization | Major | fs/adl | Steve Loughran | John Zhuge | | | ber-jira: WASB client phase II: performance and testing | Major | fs/azure | Steve Loughran | Thomas Marquardt | | | DiskChecker should perform some disk IO | Major | . | Arpit Agarwal | Arpit Agarwal | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Resource leak during failure scenario of closing of resources. | Minor | util | ramkrishna.s.vasudevan | ramkrishna.s.vasudevan | | | TestYarnVersionInfo failing if generated properties doesn't include an SVN URL | Minor | api | Steve Loughran | Steve Loughran | | | The SLS simulator is not running the preemption policy for CapacityScheduler | Minor | . | Carlo Curino | Carlo Curino | | | git should ignore .DS\_Store files on Mac OS X | Major | . | Abin Shahab | Abin Shahab | | | WebHdfsFileSystem creates too many connections for pread | Major | webhdfs | Jing Zhao | Jing Zhao | | | Prevent NN's unrecoverable death spiral after full GC | Critical | namenode | Daryn Sharp | Daryn Sharp | | | "StorageTypeStats" Metric should not count failed storage. | Major | namenode | Archana T | Surendra Singh Lilhore | |

TestAMRestart#testAMBlacklistPreventsRestartOnSameNode assumes CapacityScheduler | Major | . | Anubhav Dhoot | Anubhav Dhoot | | | Make RM and Timeline-server REST APIs more consistent | Major | . | Wangda Tan | Wangda Tan | | | Queue acl can be checked at app submission | Major | . | Jian He | Jian He | | | LocalizedResource trigger a NPE Cause the NodeManager exit | Major | . | tangshangwen | tangshangwen | | | TestHttpServer.testBindAddress bind port range is wider than expected | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestRPC.testClientBackOff failing | Major | test | Steve Loughran | Xiao Chen | | | Use SO\_REUSEADDR to avoid getting "Address already in use" when using kerberos and attempting to bind to any port on the local IP address | Major | net | Colin P. McCabe | Colin P. McCabe | | | Make app id/name available to the yarn authorizer provider for better auditing | Major | . | Jian He | Jian He | | | Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions | Minor | nodemanager | Karthik Kambatla | Sunil Govindan | | | Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes | Trivial | distcp | Wei-Chiu Chuang | Wei-Chiu Chuang | | | DataNode start slowly due to the initial DU command operations | Major | . | Yiqun Lin | Yiqun Lin | | | Fix scheduler load simulator to support multi-layer network location | Major | . | Ming Ma | Ming Ma | | | RM startup failure when AM attempts greater than max-attempts | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks | Minor | namenode | Masatake Iwasaki | Masatake Iwasaki | | | RM might fail to restart when recovering apps whose attempts are missing | Critical | . | Jun Gong | Jun Gong | | | Fix rumen and scheduler load simulator handle killed tasks properly | Major | . | Ming Ma | Ming Ma | | | Jobs that did not start are shown as starting in 1969 in the JHS web UI | Major | jobhistoryserver | Daniel Templeton | Haibo Chen | | | Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent | Major | . | Xuan Gong | Xuan Gong | | | container-executor fails to remove directory tree when chmod required | Major | nodemanager | Colin P. McCabe | Colin P. McCabe | | | Fix logging statements in resource manager's Application class | Trivial | . | Sidharta Seethana | Sidharta Seethana | | | Distributed shell breaks under strong security | Major | applications/distributed-shell, security | Daniel Templeton | Daniel Templeton | | | Disk IO imbalance in HDFS with heterogeneous storages | Major | . | Wei Zhou | Wei Zhou | | | FairScheduler: Cleanup preemptContainer to be more readable | Trivial | fairscheduler | Karthik Kambatla | Kai Sasaki | | | movetoqueue option does not documented in 'YARN Commands' | Major | documentation | Takashi Ohnishi | Takashi Ohnishi | | | Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption | Major | fairscheduler | Karthik Kambatla | Kai Sasaki | |

SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | When task logs are not available, port 8041 is referenced instead of port 8042 | Major | yarn | Haibo Chen | Haibo Chen | | | RollingFileSystemSink can throw an NPE on non-secure clusters | Major | . | Daniel Templeton | Daniel Templeton | | | container-executor should not follow symlinks in recursive\unlink\children | Blocker | . | Bibin A Chundatt | Colin P. McCabe | | | TestCompositeService should cover all enumerations of adding a service to a parent service | Major | . | Karthik Kambatla | Ray Chiang | | | Add CSRF filter support in YARN | Major | nodemanager, resourcemanager, webapp | Jonathan Maron | Jonathan Maron | | | NMs failing on DelegatingLinuxContainerRuntime init with LCE on | Blocker | . | Vinod Kumar Vavilapalli | Sidharta Seethana | | | Application submission fails when submitted queue is not available in scheduler xml | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | SSLFactory#createSSLSocketFactory exception message is wrong | Trivial | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | AuthenticatedURL should convert a 404/Not Found into an FileNotFoundException. | Minor | io, security | Steve Loughran | Steve Loughran | | | Block#toString should not output information from derived classes | Minor | . | Colin P. McCabe | Colin P. McCabe | | | SystemClock API broken in 2.9.0 | Major | . | Siddharth Seth | Siddharth Seth | | | RollingFileSystemSink doesn't work on secure clusters | Critical | . | Daniel Templeton | Daniel Templeton | | | Shell to disable bash and setsid support when running under JVM security manager | Major | security | Costin Leau | Costin Leau | | | Deadlock in AbstractService.getConfig() | Major | yarn | Steve Loughran | Steve Loughran | | | Typo in DataXceiverServer | Trivial | datanode | Michael Han | Michael Han | | | TestFairScheduler#testContinuousScheduling fails intermittently | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | JobHistoryServer CLI throws NullPointerException with job ids that do not exist | Minor | jobhistoryserver | Li Lu | Kai Sasaki | | | TaskID default constructor results in NPE on toString() | Major | mrv2 | Daniel Templeton | Daniel Templeton | | | Fix a typo (STRICT\_IE6) in Encrypted Shuffle | Trivial | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add a way to change hadoop.security.token.service.use\_ip | Major | security | Robert Kanter | Robert Kanter | | | Javadoc comment is broken for Resources.multiplyByAndAddTo() | Trivial | . | Daniel Templeton | Daniel Templeton | | | Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE | Major | . | Rohith Sharma K S | Sunil Govindan | | | RM Nodes list page takes too much time to load | Major | webapp | Bibin A Chundatt | Bibin A Chundatt | | | Capture container start/finish time in container metrics | Major | . | Jian He | Jian He | | | Namenode safemode not coming out during failover | Major | namenode | Brahma Reddy Battula | Brahma Reddy Battula | | | MR usage counters use the resources requested instead of the resources allocated | Major | . | Haibo Chen | Haibo Chen | | | FileContextMainOperationsBaseTest.testListStatusThrowsExceptionForNonExistentFile() doesnt run | Minor | test | Steve Loughran | Kai Sasaki | | | TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler | Major | test | Karthik Kambatla | Bibin A Chundatt | | | YARN WebApp ignores the configuration passed to it for keystore settings | Major |

| Sergey Shelukhin | Sergey Shelukhin | | | dataTables\_wrapper change min height | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Document missing properties in hdfs-default.xml | Major | documentation | Ray Chiang | Ray Chiang | | | NM applicationpage cause internal error 500 | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Remove duplicate logSync() and log message in FSN#enterSafemode() | Minor | . | Vinayakumar B | Vinayakumar B | | | The VolumeScanner should warn about replica files which are misplaced | Major | datanode | Colin P. McCabe | Colin P. McCabe | | | o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | SafeModeMonitor should not leave safe mode if name system is starting active service | Major | namenode | Mingliang Liu | Mingliang Liu | | | TestYarnClient#testSubmitIncorrectQueue fails with FairScheduler | Major | test | Yufei Gu | Yufei Gu | | | MRBench: setting the baseDir parameter has no effect | Minor | benchmarks | Michael Noll | Wilfred Spiegelenburg | | | Missing NullPointer check in ContainerLaunchContextPBImpl causes RM to die | Major | resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | ContainerMetrics drops records | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | JVM and UGI metrics disappear after RM is once transitioned to standby mode | Major | . | Li Lu | Li Lu | | | yetus-wrapper should fail sooner when download fails | Minor | yetus | Steven K. Wong | Steven K. Wong | | | distcp should not ignore the ignoreFailures option | Critical | tools/distcp | Gera Shegalov | Mingliang Liu | | | LocalContainerAllocator doesn't specify resource of the containers allocated. | Major | mr-am | Haibo Chen | Haibo Chen | | | getApplicationReport call may raise NPE for removed queues | Critical | . | Sumana Sathish | Jian He | | | Fix IOUtils#cleanup and IOUtils#closeStream javadoc | Trivial | io | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove synchronized input streams from Writable deserialization | Major | io | Gopal V | Gopal V | | | RedundantEditLogInputStream#LOG is set to wrong class | Minor | . | John Zhuge | John Zhuge | | | Addendum for HDFS-9579: to handle the case when client machine can't resolve network path | Major | . | Ming Ma | Ming Ma | | | application master log can not be available when clicking jobhistory's am logs link | Critical | jobhistoryserver | chenyukang | Haibo Chen | | | Fix formatting of CacheAdmin command usage help text | Major | caching | Yiqun Lin | Yiqun Lin | | | job history server can fail on startup when NameNode is in start phase | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | DataNode may format directory and lose blocks if current/VERSION is missing | Major | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | YARN web interfaces lack XFS protection | Major | nodemanager, resourcemanager, timelineserver | Jonathan Maron | Jonathan Maron | | | Remove CHANGES.txt from branch-2 | Major | . | Akira Ajisaka | Akira Ajisaka | | | Excessive log warnings for directory permission issue on NM recovery. | Major |

| Jian He | Jian He | | | FSAppAttempt demand calculation considers demands at multiple locality levels different | Major | fairscheduler | zhihai xu | zhihai xu | | | Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers | Major | . | Arun Suresh | Arun Suresh | | | Distributed shell AM fails when extra container arrives during finishing | Major | . | Sumana Sathish | Wangda Tan | | | FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | TestRollingFileSystemSinkWithHdfs needs some cleanup | Critical | . | Daniel Templeton | Daniel Templeton | | | TestRollingFileSystemSinkWithHdfs#testFailedClose() fails on branch-2 | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | HDFS read ends up ignoring an interrupt | Major | . | Siddharth Seth | Jing Zhao | | | Hadoop client displays confusing error message | Major | client | Mohammad Kamrul Islam | Gera Shegalov | | | ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats | Major | nodemanager | Sunil Govindan | Sunil Govindan | | | DFSInputStream should set thread's interrupt status after catching InterruptException from sleep | Major | . | Jing Zhao | Jing Zhao | | | TestRollingFileSystemSink.testSetInitialFlushTime() fails intermittently | Minor | test | Daniel Templeton | Daniel Templeton | | | Fix FSLeafQueue#getFairShare() for queues with zero fairshare | Major | . | Yufei Gu | Yufei Gu | | | ReconfigurationProtocol not covered by HDFSPolicyProvider. | Major | namenode | Chris Nauroth | Arpit Agarwal | | | Cache MapOutputLocations in ShuffleHandler | Major | . | Siddharth Seth | Junping Du | | | Wrong exit code while trying to get app logs using regex via CLI | Critical | yarn | Sumana Sathish | Xuan Gong | | | test\native\mini\_dfs fails by NoClassDefFoundError | Major | native, test | Akira Ajisaka | Akira Ajisaka | | | Unable to loadFSEdits due to a failure in readCachePoolInfo | Critical | caching, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Counters page display all task neverthless of task type( Map or Reduce) | Minor | jobhistoryserver | Rohith Sharma K S | Kai Sasaki | | | MockNodes.newNodes creates one more node per rack than requested | Major | test | Karthik Kambatla | Karthik Kambatla | | | Fix typos in CapacityScheduler documentation | Trivial | documentation | Ray Chiang | Ray Chiang | | | Fix TestOfflineEditsViewer#testGenerated | Blocker | test | Xiaoyu Yao | Surendra Singh Lilhore | | | NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl | Major | nodemanager | Karam Singh | Junping Du | | | Pass remote ip address down to YarnAuthorizationProvider | Major | . | Jian He | Jian He | | | Fix Lint Warnings in YARN Common | Major | yarn | Daniel Templeton | Daniel Templeton | | | TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail | Major | . | Jason Lowe | sandflee | | | PendingReconstructionBlocks.size() should be synchronized | Major | . | Eric Badger | Eric Badger | | | Fix findbugs for nodemanager & checkstyle warnings in nodemanager.ContainerExecutor | Major | nodemanager | Vrushali C | Vrushali C | | | MapReduce JerseyTest tests failing with

Address already in use" | Critical | test | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | Create framework for configurable disk checkers | Major | util | Yufei Gu | Yufei Gu | | | Handle queue names consistently in FairScheduler | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently | Major | test | Rohith Sharma K S | Xuan Gong | | | Fixing minor Scheduler test case failures | Major | . | Subru Krishnan | Subru Krishnan | | | Use AHSClient in YarnClient when TimelineServer is running | Major | . | Xuan Gong | Xuan Gong | | | Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API | Major | . | Subru Krishnan | Ellen Hui | | | Failed to Update HDFS Delegation Token for long running application in HA mode | Major | fs, ha, security | Liangliang Gu | Liangliang Gu | | | Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also) | Major | . | Zhiyuan Yang | Zhiyuan Yang | | | Fix failing unit tests in TestLinuxContainerExecutorWithMocks | Major | nodemanager | Yufei Gu | Yufei Gu | | | KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default | Minor | kms | Anthony Young-Garner | Anthony Young-Garner | | | Some recovered apps are put into default queue when RM HA | Major | . | Jun Gong | Jun Gong | | | LdapGroupsMapping getPassward shouldn't return null when IOException throws | Major | security | Zhaohao Liang | Wei-Chiu Chuang | | | AzureNativeFileSystem rename/delete performance improvements | Major | fs/azure | Subramanyam Pattipaka | Subramanyam Pattipaka | | | DataNode should not auto-format block pool directory if VERSION is missing | Major | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | EditLogTailer should call rollEdits with a timeout | Major | ha, namenode | Todd Lipcon | Lei (Eddy) Xu | | | CredentialProviderFactory fails at class loading from libhdfs (JNI) | Major | fs/s3 | Aaron Fabbri | Aaron Fabbri | | | NPE in KeyProvider.rollNewVersion | Minor | . | Colm O hEigeartaigh | Colm O hEigeartaigh | | | Document LdapGroupsMapping keystore password properties | Minor | security | Wei-Chiu Chuang | Yuanbo Liu | | | For Queue AccessControlException client retries multiple times on both RM | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Garbage snapshot records lingering forever | Critical | . | Kihwal Lee | Kihwal Lee | | | TestCapacityScheduler#testKillAllAppsInQueue fails randomly | Major | . | Varun Saxena | sandflee | | | KMS should reload whitelist and default key ACLs when hot-reloading | Major | kms | Xiao Chen | Xiao Chen | | | Test failed for TestAggregatedLogFormat on trunk | Major | . | Junping Du | Jun Gong | | | Yarn running container log fetching causes OutOfMemoryError | Major | log-aggregation | Prasanth Jayachandran | Xuan Gong | | | Fix Javadoc typos in org.apache.hadoop.fs.FileSystem | Trivial | documentation | Chen He | Andras Bokor | | | DrainDispacher#ServiceStop blocked if setDrainEventsOnStop invoked | Major | . | sandflee | sandflee | | | Open files can leak permanently due to inconsistent lease update | Critical | . | Kihwal Lee | Kihwal Lee | | | JMX AM Used metrics for queue wrong when app submited to nodelabel partition | Major | . | Bibin A Chundatt | Bibin A Chundatt | |

ControlledJob#toString failed with NPE when job status is not successfully updated | Major | . | Weiwei Yang | Weiwei Yang | | | Add detailed logging in KMS for the authentication failure of proxy user | Minor | kms | Suraj Acharya | Suraj Acharya | | | Trash does not descent into child directories to check for permissions | Critical | fs, security | Eric Yang | Weiwei Yang | | | Intermittent test failure of TestAMRMClient#testAMRMClientWithContainerResourceChange | Major | . | Varun Saxena | Bibin A Chundatt | | | Return container's ip and host from NM ContainerStatus call | Major | . | Jian He | Jian He | | | NPE listing wildcard directory in containerLaunch | Blocker | nodemanager | Haibo Chen | Daniel Templeton | | | Potential memory leak in CryptoOutputStream | Major | security | Mariappan Asokan | Mariappan Asokan | | | Typos in 4 log messages | Trivial | . | Mehran Hassani | Mehran Hassani | | | Typos in 9 log messages | Trivial | . | Mehran Hassani | Mehran Hassani | | | Fix TestDefaultContainerExecutor | Major | test | Akira Ajisaka | Arun Suresh | | | Fix some warnings by findbugs in hadoop-maven-plugin | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | YarnWebParams next.fresh.interval should be next.refresh.interval | Minor | webapp | Daniel Templeton | Daniel Templeton | | | Provide an option to enable ContainerMonitor | Major | . | Jian He | Jian He | | | Prevent IPC reader thread death | Critical | ipc | Daryn Sharp | Daryn Sharp | | | Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories | Minor | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Reduce loglevel to Debug in ContainerManagementProtocolProxy and AMRMClientImpl | Major | yarn | Yufei Gu | Yufei Gu | | | Log demand as demand in FSLeafQueue and FSParentQueue | Major | fairscheduler | Yufei Gu | Yufei Gu | | | FairScheduler: wrong queue name in log when adding application | Minor | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Network issues can cause persistent RM UI outage | Critical | webapp | Daniel Templeton | Daniel Templeton | | | Fix TestAMWebApp failure | Major | test | Akira Ajisaka | Daniel Templeton | | | Fix typos in core-default.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Fix typos in hdfs-default.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | resourcemanager.security.TestDelegationTokenRenewer fails in trunk | Major | . | Yongjun Zhang | Jason Lowe | | | LogLevel#main throws exception if no arguments provided | Major | . | Mingliang Liu | Mingliang Liu | | | Log object instance get incorrectly in TestDFSAdmin | Minor | . | Yiqun Lin | Yiqun Lin | | | Increase default value of httpfs maxHttpHeaderSize | Minor | hdfs | Nicolae Popa | Nicolae Popa | | | Remove distcp dependency on FileStatus serialization | Major | tools/distcp | Chris Douglas | Chris Douglas | | | Propogate exceptions back to client when using hedging RM failover provider | Critical | applications, resourcemanager | Subru Krishnan | Subru Krishnan | | | Null check missing for earliest in FifoPolicy | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Add documentation for HDFS-9820 distcp improvement | Major | distcp | Yongjun Zhang | Yongjun Zhang | | | fix NPE in

| Major | yarn | Haibo Chen | Haibo Chen | | | Duplicate keys in NAMENODE\SPECIFIC\KEYS | Minor | namenode | Akira Ajisaka | Akira Ajisaka | | | TestLocalResourcesTrackerImpl#testLocalResourceCache times out | Major | . | Eric Badger | Eric Badger | | | Fix inconsistent naming in class ResourceRequest | Trivial | yarn | Yufei Gu | Yufei Gu | | | DefaultContainerExecutor random working dir algorigthm skews results | Minor | . | Miklos Szegedi | Vrushali C | | | Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods() | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | RM REST API documentation is not up to date | Trivial | resourcemanager, yarn | Miklos Szegedi | Miklos Szegedi | | | Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary | Minor | resourcemanager | Sunil Govindan | Sunil Govindan | | | Update yarn daemonlog documentation due to HADOOP-12847 | Trivial | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Cleanup "mapreduce.job.hdfs-servers" from yarn-default.xml | Minor | yarn | Junping Du | Junping Du | | | Add test for leakage of reservation metrics in fair scheduler | Minor | fairscheduler | Hong Zhiguo | Hong Zhiguo | | | FairSharePolicy breaks TimSort assumption | Major | fairscheduler | Zephyr Guo | Zephyr Guo | | | AsyncDispatcher possibly invokes multiple shutdown thread when handling exception | Major | . | Jian He | Jian He | | | Add isDebugEnabled check for debug logs in nodemanager | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | MR should not schedule container requests in cases where reducer or mapper containers demand resource larger than the maximum supported | Minor | mr-am | Haibo Chen | Haibo Chen | | | BlockManagerSafeMode should respect extension period default config value (30s) | Minor | namenode | Mingliang Liu | Mingliang Liu | | | Handle duplicate completed containers in RMNodeImpl | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers | Major | . | Rohith Sharma K S | Konstantinos Karanasos | | | JHS task page search based on each individual column not working | Major | jobhistoryserver | Bibin A Chundatt | Ajith S | | | Hadoop Common includes generated test protos in both jar and test-jar | Major | build, common | Sean Busbey | Sean Busbey | | | Update NMTokens in case of requests with only opportunistic containers | Blocker | . | Konstantinos Karanasos | Konstantinos Karanasos | | | TestDiskFailures.testLocalDirsFailures failed | Major | . | Yufei Gu | Yufei Gu | | | FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource | Major | fairscheduler | sandflee | sandflee | | | Incorrect documentation for timeline service entityType/events REST end points | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value | Trivial |

| Miklos Szegedi | Chang Li | | | Fix issues related to Max App in capacity scheduler | Major | capacity scheduler | Bibin A Chundatt | Bibin A Chundatt | | | ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | RM -format-state-store and -remove-application-from-state-store commands fail with NPE | Critical | resourcemanager | Varun Saxena | Varun Saxena | | | RM crashes with NPE if generic application history is enabled | Critical | resourcemanager | Varun Saxena | Varun Saxena | | | TestPipeApplication#testSubmitter fails after HADOOP-13802 | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | yarn rmadmin -help contains a misspelled ResourceManager | Trivial | resourcemanager | Grant Sohn | Grant Sohn | | | Expose getApplications API in YarnClient with GetApplicationsRequest parameter | Major | client | Gour Saha | Jian He | | | TimelineServer doesn't catch/translate all exceptions raised | Minor | timelineserver | Steve Loughran | Steve Loughran | | | Reduce the number of default server threads for AMRMProxyService | Minor | nodemanager | Subru Krishnan | Subru Krishnan | | | io.sort.factor code default and mapred-default.xml values inconsistent | Trivial | task | Gera Shegalov | Prabhu Joseph | | | DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true | Major | . | Varun Saxena | Varun Saxena | | | Handle Opportunistic scheduling allocate request failure when NM is lost | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang | Major | test | Rohith Sharma K S | Varun Saxena | | | TestSymlinkHdfsFileSystem#testCreateLinkUsingPartQualPath2 fails after HADOOP13605 | Critical | test | Brahma Reddy Battula | Brahma Reddy Battula | | | TestContainerManagerSecurity testContainerManager[1] failed | Major | nodemanager | Haibo Chen | Haibo Chen | | | Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. | Major | . | Chris Nauroth | Li Lu | | | "Overridden" is misspelled as "overriden" in FairScheduler.md | Trivial | site | Daniel Templeton | Heather Sutherland | | | Fix race condition in TestGetGroups beforeclass setup() | Major | yarn | Haibo Chen | Haibo Chen | | | Missing scheduling policy in the FS queue metric. | Major | . | Yufei Gu | Yufei Gu | | | Bug in return value for delete() calls in WASB | Major | fs/azure | Dushyanth | Dushyanth | | | JobEndNotification info logs are missing in AM container syslog | Minor | applicationmaster | Prabhu Joseph | Haibo Chen | | | TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. | Major | common | Haibo Chen | Haibo Chen | | | Retrospect moveApplicationToQueue in align with YARN-5611 | Major | capacity scheduler, resourcemanager | Sunil Govindan | Sunil Govindan | | | Error in handling event type APP\ATTEMPT\REMOVED to the scheduler | Major | . | tangshangwen | Wilfred Spiegelenburg | | | The format of job start time in JHS is different from those of submit and finish time | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code | Trivial | client, nodemanager | Grant Sohn | Grant Sohn | | | FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations | Major |

| Vinayakumar B | Vinayakumar B | | | FairScheduler should preempt for a ResourceRequest and all preempted containers should be on the same node | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | cacheadmin,cryptoadmin and storagepolicyadmin should support generic options | Minor | tools | Archana T | Brahma Reddy Battula | | | should set HADOOP\JOB\HISTORYSERVER\_HEAPSIZE only if it's empty on branch2 | Major | scripts | Fei Hui | | | | AMRMClientAsync will stop if any exceptions thrown on allocate call | Major | . | Jian He | Jian He | | | Correct check for error code to detect Azure Storage Throttling and provide retries | Major | fs/azure | Gaurav Kanade | Gaurav Kanade | | | FsPermission string constructor does not recognize sticky bit | Major | . | Atul Sikaria | Atul Sikaria | | | FileInputStream leak on failure path in BlockSender | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. | Blocker | test, yarn | Steve Loughran | Varun Saxena | | | Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method | Major | yarn | Haibo Chen | Haibo Chen | | | MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0. | Blocker | resourcemanager | Yufei Gu | Yufei Gu | | | Make AllocationFileLoaderService.Listener public | Major | fairscheduler, yarn | Tao Jie | Tao Jie | | | A couple of spelling errors in the docs | Trivial | documentation | Grant Sohn | Grant Sohn | | | Document the missing envvars commands | Minor | documentation | Yiqun Lin | Yiqun Lin | | | TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863 | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Spelling errors in logging and exceptions for resource manager code | Trivial | resourcemanager | Grant Sohn | Grant Sohn | | | Fix unreleased resources and null dereferences | Major | . | Yufei Gu | Yufei Gu | | | Improve moveApplicationQueues command line | Major | client | Sunil Govindan | Sunil Govindan | | | Change the log level to DEBUG for recovering completed applications | Major | resourcemanager | Rohith Sharma K S | Daniel Templeton | | | ConcurrentModificationException during DataNode#refreshVolumes | Major | . | Jason Lowe | Manoj Govindassamy | | | Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Build failure due to errors of javadoc build in hadoop-azure | Major | fs/azure | Kai Sasaki | Kai Sasaki | | | RM unable to start in secure setup | Blocker | . | Ajith S | Ajith S | | | Fix Several Unsafe Practices | Major | . | Yufei Gu | Yufei Gu | | | Document the missing metrics of DataNode Volume IO operations | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Opportunistic containers : Combined patch for branch-2 | Major | . | Arun Suresh | Arun Suresh | | | Opportunistic containers minor fixes: API annotations and config parameter changes | Minor | . | Arun Suresh | Arun Suresh | | | Log aggregation get failed when NM restart even with recovery | Blocker | . | Junping Du | Junping Du | | | Misuse of format specifier in Preconditions.checkArgument | Trivial | . | Yongjun Zhang | Yuanbo Liu | | | Debug log

AbstractCSQueue#canAssignToThisQueue needs improvement | Trivial | capacity scheduler | Ying Zhang | Ying Zhang | | | TimelineServer fails to start when some LevelDb state files are missing. | Critical | . | Ravi Prakash | Ravi Prakash | | | Revert changes of AbstractResourceRequest | Blocker | . | Wangda Tan | Wangda Tan | | | When killing app, RM releases app's resource before they are released by NM | Major | resourcemanager | Jun Gong | Jason Lowe | | | simple spelling errors in yarn test code | Trivial | test | Grant Sohn | vijay | | | Improvements to KMS logging to help debug authorization errors | Minor | kms | Tristan Stevens | Tristan Stevens | | | RM unable to start in secure mode | Blocker | resourcemanager | Bibin A Chundatt | Ajith S | | | LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container | Critical | . | Wangda Tan | Wangda Tan | | | TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error | Major | fs/adl, test | John Zhuge | John Zhuge | | | The rpc to portmap service for NFS has hardcoded timeout. | Major | . | Jitendra Nath Pandey | Mukul Kumar Singh | | | yarn.scheduler.minimum-allocation-\* descriptions are incorrect about behavior when a request is out of bounds | Minor | . | Bibin A Chundatt | Julia Sommer | | | Path globbing does not match newlines | Major | . | Eric Badger | Eric Badger | | | SNN can't checkpoint when rolling upgrade is not finalized | Critical | rolling upgrades | Xiaoyu Yao | Xiaoyu Yao | | | NameNode terminated after full GC thinking QJM is unresponsive. | Major | namenode, qjm | Konstantin Shvachko | Vinitha Reddy Gankidi | | | TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk | Minor | . | Yiqun Lin | Yiqun Lin | | | Storage class member storageDirs should be private to avoid unprotected access by derived classes | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Fix opportunistic containers documentation | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | RegistrySecurity overrides zookeeper sasl system properties | Major | . | Billie Rinaldi | Billie Rinaldi | | | TestGridmixMemoryEmulation and TestResourceUsageEmulators fail on the environment other than Linux or Windows | Major | test | Akira Ajisaka | Akira Ajisaka | | | SharedCacheManager does not start up | Major | . | Chris Trezzo | Chris Trezzo | | | Invalid REST api response for getApps since queueUsagePercentage is coming as INF | Critical | . | Sunil Govindan | Sunil Govindan | | | Log portnumber in PrivilegedNfsGatewayStarter | Minor | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | Log map attempts as part of shuffle handler audit log | Major |

| Jonathan Eagles | Gerg Psztor | | | Change exception message in StorageLocationChecker | Minor | datanode | Arpit Agarwal | Arpit Agarwal | | | Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues | Major | fairscheduler | Ashwin Shankar | Yufei Gu | | | Add a test to verify Audit log entries for setfacl/getfacl commands over FS shell | Major | hdfs, test | Manoj Govindassamy | Manoj Govindassamy | | | KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser | Major | common, kms | Greg Senia | Xiaoyu Yao | | | Fix KMSClientProvider for non-secure proxyuser use case | Major | kms | Xiaoyu Yao | Xiaoyu Yao | | | Localizer leaves behind tarballs after container is complete | Major | . | Eric Badger | Eric Badger | | | Verify multiple DataNodes can be decommissioned/maintenance at the same time | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | Log updates for ZKRMStateStore | Trivial | . | Bibin A Chundatt | Daniel Sturman | | | Remove HdfsClientConfigKeys.DFS\CLIENT\SLOW\IO\WARNING\THRESHOLD\KEY usage from DNConf | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Need more diagnosis info when seeing Slow waitForAckedSeqno | Major | . | Yongjun Zhang | Xiao Chen | | | Socket reuse address option is not honored in PrivilegedNfsGatewayStarter | Major | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | Synchronization issue in delegation token cancel functionality | Major | . | Hrishikesh Gadre | Hrishikesh Gadre | | | MR AppMaster does not honor ephemeral port range | Major | mr-am, mrv2 | Frank Nguyen | Frank Nguyen | | | Balancer hung due to no available mover threads | Major | balancer & mover | yunjiong zhao | yunjiong zhao | | | Improve log message on fail over | Major | . | Jian He | Jian He | | | Application recovery has failed when node label feature is turned off during RM recovery | Minor | scheduler | Ying Zhang | Ying Zhang | | | Yarn client implicitly invoke ATS client which accesses HDFS | Major | . | Yesha Vora | Li Lu | | | Race in UGI.reloginFromKeytab | Major | security | Duo Zhang | Duo Zhang | | | UpdateCallDuration is calculated only when debug logging is enabled | Major | fairscheduler | Yufei Gu | Yufei Gu | | | FairScheduler: preempted resources can become negative | Blocker | fairscheduler, resourcemanager | Miklos Szegedi | Miklos Szegedi | | | Add javadoc for Resources.isNone | Minor | scheduler | Karthik Kambatla | Andres Perez | | | Unnecessary INFO logs in AMRMClientAsyncImpl$CallbackHandlerThread.run | Trivial | . | Grant W | Grant W | | | SwiftRestClient includes pass length in exception if auth fails | Minor | security | Marcell Hegedus | Marcell Hegedus | | | Zookeper ACLs on NN HA enabled clusters to be handled consistently | Major | hdfs | Laszlo Puskas | Hanisha Koneru | | | help of stat is confusing | Trivial | documentation, fs | Xiaohe Lan | Attila Bukor | | | Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks | Minor | fs/s3, test | Akira Ajisaka | Yiqun Lin | | | Add a regression test for sticky bit support of OIV ReverseXML processor | Major | tools | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Numeric usernames do no work with WebHDFS FS (write access) | Major | webhdfs | Pierre Villard | Pierre Villard | | | FairScheduler: Can't create a DRF queue under a FAIR policy queue | Major | . | Arun Suresh | Yufei Gu | | | Yarn client should exit with an informative error message if an incompatible Jersey library is used at client | Major | . | Weiwei Yang | Weiwei Yang | | | ConcurrentModificationException on

| Major | fairscheduler | Miklos Szegedi | Miklos Szegedi | | | Use the cached instance when edit logging SetAclOp, SetXAttrOp and RemoveXAttrOp | Major | namenode | Xiao Chen | Xiao Chen | | | Fix OOM issue with decommissioningNodesWatcher in the case of clusters with large number of nodes | Major | resourcemanager | Ajay Jadhav | Ajay Jadhav | | | 'storagepolicies -getStoragePolicy' command should accept URI based path. | Major | shell | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc | Major | . | Eric Badger | Eric Badger | | | YARNRunner#createApplicationSubmissionContext method is longer than 150 lines | Trivial | . | Chris Trezzo | Gergely Novk | | | FS: Node reservations can interfere with preemption | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | TestFairScheduler.testReservationMetrics is flaky | Major | fairscheduler | Yufei Gu | Yufei Gu | | | S3A can no longer handle unencoded + in URIs | Minor | fs/s3 | Sean Mackrory | Sean Mackrory | | | StringBuilder created and appended even if logging is disabled | Major | namenode | Staffan Friberg | Staffan Friberg | | | FailoverOnNetworkExceptionRetry does not wait when failover on certain exception | Major | . | Jian He | Jian He | | | Document missing usages of OfflineEditsViewer processors | Minor | documentation, tools | Yiqun Lin | Yiqun Lin | | | Fix occasional BindException in TestNameNodeMetricsLogger | Major | test | Arpit Agarwal | Arpit Agarwal | | | FSLeafQueue demand update needs to be atomic | Major | resourcemanager | Varun Saxena | Miklos Szegedi | | | Remove unused imports from GzipCodec.java | Minor | . | Akira Ajisaka | Yiqun Lin | | | Fix dead link in MapReduce tutorial document | Minor | documentation | Akira Ajisaka | Victor Nee | | | FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness | Major | . | Arun Suresh | Karthik Kambatla | | | Socket re-use address option should be used in SimpleUdpServer | Major | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | kms.sh creates bogus dir for tomcat logs | Minor | kms | John Zhuge | John Zhuge | | | Job#updateStatus() failed with NPE due to race condition | Major | . | Junping Du | Junping Du | | | Variable in byte printed directly in mapreduce client | Major | client | Nemo Chen | Kai Sasaki | | | io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java | Minor | . | Erik Steffl | Harsh J | | | NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe | Major | yarn | Haibo Chen | Haibo Chen | | | Fix TestAMRMClient when using FairScheduler | Minor | . | Miklos Szegedi | Miklos Szegedi | | | Fix NPE in FsDatasetImpl#checkAndUpdate | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | yarn rmadin -getGroups returns information from standby RM | Critical | yarn | Sumana Sathish | Jian He | | | user is not removed from UsersManagers when app is killed with pending container requests. | Major | . | Eric Payne | Eric Payne | | | start-build-env.sh: invalid docker image name | Major | build | Gerg Psztor | Gerg Psztor | | | Add escaping to error message in KMS web UI | Minor | security | Aaron T. Myers | Aaron T. Myers | | | Enhance documentation for

property | Trivial | documentation | Miklos Szegedi | Yufei Gu | | | Specify charset when create String in CredentialsTestJob | Minor | . | Akira Ajisaka | Kai Sasaki | | | Fix bind failure in SimpleTCPServer & Portmap where bind fails because socket is in TIME\_WAIT state | Major | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | TestRecovery.testCrashed failed | Major | test | Gerg Psztor | Gerg Psztor | | | Move application across queues should handle delayed event processing | Major | capacity scheduler | Bibin A Chundatt | Bibin A Chundatt | | | TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected | Major | . | Haibo Chen | Haibo Chen | | | Intra-queue preemption occurs even when preemption is turned off for a specific queue. | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne | | | AM not launched when a single vcore is available on the cluster | Major | fairscheduler | Yufei Gu | Yufei Gu | | | OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions | Major | yarn | Haibo Chen | Haibo Chen | | | TestResources test timeouts are too aggressive | Major | test | Jason Lowe | Eric Badger | | | DataNode reconfigure for disks doesn't remove the failed volumes | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum | Minor | . | Eric Badger | Eric Badger | | | Decommissioning stuck because of failing recovery | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode | Major | ha | Nanda kumar | Nanda kumar | | | Fix confusing block recovery message | Minor | datanode | Wei-Chiu Chuang | Yiqun Lin | | | FileSystemContractBaseTest is not cleaning up test directory clearly | Major | fs | Mingliang Liu | Mingliang Liu | | | Fix a spelling mistake in CapacityScheduler | Trivial | capacity scheduler | Jin Yibo | Jin Yibo | | | Edit file should not be processed by the same type processor in OfflineEditsViewer | Major | tools | Yiqun Lin | Yiqun Lin | | | ATS client should better handle Socket closed case | Major | timelineclient | Sumana Sathish | Li Lu | | | Make RegistrySecurity use short user names for ZK ACLs | Major | . | Billie Rinaldi | Billie Rinaldi | | | ContainerKillEvent lost when container is still recovering and application finishes | Critical | nodemanager | sandflee | sandflee | | | reuseAddress option should be used for child channels in Portmap and SimpleTcpServer | Major | nfs | Mukul Kumar Singh | Mukul Kumar Singh | | | move declaration of okhttp version from hdfs-client to hadoop-project POM | Minor | build | Steve Loughran | Xiaobing Zhou | | | Admin command line should print message to stderr in failure case | Minor | . | Kai Sasaki | Kai Sasaki | | | TestLocalCacheDirectoryManager test timeout is too aggressive | Major | test | Jason Lowe | Miklos Szegedi | | | Clean up OrderingPolicy javadoc | Minor | resourcemanager | Daniel Templeton | Daniel Templeton | | | typo in s3a rename(self, subdir) error message | Minor |

| Steve Loughran | Steve Loughran | | | Order of namenodes in ConfiguredFailoverProxyProvider is undefined | Major | ha, hdfs-client | Rafal Wojdyla | igo Goiri | | | Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files | Major | fs/adl, test | Vishwajeet Dusane | Vishwajeet Dusane | | | S3A multipart commit failing, "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort" | Critical | fs/s3 | Steve Loughran | Steve Loughran | | | No FileSystem for scheme: adl | Major | fs/adl | John Zhuge | John Zhuge | | | HttpFS doc errors | Trivial | documentation, httpfs, test | Yuanbo Liu | Yuanbo Liu | | | ViewFs should use underlying FileSystem's server side defaults | Major | fs, viewfs | Lohit Vijayarenu | Erik Krogen | | | DomainSocketWatcher::add()/delete() should not self interrupt while looping await() | Critical | hdfs-client | Mingliang Liu | Mingliang Liu | | | CredentialProviderFactory$getProviders is not thread-safe | Major | security | Vihang Karajgaonkar | Vihang Karajgaonkar | | | FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded" | Major | viewfs | Erik Krogen | Erik Krogen | | | Prevent FS state dump logger from cramming other log files | Major | fairscheduler | Yufei Gu | Yufei Gu | | | TestRMFailover#testAutomaticFailover always passes even when it should fail | Major | . | Yufei Gu | Yufei Gu | | | Fix getNumMapTasks() documentation in JobConf | Minor | documentation | Joe Mszros | Joe Mszros | | | License check for jdiff output files should be ignored | Major | build | Akira Ajisaka | Akira Ajisaka | | | OIV's ReverseXML processor cannot reconstruct some snapshot details | Major | tools | Colin P. McCabe | Akira Ajisaka | | | Client close() should not fail fast if the last block is being decommissioned | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition | Major | test | Robert Kanter | Robert Kanter | | | Memory leak in timeline server | Critical | timelineserver | Wataru Yukawa | Jonathan Eagles | | | AMs can't be scheduled on racks or nodes | Major | . | Robert Kanter | Robert Kanter | | | Typo in DataStorage exception message | Minor | datanode | Daniel Templeton | Anna Budai | | | RM configuration allows all failover methods to disabled when automatic failover is enabled | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | FileContextMainOperationsBaseTest should clean up test root path | Minor | fs, test | Mingliang Liu | Mingliang Liu | | | Header injections are possible in application proxy servlet | Major | resourcemanager, security | Naganarasimha G R | Naganarasimha G R | | | Fragments are not handled correctly by resource limit checking | Minor | . | Chris Trezzo | Chris Trezzo | | | MR Job Submission Fails if MR framework application path not on defaultFS | Minor | mrv2 | Erik Krogen | Erik Krogen | | | [S3A DOC] Correct the format for "Seoul" example | Minor | documentation, fs/s3 | Brahma Reddy Battula | Brahma Reddy Battula | | | Shuffle Handler keep-alive connections are closed from the server side | Major |

| Jonathan Eagles | Jonathan Eagles | | | exception thrown when accessing the job configuration web UI | Minor | webapps | Sangjin Lee | Haibo Chen | | | Closing a file has a wasteful preconditions in NameNode | Major | namenode | Eric Badger | Eric Badger | | | LeveldbRMStateStore can parse invalid keys when recovering reservations | Major | resourcemanager | Jason Lowe | Jason Lowe | | | ReservationAgents are not correctly configured | Major | capacity scheduler, resourcemanager | Sean Po | Manikandan R | | | Fix markdown itemization in hadoop-aws documents | Minor | documentation, fs/s3 | Akira Ajisaka | Akira Ajisaka | | | Azure: WasbRemoteCallHelper should use String equals for comparison. | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low | Major | test | Jason Lowe | Eric Badger | | | Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines | Trivial | test | Chris Trezzo | Chris Trezzo | | | RM startup failure due to wrong order in nodelabel editlog | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | TaskAttemptImpl#createCommonContainerLaunchContext is longer than 150 lines | Trivial | . | Chris Trezzo | Chris Trezzo | | | Invalid local resource request can raise NPE and make NM exit | Major | nodemanager | Tao Yang | Tao Yang | | | Move ClientProtocol HA proxies into hadoop-hdfs-client | Blocker | hdfs-client | Andrew Wang | Huafeng Wang | | | TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently | Major | test | Jason Lowe | Jason Lowe | | | Continuous scheduling thread crashes while sorting nodes | Major | . | Yufei Gu | Yufei Gu | | | Fragments specified for libjar paths are not handled correctly | Minor | . | Chris Trezzo | Chris Trezzo | | | TestThrottledAsyncChecker#testCancellation is flaky | Major | test | Arpit Agarwal | Arpit Agarwal | | | Fix some release build issues | Blocker | build | Andrew Wang | Andrew Wang | | | StorageDirectory should initialize a non-null default StorageDirType | Minor | hdfs | Hanisha Koneru | Hanisha Koneru | | | HDFS write crashed with block size greater than 2 GB | Critical | hdfs-client | Xiaobing Zhou | Xiaobing Zhou | | | Exceptions during aggregated log writes are mishandled | Critical | log-aggregation | Akira Ajisaka | Akira Ajisaka | | | Decommissioning an NM results in a -1 exit code | Minor | . | Miklos Szegedi | Miklos Szegedi | | | VersionInfo should be marked as public API | Critical | common | Thejas M Nair | Akira Ajisaka | | | Docker docs MR example is broken | Major | nodemanager | Daniel Templeton | Prashant Jha | | | Initialize FakeTimer with a less trivial value | Major | test | Andrew Wang | Andrew Wang | | | Upgrade HSQLDB to 2.3.4 | Minor | build | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | TestRMAdminCLI has very low test timeouts | Major | test | Jason Lowe | Eric Badger | | | correct spelling mistake in FileSystemRMStateStore | Trivial | . | Yeliang Cang | Yeliang Cang | | | Fix ReservationSystem creation of default ReservationQueue | Major |

| Carlo Curino | Carlo Curino | | | TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Mover should move the file blocks to default storage once policy is unset | Major | balancer & mover | Surendra Singh Lilhore | Surendra Singh Lilhore | | | TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest | Major | test | Jason Lowe | Jason Lowe | | | FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close() | Critical | nodemanager | Daryn Sharp | Haibo Chen | | | Unify Container Resizing code paths with Container Updates making it scheduler agnostic | Major | capacity scheduler, fairscheduler, resourcemanager | Arun Suresh | Arun Suresh | | | TestCapacityScheduler.testAMLimitUsage fails intermittently | Major | . | Eric Badger | Eric Badger | | | Only accessible cgroup mount directories should be selected for a controller | Major | nodemanager | Miklos Szegedi | Miklos Szegedi | | | Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart | Major | . | Eric Badger | Eric Badger | | | Add python2.7-dev to Dockerfile | Major | . | Allen Wittenauer | Allen Wittenauer | | | Skip rm.transitionToActive call to RM if RM is already active. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | FSNamesystemLock metrics can be inaccurate due to millisecond precision | Major | hdfs | Erik Krogen | Erik Krogen | | | Remove non-existent setfattr command option from FileSystemShell.md | Minor | documentation | Doris Gu | Doris Gu | | | Python example in the rack awareness document doesn't work due to bad indentation | Minor | documentation | Kengo Seki | Kengo Seki | | | Typo in metrics docs | Trivial | documentation | Daniel Templeton | Ana Krasteva | | | Fail the node if Linux Container Executor is not configured properly | Minor | . | Miklos Szegedi | Miklos Szegedi | | | TestFsDatasetCache#testPageRounder fails intermittently with AssertionError | Major | test | Andrew Wang | Andrew Wang | | | TestDistributedFileSystem.java fails to compile | Major | test | John Zhuge | John Zhuge | | | FSSchedulerNode.java fails to compile with JDK7 | Major | resourcemanager | John Zhuge | John Zhuge | | | fairscheduler-statedump.log gets generated regardless of service | Blocker | fairscheduler, scheduler | Allen Wittenauer | Yufei Gu | | | keepContainer does not work when AM retry window is set | Major | resourcemanager | kyungwan nam | kyungwan nam | | | New exception thrown by DFSClient#isHDFSEncryptionEnabled broke hacky hive code | Major | . | Yongjun Zhang | Yongjun Zhang | | | Improve and refactor user-limit calculation in capacity scheduler | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler | Major | nodemanager | Miklos Szegedi | Miklos Szegedi | | | Add a proper scheme to the datanode links in NN web UI | Major | . | Kihwal Lee | Kihwal Lee | | | Support multi-line value for ssl.server.exclude.cipher.list | Major | . | John Zhuge | John Zhuge | | | AMs only intended to run one attempt can be run more than once | Major | resourcemanager | Jason Lowe | Jason Lowe | | | Fix profs stat file warning caused by process names that includes parenthesis | Major | . | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | CryptoOutputStream throws IOException on flush() if stream is closed | Major |

| Pierre Lacave | Pierre Lacave | | | StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully | Critical | ha, namenode | Zhe Zhang | Erik Krogen | | | fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 | Blocker | yarn | Haibo Chen | Haibo Chen | | | TestIPC.testIpcWithReaderQueuing fails intermittently | Major | . | Eric Badger | Eric Badger | | | TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently | Major | . | Eric Badger | Jason Lowe | | | Fix warnings from Spotbugs in hadoop-yarn-client | Major | . | Weiwei Yang | Weiwei Yang | | | Some blocks can be permanently lost if nodes are decommissioned while dead | Blocker | namenode | Kihwal Lee | Kihwal Lee | | | Blocks can be committed with wrong size | Critical | hdfs-client | Daryn Sharp | Jing Zhao | | | Newly added NN storage directory won't get initialized and cause space exhaustion | Critical | . | Kihwal Lee | Kihwal Lee | | | Change SimpleHttpProxyHandler#exceptionCaught log level from info to debug | Minor | datanode | Xiaoyu Yao | Xiaobing Zhou | | | License error in TestLoadBalancingKMSClientProvider.java | Major | . | hu xiaodong | hu xiaodong | | | NetworkTopology calls expensive toString() when logging | Major | . | igo Goiri | igo Goiri | | | Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager | Major | resourcemanager | Weiwei Yang | Weiwei Yang | | | Yarn top shows negative container number in FS | Major | yarn | Yufei Gu | Tao Jie | | | TestLocalFileSystem tests have very low timeouts | Major | . | Eric Badger | Eric Badger | | | TestSymlinkLocalFS timeouts are too low | Major | . | Eric Badger | Eric Badger | | | Fix regression in tests caused by YARN-679 | Major | test | Steve Loughran | Steve Loughran | | | "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler | Blocker | rpc-server | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Remove indefinite caching of key provider uri in DFSClient | Major | hdfs-client | Rushabh S Shah | Rushabh S Shah | | | Quit throwing NMNotYetReadyException | Major | nodemanager | Karthik Kambatla | Manikandan R | | | License error in GridmixTestUtils.java | Major | . | lixinglong | lixinglong | | | Upgrade Jsch jar to latest version to fix vulnerability in old versions | Critical | . | Vinayakumar B | Vinayakumar B | | | NM mount cpu cgroups failed on some systems | Major | . | sandflee | Miklos Szegedi | | | Increase Common test timeouts from 1 second to 10 seconds | Major | . | Eric Badger | Eric Badger | | | YARN RM will shut down if ZKClient creation times out | Major | resourcemanager | Wilfred Spiegelenburg | Daniel Templeton | | | License error In org.apache.hadoop.metrics2.util.Servers | Major | . | hu xiaodong | hu xiaodong | | | Increase YARN test timeouts from 1 second to 10 seconds | Major | . | Eric Badger | Eric Badger | | | Increase MapReduce test timeouts from 1 second to 10 seconds | Major | . | Eric Badger | Eric Badger | | | Fix performance regression due to incorrect use of DataChecksum | Major | native, performance | LiXin Ge | LiXin Ge | | | Increase HDFS test timeouts from 1 second to 10 seconds | Major | . | Eric Badger | Eric Badger | | | Underconstruction blocks can be considered missing | Major |

| Nathan Roberts | Nathan Roberts | | | Fix JavaDoc issues in SchedulingPolicy | Trivial | fairscheduler | Daniel Templeton | Weiwei Yang | | | Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently | Major | fs/azure | Trupti Dhavle | Santhosh G Nayak | | | Correct spelling of 'beginning' and variants | Trivial | . | Dongtao Zhang | Dongtao Zhang | | | ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread | Major | capacityscheduler, resourcemanager | Min Shen | Min Shen | | | FSAppAttempt keeps redundant copy of the queue | Major | fairscheduler | Daniel Templeton | Daniel Templeton | | | reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered | Critical | datanode | Vinayakumar B | Vinayakumar B | | | Incorrect logger for FairSharePolicy | Minor | fairscheduler | Yufei Gu | Vrushali C | | | Memory leak when reading a compressed file using the native library | Major | common, io | Eli Acherkan | Eli Acherkan | | | TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently | Major | . | Eric Badger | Nathan Roberts | | | Support for querying outputstream capabilities | Major | erasure-coding | Andrew Wang | Manoj Govindassamy | | | History server getApplicationReport NPE when fetching report for pre-2.8 job | Blocker | timelineserver | Jason Lowe | Jason Lowe | | | NPE in RMAppsBlock | Major | resourcemanager | Jason Lowe | Jason Lowe | | | HDFS architecture documentation describes outdated placement policy | Minor | . | dud | Chen Liang | | | Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger | Major | fs/azure, security | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | NPE is thrown when log level changed in BlockPlacementPolicyDefault#chooseRandom() method | Major | . | xupeng | xupeng | | | HostsFileReader#getHostDetails is very expensive on large clusters | Major | util | Jason Lowe | Jason Lowe | | | Avoid reloading of Configuration in ViewFileSystem creation. | Major | viewfs | Vinayakumar B | Vinayakumar B | | | TestDataNodeOutlierDetectionViaMetrics UT fails | Major | . | Yesha Vora | Hanisha Koneru | | | Remove unused ContainerLocalization classes | Minor | nodemanager | ZhangBing Lin | ZhangBing Lin | | | TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once | Minor | test | Jason Lowe | Jason Lowe | | | TestFairSchedulerPreemption fails inconsistently. | Major | fairscheduler, resourcemanager | Sean Po | Tao Jie | | | Impersonation does not work if standby RM is contacted first | Blocker | client | Robert Kanter | Robert Kanter | | | Document missing metrics for blocks count in pending IBR | Minor | documentation | Yiqun Lin | Yiqun Lin | | | JournalNode startup failure exception should be logged in log file | Major | journal-node | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Correct license headers in hadoop-common, hdfs, yarn and mapreduce | Trivial | . | Yeliang Cang | Yeliang Cang | | | Document Metrics to track usage of memory for writes | Major | documentation | Brahma Reddy Battula | Yiqun Lin | | | AmIpFilter drops query parameters on redirect | Major | . | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | The ASF Header in ComparableVersion.java and

is not correct | Minor | common, documentation | ZhangBing Lin | ZhangBing Lin | | | Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used | Major | common | Surendra Singh Lilhore | Surendra Singh Lilhore | | | GetContentSummary uses excessive amounts of memory | Blocker | namenode | Nathan Roberts | Wei-Chiu Chuang | | | ppc64le on Linux doesn't trigger \\linux get\_executable codepath | Major | nodemanager | Sonia Garudi | Ayappan | | | Configuration does not correctly XInclude absolute file URIs | Blocker | conf | Andrew Wang | Jonathan Eagles | | | the accessTime of FileStatus returned by SFTPFileSystem's getFileStatus method is always 0 | Trivial | fs | Hongyuan Li | Hongyuan Li | | | FSCK shows overall health status as corrupt even one replica is corrupt | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | TestRMFailover fails rarely due to port conflict | Major | test | Robert Kanter | Robert Kanter | | | A faulty node can cause a lease leak and NPE on accessing data | Critical | . | Kihwal Lee | Kihwal Lee | | | Non-public resource localization on a bad disk causes subsequent containers failure | Major | . | Kuhu Shukla | Kuhu Shukla | | | ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong | Major | fs | Hongyuan Li | Hongyuan Li | | | Fix NPE in LazyPersistFileScrubber | Major | . | igo Goiri | igo Goiri | | | hadoop-aws doc header warning #5 line wrapped | Trivial | documentation, fs/s3 | John Zhuge | John Zhuge | | | TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Refactor the NodeManager DeletionService to support additional DeletionTask types. | Major | nodemanager, yarn | Shane Kumpf | Shane Kumpf | | | Completed files lost after power failure | Critical | . | Dave Latham | Vinayakumar B | | | RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception | Critical | . | Jonathan Eagles | Jonathan Eagles | | | License information is missing for native CRC32 code | Critical | . | Timothy St. Clair | Andrew Wang | | | Fix TestDFSShell.testMoveWithTargetPortEmpty failure. | Major | test | Konstantin Shvachko | Brahma Reddy Battula | | | Azure: update doc for live and contract tests | Major | documentation, fs/azure | Mingliang Liu | Mingliang Liu | | | Long running balancer may fail due to expired DataEncryptionKey | Major | balancer & mover | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updates | Major | hdfs-client, rolling upgrades | Vinayakumar B | Vinayakumar B | | | Fix license header inconsistency in hdfs | Trivial | . | Yeliang Cang | Yeliang Cang | | | Remove the redundant colon in ViewFs.md | Major | documentation | maobaolong | maobaolong | | | Use OpenJDK 7 instead of Oracle JDK 7 to avoid oracle-java7-installer failures | Major | build | Akira Ajisaka | Akira Ajisaka | | | ASF License warnings generated intermittently in trunk | Major | . | Yiqun Lin | Yiqun Lin | | | s3a: mkdir appears to be broken | Blocker | fs/s3 | Aaron Fabbri | Mingliang Liu | | | Segment overflow in FileDistributionCalculator | Major | tools | LiXin Ge | LiXin Ge | |

TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor | Major | . | Eric Badger | Eric Badger | | | Optimize NativeAzureFsInputStream for positional reads | Major | fs/azure | Rajesh Balamohan | Rajesh Balamohan | | | Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently | Major | fs/azure, test | Mingliang Liu | Mingliang Liu | | | BPServiceActor thread name is not correctly set | Major | hdfs | Chen Liang | Chen Liang | | | Positional read will fail if replicas moved to different DNs after stream is opened | Critical | hdfs-client | Vinayakumar B | Vinayakumar B | | | Document missing processor of hdfs oiv\_legacy command | Minor | documentation, tools | LiXin Ge | LiXin Ge | | | DN should not delete the block On "Too many open files" Exception | Critical | datanode | Brahma Reddy Battula | Brahma Reddy Battula | | | NNBench should Throw IOException when rename and delete fails | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails | Major | fs/azure, test | Mingliang Liu | Rajesh Balamohan | | | Upgrade AWS SDK to 1.11.134 | Critical | fs/s3 | Aaron Fabbri | Aaron Fabbri | | | WritableRpcEngine.Invocation#toString NPE on null parameters | Minor | ipc | John Zhuge | John Zhuge | | | RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels. | Blocker | . | Eric Payne | Sunil Govindan | | | RM startup failure with old state store due to version mismatch | Critical | . | Bibin A Chundatt | Varun Saxena | | | Switch from aalto-xml to woodstox to handle odd XML features | Blocker | conf | Andrew Wang | Jonathan Eagles | | | TestJMXGet fails occasionally | Major | . | Arpit Agarwal | Arpit Agarwal | | | When constructing a thread name, BPOfferService may print a bogus warning message | Minor | datanode | Tsz Wo Nicholas Sze | Weiwei Yang | | | Job end notification not send due to YarnRuntimeException | Major | applicationmaster | yunjiong zhao | yunjiong zhao | | | TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0 | Major | fs | Sonia Garudi | Hongyuan Li | | | Add Unit Test to make sure Job end notification get sent even appMaster stop get YarnRuntimeException | Minor | . | Junping Du | Gergely Novk | | | Fix warnings from Spotbugs in hadoop-yarn-common | Major | . | Weiwei Yang | Weiwei Yang | | | Fix findbugs warnings in SLSCapacityScheduler.java | Major | . | Akira Ajisaka | Akira Ajisaka | | | Replace MRv1 specific terms in HostsFileReader | Minor | documentation | Akira Ajisaka | hu xiaodong | | | HDFS Architecture documentation incorrectly describes writing to a local temporary file. | Minor | documentation | Chris Nauroth | Nanda kumar | | | OIV tests should not write outside 'target' directory. | Major | . | Konstantin Shvachko | Yiqun Lin | | | Fix dead link in the Javadoc of FairSchedulerEventLog.java | Minor | documentation | Akira Ajisaka | Weiwei Yang | | | Size of args cannot be less than zero in TraceAdmin#run as its linkedlist | Trivial | common, tracing | Weisen Han | Weisen Han | | | Successfully closed files can stay under-replicated. | Critical | . | Kihwal Lee | Kihwal Lee | | | FTPFileSystem#getFsAction always returns

| Major | fs | Hongyuan Li | Hongyuan Li | | | TestCopyPreserveFlag fails consistently because of mismatch in access time | Major | hdfs, test | Mukul Kumar Singh | Mukul Kumar Singh | | | GenericTestUtils#waitFor missing parameter verification | Major | test | Yiqun Lin | Yiqun Lin | | | KerberosAuthenticationHandler should authenticate with SPN in AP-REQ | Major | security | Daryn Sharp | Daryn Sharp | | | TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk | Major | . | Varun Saxena | Robert Kanter | | | ZKFC should use getAversion() while setting the zkacl | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk | Critical | resourcemanager | dongtingting | Bibin A Chundatt | | | Azure: handle failure gracefully in case of missing account access key | Major | fs/azure | Mingliang Liu | Mingliang Liu | | | TestFsDatasetImpl.testCleanShutdownOfVolume fails | Major | test | Akira Ajisaka | hu xiaodong | | | ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions | Major | fs/s3 | Mingliang Liu | Mingliang Liu | | | CSQueueMetrics needs to update the current metrics for default partition only | Major | capacity scheduler | Naganarasimha G R | Manikandan R | | | KMS JMX endpoint throws ClassNotFoundException | Critical | kms | Andrew Wang | John Zhuge | | | Add counters for block re-replication | Major | . | Chen Liang | Chen Liang | | | Add parameter for rack locality delay in CapacityScheduler | Major | capacityscheduler | Konstantinos Karanasos | Konstantinos Karanasos | | | Fix meaningless operations in TestDFSIO in some situation. | Major | tools/rumen | LiXin Ge | LiXin Ge | | | Description of dfs.block.invalidate.limit is incorrect in hdfs-default.xml | Minor | documentation | Weiwei Yang | Weiwei Yang | | | LocalJobRunner fails when run on a node from multiple users | Blocker | client | Jason Lowe | Jason Lowe | | | Hide Jetty Server version header in HTTP responses | Major | security | Vinayakumar B | Surendra Singh Lilhore | | | TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently | Major | . | Eric Badger | Eric Badger | | | Nodemanager container crash after ext3 folder limit | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException | Major | . | Rushabh S Shah | Rushabh S Shah | | | Queue AM limit is not honored in CS always | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | [Docs] A small mistake in the example of TimelineClient | Trivial | docs | Jinjiang Ling | Jinjiang Ling | | | Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe | Major | security | Benoy Antony | Benoy Antony | | | Restrict setOwner to list of user when security is enabled in wasb | Major | fs/azure | Varada Hemeswari | Varada Hemeswari | | | Fix typo in ResourceManagerHA.md | Trivial | documentation | Akira Ajisaka | Yeliang Cang | | | TimelineWriter does not fully consume the POST response | Major | timelineclient | Jason Lowe | Jason Lowe | | | Datanode UI should display hostname based on JMX bean instead of window.location.hostname | Major | hdfs | Jeffrey E Rodriguez | Jeffrey E Rodriguez | |

FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs | Minor | test | Andras Bokor | Andras Bokor | | | branch-2 compilation is broken in hadoop-azure | Blocker | build, fs/azure | Sunil Govindan | Sunil Govindan | | | Concurrent task limits should only be applied when necessary | Major | mrv2 | Jason Lowe | Nathan Roberts | | | RollingLevelDBTimelineStore backwards incompatible after fst upgrade | Blocker | . | Jonathan Eagles | Jonathan Eagles | | | NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code | Major | nodemanager | Jason Lowe | Jason Lowe | | | Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo | Blocker | resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Make schedulables without demand less needy in FairSharePolicy#compare | Major | fairscheduler | YunFan Zhou | YunFan Zhou | | | Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure | Major | . | Naganarasimha G R | Naganarasimha G R | | | AM attempt fail to register before RM processes launch event | Critical | resourcemanager | Jason Lowe | Bibin A Chundatt | | | Remove BPOfferService lock contention to get block pool id | Critical | datanode | Daryn Sharp | Daryn Sharp | | | TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | IllegalStateException while handling APP\ATTEMPT\REMOVED event when async-scheduling enabled in CapacityScheduler | Major | . | Tao Yang | Tao Yang | | | Add Job#close API to shutdown MR client services. | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS | Major | jobhistoryserver | Lantao Jin | Lantao Jin | | | Incorrect javadoc description in StorageLocationChecker#check | Major | . | Nanda kumar | Nanda kumar | | | Fix NM startup failure with old state store due to version mismatch | Major | nodemanager | Ray Chiang | Botong Huang | | | GenericTestUtils.waitFor needs to check condition again after max wait time | Major | . | Daniel Templeton | Daniel Templeton | | | Application report fails if app rejected due to nodesize | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | GenericTestUtils.waitFor should use monotonic time | Trivial | test | Jason Lowe | Daniel Templeton | | | Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection | Major | namenode | Mukul Kumar Singh | Mukul Kumar Singh | | | HTTPFS liststatus returns incorrect pathSuffix for path of file | Major | httpfs | Yongjun Zhang | Yongjun Zhang | | | Null LocalResource visibility or resource type can crash the nodemanager | Major | . | Jinjiang Ling | Jinjiang Ling | | | Fix inconsistent replica size after a data pipeline failure | Critical | datanode | Wei-Chiu Chuang | Erik Krogen | | | Improve balancer usability after HDFS-8818 | Blocker | . | Kihwal Lee | Kihwal Lee | | | dfsadmin shows DFS Used%: NaN% if the cluster has zero block. | Trivial | . | Wei-Chiu Chuang | Weiwei Yang | | | AMRMClientImpl.checkNodeLabelExpression() has wrong error message | Minor | . | Daniel Templeton | Manikandan R | | | TestContainerManagerSecurity tests for Yarn Server are flakey | Major | test | Daniel Sturman | Daniel Sturman | | |

FairShareComparator#compare | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Bind IPC connections to kerberos UPN host for proxy users | Major | ipc | Daryn Sharp | Daryn Sharp | | | Non-dfsUsed will be doubled on dead node re-registration | Blocker | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker | Major | api, nodemanager | Brook Zhou | Brook Zhou | | | TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout | Major | test | Akira Ajisaka | Akira Ajisaka | | | mvn clean compile fails | Major | build | Andras Bokor | Andras Bokor | | | BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured | Major | hdfs, namenode | Weiwei Yang | Weiwei Yang | | | Unexpected jackson-core-2.2.3 dependency introduced | Blocker | timelineserver | Jason Lowe | Jonathan Eagles | | | Increase max heap size of Maven javadoc plugin | Major | test | Andras Bokor | Andras Bokor | | | TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails | Major | . | Sonia Garudi | Sonia Garudi | | | Wrong default value for "fs.df.interval" | Major | common, conf, fs | Konstantin Shvachko | Sherwood Zheng | | | Configuration can log misleading warnings about an attempt to override final parameter | Major | conf | Andrew Sherman | Andrew Sherman | | | Preemption calculation is not accurate when reserved containers are present in queue. | Major | capacity scheduler | Sunil Govindan | Wangda Tan | | | FileStatus.compareTo binary compatible issue | Blocker | . | Sergey Shelukhin | Akira Ajisaka | | | Fix formatting issue and regression caused by conversion from APT to Markdown | Minor | documentation | Doris Gu | Doris Gu | | | Ensure apps could run given NodeLabels are disabled post RM switchover/restart | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | HDFS snapshots doesn't capture all open files when one of the open files is deleted | Major | snapshots | Manoj Govindassamy | Manoj Govindassamy | | | Nodemanager can fail to fully delete application local directories when applications are killed | Critical | nodemanager | Jason Lowe | Jason Lowe | | | Handle IllegalStateException in Async Scheduling mode of CapacityScheduler | Major | capacityscheduler | Tao Yang | Tao Yang | | | [FairScheduler] Preemption reservation may cause regular reservation leaks | Blocker | fairscheduler | Miklos Szegedi | Miklos Szegedi | | | Azure: BlockBlobInputStream position incorrect after seek | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | Fix debug log when Resource Handler chain is enabled | Minor | . | Yang Wang | Yang Wang | | | Socket not closed properly when reading Configurations with BlockReaderRemote | Blocker | conf | Xiao Chen | Jonathan Eagles | | | Fix resource leak that happens during container re-initialization. | Major | nodemanager | Arun Suresh | Arun Suresh | | | Document missing namenode metrics that were added recently | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Azure: IndexOutOfBoundsException in BlockBlobInputStream | Minor | fs/azure | Rajesh Balamohan | Thomas Marquardt | | | MR job should only set tracking url if history was successfully written | Major | . | Eric Badger | Eric Badger | | | Disable setting tcp socket send/receive buffers for write pipelines | Major | datanode, hdfs | Daryn Sharp | Daryn Sharp | | | Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory | Major | fs/azure, test | Steve Loughran | Steve Loughran | | | Do

outside of FSDataset lock | Critical | datanode | Vinayakumar B | Vinayakumar B | | | LeaseManager operations are inefficient in 2.8. | Blocker | namenode | Rushabh S Shah | Rushabh S Shah | | | CompositeGroupsMapping should not swallow exceptions | Major | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes | Minor | common, fs/azure | John Zhuge | John Zhuge | | | BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate | Trivial | . | Miklos Szegedi | Kai Sasaki | | | Limit application attempt's diagnostic message size thoroughly | Major | resourcemanager | Chengbing Liu | Chengbing Liu | | | Hedged read might hang infinitely if read data from all DN failed | Major | hdfs-client | Chen Zhang | Chen Zhang | | | Change javax.cache library implementation from JSR107 to Apache Geronimo | Blocker | . | Ray Chiang | Ray Chiang | | | Log app attempt during InvalidStateTransition | Major | . | Jonathan Eagles | Jonathan Eagles | | | Extend ZKCuratorManager API for more reusability | Major | . | igo Goiri | igo Goiri | | | Duplicate instantiation in FairSchedulerQueueInfo | Minor | fairscheduler | Masahiro Tanaka | Masahiro Tanaka | | | TestAMRMProxy#testAMRMProxyTokenRenewal is flakey | Major | . | Robert Kanter | Robert Kanter | | | Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor | Minor | . | Oleg Danilov | Oleg Danilov | | | Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common | Major | mrv2 | Haibo Chen | Haibo Chen | | | NPE in RM while using YarnClient.getApplications() | Major | . | Lingfeng Su | Lingfeng Su | | | SFTPFileSystem operations should restore cwd | Major | . | Namit Maheshwari | Chen Liang | | | Hedged pread takes more time when block moved from initial locations | Major | hdfs-client | Vinayakumar B | Vinayakumar B | | | InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time | Critical | resourcemanager | Jian Fang | Jonathan Eagles | | | Fix tests faking kerberos to explicitly set ugi auth type | Major | yarn | Daryn Sharp | Daryn Sharp | | | AuthenticatedURL will reuse bad/expired session cookies | Critical | common | Daryn Sharp | Daryn Sharp | | | Do async container release to prevent deadlock during container updates | Major | . | Arun Suresh | Arun Suresh | | | testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured | Major | test | Yesha Vora | Wangda Tan | | | Fix NM state store update comment | Minor | nodemanager | Botong Huang | Botong Huang | | | AM heartbeat stuck when responseId overflows MAX\_INT | Blocker | . | Botong Huang | Botong Huang | | | DirectoryScanner will throw IllegalStateException when Multiple BP's are present | Blocker | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Race Between update pipeline and DN Re-Registration | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | RM SchedulingMonitor gives no indication why the spawned thread crashed. | Critical | yarn | Eric Payne | Eric Payne | | | NM failed to perform log aggregation due to absent container | Blocker | log-aggregation | Jason Lowe | Jason Lowe | |

DataNode#transferBlock does not create its daemon in the xceiver thread group | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin | Critical | capacity scheduler, scheduler preemption, yarn | Eric Payne | Eric Payne | | | ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. | Minor | nodemanager | Miklos Szegedi | Miklos Szegedi | | | TestAMRMProxy is failing with invalid request | Major | . | Jason Lowe | Jason Lowe | | | DataNode should log file name on disk error | Major | datanode | Wei-Chiu Chuang | Ajay Kumar | | | yarn application -list -appTypes \<appType\> is not working | Blocker | . | Jian He | Jian He | | | TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey | Minor | resourcemanager, test | Daniel Templeton | Robert Kanter | | | TestTaskAttempt fails in trunk | Major | test | Tsuyoshi Ozawa | Haibo Chen | | | ContainerRequest#executionTypeRequest causes NPE | Critical | . | Jian He | Jian He | | | Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done to this file | Major | hdfs, namenode | Yongjun Zhang | Yongjun Zhang | | | Potential issue on setting AMContainerSpec#tokenConf to null before app is completed | Major | . | Rohith Sharma K S | Manikandan R | | | Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory | Minor | encryption, hdfs | Wellington Chevreuil | Wellington Chevreuil | | | Fix incompatible API change on FsServerDefaults to HADOOP-14104 | Blocker | . | Junping Du | Junping Du | | | Remove TestDFSIO "Total Throughput" calculation | Critical | benchmarks, test | Dennis Huo | Dennis Huo | | | TestAMSimulator and TestNMSimulator fail | Major | test | Akira Ajisaka | Akira Ajisaka | | | CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. | Major | capacity scheduler, webapp | Wangda Tan | Wangda Tan | | | HDFS metrics render error in the page of Github | Minor | documentation, metrics | Yiqun Lin | Yiqun Lin | | | Update ADLS SDK to 2.2.2 for MSI fix | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | CapacitySchedulerPage NPE in "Aggregate scheduler counts" section | Minor | . | Eric Payne | Eric Payne | | | Incorrect ReservationId.compareTo() implementation | Minor | reservation system | Oleg Danilov | Oleg Danilov | | | Wasb mkdirs security checks inconsistent with HDFS | Major | fs/azure | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | TestAMRMClientOnRMRestart fails sporadically with bind address in use | Major | test | Jason Lowe | Jason Lowe | | | Kill application button is visible even if the application is FINISHED in RM UI | Major | . | Sumana Sathish | Suma Shivaprasad | | | Let NameNode to bypass external attribute provider for special user | Major | . | Yongjun Zhang | Yongjun Zhang | | | Edit log corruption due to hard lease recovery of not-closed file which has snapshots | Major | namenode | Xiao Chen | Xiao Chen | | | Hadoop 2.8.2 release build process get stuck due to java issue | Blocker | build | Junping Du | Junping Du | | | NM web server related UT fails with "NMWebapps failed to start." | Major |

| Yesha Vora | Jason Lowe | | | Journal nodes fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start | Major | . | Ajay Kumar | Ajay Kumar | | | ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. | Blocker | yarn | Sumana Sathish | Tan, Wangda | | | Update HDFS Federation setup document, for incorrect property name for secondary name node http address | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Unable to override the $HADOOP\CONF\DIR env variable for container | Major | nodemanager | Terence Yim | Jason Lowe | | | shuffle hangs after a node manager connection timeout | Major | mrv2 | Jooseong Kim | Jooseong Kim | | | RMContext need not to be injected to webapp and other Always Running services. | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Datatable sorting on the Datanode Information page in the Namenode UI is broken | Major | . | Shawna Martell | Shawna Martell | | | NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout | Major | namenode, qjm | Erik Krogen | Erik Krogen | | | Cross-queue preemption sometimes starves an underserved queue | Major | capacity scheduler | Eric Payne | Eric Payne | | | Add a pluggable StateMachine Listener that is notified of NM Container State changes | Major | . | Arun Suresh | Arun Suresh | | | Shuffle Handler prints disk error stack traces for every read failure. | Major | . | Kuhu Shukla | Kuhu Shukla | | | Introduce a config to allow setting up write pipeline with fewer nodes than replication factor | Major | . | Yongjun Zhang | Brahma Reddy Battula | | | Fix finicky TestContainerManager tests | Major | . | Arun Suresh | Arun Suresh | | | Use classloader inside configuration class to make new classes | Major | . | Jongyoul Lee | Jongyoul Lee | | | FSDirectory should use Time.monotonicNow for durations | Minor | . | Chetna Chaudhari | Bharat Viswanadham | | | Hardcoded absolute pathname in DockerLinuxContainerRuntime | Major | nodemanager | Miklos Szegedi | Eric Badger | | | "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX | Major | metrics | Sai Nukavarapu | Hanisha Koneru | | | BaileyBorweinPlouffe should use Time.monotonicNow for measuring durations | Minor | examples | Chetna Chaudhari | Chetna Chaudhari | | | RM intermittently crashed with NPE while handling APP\ATTEMPT\REMOVED event when async-scheduling enabled in CapacityScheduler | Major | resourcemanager | Karam Singh | sandflee | | | DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor | Blocker | nodemanager | Miklos Szegedi | Miklos Szegedi | | | DistSum should use Time.monotonicNow for measuring durations | Minor | . | Chetna Chaudhari | Chetna Chaudhari | | | QuasiMonteCarlo should use Time.monotonicNow for measuring durations | Minor | examples | Chetna Chaudhari | Chetna Chaudhari | | | gridmix/SleepReducer should use Time.monotonicNow for measuring durations | Minor | . | Chetna Chaudhari | Chetna Chaudhari | | | Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2 | Minor | documentation | Akira Ajisaka | Bharat Viswanadham | | | Remove duplicated code in AMRMClientAsyncImpl.java | Minor | client | Sen Zhao | Sen Zhao | | | ReuseObjectMapper in Hadoop Common | Minor |

| Hanisha Koneru | Hanisha Koneru | | | AHS REST API can return NullPointerException | Major | . | Prabhu Joseph | Billie Rinaldi | | | TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently | Major | . | Eric Badger | Eric Badger | | | Remove references to Guava Objects.toStringHelper | Major | . | Jonathan Eagles | Jonathan Eagles | | | hadoop-project/pom.xml is executable | Minor | . | Akira Ajisaka | Ajay Kumar | | | Add admin configuration to filter per-user's apps in secure cluster | Major | webapp | Sunil Govindan | Sunil Govindan | | | AggregatedLogsBlock reports a bad 'end' value as a bad 'start' value | Major | log-aggregation | Jason Lowe | Jason Lowe | | | NM returns new SCHEDULED container status to older clients | Blocker | nodemanager | Jason Lowe | Arun Suresh | | | LoadGenerator#genFile write close timing is incorrectly calculated | Major | fs | Jason Lowe | Hanisha Koneru | | | yarn.router.pipeline.cache-max-size is missing in yarn-default.xml | Major | . | Rohith Sharma K S | Jason Lowe | | | TestSchedulingMonitor#testRMStarts fails sporadically | Major | . | Jason Lowe | Jason Lowe | | | yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment | Major | amrmproxy | Yufei Gu | Yufei Gu | | | TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails | Major | capacity scheduler, test | Wangda Tan | Akira Ajisaka | | | Whitelisted variables do not support delayed variable expansion | Major | nodemanager | Jason Lowe | Jason Lowe | | | Add Node and Rack Hints to Opportunistic Scheduler | Major | . | Arun Suresh | kartheek muthyala | | | ContainerExecutor always launches with priorities due to yarn-default property | Minor | nodemanager | Jason Lowe | Jason Lowe | | | Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne | | | TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | FairCallQueue may defer servicing calls | Major | ipc | Daryn Sharp | Daryn Sharp | | | Improve exception message when mapreduce.jobhistory.webapp.address is in wrong format | Major | applicationmaster | Prabhu Joseph | Prabhu Joseph | | | Change Configuration.getPropsWithPrefix to use getProps instead of iterator | Major | . | Larry McCay | Larry McCay | | | Update GroupsMapping documentation to reflect the new changes | Major | documentation | Anu Engineer | Esther Kundin | | | Fix unsafe casting from long to int for class Resource and its sub-classes | Major | resourcemanager | Yufei Gu | Yufei Gu | | | Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications | Critical | . | Sumana Sathish | Tan, Wangda | | | Backport container-executor changes from YARN-6852 to branch-2 | Major | . | Varun Vasudev | Varun Vasudev | | | Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval | Minor | . | Ajay Kumar | Ajay Kumar | | | expunge may fail to remove trash from encryption zone | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | |

TestKMS#testACLs fails intermittently | Major | kms, test | Xiao Chen | Xiao Chen | | | LogAggregationTFileController deletes/renames while file is open | Critical | nodemanager | Daryn Sharp | Jason Lowe | | | container-executor fails to remove entries from a directory that is not writable or executable | Critical | . | Jason Lowe | Jason Lowe | | | TestRouterWebServiceUtil#testMergeMetrics is flakey | Major | federation | Robert Kanter | Robert Kanter | | | CLONE - Fix source-level compatibility after HADOOP-11252 | Blocker | . | Junping Du | Junping Du | | | TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair scheduler | Major | . | Miklos Szegedi | Miklos Szegedi | | | Improve bower dependencies for YARN UI v2 | Critical | webapp | Sunil Govindan | Sunil Govindan | | | TestDistributedShell should be scheduler agnostic | Major | . | Haibo Chen | Haibo Chen | | | DistCp#cleanup() should check whether jobFS is null | Minor | . | Ted Yu | Andras Bokor | | | DFSZKFailOverController re-order logic for logging Exception | Major | . | Bharat Viswanadham | Bharat Viswanadham | | | Handle JDK-8071638 for hadoop-common | Blocker | . | Bibin A Chundatt | Bibin A Chundatt | | | Add a link to HDFS router federation document in site.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | NoSuchElementException in FairScheduler after failover causes RM crash | Blocker | fairscheduler | Robert Kanter | Robert Kanter | | | TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE | Major | test | Robert Kanter | Yufei Gu | | | LocalityMulticastAMRMProxyPolicy should handle cancel request properly | Minor | . | Botong Huang | Botong Huang | | | HTTPFS HTTP max header size env variable is not respected in branch-2 | Minor | . | Istvan Fajth | Istvan Fajth | | | Fix configurable WebhDFS connect/read timeout | Blocker | hdfs-client, webhdfs | Xiaoyu Yao | Xiaoyu Yao | | | Possible NPE in RMWebapp when HA is enabled and the active RM fails | Major | . | Chandni Singh | Chandni Singh | | | Enforce JDK limitations | Major | build | Chris Douglas | Chris Douglas | | | Add missing figures to Resource Estimator tool | Major | . | Subru Krishnan | Rui Li | | | Fix TestDistributedScheduler | Major | . | Jason Lowe | Arun Suresh | | | TestFSAppStarvation.testPreemptionEnable fails intermittently | Major | . | Sunil Govindan | Miklos Szegedi | | | ShuffleHandler is not aware of disks that are added | Major | . | Kuhu Shukla | Kuhu Shukla | | | Clean up jdiff xml files added for 2.8.2 release | Blocker | . | Subru Krishnan | Junping Du | | | BZip2 drops records when reading data in splits | Critical | . | Aki Tanaka | Jason Lowe | | | TestMountTable fails with Java 7 | Major | . | igo Goiri | igo Goiri | | | Many RM unit tests failing with FairScheduler | Major | test | Robert Kanter | Robert Kanter | | | Fix typo "onf" in DynamicInputChunk.java | Trivial |

| fang zhenyi | fang zhenyi | | | Queue dash board in new YARN UI has incorrect values | Critical | webapp | Sunil Govindan | Sunil Govindan | | | Preemption properties should be refreshable | Major | capacity scheduler, scheduler preemption | Eric Payne | Gergely Novk | | | incorrect log preview displayed in jobhistory server ui | Major | yarn | Santhosh B Gowda | Xuan Gong | | | Fix ResourceEstimator findbugs issues | Blocker | . | Allen Wittenauer | Arun Suresh | | | Fix DominantResourceFairnessPolicy serializable findbugs issues | Blocker | . | Allen Wittenauer | Daniel Templeton | | | Router getApps REST invocation fails with multiple RMs | Critical | . | Subru Krishnan | igo Goiri | | | [Umbrella] Enhancements to Distributed Scheduling | Major | . | Arun Suresh | Arun Suresh | | | Add support for change of container ExecutionType | Major | . | Arun Suresh | Arun Suresh | | | [branch-2] "dfsrouter" should use hdfsScript | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix issue where RM fails to switch to active after first successful start | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Ensure singleton for ResourceEstimatorService | Major | . | Subru Krishnan | Rui Li | | | Rebase ResourceEstimator start/stop scripts for branch-2 | Major | . | Subru Krishnan | Rui Li | | | RMActiveService context to be updated with new RMContext on failover | Critical | . | Ajith S | Rohith Sharma K S | | | [branch-2] Include hadoop-cloud-storage-project in the main hadoop pom modules | Critical | . | Subru Krishnan | Subru Krishnan | | | Fix Container log link in new YARN UI | Major | . | Sunil Govindan | Sunil Govindan | | | Fix miscellaneous issues in ATSv2 after merge to branch-2 | Major | . | Varun Saxena | Varun Saxena | | | Update LICENSE.txt for HADOOP-14840 | Major | . | Arun Suresh | Arun Suresh | | | Handling PathIsNotEmptyDirectoryException in DFSClient delete call | Major | hdfs-client | Nanda kumar | Nanda kumar | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Pretty-format the output for DFSIO | Major | . | Kai Zheng | Kai Zheng | | | TestDirectoryScanner.testThrottle() is still a little flakey | Major | balancer & mover, test | Daniel Templeton | Daniel Templeton | | | TestResourceManager#testResourceAllocation() fails when using FairScheduler | Major | fairscheduler, test | Ray Chiang | Yufei Gu | | | Run checkstyle on test source files | Minor | . | John Zhuge | John Zhuge | | | TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X | Major | test | Allen Wittenauer | Kai Sasaki | | | TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure | Major | test | Bibin A Chundatt | Bibin A Chundatt | | | TestRMWebServices#testDumpingSchedulerLogs fails randomly | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Tests fails with localizer port bind exception. | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently | Major | test | Rohith Sharma K S | Ajith S | | | Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use\_ip enabled | Blocker | test | Rohith Sharma K S | Rohith Sharma K S | | | TestContinuousScheduling#testSortedNodes fails intermittently | Minor | . | sandflee | Yufei Gu | | | TestResourceTrackerOnHA crashes | Major | test | Jason Lowe | Ajith S | | | Fix ReservationACLsTestBase | Major | . | Sean Po | Sean Po | |

TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment | Minor | nodemanager, test | Takashi Ohnishi | Takashi Ohnishi | | | In TestFairScheduler, testcase should not create FairScheduler redundantly | Trivial | fairscheduler | Tao Jie | Tao Jie | | | Flaky test TestJobImpl.testKilledDuringKillAbort | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | TestFSNameSystemMBean should wait until JMX cache is cleared | Major | test | Akira Ajisaka | Erik Krogen | | | Unit test for NameNodeStatusMXBean | Major | hdfs, test | Hanisha Koneru | Hanisha Koneru | | | Replace assertThat with assertTrue in MetricsAsserts | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Use MockRMMemoryStateStore to reduce test failures | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Use Mockito.when instead of Mockito.stub | Minor | test | Akira Ajisaka | Andras Bokor | | | TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently | Minor | . | sandflee | Jason Lowe | | | TestCapacityScheduler.testRefreshQueuesWithQueueDelete fails randomly | Major | test | Sunil Govindan | Naganarasimha G R | | | Verify HDFS Snapshots with open files captured are safe across truncates and appends on current version file | Major | hdfs, snapshots | Manoj Govindassamy | Manoj Govindassamy | | | Skip the testcase testJobWithChangePriority if FairScheduler is used | Major | client | Peter Bacsko | Peter Bacsko | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Bootstrap TimelineServer Next Gen Module | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | [Collector wireup] Set up timeline collector with basic request serving structure and lifecycle | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | [Data Model] create overall data objects of TS next gen | Major | timelineserver | Sangjin Lee | Zhijie Shen | | | [Data Mode] Implement client API to put generic entities | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | [Collector implementation] the REST server (web server) for per-node collector does not work if it runs inside node manager | Major | timelineserver | Sangjin Lee | Li Lu | | | [Event producers] Change distributed shell to use new timeline service | Major | timelineserver | Zhijie Shen | Junping Du | | | [Source organization] Refactor timeline collector according to new code organization | Major | timelineserver | Li Lu | Li Lu | | | [Storage implementation] Create backing storage write interface and a POC only file based storage implementation | Major | timelineserver | Vrushali C | Vrushali C | | | [Collector wireup] Implement timeline app-level collector service discovery | Major | timelineserver | Sangjin Lee | Junping Du | | | rename TimelineAggregator etc. to TimelineCollector | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | TestTimelineServiceClientIntegration fails | Minor | timelineserver | Sangjin Lee | Sangjin Lee | | | [Collector wireup] Implement RM starting its timeline collector | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | [Data Model] Make putEntities operation be aware of the app's context | Major | timelineserver | Sangjin Lee | Zhijie Shen | | | Collector's web server should randomly bind an available port | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | [Event Producers] NM TimelineClient container metrics posting to new timeline

| Major | nodemanager | Junping Du | Junping Du | | | Clearly define flow ID/ flow run / flow version in API and storage | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | [Event producers] Implement MapReduce AM writing MR events/counters to v2 ATS | Major | . | Sangjin Lee | Junping Du | | | Reuse TimelineCollectorManager for RM | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Sub resources of timeline entity needs to be passed to a separate endpoint. | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | convert load test driver to timeline service v.2 | Major | . | Sangjin Lee | Sangjin Lee | | | Consolidate data model change according to the backend implementation | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | unit tests failures and issues found from findbug from earlier ATS checkins | Minor | timelineserver | Sangjin Lee | Naganarasimha G R | | | [Storage implementation] Exploiting the option of using Phoenix to access HBase backend | Major | timelineserver | Zhijie Shen | Li Lu | | | Add miniHBase cluster and Phoenix support to ATS v2 unit tests | Major | timelineserver | Li Lu | Li Lu | | | TestMRTimelineEventHandling and TestApplication are broken | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | add a mode to replay MR job history files to the timeline service | Major | . | Sangjin Lee | Sangjin Lee | | | [Storage implementation] explore & create the native HBase schema for writes | Critical | timelineserver | Sangjin Lee | Vrushali C | | | Timeline service v2 load generator needs to write event id | Major | . | Li Lu | Li Lu | | | Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data | Major | timelineserver | Vrushali C | Vrushali C | | | build is broken on YARN-2928 branch due to possible dependency cycle | Blocker | timelineserver | Sangjin Lee | Li Lu | | | Refactor and fix null casting in some map cast for TimelineEntity (old and new) and fix findbug warnings | Major | timelineserver | Junping Du | Junping Du | | | [Event producers] Implement RM writing app lifecycle events to ATS | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | Exclude

from hbase-client and hbase-testing-util | Major | timelineserver | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Generalize native HBase writer for additional tables | Minor | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | Test case failures in TestDistributedShell and some issue fixes related to ATSV2 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | [Storage abstraction] Create backing storage read interface for ATS readers | Major | timelineserver | Sangjin Lee | Varun Saxena | | | [Data Serving] Set up ATS reader with basic request serving structure and lifecycle | Major | timelineserver | Sangjin Lee | Varun Saxena | | | add equals and hashCode to TimelineEntity and other classes in the data model | Major | timelineserver | Sangjin Lee | Li Lu | | | Cache runningApps in RMNode for getting running apps on given NodeId | Major | nodemanager, resourcemanager | Junping Du | Junping Du | | | [Collector wireup] We need an assured way to determine if a container is an AM container on NM | Major | nodemanager, timelineserver | Zhijie Shen | Giovanni Matteo Fumarola | | | ensure timely flush of timeline writes | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Bugs in HBaseTimelineWriterImpl | Major | timelineserver | Zhijie Shen | Vrushali C | | | Change to use the AM flag in ContainerContext determine AM container | Major | timelineserver | Zhijie Shen | Sunil Govindan | | | Rethink event column key issue | Major | timelineserver | Zhijie Shen | Vrushali C | | | [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend | Major | timelineserver | Sangjin Lee | Zhijie Shen | | | split the application table from the entity table | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Refactor timelineservice.storage to add support to online and offline aggregation writers | Major | timelineserver | Li Lu | Li Lu | | | [Event producers] Implement NM writing container lifecycle events to ATS | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | Deal with byte representations of Longs in writer code | Major | timelineserver | Vrushali C | Sangjin Lee | | | build is broken at TestHBaseTimelineWriterImpl.java | Blocker | timelineserver | Sangjin Lee | Sangjin Lee | | | REST API implementation for getting raw entities in TimelineReader | Major | timelineserver | Varun Saxena | Varun Saxena | | | Miscellaneous issues in NodeManager project | Minor | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Proxying all AM-RM communications | Major | nodemanager, resourcemanager | Carlo Curino | Kishore Chaliparambil | | | Add a "skip existing table" mode for timeline schema creator | Major | timelineserver | Li Lu | Li Lu | | | Populate flow run data in the flow\_run & flow activity tables | Major | timelineserver | Vrushali C | Vrushali C | | | [timeline reader] implement support for querying for flows and flow runs | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | [reader REST API] implement support for querying for flows and flow runs | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Add request/response logging & timing for each REST endpoint call | Minor | timelineserver | Vrushali C | Varun Saxena | | | HBase reader throws NPE if Get returns no rows | Major | timelineserver | Varun Saxena | Varun Saxena | | | Implement support for querying single app and all apps for a flow run | Blocker | timelineserver | Varun Saxena | Varun Saxena | | | [storage implementation] app id as string in row keys can cause incorrect ordering | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Support additional queries for ATSv2 Web UI | Major | . | Varun Saxena | Varun Saxena | | | [reader implementation] support flow activity queries based on time | Minor | timelineserver | Sangjin Lee | Varun Saxena | | | Refactor the SystemMetricPublisher in RM to better support newer events | Major | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Store user in app to flow table | Major | timelineserver | Varun Saxena | Varun Saxena | | | Change the way metric values are stored in HBase Storage | Major | timelineserver | Varun Saxena | Varun Saxena | | | TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch | Major |

| Varun Saxena | Varun Saxena | | | Support for fetching specific configs and metrics based on prefixes | Major | timelineserver | Varun Saxena | Varun Saxena | | | Move the safemode block count into BlockManager | Major | namenode | Haohui Mai | Mingliang Liu | | | Refactor reconfiguration of ClientDatanodeProtocol for reusability | Major | . | Xiaobing Zhou | Xiaobing Zhou | | | ensure the timeline service v.2 is disabled cleanly and has no impact when it's turned off | Critical | timelineserver | Sangjin Lee | Sangjin Lee | | | TestTimelineAuthenticationFilter and TestYarnConfigurationFields fail | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Code cleanup for DatanodeManager | Major | namenode | Jing Zhao | Jing Zhao | | | [Bug fix] RM fails to start when SMP is enabled | Major | timelineserver | Li Lu | Li Lu | | | Unify the term flowId and flowName in timeline v2 codebase | Major | timelineserver | Li Lu | Zhan Zhang | | | cgroups based memory monitoring for containers | Major | nodemanager | Karthik Kambatla | Varun Vasudev | | | TestDistributedShell fails for V2 scenarios | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | RM only get back addresses of Collectors that NM needs to know. | Critical | resourcemanager, timelineserver | Junping Du | Junping Du | | | Add an OPPORTUNISTIC ExecutionType | Major | nodemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Recovery may get very slow with lots of services with lots of app-attempts | Major | resourcemanager | Jun Gong | Jun Gong | | | Move code that tracks blocks with future generation stamps to BlockManagerSafeMode | Major | namenode | Mingliang Liu | Mingliang Liu | | | some tests in TestContainerLanch fails on non-english locale environment | Minor | nodemanager, test | Takashi Ohnishi | Takashi Ohnishi | | | Allow ResourceRequests to specify ExecutionType of a request ask | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Some of the NM events are not getting published due race condition when AM container finishes in NM | Major | nodemanager, timelineserver | Naganarasimha G R | Naganarasimha G R | | | Refactor reader classes in storage to nest under hbase specific package name | Minor | timelineserver | Vrushali C | Li Lu | | | Add cgroups support for docker containers | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Make SystemClock singleton so AppSchedulingInfo could use it | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Move BlockIdManager from FSNamesystem to BlockManager | Major | namenode | Jing Zhao | Jing Zhao | | | TestDistributedShell fails for v2 test cases after modifications for 1.5 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Directories that are mounted in docker containers need to be more restrictive/container-specific | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | TestAMRMClientOnRMRestart fails on trunk | Major | client, test | Takashi Ohnishi | Takashi Ohnishi | | | TestClientRMService#testGetClusterNodes fails occasionally | Major | test | Jason Lowe | Takashi Ohnishi | | | Add command line option to ask NameNode reload

| Major | namenode | Xiaobing Zhou | Xiaobing Zhou | | | correctly set createdTime and remove modifiedTime when publishing entities | Major | timelineserver | Varun Saxena | Varun Saxena | | | Support fetching entities by UID and change the REST interface to conform to current REST APIs' in YARN | Major | timelineserver | Varun Saxena | Varun Saxena | | | TestRMRestart fails and findbugs issue in YARN-2928 branch | Major | timelineserver | Varun Saxena | Varun Saxena | | | TestRMRestart.testRMRestartAfterPreemption fails intermittently in trunk | Major | test | Rohith Sharma K S | Bibin A Chundatt | | | TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt fails occasionally | Major | test | Jason Lowe | Sunil Govindan | | | Refactor reader API for better extensibility | Major | timelineserver | Varun Saxena | Varun Saxena | | | Fix javadoc and checkstyle issues in timelineservice code | Major | timelineserver | Varun Saxena | Varun Saxena | | | Replace starting a separate thread for post entity with event loop in TimelineClient | Major | timelineserver | Junping Du | Naganarasimha G R | | | TestYarnCLI#testGetContainers failing in CN locale | Major | yarn | Bibin A Chundatt | Bibin A Chundatt | | | TestBlockManagerSafeMode#testCheckSafeMode fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | Remove dependency on commons-httpclient for TestHttpServerLogs | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | ATS storage has one extra record each time the RM got restarted | Major | timelineserver | Li Lu | Naganarasimha G R | | | reconcile the two versions of the timeline service performance tests | Minor | . | Sangjin Lee | Sangjin Lee | | | Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart | Major | namenode | Patrick Kling | Xiaobing Zhou | | | Add the flush and compaction functionality via coprocessors and scanners for flow run table | Major | timelineserver | Vrushali C | Vrushali C | | | CPU Usage Metric is not captured properly in YARN-2928 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | lz4.c does not detect 64-bit mode properly | Major | native | Alan Burlison | Alan Burlison | | | Document Reservation ACLs | Minor | . | Sean Po | Sean Po | | | Don't go through all schedulers in ParameterizedTestBase | Major | . | Karthik Kambatla | Karthik Kambatla | | | NM is going down with NPE's due to single thread processing of events by Timeline client | Critical | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Support reconfiguring fs.protected.directories without NN restart | Major | namenode | Xiaobing Zhou | Xiaobing Zhou | | | Generate histograms in ContainerMetrics for actual container resource usage | Major | . | Varun Vasudev | Varun Vasudev | | | Consolidate flow name/version/run defaults | Major | timelineserver | Zhijie Shen | Sangjin Lee | | | S3A JUnit tests failing if using HTTP proxy | Minor | fs/s3 | Zoran Rajic | Zoran Rajic | | | Support complex filters in TimelineReader | Major | . | Varun Saxena | Varun Saxena | | | Support enable caller context in HDFS namenode audit log without restart namenode | Major | . | Xiaoyu Yao | Xiaobing Zhou | | | Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties | Major | . | Xiaoyu Yao | Xiaobing Zhou | | |

fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | Queuing of container requests in the NM | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate | Major | . | Wangda Tan | Sunil Govindan | | | Support enable Hadoop IPC backoff without namenode restart | Major | . | Xiaoyu Yao | Xiaobing Zhou | | | A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. | Major | . | Wangda Tan | Wangda Tan | | | [Aggregation] App-level aggregation and accumulation for YARN system metrics | Major | timelineserver | Junping Du | Li Lu | | | Random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Arun Suresh | | | Fix ContainerRequest Constructor to set nodelabelExpression correctly | Major | test | Bibin A Chundatt | Bibin A Chundatt | | | Improve yarn logs to fetch container logs without specifying nodeId | Major | . | Xuan Gong | Xuan Gong | | | MockAM#waitForState sleep duration is too long | Major | . | Karthik Kambatla | Yufei Gu | | | Add support in the NodeManager to re-launch containers | Major | . | Jun Gong | Jun Gong | | | Add a check in the coprocessor for table to operated on | Major | timelineserver | Vrushali C | Vrushali C | | | Ensure non-metric values are returned as is for flow run table from the coprocessor | Major | timelineserver | Vrushali C | Vrushali C | | | [Documentation] Documenting the timeline service v2 | Major | . | Zhijie Shen | Sangjin Lee | | | Store MR counters as timeline metrics instead of event | Major | . | Junping Du | Naganarasimha G R | | | Provide a mechanism to represent complex filters and parse them at the REST layer | Major | timelineserver | Varun Saxena | Varun Saxena | | | Store job configurations in Timeline Service v2 | Major | applicationmaster | Junping Du | Varun Saxena | | | ATS/NM should support a link to dowload/get the logs in text format | Major | yarn | Xuan Gong | Xuan Gong | | | Improve "yarn logs" command-line to optionally show log metadata also | Major | . | Xuan Gong | Xuan Gong | | | Add support for configurable read-only mounts when launching Docker containers | Major | yarn | Billie Rinaldi | Billie Rinaldi | | | hbase unit tests fail due to dependency issues | Blocker | timelineserver | Sangjin Lee | Sangjin Lee | | | Support specifying resources for task containers in SLS | Major | . | Wangda Tan | Wangda Tan | | | "yarn logs" command should not require the appOwner argument | Major | . | Ram Venkatesh | Xuan Gong | | | Refactor startContainerInternal() in ContainerManager to remove unused parameter | Minor | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Add maintenance states to AdminStates | Major | . | Ming Ma | Ming Ma | | | Corrective mechanisms for rebalancing NM container queues | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Arun Suresh | | | Notify the RM about the status of OPPORTUNISTIC containers | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Fix findbugs warning in hadoop-yarn-common module | Major |

| Akira Ajisaka | Arun Suresh | | | Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers | Major | nodemanager, resourcemanager | Arun Suresh | Arun Suresh | | | timeline service build fails with java 8 | Blocker | timelineserver | Sangjin Lee | Sangjin Lee | | | Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster | Major | . | Arun Suresh | Arun Suresh | | | timelinereader has a lot of logging that's not useful | Minor | timelineserver | Sangjin Lee | Sangjin Lee | | | Yarn logs should take a -out option to write to a directory | Major | . | Xuan Gong | Xuan Gong | | | saveNamespace should not throw IOE when only one storage directory fails to write VERSION file | Major | . | Todd Lipcon | Andras Bokor | | | Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds | Major | . | Arun Suresh | Konstantinos Karanasos | | | Code cleanup for TestDistributedShell | Major | timelineserver | Li Lu | Li Lu | | | Add support for a minimum retry interval for container retries | Major | . | Varun Vasudev | Jun Gong | | | Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect | Major | ui, webhdfs | Ravi Prakash | Ravi Prakash | | | created time shows 0 in most REST output | Critical | timelineserver | Sangjin Lee | Varun Saxena | | | Online aggregation logic should not run immediately after collectors got started | Major | timelineserver | Li Lu | Li Lu | | | Avoid setting CONTENT-DISPOSITION header in the container-logs web-service | Major | . | Xuan Gong | Xuan Gong | | | NPE in Separator.joinEncoded() | Critical | timelineserver | Sangjin Lee | Vrushali C | | | flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled | Critical | timelineserver | Sangjin Lee | Varun Saxena | | | fix "no findbugs output file" error for hadoop-yarn-server-timelineservice-hbase-tests | Major | timelineserver | Vrushali C | Vrushali C | | | timestamps are stored unencoded causing parse errors | Blocker | timelineserver | Sangjin Lee | Varun Saxena | | | Improve "yarn log" command-line "logFiles" option to support regex | Major | . | Xuan Gong | Xuan Gong | | | Expose ExecutionType in Container api record | Major | . | Arun Suresh | Hitesh Sharma | | | QueuingContainerManager does not start GUARANTEED Container even if Resources are available | Major | . | Arun Suresh | Konstantinos Karanasos | | | YARN container system metrics are not aggregated to application | Critical | timelineserver | Sangjin Lee | Naganarasimha G R | | | Add support for different network setups when launching the docker container | Major | nodemanager | Varun Vasudev | Sidharta Seethana | | | Get Container logs for the Running application from Yarn Logs CommandLine | Major | . | Xuan Gong | Xuan Gong | | | entire time series is returned for YARN container system metrics (CPU and memory) | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Improve "yarn log" command-line to read the last K bytes for the log files | Major |

| Xuan Gong | Xuan Gong | | | Fix Exceptions thrown during registerAM call when Distributed Scheduling is Enabled | Major | . | Arun Suresh | Hitesh Sharma | | | Make HBaseTimeline[Reader\|Writer]Impl default and move FileSystemTimeline\*Impl | Major | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | HDFS Namenode UI: Fix browsing directories that need to be encoded | Major | ui | Ravi Prakash | Ravi Prakash | | | Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI | Major | ui, webhdfs | Ravi Prakash | Ravi Prakash | | | fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests | Major | timelineserver | Vrushali C | Vrushali C | | | Escaping occurences of encodedValues | Critical | timelineserver | Joep Rottinghuis | Sangjin Lee | | | Allow ResourceRequest to specify an enforceExecutionType flag | Major | . | Arun Suresh | Arun Suresh | | | More test cases for queuing of containers at the NM | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Properly report status of killed/stopped queued containers | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | NPE in Distributed Shell while publishing DS\CONTAINER\START event and other miscellaneous issues | Major | timelineserver | Varun Saxena | Varun Saxena | | | Rename the download=true option for getLogs in NMWebServices and AHSWebServices | Major | . | Xuan Gong | Xuan Gong | | | Eliminate singleton converters and static method access | Major | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | Allow files / directories to be moved | Major | ui | Ravi Prakash | Ravi Prakash | | | [Documentation] Update timeline service v2 documentation to capture information about filters | Major | timelineserver | Varun Saxena | Varun Saxena | | | Modify AMRMClient to set the ExecutionType in the ResourceRequest | Major | . | Arun Suresh | Arun Suresh | | | Run existing ContainerManager tests using QueuingContainerManagerImpl | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | AM-RM protocol changes for identifying resource-requests explicitly | Major | applications, resourcemanager | Subru Krishnan | Subru Krishnan | | | Container line in yarn logs output for a live application should include the hostname for the container | Major | . | Siddharth Seth | Xuan Gong | | | "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete | Major | . | Vinod Kumar Vavilapalli | Jian He | | | fix several rebase and other miscellaneous issues before merge | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | upgrade HBase version for first merge | Critical | timelineserver | Sangjin Lee | Vrushali C | | | Yarn CLI to obtain App logs for last 'n' bytes fails with 'java.io.IOException' and for 'n' bytes fails with NumberFormatException | Blocker | . | Sumana Sathish | Xuan Gong | | | Add per-cache-pool default replication num configuration | Minor | caching | xupeng | xupeng | | | Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node | Major | . | Arun Suresh | igo Goiri | | | Inconsistent values of counters across tasks and job reported to timeline

| Major | applicationmaster | Varun Saxena | Varun Saxena | | | Exclude generated federation protobuf sources from YARN Javadoc/findbugs build | Minor | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | [documentation] several updates/corrections to timeline service documentation | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | yarn logs command: no need to specify -applicationId when specifying containerId | Major | . | Jian He | Gergely Novk | | | Logs for a completed container are not available in the yarn logs output for a live application | Major | . | Siddharth Seth | Xuan Gong | | | fix hadoop-aws pom not to do the exclusion | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Support for specifying a path for ATS plugin jars | Major | timelineserver | Li Lu | Li Lu | | | Improve yarn logs to get Container List | Major | . | Xuan Gong | Xuan Gong | | | Log Docker run command when container fails | Major | yarn | Varun Vasudev | Varun Vasudev | | | Fix signal handling for docker containers | Major | yarn | Sidharta Seethana | Shane Kumpf | | | For AM containers, or for containers of running-apps, "yarn logs" incorrectly only (tries to) shows syslog file-type by default | Major | log-aggregation | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | Mount usercache and NM filecache directories into Docker container | Major | yarn | Varun Vasudev | Sidharta Seethana | | | Obtaining logs for completed container says 'file belongs to a running container ' at the end | Critical | . | Sumana Sathish | Xuan Gong | | | YARN\CONTAINER\FINISHED of YARN\_CONTAINERs will always have running state | Major | timelineserver | Li Lu | Vrushali C | | | Distributed Scheduling: Ensure sort order of allocatable nodes returned by the RM is not lost | Major | . | Arun Suresh | Arun Suresh | | | Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey | Major | . | Arun Suresh | Arun Suresh | | | ResourceRequest should take ExecutionType into account during comparison | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Federation Membership State Store internal APIs | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl | Minor | timelineserver | Joep Rottinghuis | Vrushali C | | | Refactoring and other clean-up for distributed scheduling | Major | . | Arun Suresh | Konstantinos Karanasos | | | Rename DockerStopCommandTest to TestDockerStopCommand | Trivial | . | Shane Kumpf | Shane Kumpf | | | Add support for docker inspect command | Major | yarn | Shane Kumpf | Shane Kumpf | | | remove AHS enable check from LogsCLI#fetchAMContainerLogs | Major | . | Xuan Gong | Xuan Gong | | | Add support for docker rm | Minor | yarn | Shane Kumpf | Shane Kumpf | | | In-memory based implementation of the FederationMembershipStateStore | Major | nodemanager, resourcemanager | Subru Krishnan | Ellen Hui | | | Federation Subcluster Resolver | Major | nodemanager, resourcemanager | Carlo Curino | Ellen Hui | | | Fix @return related javadoc warnings in yarn-api | Major |

| Vrushali C | Vrushali C | | | Changes in scheduler to identify resource-requests explicitly by allocation-id | Major | resourcemanager | Subru Krishnan | Subru Krishnan | | | Federation Application State Store internal APIs | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Federation PolicyStore internal APIs | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Differentiate exactly match with regex in yarn log CLI | Major | . | Xuan Gong | Xuan Gong | | | Compose Federation membership/application/policy APIs into an uber FederationStateStore API | Major | nodemanager, resourcemanager | Subru Krishnan | Ellen Hui | | | Remove bind-mount /etc/passwd for Docker containers | Major | yarn | Zhankun Tang | Zhankun Tang | | | In-memory based implementation of the FederationApplicationStateStore, FederationPolicyStateStore | Major | nodemanager, resourcemanager | Subru Krishnan | Ellen Hui | | | Make DiskChecker pluggable in NodeManager | Major | nodemanager | Ray Chiang | Yufei Gu | | | Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | Ellen Hui | | | Clarify DecommissionType.FORCEFUL comment | Minor | documentation | Robert Kanter | Vrushali C | | | Create Facade for Federation State and Policy Store | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | InputValidator for the FederationStateStore internal APIs | Major | nodemanager, resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Automatic and Asynchronous Decommissioning Nodes Status Tracking | Major | resourcemanager | Daniel Zhi | Daniel Zhi | | | Create a FailoverProxy for Federation services | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Refactor DistributedScheduling framework to pull out common functionality | Major | resourcemanager | Arun Suresh | Arun Suresh | | | Mount /sys/fs/cgroup into Docker containers as read only mount | Major | yarn | Varun Vasudev | luhuichun | | | Fix typo in RM\SCHEDULER\RESERVATION\THRESHOLD\INCREMENT\_MULTIPLE | Trivial | fairscheduler | Ray Chiang | Ray Chiang | | | Add localize API to the ContainerManagementProtocol | Major | . | Jian He | Jian He | | | API changes required to support recurring reservations in the YARN ReservationSystem | Major | resourcemanager | Subru Krishnan | Sangeetha Abdu Jyothi | | | Changes in AMRMClient for identifying resource-requests explicitly | Major | resourcemanager | Subru Krishnan | Arun Suresh | | | Integrate Federation services with ResourceManager | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Admins support for maintenance state | Major | . | Ming Ma | Ming Ma | | | DiskBalancer: Add the getNodeList method in Command | Minor | balancer & mover | Yiqun Lin | Yiqun Lin | | | Expose UpdateResourceRequest API to allow AM to request for change in container properties | Major | . | Arun Suresh | Arun Suresh | | | Fix failing unit test in TestDockerContainerRuntime | Minor | nodemanager, yarn | Sidharta Seethana | Sidharta Seethana | | | Return SubClusterId in FederationStateStoreFacade#addApplicationHomeSubCluster for Router Failover | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Make the RM epoch base value configurable | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Store all queue-specific information in FSQueue | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Allow resource localization while container is running | Major | . | Jian He | Jian He | | | Policies APIs (for Router and AMRMProxy policies) | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | Core changes in NodeManager to support re-initialization of Containers with new launchContext | Major |

| Arun Suresh | Arun Suresh | | | Add REST API to retrieve scheduler activity | Major | capacity scheduler, resourcemanager | Sunil Govindan | Chen Ge | | | Changes in NodeManager to support Container rollback and commit | Major | . | Arun Suresh | Arun Suresh | | | Improve locks in AbstractCSQueue/LeafQueue/ParentQueue | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | Stateless Federation router policies implementation | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | Expose upgrade and restart API in ContainerManagementProtocol | Major | . | Arun Suresh | Arun Suresh | | | Remove unnecessary conf cache from DataNode | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | Improve locks in AppSchedulingInfo | Major | resourcemanager, scheduler | Wangda Tan | Varun Saxena | | | Add a service for monitoring application life time out | Major | scheduler | nijel | Rohith Sharma K S | | | Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests | Major | resourcemanager | Arun Suresh | Konstantinos Karanasos | | | Expose priority in ReservationSystem submission APIs | Major | capacity scheduler, fairscheduler, resourcemanager | Sean Po | Sean Po | | | Refactor TestPBImplRecords so that we can reuse for testing protocol records in other YARN modules | Major | . | Subru Krishnan | Subru Krishnan | | | Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test | Major | test | Mingliang Liu | Mingliang Liu | | | Have an explicit KerberosAuthException for UGI to throw, text from public constants | Major | security | Steve Loughran | Xiao Chen | | | Stateless ARMRMProxy policies implementation | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery | Major | timelineserver | Li Lu | Li Lu | | | Retrospect yarn entity fields which are publishing in events info fields. | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | Block management for maintenance states | Major |

| Ming Ma | Ming Ma | | | [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | Bad placement policy in FairScheduler causes the RM to crash | Major | fairscheduler | Ray Chiang | Ray Chiang | | | Refactor nodeUpdate across schedulers | Major | capacityscheduler, fairscheduler, scheduler | Ray Chiang | Ray Chiang | | | Allow configurable prefix for hbase table names (like prod, exp, test etc) | Major | timelineserver | Vrushali C | Vrushali C | | | introduce entity prefix for return and sort order | Critical | timelineserver | Sangjin Lee | Rohith Sharma K S | | | Create POC timeline web UI with new YARN web UI framework | Major | timelineserver | Li Lu | Li Lu | | | Fix Opportunistic Allocation to set the correct value of Node Http Address | Major | resourcemanager | Arun Suresh | Arun Suresh | | | Split TestHBaseTimelineStorage into multiple test classes | Major | timelineserver | Varun Saxena | Varun Saxena | | | Trim configuration values in DockerLinuxContainerRuntime | Minor | nodemanager | Tianyin Xu | Tianyin Xu | | | PolicyManager to tie together Router/AMRM Federation policies | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | Make HBase configuration for the timeline service configurable | Major | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | Enhance UI to show cluster resource utilization of various container Execution types | Blocker | resourcemanager | Sriram Rao | Konstantinos Karanasos | | | Recover collector list when RM fails over | Major | resourcemanager | Junping Du | Li Lu | | | Add global scheduler interface definition and update CapacityScheduler to use it. | Major | capacityscheduler | Wangda Tan | Wangda Tan | | | Add validation to ensure default ports are unique in Configuration | Major | yarn | Konstantinos Karanasos | Konstantinos Karanasos | | | Allow fetching exact reason as to why a submitted app is in ACCEPTED state in Fair Scheduler | Major | fairscheduler, resourcemanager | Naganarasimha G R | Yufei Gu | | | Provide an API to update lifetime of an application. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Support for parallel checking of StorageLocations on DataNode startup | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Support for running async disk checks in DataNode | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Simplify initialization/use of RouterPolicy via a RouterPolicyFacade | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | adopt the id prefix for YARN, MR, and DS entities | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Update DataNode to use StorageLocationChecker at startup | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Add a HashBasedRouterPolicy, and small policies and test refactoring. | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | Add AlwayReject policies for router and amrmproxy. | Major | nodemanager, resourcemanager | Carlo Curino | Carlo Curino | | | Update the RM webapp host that is reported as part of Federation membership to current primary RM's IP | Minor | federation, resourcemanager | Subru Krishnan | Subru Krishnan | | | Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611 | Major | . | Sunil Govindan | Sunil Govindan | | | Add REST endpoints for updating application timeouts | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Add Application timeouts in Application report and CLI | Major | scheduler | nijel | Rohith Sharma K S | | | Support for parallel checking of FsVolumes | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Bring back the feature to view chunks of files in the HDFS file browser | Major | ui | Ravi Prakash | Ravi Prakash | | | Separate QueueManager from Scheduler | Major | capacityscheduler | Xuan Gong | Xuan Gong | | | The state of the parentQueue and its childQueues should be synchronized. | Major | capacity scheduler, resourcemanager | Xuan Gong | Xuan Gong | | |

ApplicationReport#getApplicationTimeouts | Major | scheduler | Jian He | Rohith Sharma K S | | | Provide timeline reader API to list available timeline entity types for one application | Major | timelinereader | Li Lu | Li Lu | | | Remove direct reference to TimelineClientImpl | Major | . | Li Lu | Li Lu | | | Simplify opportunistic container parameters and metrics | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Validate application update timeout request parameters | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Yarn live log aggregation does not throw if command line arg is wrong | Major | log-aggregation | Prasanth Jayachandran | Xuan Gong | | | Render Application Timeout value in web UI | Major | scheduler | Rohith Sharma K S | Akhil PB | | | Update DataNode to use DatasetVolumeChecker | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Update hbase version to 1.2 (removes phoenix dependencies) | Major | . | Vrushali C | Vrushali C | | | Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM | Major | distributed-scheduling | Arun Suresh | Hitesh Sharma | | | Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes | Major | . | Arun Suresh | Arun Suresh | | | Add state-machine implementation for scheduler queues | Major | . | Xuan Gong | Xuan Gong | | | Document timeout interfaces CLI and REST APIs | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Create new DiskValidator class with metrics | Major | nodemanager | Ray Chiang | Yufei Gu | | | Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations | Major | capacity scheduler, scheduler | Naganarasimha G R | Naganarasimha G R | | | Unable to access logs for a running application if YARN\ACL\ENABLE is enabled | Major | . | Xuan Gong | Xuan Gong | | | Update AppSchedulingInfo to use SchedulingPlacementSet | Major | . | Wangda Tan | Wangda Tan | | | RM fails to start during an upgrade - Failed to load/recover state (YarnException: Invalid application timeout, value=0 for type=LIFETIME) | Critical | resourcemanager | Gour Saha | Rohith Sharma K S | | | [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters | Critical | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | FlowRunEntity does not deserialize long values correctly | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | Support for application priority ACLs in queues of CapacityScheduler | Major | capacityscheduler | Sunil Govindan | Sunil Govindan | | | Update webUI/JMX to display maintenance state info | Major | . | Ming Ma | Manoj Govindassamy | | | TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped | Minor | test, yarn | Junping Du | Junping Du | | | Update documentation for single node hbase deploy | Major | timelineserver | Vrushali C | Vrushali C | | | Add a new web service to list the files on a container in AHSWebService | Major |

| Xuan Gong | Xuan Gong | | | Accommodate app-id-\>cluster mapping | Major | timelineserver | Joep Rottinghuis | Sangjin Lee | | | Fix minor bugs in handling of local AMRMToken in AMRMProxy | Minor | federation | Botong Huang | Botong Huang | | | CapacityScheduler: Support deleting queues without requiring a RM restart | Major | capacity scheduler | Xuan Gong | Naganarasimha G R | | | Support fromId for flowRuns and flow/flowRun apps REST API's | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | Update fsck to display maintenance state info | Major | datanode, namenode | Manoj Govindassamy | Manoj Govindassamy | | | Update the coprocessor to be a dynamically loaded one | Major | timelineserver | Vrushali C | Vrushali C | | | Propagate allowPreemptionFrom flag all the way down to the app | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | Maintenance state expiry should be an epoch time and not jvm monotonic | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Improve webservice to list aggregated log files | Major | . | Xuan Gong | Xuan Gong | | | FairScheduler: Avoid preempting AM containers | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | Handle localization sym-linking correctly at the YARN level | Major | . | Chris Trezzo | Chris Trezzo | | | Obtaining app logs for Running application fails with "Unable to parse json from webservice. Error:" | Major | . | Sumana Sathish | Xuan Gong | | | improve YARN webservice to output aggregated container logs | Major | . | Xuan Gong | Xuan Gong | | | Changes to allow CapacityScheduler to use configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Improve AHS webservice to accept NM address as a parameter to get container logs | Major | . | Xuan Gong | Xuan Gong | | | Document Use of Docker with LinuxContainerExecutor | Critical | documentation | Daniel Templeton | Daniel Templeton | | | Reduce fair call queue priority inversion | Major | ipc | Daryn Sharp | Daryn Sharp | | | Allow ipc layer exceptions to selectively close connections | Major | ipc | Daryn Sharp | Daryn Sharp | | | Reduce fair call queue lock contention | Major | ipc | Daryn Sharp | Daryn Sharp | | | TimelineReaderServer should wait to join with HttpServer2 | Minor | timelinereader | Sangjin Lee | Sangjin Lee | | | Add support for SSE-KMS and SSE-C in s3a filesystem | Major | fs/s3 | Andrew Olson | Steve Moist | | | re-direct NM Web Service to get container logs for finished applications | Major | . | Xuan Gong | Xuan Gong | | | AMRMClient changes to support ExecutionType update | Major | . | Arun Suresh | Arun Suresh | | | AM blacklisting to consider node label partition | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Extend visualization for Maintenance Mode under Datanode tab in the NameNode UI | Major | datanode, namenode | Manoj Govindassamy | Elek, Marton | | | FS Preemption is a trickle for severely starved applications | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Set UncaughtExceptionHandler for all FairScheduler threads | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Use shaded aws-sdk uber-JAR

| Major | build, fs/s3 | Steve Loughran | Steve Loughran | | | FairScheduler might not trigger preemption when using DRF | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Documentation changes for TimelineV2Client | Minor | documentation | Varun Saxena | Naganarasimha G R | | | Separate class InnerNode from class NetworkTopology and make it extendable | Major | namenode | Chen Liang | Tsz Wo Nicholas Sze | | | Split S3 testing documentation out into its own file | Minor | documentation, fs/s3 | Steve Loughran | Steve Loughran | | | Avoid OutOfMemoryError in TestMaintenanceState test runs | Major | datanode, namenode | Manoj Govindassamy | Manoj Govindassamy | | | Relax error message assertion in S3A test ITestS3AEncryptionSSEC | Minor | fs/s3 | Mingliang Liu | Mingliang Liu | | | CORS support in timeline v2 | Major | timelinereader | Sreenath Somarajapuram | Rohith Sharma K S | | | RM changes to support change of container ExecutionType | Major | . | Arun Suresh | Arun Suresh | | | EntityGroupFSTimelineStore should allow configurable cache stores. | Major | timelineserver | Li Lu | Li Lu | | | Bootstrap Router server module | Major | nodemanager, resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Create YarnConfigurationStore interface and InMemoryConfigurationStore class | Major | . | Jonathan Hung | Jonathan Hung | | | FairScheduler preemption and update should not run concurrently | Major | fairscheduler, test | Sunil Govindan | Tao Jie | | | [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. | Major | . | Wangda Tan | Wangda Tan | | | FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp | Major | . | Haibo Chen | Haibo Chen | | | Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy | Minor | federation | Botong Huang | Botong Huang | | | Support fromid(offset) filter for /flows API | Major | timelineserver | Rohith Sharma K S | Rohith Sharma K S | | | HDFS specific network topology classes with storage type info included | Major | namenode | Chen Liang | Chen Liang | | | Maintenance minimum replication config value allowable range should be [0, DefaultReplication] | Major | datanode, namenode | Manoj Govindassamy | Manoj Govindassamy | | | Share a single instance of SubClusterResolver instead of instantiating one per AM | Minor | . | Botong Huang | Botong Huang | | | Add FROM\_ID info key for timeline entities in reader response. | Major | timelineserver | Rohith Sharma K S | Rohith Sharma K S | | | Move UID constant to TimelineReaderUtils | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | Add support for Docker pull | Major | yarn | Zhankun Tang | luhuichun | | | Implement MutableConfigurationManager for handling storage into configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Expose # of pre-emptions as a queue counter | Major | fairscheduler | Philip Zeyliger | Karthik Kambatla | | | Cleanup when AMRMProxy fails to initialize a new interceptor chain | Minor | . | Botong Huang | Botong Huang | | | timeline service schema creator fails if executed from a remote machine | Minor | timelineserver | Sangjin Lee | Sangjin Lee | | | Add storage type demand to into DFSNetworkTopology#chooseRandom | Major | namenode | Chen Liang | Chen Liang | | | Potential infinite redirection on YARN log redirection web service | Major |

| Xuan Gong | Xuan Gong | | | yarn logs cli should provide logs for a completed container even when application is still running | Major | . | Siddharth Seth | Xuan Gong | | | DFSTopologyNodeImpl#chooseRandom optimizations | Major | namenode | Chen Liang | Chen Liang | | | YARN logs CLI needs alway check containerLogsInfo/containerLogInfo before parse the JSON object from NMWebService | Major | . | Siddharth Seth | Xuan Gong | | | Add Builder methods for TimelineEntityFilters | Major | timelineserver | Rohith Sharma K S | Haibo Chen | | | needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject() | Minor | fs/s3 | Steve Loughran | Yuanbo Liu | | | Remove URI parameter in AWSCredentialProvider constructors | Major | fs/s3 | Mingliang Liu | Mingliang Liu | | | Azure Data Lake doc is missing required config entry | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | Fix ADLS doc for credential provider | Major | documentation, fs/adl | John Zhuge | John Zhuge | | | TestAdlFileSystemContractLive fails to clean up | Minor | fs/adl, test | John Zhuge | John Zhuge | | | Rename ADLS credential properties | Minor | fs/adl | John Zhuge | John Zhuge | | | Combine the old and the new chooseRandom for better performance | Major | namenode | Chen Liang | Chen Liang | | | Implement putEntitiesAsync API in TimelineCollector | Major | ATSv2, timelineserver | Joep Rottinghuis | Haibo Chen | | | Make TimelineV2Client's drain timeout after stop configurable | Major | . | Jian He | Haibo Chen | | | Exceptions caused by synchronous putEntities requests can be swallowed | Critical | ATSv2 | Haibo Chen | Haibo Chen | | | ATSv2 HBase related tests fail due to guava version upgrade | Major | timelineserver | Sonia Garudi | Haibo Chen | | | NMTimelinePublisher#serviceStop does not stop timeline clients | Major | yarn | Haibo Chen | Haibo Chen | | | Create REST API for changing YARN scheduler configurations | Major | . | Jonathan Hung | Jonathan Hung | | | Add an ability to convert ChildQueue to ParentQueue | Major | capacity scheduler | Xuan Gong | Xuan Gong | | | TimelineCollector is not stopped when an app finishes in RM | Critical | timelineserver | Varun Saxena | Varun Saxena | | | Utils for Federation State and Policy Store | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Federation State Store Interface API | Major | fs | Jason Kace | Jason Kace | | | Update SLF4J from 1.7.10 to 1.7.25 | Major | . | Akira Ajisaka | Akira Ajisaka | | | Add default value for NM disk validator | Major | nodemanager | Yufei Gu | Yufei Gu | | | Deprecate SharedInstanceProfileCredentialsProvider in branch-2. | Major | fs/s3 | Mingliang Liu | Mingliang Liu | | | S3A to delete unnecessary fake directory objects in mkdirs() | Major | fs/s3 | Mingliang Liu | Mingliang Liu | | | Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes | Major | . | Wangda Tan | Wangda Tan | | | Occasional test failure in TestWeightedRandomRouterPolicy | Minor | federation | Botong Huang | Carlo Curino | | | FairScheduler: Reserve preempted resources for corresponding applications | Major |

| Miklos Szegedi | Miklos Szegedi | | | Explicitly exclude S3A root dir ITests from parallel runs | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | Add ADLS sensitive config keys to default list | Minor | fs, fs/adl, security | John Zhuge | John Zhuge | | | Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus | Minor | fs/s3, test | Steve Loughran | Steve Moist | | | Federation State and Policy Store (DBMS implementation) | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Rename ADLS CONTRACT\ENABLE\KEY | Minor | fs/adl | Mingliang Liu | Mingliang Liu | | | Enhance the timelinewriter.flush() race condition fix | Major | yarn | Haibo Chen | Haibo Chen | | | Add support for async edit logging | Major | namenode | Daryn Sharp | Daryn Sharp | | | Update LowCost agents logic to take advantage of YARN-4358 | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Ishai Menache | | | Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent | Major | reservation system | Subru Krishnan | Subru Krishnan | | | Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem | Major | resourcemanager | Subru Krishnan | Sangeetha Abdu Jyothi | | | Provide configuration support for upgrade domain | Major | . | Ming Ma | Ming Ma | | | Display upgrade domain information in fsck | Major | . | Ming Ma | Ming Ma | | | Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes | Minor | . | Chris Trezzo | Chris Trezzo | | | Federation State Store FS Implementation | Major | hdfs | igo Goiri | Jason Kace | | | Create a proxy chain for ApplicationClientProtocol in the Router | Major | nodemanager, resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Improve test coverage and add utility classes for common Docker operations | Major | nodemanager, yarn | Shane Kumpf | Shane Kumpf | | | App level aggregation should not consider metric values reported in the previous aggregation cycle | Major | timelineserver | Varun Saxena | Varun Saxena | | | Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault | Major | namenode | Yiqun Lin | Yiqun Lin | | | Fix memory leak and finish app trigger in AMRMProxy | Critical | . | Botong Huang | Botong Huang | | | Support multiple attempts on the node when AMRMProxy is enabled | Major | amrmproxy, federation, nodemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Reduce the visibility

FileSystem#newFSDataOutputStreamBuilder before the API becomes stable | Blocker | fs | Lei (Eddy) Xu | Lei (Eddy) Xu | | | ConcurrentModificationException in TimelineCollectorManager while stopping RM | Major | resourcemanager | Rohith Sharma K S | Haibo Chen | | | Create a proxy chain for ResourceManager Admin API in the Router | Major | nodemanager, resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | [ATSv2] Can't retrieve more than 1000 versions of metrics in time series | Critical | timelineserver | Rohith Sharma K S | Vrushali C | | | Update exception message during timeline collector aux service initialization | Minor | timelineserver | Vrushali C | Vrushali C | | | NMClient API change for container upgrade | Major | . | Jian He | Arun Suresh | | | s3a delete() operation fails during a concurrent delete of child entries | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add pluggable configuration ACL policy interface and implementation | Major | . | Jonathan Hung | Jonathan Hung | | | Refactor of ResourceManager#startWebApp in a Util class | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Store application flow context in NM state store for work-preserving restart | Major | timelineserver | Vrushali C | Rohith Sharma K S | | | UnmanagedAM pool manager for federating application across clusters | Major | nodemanager, resourcemanager | Subru Krishnan | Botong Huang | | | TestMaintenanceState#testWithNNAndDNRestart fails intermittently | Major | . | Yiqun Lin | Yiqun Lin | | | Fix unit test failure in TestRouterClientRMService | Minor | . | Botong Huang | Botong Huang | | | [Documentation] Documenting the YARN Federation feature | Major | nodemanager, resourcemanager | Subru Krishnan | Carlo Curino | | | Remove columnFor() methods of Columns in HBaseTimeline backend | Major | . | Haibo Chen | Haibo Chen | | | Federation Intercepting and propagating AM- home RM communications | Major | nodemanager, resourcemanager | Kishore Chaliparambil | Botong Huang | | | Identifying starved apps does not need the scheduler writelock | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Reuse iip in unprotectedRemoveXAttrs calls | Major | . | Xiao Chen | Xiao Chen | | | DFSAdmin report command supports displaying maintenance state datanodes | Major | datanode, namenode | Yiqun Lin | Yiqun Lin | | | Provide help information and documentation for TimelineSchemaCreator | Major | timelineserver | Li Lu | Haibo Chen | | | Support global configuration mutation in MutableConfProvider | Major | . | Jonathan Hung | Jonathan Hung | | | Reduce fair call queue backoff's impact on clients | Major | ipc | Daryn Sharp | Daryn Sharp | | | Allow metric TTL for Application table to be specified through cmd | Major | ATSv2 | Haibo Chen | Haibo Chen | | | [ATSv2 Security] Collector side changes for loading auth filters and principals | Major | timelineserver | Varun Saxena | Varun Saxena | | | Federation: transparently spanning application across multiple sub-clusters | Major | . | Botong Huang | Botong Huang | | | Reduce Resource instance overhead via non-PBImpl | Major | resourcemanager | Daryn Sharp | Daryn Sharp | | | Add more debug logs for delegation tokens and authentication | Minor | security | Xiao Chen | Xiao Chen | | | KMS should log error details in KMSExceptionsProvider | Major | kms | Xiao Chen | Xiao Chen | | | Improve performance of AssignmentInformation datastructures | Major | . | Daryn Sharp | Daryn Sharp | | | [ATSv2 Security] Timeline reader side changes for loading auth filters and principals | Major | timelineserver | Varun Saxena | Varun Saxena | | | [ATSv2] Update HBase version to 1.2.6 | Major | timelineserver | Varun Saxena | Vrushali C | | | Provide Builder pattern for DistributedFileSystem.create | Major | fs | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Move log4j APIs over to slf4j in hadoop-common | Major | . | Akira Ajisaka | Akira Ajisaka | | | Provide Builder pattern for

| Major | fs | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API | Major | fs | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Avoid locking overhead for NO\_LABEL lookups | Major | resourcemanager | Daryn Sharp | Daryn Sharp | | | Move logging APIs over to slf4j in hadoop-tools | Major | . | Akira Ajisaka | Akira Ajisaka | | | Add ability to blacklist sub-clusters when invoking Routing policies | Major | router | Subru Krishnan | Giovanni Matteo Fumarola | | | Enable DFSNetworkTopology as default | Major | namenode | Chen Liang | Chen Liang | | | Add IOUtils.cleanupWithLogger that accepts slf4j logger API | Major | . | Akira Ajisaka | Chen Liang | | | [WASB] the configured retry policy is not used for all storage operations. | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | [ATSv2 Security] Client side changes for authentication | Major | timelineserver | Varun Saxena | Varun Saxena | | | Use GenericTestUtils.setLogLevel when available in hadoop-tools | Major | . | Akira Ajisaka | Wenxin He | | | Federation: routing client invocations transparently to multiple RMs | Major | client, resourcemanager, router | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | regression: Azure tests which capture logs failing with move to SLF4J | Major | fs/azure, test | Steve Loughran | Steve Loughran | | | Azure: Concurrent I/O does not work when secure.mode is enabled | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | add more on s3 regions to the s3a documentation | Minor | documentation, fs/s3 | Steve Loughran | Steve Loughran | | | NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code | Major | fs/azure | Steve Loughran | Steve Loughran | | | Document graceful decommission CLI and usage | Major | documentation | Junping Du | Elek, Marton | | | AWS SDK 1.11+ aborts() on close() if \> 0 bytes in stream; logs error | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Eliminate double-copy of child queues in canAssignToThisQueue | Major | resourcemanager | Daryn Sharp | Daryn Sharp | | | Add ServiceOperations.stopQuietly that accept slf4j logger API | Major | . | Wenxin He | Wenxin He | | | Add ReflectionUtils.logThreadInfo that accept slf4j logger API | Major | . | Wenxin He | Wenxin He | | | Deprecate public APIs relate to log4j1 | Major | . | Akira Ajisaka | Wenxin He | | | Use GenericTestUtils.setLogLevel when available in hadoop-common | Major | . | Wenxin He | Wenxin He | | | Create CLI for changing YARN configurations | Major | . | Jonathan Hung | Jonathan Hung | | | Refactor ApplicaitonMasterService to move actual processing logic to a separate class | Minor | . | Arun Suresh | Arun Suresh | | | Replace commons-logging APIs with slf4j in StreamPumper | Major |

| Wenxin He | Wenxin He | | | NPE in RM while setting collectors map in NodeHeartbeatResponse | Major | timelineserver | Vrushali C | Vrushali C | | | Adding required missing configs to Federation configuration guide based on e2e testing | Major | documentation, federation | Subru Krishnan | Tanuj Nayak | | | [Bug] FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Move FederationStateStore SQL DDL files from test resource to sbin | Major | nodemanager, resourcemanager | Subru Krishnan | Subru Krishnan | | | Add cross-user preemption within CapacityScheduler's leaf-queue | Major | capacity scheduler | Vinod Kumar Vavilapalli | Sunil Govindan | | | NM Container lifecycle and state transitions to support for PAUSED container state. | Major | . | Hitesh Sharma | Hitesh Sharma | | | wasb: add support for caching Authorization and SASKeys | Major | fs/azure | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Support for ApplicationMasterService processing chain of interceptors | Major | . | Arun Suresh | Arun Suresh | | | CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls | Major | capacityscheduler | Nathan Roberts | Nathan Roberts | | | Support fetching metrics by time range | Major | timelineserver | Varun Saxena | Varun Saxena | | | Ensure that supplemented timestamp is stored only for flow run metrics | Major | timelineserver | Vrushali C | Varun Saxena | | | Add table for storing sub-application entities | Major | timelineserver | Vrushali C | Vrushali C | | | Customize User-Agent header sent in HTTP/HTTPS requests by WASB. | Minor | fs/azure | Georgi Chalakov | Georgi Chalakov | | | Allow custom hostname for docker containers in native services | Major | yarn-native-services | Billie Rinaldi | Billie Rinaldi | | | Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk | Major | federation | Subru Krishnan | Botong Huang | | | Create a proxy chain for ResourceManager REST API in the Router | Major | nodemanager, resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created | Major | timelineserver | Vrushali C | Vrushali C | | | AdlFilesystem#initialize swallows exception when getting user name | Minor | fs/adl | John Zhuge | John Zhuge | | | Ensure sub-application user is extracted & sent to timeline service | Major | timelineserver | Vrushali C | Rohith Sharma K S | | | Rebasing HDFS-10467 | Major | fs | igo Goiri | igo Goiri | | | Update Microsoft JDBC Driver for SQL Server version in License.txt | Minor | federation | Botong Huang | Botong Huang | | | Federation Membership State Store internal API | Major | hdfs | igo Goiri | Jason Kace | | | Create LeveldbConfigurationStore class using Leveldb as backing store | Major | . | Jonathan Hung | Jonathan Hung | | | Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it | Major | common, fs, hdfs-client | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Disable queue refresh when configuration mutation is enabled | Major | . | Jonathan Hung | Jonathan Hung | | | Federation Namenode Heartbeat | Major | fs | igo Goiri | igo Goiri | | | Add MySql Scripts for FederationStateStore | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Refactor decommission related code to support maintenance state for datanodes | Major | . | Ming Ma | Manoj Govindassamy | | | ContainerManagementProtocol changes to support change of container ExecutionType | Major |

| Arun Suresh | kartheek muthyala | | | remove jackson, joda and other transient aws SDK dependencies from hadoop-aws | Minor | build, fs/s3 | Steve Loughran | Steve Loughran | | | Federation Mount Table State Store internal API | Major | fs | Jason Kace | igo Goiri | | | Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager | Major | . | Yeliang Cang | Yeliang Cang | | | Handle concurrent register AM requests in FederationInterceptor | Minor | . | Botong Huang | Botong Huang | | | Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice | Major | . | Yeliang Cang | Yeliang Cang | | | Refactoring RMWebServices by moving some util methods to RMWebAppUtil | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Federation admin tool | Major | fs | igo Goiri | igo Goiri | | | TestWasbRemoteCallHelper failing | Major | fs/azure, test | Steve Loughran | Esfandiar Manii | | | Add PoolInitializationException as retriable exception in FederationFacade | Major | federation | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite | Major | timelineserver | Varun Saxena | Vrushali C | | | Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice | Major | . | Yeliang Cang | Yeliang Cang | | | Remove service loader config file for wasb fs | Minor | fs/azure | John Zhuge | Esfandiar Manii | | | [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM | Major | timelineserver | Varun Saxena | Varun Saxena | | | [ATSv2 Security] Renew delegation token for app automatically if an app collector is active | Major | timelineserver | Varun Saxena | Varun Saxena | | | Restrict read access to timelineservice v2 data | Major | timelinereader | Vrushali C | Vrushali C | | | Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution) | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Validate that the duration of the periodic reservation is less than the periodicity | Major | reservation system | Subru Krishnan | Subru Krishnan | | | Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat | Major | timelineserver | Sonia Garudi | Haibo Chen | | | ContainerScheduler and ContainerManager changes to support ExecType update | Major | . | Arun Suresh | kartheek muthyala | | | Deleting all children of a Parent Queue on refresh throws exception | Major | capacity scheduler | Naganarasimha G R | Naganarasimha G R | | | wasb: improve throughput by 34% when account limit exceeded | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol | Major | timelineserver | Varun Saxena | Varun Saxena | | | ZooKeeper based implementation of the FederationStateStore | Major | federation, nodemanager, resourcemanager | Subru Krishnan | igo Goiri | | | [Documentation] Router-based federation documentation | Minor | fs | igo Goiri | igo Goiri | | | Rebasing HDFS-10467 (2) | Major |

| igo Goiri | igo Goiri | | | Backport HADOOP-14386 to YARN-5355 branch | Major | timelineserver | Vrushali C | Vrushali C | | | container-executor fails for docker when command length \> 4096 B | Major | yarn | Eric Badger | Eric Badger | | | [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled | Major | . | Rohith Sharma K S | Varun Saxena | | | WASB: delete recursive should not fail if a file is deleted | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | Nodemanager NPE running jobs with security off | Major | timelineserver | Aaron Gresch | Varun Saxena | | | [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active | Major | timelineserver | Varun Saxena | Varun Saxena | | | Add flag to notify all types of container updates to NM via NodeHeartbeatResponse | Major | . | Arun Suresh | kartheek muthyala | | | Reader API for sub application entities | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | Rolling upgrade/config change is broken on timeline v2. | Major | timelineserver | Li Lu | Vrushali C | | | Federation State Store ZooKeeper implementation | Major | fs | igo Goiri | Jason Kace | | | Documentation updates for TimelineService v2 | Major | documentation, timelineserver | Varun Saxena | Rohith Sharma K S | | | [ATSv2 Security] Add timeline delegation token received in allocate response to UGI | Major | . | Varun Saxena | Varun Saxena | | | [Security] Review and implement authentication in ATS v.2 | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Metrics for Federation StateStore | Major | . | Subru Krishnan | Ellen Hui | | | Metrics for Federation Router | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Fix issues on recovery in LevelDB store | Major | . | Jonathan Hung | Jonathan Hung | | | Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager | Major | . | Yeliang Cang | Yeliang Cang | | | Create an abstract log writer for extendability | Major | . | Xuan Gong | Xuan Gong | | | Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps) | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | When an export var command fails in launch\_container.sh, the full container launch should fail | Major | . | Hitesh Shah | Sunil Govindan | | | Add support for using container saskeys for all accesses | Major | fs/azure | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Document the current known issue with server-side NM graceful decom | Blocker | graceful | Robert Kanter | Robert Kanter | | | Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Fix issues with docker commands executed by container-executor | Major | nodemanager | Shane Kumpf | Shane Kumpf | | | Create an abstract log reader for extendability | Major | . | Xuan Gong | Xuan Gong | | | Sort out hadoop-aws contract-test-options.xml | Minor | fs/s3, test | Steve Loughran | John Zhuge | | | TestLogsCLI fails in trunk and branch-2 and javadoc error | Major | . | Xuan Gong | Xuan Gong | | | Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem | Major | resourcemanager | Subru Krishnan | Subru Krishnan | | | Add updateContainer API to NMClient. | Major |

| Arun Suresh | kartheek muthyala | | | S3A case "testRandomReadOverBuffer" failed due to improper range parameter | Minor | fs/s3 | Yonger | Yonger | | | Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. | Major | . | Xuan Gong | Xuan Gong | | | Fixing compilation issue with BanDuplicateClasses | Major | fs | igo Goiri | igo Goiri | | | Introduce default and max lifetime of application at LeafQueue level | Major | capacity scheduler | Rohith Sharma K S | Rohith Sharma K S | | | NMContainerStatus should have the Container ExecutionType. | Major | . | Arun Suresh | kartheek muthyala | | | Federation Metrics | Major | fs | Giovanni Matteo Fumarola | igo Goiri | | | SharingPolicy enhancements required to support recurring reservations in ReservationSystem | Major | resourcemanager | Subru Krishnan | Carlo Curino | | | Add a new log aggregation file format controller | Major | . | Xuan Gong | Xuan Gong | | | Container update RM-NM communication fix for backward compatibility | Major | . | Arun Suresh | Arun Suresh | | | TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2 | Major | timelineserver | Vrushali C | Varun Saxena | | | Update paused container state in the NM state store | Blocker | . | Hitesh Sharma | Hitesh Sharma | | | Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218 | Major | fs | igo Goiri | igo Goiri | | | Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat | Major | . | Xuan Gong | Xuan Gong | | | Remove XML excludes file format | Blocker | graceful | Robert Kanter | Robert Kanter | | | Add retry logic in LogsCLI when fetch running application logs | Major | . | Xuan Gong | Xuan Gong | | | Fixing TestNamenodeHeartbeat and support non-HA | Major | fs | igo Goiri | igo Goiri | | | Implement zookeeper based store for scheduler configuration updates | Major | . | Wangda Tan | Jonathan Hung | | | wasb throws an exception if you try to create a file and there's no parent directory | Minor | fs/azure | Steve Loughran | Esfandiar Manii | | | Change hosts JSON file format | Major | . | Ming Ma | Ming Ma | | | Better documentation for maintenace mode and upgrade domain | Major | datanode, documentation | Wei-Chiu Chuang | Ming Ma | | | Add closing logic to configuration store | Major | . | Jonathan Hung | Jonathan Hung | | | Allow users to enter containers as UID:GID pair instead of by username | Major | yarn | Sidharta Seethana | luhuichun | | | Moving logging APIs over to slf4j in hadoop-mapreduce-examples | Major | . | Gergely Novk | Gergely Novk | | | [Documentation] Adding configuration keys for the Router | Minor | fs | igo Goiri | igo Goiri | | | MetricsSystemImpl should use Time.monotonicNow for measuring durations | Minor | . | Chetna Chaudhari | Chetna Chaudhari | | | LoadGenerator should use Time.monotonicNow() to measure durations | Major | . | Chetna Chaudhari | Bharat Viswanadham | | | Documentation for API based scheduler configuration management | Major | . | Jonathan Hung | Jonathan Hung | | | WritableRpcEngine should use Time.monotonicNow | Minor |

| Chetna Chaudhari | Chetna Chaudhari | | | Move up to AWS SDK 1.11.199 | Blocker | build, fs/s3 | Steve Loughran | Steve Loughran | | | Add fsserver defaults call to WebhdfsFileSystem. | Minor | webhdfs | Rushabh S Shah | Rushabh S Shah | | | Removing queue then failing over results in exception | Critical | . | Jonathan Hung | Jonathan Hung | | | Misc changes to YARN-5734 | Major | . | Jonathan Hung | Jonathan Hung | | | Update YARN daemon startup/shutdown scripts to include Router service | Major | nodemanager, resourcemanager | Subru Krishnan | Giovanni Matteo Fumarola | | | Add support for updateContainers when allocating using FederationInterceptor | Minor | . | Botong Huang | Botong Huang | | | Update Shared cache client api to use URLs | Minor | . | Chris Trezzo | Chris Trezzo | | | Add a size threshold beyond which yarn logs will require a force option | Major | . | Siddharth Seth | Xuan Gong | | | Honoring sticky bit during Deletion when authorization is enabled in WASB | Major | fs/azure | Varada Hemeswari | Varada Hemeswari | | | Add size-based rolling policy to LogAggregationIndexedFileController | Major | . | Xuan Gong | Xuan Gong | | | Capture launch\_container.sh logs to a separate log file | Major | . | Wangda Tan | Suma Shivaprasad | | | Rebasing HDFS-10467 after HDFS-12447 | Major | fs | igo Goiri | igo Goiri | | | Enhance S3GuardTool with bucket-info and set-capacity commands, tests | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add work preserving restart support for Unmanaged AMs | Major | resourcemanager | Karthik Kambatla | Botong Huang | | | Placement Agent enhancements required to support recurring reservations in ReservationSystem | Blocker | resourcemanager | Subru Krishnan | Carlo Curino | | | Azure wasb: getFileStatus not making any auth checks | Major | fs/azure, security | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Add documentation for the YARN shared cache | Major | . | Chris Trezzo | Chris Trezzo | | | Federation UI | Major | fs | igo Goiri | igo Goiri | | | Rename Router tooling | Major | fs | igo Goiri | igo Goiri | | | Restrict Access to setPermission operation when authorization is enabled in WASB | Major | fs/azure | Kannapiran Srinivasan | Kannapiran Srinivasan | | | Merge YARN-5734 branch to branch-2 | Major | . | Xuan Gong | Xuan Gong | | | Sticky bit implementation for rename() operation in Azure WASB | Major | fs, fs/azure | Varada Hemeswari | Varada Hemeswari | | | Add support for sections in container-executor configuration file | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | Azure: POSIX permissions are taking effect in access() method even when authorization is enabled | Major | fs/azure | Santhosh G Nayak | Santhosh G Nayak | | | NM Statestore cleanup for Container updates | Blocker | . | Arun Suresh | kartheek muthyala | | | Fix TestRMWebServicesReservation parametrization for fair scheduler | Blocker | fairscheduler, reservation system | Yufei Gu | Yufei Gu | | | Backport ATSv2 to branch-2 | Critical | . | Varun Saxena | Varun Saxena | | | Support same origin policy for cross site scripting prevention. | Major | yarn-ui-v2 | Vrushali C | Sunil Govindan | | | Backporting HDFS-10467 to branch-2 | Major |

| igo Goiri | igo Goiri | | | Make Collections.sort() more efficient by caching resource usage | Major | fairscheduler | Xianyin Xin | Yufei Gu | | | Docker permitted volumes don't properly check for directories | Major | yarn | Eric Badger | Eric Badger | | | Backport new yarn-ui to branch2 code (starting with YARN-5355\_branch2) | Critical | timelineclient, timelinereader, timelineserver | Vrushali C | Vrushali C | | | Basic Federation UI | Major | nodemanager, resourcemanager | Giovanni Matteo Fumarola | igo Goiri | | | Document configuration of ReservationSystem for FairScheduler | Blocker | capacity scheduler | Subru Krishnan | Yufei Gu | | | Add REST API for supporting recurring reservations | Major | resourcemanager | Sangeetha Abdu Jyothi | Sean Po | | | Add documentation for Container Update API | Blocker | . | Arun Suresh | Arun Suresh | | | Fix findbugs warning in timeline service on branch-2 | Major | timelineclient, timelinereader, timelineserver | Vrushali C | Vrushali C | | | Documentation changes post branch-2 merge | Major | timelineclient, timelinereader, timelineserver | Varun Saxena | Vrushali C | | | Update LICENSE.txt with respect to new YARN UI merge | Blocker | . | Subru Krishnan | Varun Saxena | | | Federation Router Web Service fixes | Major | . | igo Goiri | igo Goiri | | | Application lifetime does not work with FairScheduler | Major | resourcemanager | Miklos Szegedi | Miklos Szegedi | | | Add support for docker to have no capabilities | Major | yarn | Eric Badger | Eric Badger | | | Use deployed webapp folder to launch new YARN UI | Major | yarn-ui-v2 | Sunil Govindan | Sunil Govindan | | | Avoid unnecessary getBlockCollection calls in BlockManager | Major | namenode | Jing Zhao | Jing Zhao | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Fix checkstyle error in CapacityScheduler.java | Trivial | . | Rohith Sharma K S | Naganarasimha G R | | | Avoid printing a stack trace when recovering an app after the RM restarts | Major | . | Siddharth Seth | Junping Du | | | Add tests for container-executor's is\feature\enabled function | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling | Major | fairscheduler, test | Karthik Kambatla | Kai Sasaki | | | Log ContainerRuntime initialization error in LinuxContainerExecutor | Trivial | nodemanager | Sidharta Seethana | Sidharta Seethana | | | Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers | Blocker | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Clean up the overwrite of createDispatcher() in subclass of MockRM | Minor | resourcemanager | Yufei Gu | Yufei Gu | | | Revert HADOOP-13606 swift FS to add a service load metadata file | Major | . | John Zhuge | John Zhuge | | | Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully | Minor | ha, namenode | Erik Krogen | Erik Krogen | | | Clean up unused code in class FairSchedulerQueueInfo | Trivial | fairscheduler | Yufei Gu | Larry Lo | | | Document Docker work as experimental | Blocker | documentation | Varun Vasudev | Varun Vasudev | | | Add container ExecutionType into ContainerReport | Minor | . | Botong Huang | Botong Huang | | | Remove