Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.0.0-alpha2.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Modify hadoop\verify\user to be consistent with hadoop\subcommand\opts (ie more granularity) | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | ACL inheritance conflicts with umaskmode | Critical | security | LINTE | John Zhuge | | | Deprecate HADOOP\SERVERNAME\OPTS; replace with (command)\(subcommand)\OPTS | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | ConfServlet should respect Accept request header | Major | conf | Weiwei Yang | Weiwei Yang | | | Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. | Major | datanode, fs | Virajith Jalaparti | Virajith Jalaparti | | | Migrate other Hadoop side tests to prepare for removing WritableRPCEngine | Major | test | Kai Zheng | Wei Zhou | | | Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers | Major | qjm | Sean Mackrory | Sean Mackrory | | | Reduce Kafka dependencies in hadoop-kafka module | Major | metrics | Grant Henke | Grant Henke | | | Update jackson from 1.9.13 to 2.x in hadoop-tools | Major | tools | Akira Ajisaka | Akira Ajisaka | | | yarn.app.mapreduce.client.job.max-retries should have a more useful default | Major | client | Daniel Templeton | Miklos Szegedi | | | Configuration does not substitute multiple references to the same var | Critical | conf | Andrew Wang | Andrew Wang | | | Modifications to remove the assumption that FsVolumes are backed by java.io.File. | Major | datanode, fs | Virajith Jalaparti | Virajith Jalaparti | | | Switch from "raw" to "system" xattr namespace for erasure coding policy | Major | erasure-coding | Andrew Wang | Andrew Wang | | | Lower the default max applications stored in the RM and store | Blocker | resourcemanager | KWON BYUNGCHANG | Daniel Templeton | | | Remove stale method ViewFileSystem#getTrashCanLocation | Minor | viewfs | Manoj Govindassamy | Manoj Govindassamy | | | Retire BKJM from trunk | Major | ha | Vinayakumar B | Vinayakumar B | | | TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior | Major | resourcemanager, timelineclient | Junping Du | Junping Du | | | S3ABlockOutputStream to support huge (many GB) file writes | Major | fs/s3 | Steve Loughran | Steve Loughran | | | remove unnecessary dependency from hadoop-mapreduce-client-jobclient to hadoop-mapreduce-client-shuffle | Minor | mrv2 | Haibo Chen | Haibo Chen | | | FileSystem#listStatus should throw IOE upon access error | Major | fs | Matt Foley | John Zhuge | | | Remove the message about HTTP OPTIONS in SPNEGO initialization message from kms audit log | Minor | kms | Xiao Chen | Xiao Chen | | | Deprecate and remove DockerContainerExecutor | Critical | nodemanager | Daniel Templeton | Daniel Templeton | | | Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes | Minor | . | Devaraj K | Devaraj K | | | Audit Log should escape control characters | Major | . | Eric Badger | Eric Badger | | | Stackoverflow for schemeless defaultFS with trailing slash | Major | fs | Darius Murawski | John Zhuge | | | Update jackson from 1.9.13 to 2.x in hadoop-hdfs | Major | build | Akira Ajisaka | Akira Ajisaka | | | Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue | Minor |

| Yiqun Lin | Yiqun Lin | | | Incorrect error message by fs -put local dir without permission | Major | . | John Zhuge | John Zhuge | | | Upgrade commons-configuration version to 2.1 | Major | build | Sean Mackrory | Sean Mackrory | | | Upgrade Jackson 2.2.3 to 2.7.8 | Major | build | Steve Loughran | Sean Mackrory | | | Update jackson from 1.9.13 to 2.x in hadoop-yarn | Major | build, timelineserver | Akira Ajisaka | Akira Ajisaka | | | Upgrade to AWS SDK 1.11.45 | Blocker | build, fs/s3 | Steve Loughran | Steve Loughran | | | Update jackson from 1.9.13 to 2.x in hadoop-common-project | Major | build | Akira Ajisaka | Akira Ajisaka | | | Upgrade Tomcat to 6.0.48 | Blocker | kms | John Zhuge | John Zhuge | | | Lower the default maximum number of blocks per file | Major | namenode | Aaron T. Myers | Aaron T. Myers | | | Update jackson from 1.9.13 to 2.x in hadoop-maven-plugins | Minor | build | Akira Ajisaka | Akira Ajisaka | | | Create and distribute hadoop-mapreduce-client-core-tests.jar | Critical | build | Arun C Murthy | Akira Ajisaka | | | Switch KMS from Tomcat to Jetty | Major | kms | John Zhuge | John Zhuge | | | Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only) | Blocker | . | Junping Du | Wangda Tan | | | Remove vestigal templates directories creation | Major | build | Allen Wittenauer | Allen Wittenauer | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Incorporate Aliyun OSS file system implementation | Major | fs, fs/oss | shimingfei | mingfei.shi | | | Add support for HDFS erasure code policy to TestDFSIO | Major | . | Sammi Chen | Sammi Chen | | | Add a tool to get FileEncryptionInfo from CLI | Major | encryption | Xiao Chen | Xiao Chen | | | hadoop-aliyun: merge HADOOP-12756 branch back | Major | fs | shimingfei | Genmao Yu | | | Improve distcp to support efficient restore to an earlier snapshot | Major | distcp | Yongjun Zhang | Yongjun Zhang | | | Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup | Major | test | Steve Loughran | Steve Loughran | | | Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle | Major | nodemanager | Chris Douglas | Arun Suresh | | | hadoop build to allow hadoop version property to be explicitly set | Minor | build | Steve Loughran | Steve Loughran | | | Add Codec for ZStandard Compression | Major |

| churro morales | churro morales | | | Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers | Major | tools | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Update scripts to be smarter when running with privilege | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Extend YARN to support distributed scheduling | Major | nodemanager, resourcemanager | Sriram Rao | Konstantinos Karanasos | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add assertion for getNumEncryptionZones when no EZ is created | Minor | test | Vinitha Reddy Gankidi | Vinitha Reddy Gankidi | | | Implement WebHdfsFileSystem#listStatusIterator | Major | webhdfs | Andrew Wang | Andrew Wang | | | Add Logging for Long-held NN Read Locks | Major | logging, namenode | Erik Krogen | Erik Krogen | | | Design Server.Call to be extensible for unified call queue | Major | ipc | Daryn Sharp | Daryn Sharp | | | Log DataNodes in the write pipeline | Trivial | hdfs-client | John Zhuge | John Zhuge | | | Fix JSON errors in WebHDFS.md examples | Trivial | documentation | Andrew Wang | Andrew Wang | | | Clean up WeightAdjuster | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer | Major | tools | Yiqun Lin | Yiqun Lin | | | Complete the document for FileDistribution processor in OfflineImageViewer | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Make Path serializable | Minor | io | Steve Loughran | Steve Loughran | | | Measure lock time in FsDatasetImpl | Major | datanode | Chen Liang | Chen Liang | | | Add log when URLConnectionFactory.openConnection failed | Minor | webhdfs | yunjiong zhao | yunjiong zhao | | | Fix typos in HDFS documents | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Standardize serializiation of WebHDFS DirectoryListing | Major | webhdfs | Andrew Wang | Andrew Wang | | | Add eol=lf for unix format files in .gitattributes | Major | . | Akira Ajisaka | Yiqun Lin | | | Checkstyle suppression by annotation or comment | Minor | . | John Zhuge | John Zhuge | | | Move dev-support/checkstyle/suppressions.xml to hadoop-build-tools | Trivial | . | John Zhuge | John Zhuge | | | If user is unauthorized, log "unauthorized" instead of "Invalid signed text:" | Minor | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Implement HttpFSFileSystem#listStatusIterator | Major | httpfs | Andrew Wang | Andrew Wang | | | Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones | Minor | . | Xiao Chen | Xiao Chen | | | Remove stray references to DFS\HDFS\BLOCKS\METADATA\ENABLED | Trivial | . | Andrew Wang | Andrew Wang | | | scheduler spends too much time looking at empty priorities | Major | capacity scheduler, fairscheduler, resourcemanager | Nathan Roberts | Jason Lowe | | | Optimize du -x to cache intermediate result | Major | snapshots | Xiao Chen | Xiao Chen | | | YARN Web UIs should provide a robots.txt | Trivial | . | Lars Francke | Sidharta Seethana | | | Master.getMasterAddress() should be updated to use YARN-4629 | Minor | applicationmaster | Daniel Templeton | Daniel Templeton | | | TestYarnCLI.testGetContainers fails due to mismatched date formats | Minor | . | Eric Badger | Eric Badger | | | Dispatcher#dispatch should log IOException stacktrace | Trivial | balancer & mover | Wei-Chiu Chuang | Manoj Govindassamy | | | Allow REST API to set a user generated message when killing an application | Major | . | Rajat Jain | Rohith Sharma K S | | | Remove the unused method InodeId#checkId() | Major | namenode | Jagadesh Kiran N | Jagadesh Kiran N | | |

swallows exceptions | Minor | client | Steve Loughran | Varun Saxena | | | Replace config key literal strings with config key names I: hadoop common | Minor | conf | Chen Liang | Chen Liang | | | Light cleanup in ZKRMStateStore | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | add progress log to JHS during startup | Minor | jobhistoryserver | Haibo Chen | Haibo Chen | | | Support external calls in the RPC call queue | Major | ipc | Daryn Sharp | Daryn Sharp | | | Improve StripedBlockReader#createBlockReader error logging | Minor | erasure-coding | Wei-Chiu Chuang | Manoj Govindassamy | | | HDFS Erasure Coding doc should state its currently supported erasure coding policies | Major | documentation, erasure-coding | Wei-Chiu Chuang | Yiqun Lin | | | Add logs to KMS server-side to improve supportability | Minor | kms | Xiao Chen | Suraj Acharya | | | Reduce performance penalty of block caching when not used | Major | namenode | Daryn Sharp | Daryn Sharp | | | Do not attempt to recover progress from previous job attempts if spill encryption is enabled | Major | applicationmaster | Karthik Kambatla | Haibo Chen | | | Should check if node exists when replace nodelabels | Minor | . | Tao Jie | Tao Jie | | | Optimize insertion/removal of replica in ShortCircuitCache | Major | hdfs-client | Fenghua Hu | Fenghua Hu | | | Document -safely option of rm command. | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | Support to retrieve specific property from configuration via REST API | Major | conf | Weiwei Yang | Weiwei Yang | | | Make class Token$PrivateToken private | Minor | . | John Zhuge | John Zhuge | | | Reduce log level when network topology cannot find enough datanodes. | Minor | . | Xiao Chen | Xiao Chen | | | Downgrade stack trace on FS load from Warn to debug | Minor | fs | Steve Loughran | Steve Loughran | | | Stop bundling HTML source code in javadoc JARs | Major | build | Andrew Wang | Andrew Wang | | | Avoid use of toString() in output of HDFS ACL shell commands. | Minor | . | Chris Nauroth | Chris Nauroth | | | Do not attach javadoc and sources jars during non-dist build | Major | . | Andrew Wang | Andrew Wang | | | Deprecate WriteableRPCEngine | Major | . | Haohui Mai | Wei Zhou | | | Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentation | Minor | . | Xiao Chen | Huafeng Wang | | | Ignore file backed pages from memory computation when smaps is enabled | Minor | . | Rajesh Balamohan | Rajesh Balamohan | | | Refactor TestFsck | Minor | . | Takanobu Asanuma | Takanobu Asanuma | | | Snappy may complain Hadoop is built without snappy if libhadoop is not found. | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Revert HADOOP-13534 Remove unused TrashPolicy#getInstance and initialize code | Major | . | Andrew Wang | Andrew Wang | | | Document caveat for KeyShell when underlying KeyProvider does not delete a key | Minor | documentation, kms | Xiao Chen | Xiao Chen | | | Route webhdfs through the RPC call queue | Major | ipc, webhdfs | Daryn Sharp | Daryn Sharp | | | Replace config key literal strings with config key names II: hadoop hdfs | Minor |

| Mingliang Liu | Chen Liang | | | Supress CachingGetSpaceUsed from logging interrupted exception stacktrace | Minor | fs | Wei-Chiu Chuang | Hanisha Koneru | | | Publish AM launch command to ATS | Major | . | Daniel Templeton | Rohith Sharma K S | | | Unnecessary INFO logging on DFSClients for InvalidToken | Minor | fs | Harsh J | Harsh J | | | Expose "XmitsInProgress" through DataNodeMXBean | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Upgrade HTrace version | Major | . | Sean Mackrory | Sean Mackrory | | | Code cleanup -- ViewFileSystem and InodeTree | Minor | viewfs | Manoj Govindassamy | Manoj Govindassamy | | | Fix a few typos in site markdown documents | Minor | documentation | Andrew Wang | Ding Fei | | | DefaultContainerExecutor needs JavaDocs | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Add %A and %a formats for fs -stat command to print permissions | Major | fs | Alex Garbarini | Alex Garbarini | | | Add a tool to reconstruct block meta file from CLI | Major | datanode | Xiao Chen | Xiao Chen | | | Expose nonDfsUsed via StorageTypeStats | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Cleanup DiskChecker interface | Major | util | Arpit Agarwal | Arpit Agarwal | | | Add a new instrumented read-write lock | Major | common | Jingcheng Du | Jingcheng Du | | | Upgrade OWASP dependency-check plugin version | Minor | security | Mike Yoder | Mike Yoder | | | Support multiple authentication schemes via AuthenticationFilter | Major | security | Hrishikesh Gadre | Hrishikesh Gadre | | | KMS Server should log exceptions before throwing | Major | kms | Xiao Chen | Suraj Acharya | | | Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir() | Major | client | Santhosh G Nayak | Santhosh G Nayak | | | Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as VisibleForTesting | Minor | hdfs-client | Keith Turner | Manoj Govindassamy | | | Many classes use bare yarn. properties instead of the defined constants | Major | . | Daniel Templeton | Daniel Templeton | | | Upgrade jaxb-api version | Major | build | Sean Mackrory | Sean Mackrory | | | Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. | Minor | test | Chris Nauroth | Chris Nauroth | | | The description of dfs.block.replicator.classname is not clear | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Tighten the authorization of datanode RPC | Major | datanode, security | Kihwal Lee | Kihwal Lee | | | Clean up Lint warnings in nodemanager | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Update default-log4j.properties for httpfs to imporve test logging | Major | httpfs, test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Reduce logging application reserved debug info in FSAppAttempt#assignContainer | Minor | fairscheduler | Yiqun Lin | Yiqun Lin | | | Reuse objectMapper instance in Yarn | Major | timelineclient | Yiqun Lin | Yiqun Lin | | | Mention the default NN rpc ports in hdfs-default.xml | Minor | documentation | Andrew Wang | Yiqun Lin | | | Update staled configuration properties related to erasure coding | Major |

| Sammi Chen | Sammi Chen | | | capacity scheduler: Make number of OFF\_SWITCH assignments per heartbeat configurable | Major | capacityscheduler | Nathan Roberts | Nathan Roberts | | | Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | Update jetty dependency to version 9 | Critical | . | Robert Rati | Robert Kanter | | | Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml | Minor | documentation, jobhistoryserver | Akira Ajisaka | Yiqun Lin | | | Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage | Major | datanode | Arpit Agarwal | Hanisha Koneru | | | Make all MockRM#waitForState consistent. | Major | resourcemanager | Yufei Gu | Yufei Gu | | | Log the trace information on FSAppAttempt#assignContainer | Major | applications, fairscheduler | Yiqun Lin | Yiqun Lin | | | Update the document for JobConf#setNumReduceTasks | Major | documentation | Akira Ajisaka | Yiqun Lin | | | Ignore package line length checkstyle rule | Major | build | Shane Kumpf | Shane Kumpf | | | Incorporate checkcompatibility script which runs Java API Compliance Checker | Major | scripts | Andrew Wang | Andrew Wang | | | Fix typing mistake of inline document in hadoop-metrics2.properties | Major | documentation | Rui Gao | Rui Gao | | | De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads | Minor | . | Wei-Chiu Chuang | Manoj Govindassamy | | | Output javadoc inside the target directory | Major | documentation | Andrew Wang | Andrew Wang | | | Update HttpFS to use ConfigRedactor | Major | . | Sean Mackrory | Sean Mackrory | | | Use CliParser to parse options in RMAdminCLI | Major | resourcemanager | Tao Jie | Tao Jie | | | Support decoding KMS Delegation Token with its own Identifier | Major | kms, security | Xiaoyu Yao | Xiaoyu Yao | | | Quash unnecessary safemode WARN message during NameNode startup | Trivial | . | Andrew Wang | Yiqun Lin | | | Remove unused properties from JTConfig.java | Major | . | Akira Ajisaka | Haibo Chen | | | Minor cleanup to UGI use in AdminService | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | Expose getTrashRoot to HTTPFS and WebHDFS | Major | encryption, httpfs, webhdfs | Xiao Chen | Yuanbo Liu | | | Update jackson from 1.9.13 to 2.x in hadoop-mapreduce | Major | build | Akira Ajisaka | Akira Ajisaka | | | Update document for "rmadmin -replaceLabelOnNode" | Minor | . | Tao Jie | Tao Jie | | | NodeManager should communicate physical resource capability to ResourceManager | Major | nodemanager, resourcemanager | Nathan Roberts | igo Goiri | | | Make generic options help more consistent, and aligned | Minor | . | Grant Sohn | Grant Sohn | | | Make MutableRates metrics thread-local write, aggregate-on-read | Major | metrics | Erik Krogen | Erik Krogen | | | Replace DatanodeInfo constructors with a builder pattern | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Remove unused HADOOP\AUDIT\LOGGER from hadoop-env.sh | Minor | scripts | Akira Ajisaka | Yiqun Lin | | | Retry until TGT expires even if the UGI renewal thread encountered exception | Major | security | Xiao Chen | Xiao Chen | | | TestEncryptionZones should waitActive | Minor | test | Xiao Chen | John Zhuge | | | Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with

| Major | build | Chris Nauroth | Chris Nauroth | | | Add Builder methods for common yarn API records | Major | . | Arun Suresh | Tao Jie | | | Improve BlockManager#processMisReplicatesAsync log | Major | namenode | Xiaoyu Yao | Chen Liang | | | In FairScheduler, parent queues should also display queue status | Major | fairscheduler | Siqi Li | Siqi Li | | | Add a test to verify that Configuration handles &-encoded characters | Minor | test | Steve Loughran | Steve Loughran | | | Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()} | Major | fs | Steve Loughran | Mingliang Liu | | | YARN container executor config does not handle white space | Trivial | nodemanager | Miklos Szegedi | Miklos Szegedi | | | invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures | Major | resourcemanager | sandflee | sandflee | | | Remove confusing log output in FsDatasetImpl#getInitialVolumeFailureInfos | Minor | datanode | Chen Liang | Chen Liang | | | Expose "NumOpenConnectionsPerUser" as a metric | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Refactor striped file tests to allow flexibly test erasure coding policy | Major | . | Sammi Chen | Sammi Chen | | | Remove outdated overview.html | Minor | . | Akira Ajisaka | Brahma Reddy Battula | | | FairScheduler hides group resolution exceptions when assigning queue | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | add getFileStatus("/") test to AbstractContractGetFileStatusTest | Minor | fs, test | Steve Loughran | Steve Loughran | | | Allow handoff on the server side for RPC requests | Major | ipc | Siddharth Seth | Siddharth Seth | | | Clean up FileSystem javadocs, logging; improve diagnostics on FS load | Major | fs | Steve Loughran | Steve Loughran | | | Make Kdiag check whether hadoop.token.files points to existent and valid files | Major | . | Ravi Prakash | Ravi Prakash | | | Metric Snapshot log can be changed #MetricsSystemImpl.java since all the services will be initialized | Minor | metrics | Brahma Reddy Battula | Brahma Reddy Battula | | | Open up already widely-used APIs for delegation-token fetching & renewal to ecosystem projects | Blocker | . | Robert Joseph Evans | Vinod Kumar Vavilapalli | | | Document uppercase key names are not supported in TransparentEncryption.md | Minor | documentation | Yuanbo Liu | Yiqun Lin | | | The distance between sync blocks in SequenceFiles should be configurable | Major | io | Owen O'Malley | Harsh J | | | Redundant groupid warning in child projects | Minor | . | Kai Sasaki | Kai Sasaki | | | FairScheduler should log information about AM-resource-usage and max-AM-share for queues | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Update fair scheduler to use pluggable auth provider | Major | fairscheduler | Daniel Templeton | Tao Jie | | | Make qbt script executable | Trivial | scripts | Andrew Wang | Andrew Wang | | | Improve performance of postponed block scans | Critical | namenode | Daryn Sharp | Daryn Sharp | | | Allow

to be downloadable on the job overview page in JHS | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | Hide redundant table on NameNode WebUI when no nodes are decomissioning | Trivial | hdfs, ui | Weiwei Yang | Weiwei Yang | | | Add a time unit to the DataNode client trace format | Minor | datanode | Akira Ajisaka | Jagadesh Kiran N | | | Add reencryptEncryptedKey interface to KMS | Major | kms | Xiao Chen | Xiao Chen | | | Datanodes not sorted properly by distance when the reader isn't a datanode | Major | . | Ming Ma | Nanda kumar | | | Annotate NameNode and DataNode MXBean interfaces as Private/Stable | Major | . | Akira Ajisaka | Jagadesh Kiran N | | | Cleanup unchecked types for EventHandler | Major | . | Daniel Templeton | Daniel Templeton | | | Remove snapshot version of SDK dependency from Azure Data Lake Store File System | Major | fs/adl | Vishwajeet Dusane | Vishwajeet Dusane | | | Add instrumentation hooks around Datanode disk IO | Major | datanode | Xiaoyu Yao | Arpit Agarwal | | | Redundant toString() in DFSConfigKeys.java | Trivial | . | Akira Ajisaka | Jagadesh Kiran N | | | Test only changes from YARN-4126 | Major | . | Andrew Wang | Jian He | | | Ability to clean up subprocesses spawned by Shell when the process exits | Major | . | Eric Badger | Eric Badger | | | Remove unused variables in FSImage.java | Trivial | . | Akira Ajisaka | Jagadesh Kiran N | | | Refactor: Wrap Datanode IO related operations | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Adding per disk IO statistics and metrics in DataNode. | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Introduce fault injectors to simulate slow mirrors | Major | datanode, test | Xiaobing Zhou | Xiaobing Zhou | | | Remove TRUSTSTORE\_PASSWORD related scripts from KMS | Minor | kms | Xiao Chen | John Zhuge | | | Use MiniYARNCluster with try-with-resources in tests | Trivial | test | Andras Bokor | Andras Bokor | | | Azure: Add a new SAS key mode for WASB. | Major | fs/azure | Dushyanth | Dushyanth | | | Collect peer performance statistics on DataNode. | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | FairShareComparator: Cache value of getResourceUsage for better performance | Major | fairscheduler | zhangshilong | zhangshilong | | | Check groupEntryIndex and throw a helpful exception on failures when removing ACL. | Major | namenode | Xiao Chen | Xiao Chen | | | Cleanup leader election configs and pluggability | Critical | resourcemanager | Karthik Kambatla | Karthik Kambatla | | | Cleanup unused DataNode#checkDiskErrorAsync() | Minor | . | Xiaoyu Yao | Hanisha Koneru | | | Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. | Major | documentation | Chris Nauroth | Surendra Singh Lilhore | | | Yarn Distributed Shell does not print throwable t to App Master When failed to start container | Minor | . | dashwang | Jim Frankola | | | log lastWrittenTxId etc info in logSyncAll | Major | hdfs | Yongjun Zhang | Yongjun Zhang | | | Move TransferFsImage#doGetUrl function to a Util class | Major | . | Hanisha Koneru | Hanisha Koneru | | | AsyncDispatcher thread name can be set to improved debugging | Major |

| Ajith S | Ajith S | | | Make FTPFileSystem's data connection mode and transfer mode configurable | Major | fs | Xiao Chen | Xiao Chen | | | Support multiple Datanode File IO hooks | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Automatically create YARN control group for pre-mounted cgroups | Minor | . | Miklos Szegedi | Miklos Szegedi | | | Support File IO sampling for Datanode IO profiling hooks | Major | datanode | Hanisha Koneru | Hanisha Koneru | | | Datanode should only check the failed volume upon IO errors | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Replace UTF8 with Text in MRBench | Minor | test | Akira Ajisaka | Peter Bacsko | | | Print remaining edit logs from buffer if edit log can't be rolled. | Major | ha, namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Update project release notes for 3.0.0-alpha2 | Major | documentation | Andrew Wang | Andrew Wang | | | Replace deprecated HttpServer2 and SSLFactory constants | Minor | . | John Zhuge | John Zhuge | | | Include file lengths in Mismatch in length error for distcp | Minor | . | Ted Yu | Ted Yu | | | Add -DskipShade maven profile to disable jar shading to reduce compile time | Minor | build | Arun Suresh | Arun Suresh | | | Add document for container metrics | Major | documentation, nodemanager | Weiwei Yang | Weiwei Yang | | | Give fetchers hint when ShuffleHandler rejects a shuffling connection | Major | mrv2 | Haibo Chen | Haibo Chen | | | swift FS to add a service load metadata file | Major | fs/swift | Steve Loughran | Steve Loughran | | | Refactor Azure Data Lake Store as an independent FileSystem | Major | fs/adl | Shrikant Naidu | Vishwajeet Dusane | | | Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API | Major | webhdfs | Weiwei Yang | Weiwei Yang | | | DiskChecker should perform some disk IO | Major | . | Arpit Agarwal | Arpit Agarwal | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | "StorageTypeStats" Metric should not count failed storage. | Major | namenode | Archana T | Surendra Singh Lilhore | | | TestRecovery occasionally fails | Major | mrv2, test | Jason Lowe | Haibo Chen | | | DataXceiver#run() should not log InvalidToken exception as an error | Major | . | Pan Yuxuan | Pan Yuxuan | | | Improve log message for edit loading failures caused by FS limit checks. | Major | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky | Major | security, test | Mingliang Liu | Weiwei Yang | | | Reuse closeResponder to reset the response variable in DataStreamer#run | Minor | . | Yiqun Lin | Yiqun Lin | | | Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. | Minor | . | Eric Payne | Eric Payne | | | AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | Fix typos in httpfs.sh | Trivial | httpfs | John Zhuge | John Zhuge | | | Remove duplicate or unused variable in appendFile() | Minor |

| Kihwal Lee | Kihwal Lee | | | UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket | Major | security | Alejandro Abdelnur | Xiao Chen | | | Clean up TestLocalFileSystemPermission | Trivial | fs | Andras Bokor | Andras Bokor | | | test\libhdfs\threaded\hdfs\static and test\libhdfs\zerocopy\hdfs\static are failing | Major | libhdfs | Akira Ajisaka | Akira Ajisaka | | | DFS reserved space is erroneously counted towards non-DFS used. | Major | datanode | Chris Nauroth | Brahma Reddy Battula | | | Potential memory leak in CryptoOutputStream | Major | security | Mariappan Asokan | Mariappan Asokan | | | Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions | Critical | httpfs | Andrew Wang | Andrew Wang | | | FsDatasetImpl#getBlockReports can occasionally throw NullPointerException | Major | datanode | Wei-Chiu Chuang | Manoj Govindassamy | | | FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use | Major | hdfs | Manoj Govindassamy | Arpit Agarwal | | | distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't set | Minor | tools/distcp | Zoran Dimitrijevic | Zoran Dimitrijevic | | | Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport | Minor | documentation | Akira Ajisaka | Yiqun Lin | | | NM fails to start after downgrade from 2.8 to 2.7 | Blocker | nodemanager | Jason Lowe | Jason Lowe | | | Broken code snippet area in Hadoop Benchmarking | Minor | documentation | Kai Sasaki | Kai Sasaki | | | Typos in 4 log messages | Trivial | . | Mehran Hassani | Mehran Hassani | | | Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine | Major | test | Arun Suresh | Wei Zhou | | | TopCLI console support for HA mode | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Typos in 4 log messages | Trivial | . | Mehran Hassani | Mehran Hassani | | | Typos in 9 log messages | Trivial | . | Mehran Hassani | Mehran Hassani | | | Fix TestDefaultContainerExecutor | Major | test | Akira Ajisaka | Arun Suresh | | | [Atsv2] Document object passing in infofilters with an example | Major | timelinereader, timelineserver | Rohith Sharma K S | Rohith Sharma K S | | | TestContainerManagerSecurity#testNMTokens is asserting | Major | . | Jason Lowe | Robert Kanter | | | Fix typo in a log messages of AbstractDelegationTokenSecretManager | Trivial | . | Mehran Hassani | Mehran Hassani | | | TestEncryptionZonesWithKMS#testReadWrite fails intermittently | Major | . | Xiao Chen | Xiao Chen | | | Fix some warnings by findbugs in hadoop-maven-plugin | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | YarnWebParams next.fresh.interval should be next.refresh.interval | Minor | webapp | Daniel Templeton | Daniel Templeton | | | TimelineClient failed to retry on "java.net.SocketTimeoutException: Read timed out" | Critical | yarn | Sumana Sathish | Junping Du | | | Math error in AbstractContractDistCpTest | Minor | . | Aaron Fabbri | Aaron Fabbri | | | Remove the redundant charactors for command -saveNamespace in HDFSCommands.md | Trivial | documentation | Yiqun Lin | Yiqun Lin | | | Replace "fs.default.name" with "fs.defaultFS" in viewfs document | Minor | documentation, federation | Brahma Reddy Battula | Brahma Reddy Battula | | | Update space quota when a UC block is completed rather than committed. | Major | hdfs, namenode | Erik Krogen | Erik Krogen | | | Fix Eclipse Java 8 compile errors related to generic parameters. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Fix Yarn documentation to link to correct

| Minor | . | Xiao Chen | Xiao Chen | | | Small refactor in ZKRMStateStore | Minor | resourcemanager | Oleksii Dymytrov | Oleksii Dymytrov | | | KMS should set UGI's Configuration object properly | Major | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Correct the distcp counters name while displaying counters | Minor | distcp | Archana T | Surendra Singh Lilhore | | | Wrong audit constants are used in Get/Put of priority in RMWebService | Trivial | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Remove outdated Fault Injection Framework documentaion | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | Throttle FsNameSystem lock warnings | Major | logging, namenode | Arpit Agarwal | Hanisha Koneru | | | TestPendingInvalidateBlock failed in trunk | Major | test | Yiqun Lin | Yiqun Lin | | | Fix usage of FsDatasetImpl object lock in ReplicaMap | Blocker | . | Arpit Agarwal | Arpit Agarwal | | | Missing refreshClusterMaxPriority usage in rmadmin help message | Minor | . | Kai Sasaki | Kai Sasaki | | | Enhance setOwner testing | Major | . | Yongjun Zhang | John Zhuge | | | Fix time measurement bug in TestDatanodeRestart#testWaitForRegistrationOnRestart | Minor | test | Xiaobing Zhou | Xiaobing Zhou | | | Add utility to find set of available ephemeral ports to ServerSocketUtil | Major | . | Brahma Reddy Battula | Masatake Iwasaki | | | Provide an option to enable ContainerMonitor | Major | . | Jian He | Jian He | | | Prevent IPC reader thread death | Critical | ipc | Daryn Sharp | Daryn Sharp | | | MiniDFSCluster#storageCapacities has no effects on real capacity | Major | . | Xiaobing Zhou | Xiaobing Zhou | | | Move remnants of oah.hdfs.client to hadoop-hdfs-client | Critical | hdfs-client | Andrew Wang | Andrew Wang | | | Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories | Minor | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | RMContainerAllocator sends container diagnostics event after corresponding completion event | Major | mrv2 | Haibo Chen | Haibo Chen | | | Fix findbugs warning in VersionInfoMojo.java | Major | . | Tsuyoshi Ozawa | Yuanbo Liu | | | getEZForPath should NOT throw FNF | Blocker | hdfs | Daryn Sharp | Andrew Wang | | | Fix ClassFormatException in trunk build. | Major | build | Kihwal Lee | Kihwal Lee | | | Fix Erasure Coding documentation | Trivial | documentation, erasure-coding | Wei-Chiu Chuang | Manoj Govindassamy | | | Reduce loglevel to Debug in ContainerManagementProtocolProxy and AMRMClientImpl | Major | yarn | Yufei Gu | Yufei Gu | | | Log demand as demand in FSLeafQueue and FSParentQueue | Major | fairscheduler | Yufei Gu | Yufei Gu | | | FairScheduler: wrong queue name in log when adding application | Minor | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Network issues can cause persistent RM UI outage | Critical | webapp | Daniel Templeton | Daniel Templeton | | | Setreplication removing block from underconstrcution temporarily when batch IBR is enabled. | Major | namenode | Brahma Reddy Battula | Brahma Reddy Battula | | | Correct the javadoc

dfsadmin#disallowSnapshot | Minor | documentation | Jagadesh Kiran N | Jagadesh Kiran N | | | Correct the API name for truncate in webhdfs document | Major | documentation | Surendra Singh Lilhore | Surendra Singh Lilhore | | | TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException | Major | hdfs-client | Rushabh S Shah | Rushabh S Shah | | | s3a: Support createNonRecursive API | Major | fs/s3 | Sean Mackrory | Sean Mackrory | | | Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications | Major | encryption | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestRequestHedgingProxyProvider is flaky | Major | test | Andrew Wang | Andrew Wang | | | Fix TestAMWebApp failure | Major | test | Akira Ajisaka | Daniel Templeton | | | Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval | Minor | mr-am | Haibo Chen | Haibo Chen | | | Fix typos in core-default.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | YARN\APPLICATION\UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order | Major | . | Xuan Gong | Sunil Govindan | | | getPathFromYarnURL should use standard methods | Major | . | Sergey Shelukhin | Sergey Shelukhin | | | TestZKSignerSecretProvider#testMultipleInit occasionally fail | Major | . | Wei-Chiu Chuang | Eric Badger | | | Fix typos in hdfs-default.xml | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Disk usage summary of snapshots causes renamed blocks to get counted twice | Major | snapshots | Sean Mackrory | Sean Mackrory | | | resourcemanager.security.TestDelegationTokenRenewer fails in trunk | Major | . | Yongjun Zhang | Jason Lowe | | | LogLevel#main throws exception if no arguments provided | Major | . | Mingliang Liu | Mingliang Liu | | | Export hdfsTruncateFile symbol in libhdfs | Blocker | libhdfs | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Remove unthrown IOException from TrashPolicy#initialize and #getInstance signatures | Critical | fs | Haibo Chen | Andrew Wang | | | Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md | Major | documentation | Mingliang Liu | Mingliang Liu | | | webhdfs PUT does not work if requests are routed to call queue. | Major | . | Kihwal Lee | Kihwal Lee | | | Make Decommission less expensive when lot of blocks present. | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Distcp with -delete feature on raw data not implemented | Major | tools/distcp | Mavin Martin | Mavin Martin | | | DFSAdmin should log detailed error message if any | Major | tools | Mingliang Liu | Mingliang Liu | | | AliyunOSSInputStream#read() should update read bytes stat correctly | Major | tools | Mingliang Liu | Mingliang Liu | | | TestPendingInvalidateBlock should wait for IBRs | Minor | . | Yiqun Lin | Yiqun Lin | | | Distcp using webhdfs on secure HA clusters fails with StandbyException | Major | webhdfs | Benoy Antony | Benoy Antony | | | \`getTrashRoot\`'s behavior is not consistent in DFS after enabling EZ. | Major | . | Yuanbo Liu | Yuanbo Liu | | | If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed | Major | . | Yuanbo Liu | Yuanbo Liu | | | BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order | Critical | namenode | Konstantin Shvachko | Vinitha Reddy Gankidi | | | TestStorageMover#testNoSpaceDisk is failing intermittently | Major | test | Rakesh R | Rakesh R | | | [Atsv2] Publish queue name and RMAppMetrics to ATS | Major |

| Rohith Sharma K S | Rohith Sharma K S | | | TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove | Minor | hdfs | Manoj Govindassamy | Manoj Govindassamy | | | Several log refactoring/improvement suggestion in HDFS | Major | . | Nemo Chen | Hanisha Koneru | | | TestDiskspaceQuotaUpdate fails in trunk due to Bind exception | Minor | . | Yiqun Lin | Yiqun Lin | | | Volume Scanner marks a block as "suspect" even if the exception is network-related | Major | hdfs | Rushabh S Shah | Rushabh S Shah | | | Log object instance get incorrectly in TestDFSAdmin | Minor | . | Yiqun Lin | Yiqun Lin | | | Incorrect check and message in FsDatasetImpl#invalidate | Major | datanode | Wei-Chiu Chuang | Yiqun Lin | | | truncate will fail when we use viewfilesystem | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix some failed tests due to BindException | Major | . | Yiqun Lin | Yiqun Lin | | | Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes | Major | timelineserver | Li Lu | Li Lu | | | TestAHSWebServices is failing | Major | timelineserver | Akira Ajisaka | Akira Ajisaka | | | KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used | Critical | . | Sergey Shelukhin | Xiaoyu Yao | | | Add missing cleanupSSLConfig() call for tests that use setupSSLConfig() | Major | test | Kuhu Shukla | Kuhu Shukla | | | Remove distcp dependency on FileStatus serialization | Major | tools/distcp | Chris Douglas | Chris Douglas | | | Duplicate '-' in the daemon log name | Major | logging | Akira Ajisaka | Akira Ajisaka | | | Propogate exceptions back to client when using hedging RM failover provider | Critical | applications, resourcemanager | Subru Krishnan | Subru Krishnan | | | Null check missing for earliest in FifoPolicy | Major | fairscheduler | Yufei Gu | Yufei Gu | | | TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Enforce timeout in balancer | Major | balancer & mover | Kihwal Lee | Kihwal Lee | | | Add documentation for HDFS-9820 distcp improvement | Major | distcp | Yongjun Zhang | Yongjun Zhang | | | RM should transition to standby when connection is lost for an extended period | Critical | resourcemanager | Daniel Templeton | Daniel Templeton | | | Suppress verbose log message in BlockPlacementPolicyDefault | Major | . | Arpit Agarwal | Chen Liang | | | TestFileChecksum fails in some cases | Major | . | Wei-Chiu Chuang | Sammi Chen | | | fix NPE in AMRMClientImpl.getMatchingRequests() | Major | yarn | Haibo Chen | Haibo Chen | | | Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT\TIMEOUT\CONN\_CONFIGURATOR | Major | . | Kuhu Shukla | Kuhu Shukla | | | Unnecessary superuser check in versionRequest() | Major | namenode, security | Kihwal Lee | Kihwal Lee | | | Audit dependencies for Category-X | Blocker | timelineserver | Sean Busbey | Sangjin Lee | | | TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode | Major |

| Eric Badger | Eric Badger | | | ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException | Major | viewfs | Eli Collins | Manoj Govindassamy | | | Duplicate keys in NAMENODE\SPECIFIC\KEYS | Minor | namenode | Akira Ajisaka | Akira Ajisaka | | | TestLocalResourcesTrackerImpl#testLocalResourceCache times out | Major | . | Eric Badger | Eric Badger | | | Fix inconsistent naming in class ResourceRequest | Trivial | yarn | Yufei Gu | Yufei Gu | | | DefaultContainerExecutor random working dir algorigthm skews results | Minor | . | Miklos Szegedi | Vrushali C | | | BlockIdManager.clear doesn't reset the counter for blockGroupIdGenerator | Minor | hdfs | Ewan Higgs | Yiqun Lin | | | Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods() | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | DN should notify NN when client requests a missing block | Major | . | Daryn Sharp | Walter Su | | | RM REST API documentation is not up to date | Trivial | resourcemanager, yarn | Miklos Szegedi | Miklos Szegedi | | | Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock | Major | . | Wangda Tan | Varun Saxena | | | TestNodeLabelContainerAllocation is not timing out | Major | test | Jason Lowe | Varun Saxena | | | Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary | Minor | resourcemanager | Sunil Govindan | Sunil Govindan | | | Print the directory paths when ViewFs denies the rename operation on internal dirs | Major | viewfs | Tianyin Xu | Rakesh R | | | Update yarn daemonlog documentation due to HADOOP-12847 | Trivial | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Recovered containers will be killed after NM stateful restart | Major | nodemanager | Siqi Li | Siqi Li | | | Cleanup "mapreduce.job.hdfs-servers" from yarn-default.xml | Minor | yarn | Junping Du | Junping Du | | | Checkstyle: MonitoringThread.Run method length is too long | Trivial | nodemanager | Miklos Szegedi | Miklos Szegedi | | | Cluster metrics have wrong Total Memory when there is reserved memory on CS | Major | capacityscheduler, resourcemanager | Thomas Graves | Brahma Reddy Battula | | | datanodesSoftwareVersions map may counting wrong when rolling upgrade | Major | . | Phil Yang | Erik Krogen | | | Potential resource leaks in BinaryProtocol$TeeOutputStream.java | Major | . | Ravi Teja Ch N V | Sunil Govindan | | | Add test for leakage of reservation metrics in fair scheduler | Minor | fairscheduler | Hong Zhiguo | Hong Zhiguo | | | FairSharePolicy breaks TimSort assumption | Major | fairscheduler | Zephyr Guo | Zephyr Guo | | | KMS REST API Documentation Decrypt URL typo | Minor | documentation, kms | Jeffrey E Rodriguez | Jeffrey E Rodriguez | | | Fix the asflicense warnings | Major | . | Yiqun Lin | Yiqun Lin | | | Shell.checkIsBashSupported swallowed an interrupted exception | Minor | util | Wei-Chiu Chuang | Wei-Chiu Chuang | | | NM should clean up app log dirs after NM restart | Major | nodemanager | sandflee | sandflee | | | Fix the order that resources are cleaned up from the local Public/Private caches | Major | . | Chris Trezzo | Chris Trezzo | | | Update dfs -count -t command line help and documentation | Minor | documentation, fs | Wei-Chiu Chuang | Yiqun Lin | | | RM recovery too slow due

LeafQueue#activateApplication() | Critical | capacity scheduler, rolling upgrade | Bibin A Chundatt | Bibin A Chundatt | | | Delete LinuxContainerExecutor comment from yarn-default.xml | Trivial | yarn | Daniel Templeton | Jan Hentschel | | | AsyncDispatcher possibly invokes multiple shutdown thread when handling exception | Major | . | Jian He | Jian He | | | Fix javadoc jars to not contain entire target directory | Minor | build | EJ Ciramella | Andrew Wang | | | Logging the username when deny the setOwner operation | Minor | namenode | Tianyin Xu | Rakesh R | | | Add isDebugEnabled check for debug logs in nodemanager | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Aggregated Logs root directory is created with wrong group if nonexistent | Major | log-aggregation, nodemanager, security | Haibo Chen | Haibo Chen | | | MR should not schedule container requests in cases where reducer or mapper containers demand resource larger than the maximum supported | Minor | mr-am | Haibo Chen | Haibo Chen | | | Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | BlockManagerSafeMode should respect extension period default config value (30s) | Minor | namenode | Mingliang Liu | Mingliang Liu | | | Handle duplicate completed containers in RMNodeImpl | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Fix the jenkins warning related to the deprecated method StorageReceivedDeletedBlocks | Major | . | Yiqun Lin | Yiqun Lin | | | Azure testGlobStatusThrowsExceptionForUnreadableDir fails | Minor | fs/azure | John Zhuge | John Zhuge | | | Datanode in tests cannot start in Windows after HDFS-10638 | Blocker | datanode | Vinayakumar B | Vinayakumar B | | | NPE when getting node status of a decommissioned node after an RM restart | Major | . | Robert Kanter | Robert Kanter | | | TestHadoopArchives times out | Major | test | Akira Ajisaka | Akira Ajisaka | | | Remove hardcoded absolute path for ls | Major | util | Christine Koppelt | Christine Koppelt | | | Skip testGlobStatusThrowsExceptionForUnreadableDir in TestFSMainOperationsSwift | Major | fs/swift, test | John Zhuge | John Zhuge | | | Revert HADOOP-13081 add the ability to create multiple UGIs/subjects from one kerberos login | Major | security | Andrew Wang | Chris Nauroth | | | Revert health check exit code check | Major | nodemanager | Allen Wittenauer | Allen Wittenauer | | | MutableStat mean loses accuracy if add(long, long) is used | Minor | metrics | Erik Krogen | Erik Krogen | | | Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers | Major | . | Rohith Sharma K S | Konstantinos Karanasos | | | JHS task page search based on each individual column not working | Major | jobhistoryserver | Bibin A Chundatt | Ajith S | | | Hadoop Common includes generated test protos in both jar and test-jar | Major | build, common | Sean Busbey | Sean Busbey | | | Update NMTokens in case of requests with only opportunistic containers | Blocker | . | Konstantinos Karanasos | Konstantinos Karanasos | | | DelegationTokenAuthenticationHandler writes via closed writer | Minor | security | Gregory Chanan | Hrishikesh Gadre | | | Unnecessary duplicate start container request sent to NM State store | Major |

| Varun Saxena | Varun Saxena | | | TestDiskFailures.testLocalDirsFailures failed | Major | . | Yufei Gu | Yufei Gu | | | FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource | Major | fairscheduler | sandflee | sandflee | | | Incorrect documentation for timeline service entityType/events REST end points | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Validate required params for WebHDFS requests | Major | . | Jagadesh Kiran N | Jagadesh Kiran N | | | TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value | Trivial | . | Miklos Szegedi | Chang Li | | | CreateEditsLog throws NullPointerException | Major | hdfs | Hanisha Koneru | Hanisha Koneru | | | Update npm to latest version in Dockerfile to avoid random failures of npm while run maven build | Major | yarn-ui-v2 | Sunil Govindan | Sunil Govindan | | | Fix issues related to Max App in capacity scheduler | Major | capacity scheduler | Bibin A Chundatt | Bibin A Chundatt | | | ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | TestAppendSnapshotTruncate fails with bind exception | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing | Major | security, test | Mingliang Liu | Mingliang Liu | | | The tests in TestBalancer run fails due to NPE | Major | test | Yiqun Lin | Yiqun Lin | | | Concurrent append and read operations lead to checksum error | Major | datanode, httpfs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Job history server scans can become blocked on a single, slow entry | Critical | jobhistoryserver | Prabhu Joseph | Prabhu Joseph | | | RM -format-state-store and -remove-application-from-state-store commands fail with NPE | Critical | resourcemanager | Varun Saxena | Varun Saxena | | | RM crashes with NPE if generic application history is enabled | Critical | resourcemanager | Varun Saxena | Varun Saxena | | | NPE while processing localizer heartbeat | Major | nodemanager | Jason Lowe | Varun Saxena | | | Metric for resource\*time that was preempted | Major | resourcemanager | Chang Li | Chang Li | | | TestTokenClientRMService#testTokenRenewalWrongUser fails | Major | . | Varun Saxena | Gergely Novk | | | TestKMS#testDelegationTokensOpsSimple and TestKMS#testDelegationTokensOpsKerberized Fails in Trunk | Major | test | Brahma Reddy Battula | Xiao Chen | | | Revert CHMOD on the new dirs created-LinuxContainerExecutor creates appcache and its subdirectories with wrong group owner. | Blocker | . | Haibo Chen | Naganarasimha G R | | | TestPipeApplication#testSubmitter fails after HADOOP-13802 | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | yarn rmadmin -help contains a misspelled ResourceManager | Trivial | resourcemanager | Grant Sohn | Grant Sohn | | | Malicious AM can kill containers of other apps running in any node its containers are running | Minor | nodemanager | Botong Huang | Botong Huang | | | Expose getApplications API in YarnClient with GetApplicationsRequest parameter | Major | client | Gour Saha | Jian He | | | Fix bind exception threw in TestRenameWhileOpen | Major |

| Yiqun Lin | Yiqun Lin | | | TimelineServer doesn't catch/translate all exceptions raised | Minor | timelineserver | Steve Loughran | Steve Loughran | | | Reduce the number of default server threads for AMRMProxyService | Minor | nodemanager | Subru Krishnan | Subru Krishnan | | | Fix flaky TestKill.testKillJob() | Major | mrv2 | Haibo Chen | Haibo Chen | | | Sample configuration of KMS HTTP Authentication signature is misleading | Minor | conf, documentation, kms | Masahiro Tanaka | Masahiro Tanaka | | | TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | TestDFSShell#testMoveWithTargetPortEmpty fails intermittently | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails | Major | test | Jason Lowe | Eric Badger | | | io.sort.factor code default and mapred-default.xml values inconsistent | Trivial | task | Gera Shegalov | Prabhu Joseph | | | Enhance Dispatcher logic on deciding when to give up a source DataNode | Major | balancer & mover | Zhe Zhang | Mark Wagner | | | Index out of range in SysInfoWindows | Major | native, util | igo Goiri | igo Goiri | | | DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true | Major | . | Varun Saxena | Varun Saxena | | | Handle Opportunistic scheduling allocate request failure when NM is lost | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Fix a typo in the comments of RPC.getProtocolVersion | Trivial | common | Ethan Li | Ethan Li | | | Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang | Major | test | Rohith Sharma K S | Varun Saxena | | | TestSymlinkHdfsFileSystem#testCreateLinkUsingPartQualPath2 fails after HADOOP13605 | Critical | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Wrong HttpFS test command in doc | Minor | documentation, httpfs | John Zhuge | John Zhuge | | | Replace ugi.getUsername() with ugi.getShortUserName() in viewFS | Minor | viewfs | Archana T | Brahma Reddy Battula | | | HBaseTimelineWriterImpl appears to reference a bad property name | Major | . | Daniel Templeton | Varun Saxena | | | Ambiguous plugin version warning from maven build. | Minor | . | Kai Sasaki | Kai Sasaki | | | KMSTokenRenewer should close providers | Critical | kms | Xiao Chen | Xiao Chen | | | Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange when setting IP and host | Minor | nodemanager | Miklos Szegedi | Miklos Szegedi | | | TestContainerManagerSecurity testContainerManager[1] failed | Major | nodemanager | Haibo Chen | Haibo Chen | | | Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. | Major | . | Chris Nauroth | Li Lu | | | "Overridden" is misspelled as "overriden" in FairScheduler.md | Trivial | site | Daniel Templeton | Heather Sutherland | | | Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string" | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write | Major | timelineserver | Atul Sikaria | Atul Sikaria | | | Fix flaky TestKill.testKillTask() | Major | mrv2 | Haibo Chen | Haibo Chen | | | Fix race condition in TestGetGroups beforeclass setup() | Major | yarn | Haibo Chen | Haibo Chen | | | Missing scheduling policy in the FS queue metric. | Major |

| Yufei Gu | Yufei Gu | | | Fuse wrapper has a typo | Trivial | fuse-dfs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | KMSWebApp should close KeyProviderCryptoExtension | Major | kms | Anthony Young-Garner | John Zhuge | | | Analyse 2.8.0/3.0.0 jdiff reports and fix any issues | Blocker | resourcemanager | Wangda Tan | Akira Ajisaka | | | Bug in return value for delete() calls in WASB | Major | fs/azure | Dushyanth | Dushyanth | | | KMS should not require truststore password | Major | kms, security | Mike Yoder | Mike Yoder | | | Spelling errors in logging and exceptions for code | Major | common, fs, io, security | Grant Sohn | Grant Sohn | | | Intermittent deadlock in NameNode when failover happens. | Blocker | namenode | Abhishek Modi | Akira Ajisaka | | | NN UI should link DN web address using hostnames | Critical | . | Kihwal Lee | Weiwei Yang | | | JobEndNotification info logs are missing in AM container syslog | Minor | applicationmaster | Prabhu Joseph | Haibo Chen | | | Progress bars in Web UI always at 100% | Blocker | webapps | Shen Yinjie | Shen Yinjie | | | TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. | Major | common | Haibo Chen | Haibo Chen | | | Fix up incompatible changes introduced on ContainerStatus and NodeReport | Blocker | api | Karthik Kambatla | Sangjin Lee | | | Retrospect moveApplicationToQueue in align with YARN-5611 | Major | capacity scheduler, resourcemanager | Sunil Govindan | Sunil Govindan | | | Error in handling event type APP\ATTEMPT\REMOVED to the scheduler | Major | . | tangshangwen | Wilfred Spiegelenburg | | | The format of job start time in JHS is different from those of submit and finish time | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code | Trivial | client, nodemanager | Grant Sohn | Grant Sohn | | | Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState | Major | . | Varun Saxena | Varun Saxena | | | Directory Scanner should log startup message time correctly | Minor | . | Xiao Chen | Yiqun Lin | | | Fix typos in HttpFs documentations | Trivial | documentation | Yiqun Lin | Yiqun Lin | | | FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations | Major | . | Vinayakumar B | Vinayakumar B | | | Listing encryption zones fails when deleting a EZ that is on a snapshotted directory | Minor | hdfs | Wellington Chevreuil | Wellington Chevreuil | | | Lifeline message should be ignored for dead nodes | Critical | . | Vinayakumar B | Vinayakumar B | | | FairScheduler should preempt for a ResourceRequest and all preempted containers should be on the same node | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | FsShell can suppress the real error if no error message is present | Major | fs | Rob Vesse | John Zhuge | | | Fix dead links in Job relevant classes | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Lease is leaked on write failure | Major | hdfs-client | Rushabh S Shah | Kuhu Shukla | |

KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request | Major | security | Xiaoyu Yao | Xiaoyu Yao | | | HDFS-11056 failed to close meta file | Blocker | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | cacheadmin,cryptoadmin and storagepolicyadmin should support generic options | Minor | tools | Archana T | Brahma Reddy Battula | | | Fix javac warnings related to the deprecated APIs after upgrading Jackson | Minor | . | Yiqun Lin | Yiqun Lin | | | Fix javac warning related to the deprecated APIs after upgrading Jackson | Minor | . | Yiqun Lin | Yiqun Lin | | | Mover should avoid unnecessary retries if the block is pinned | Major | balancer & mover | Rakesh R | Rakesh R | | | WebHDFS DataNode calls fail without parameter createparent | Blocker | webhdfs | Samuel Low | John Zhuge | | | Document the missing options of hdfs zkfc command | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing | Major | test | Brahma Reddy Battula | Xiaoyu Yao | | | AMRMClientAsync will stop if any exceptions thrown on allocate call | Major | . | Jian He | Jian He | | | Correct check for error code to detect Azure Storage Throttling and provide retries | Major | fs/azure | Gaurav Kanade | Gaurav Kanade | | | FsPermission string constructor does not recognize sticky bit | Major | . | Atul Sikaria | Atul Sikaria | | | Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter | Major | datanode | Eric Badger | Eric Badger | | | FileInputStream leak on failure path in BlockSender | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | TestDataNodeLifeline Fails intermittently | Major | datanode | Anu Engineer | Yiqun Lin | | | VolumeScanner reports write-in-progress replicas as corrupt incorrectly | Major | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | maven issuing pom.version being deprecation warnings | Minor | build | Joe Pallas | Steve Loughran | | | MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. | Blocker | test, yarn | Steve Loughran | Varun Saxena | | | Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launch | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | ClassCastException when we use Bzipcodec for Fsimage compression | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Return error when appending files by webhdfs rest api fails | Major | . | Yuanbo Liu | Yuanbo Liu | | | Add a test to verify NameNodeMXBean#getDecomNodes() and Live/Dead Decom Nodes shown in NameNode WebUI | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Document missing NameNode metrics | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method | Major | yarn | Haibo Chen | Haibo Chen | | | MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0. | Blocker | resourcemanager | Yufei Gu | Yufei Gu | | | File mtime change could not save to editlog | Critical | . | Jimmy Xiang | Jimmy Xiang | | | Update the documents to run MapReduce application | Blocker | documentation | Bibin A Chundatt | Bibin A Chundatt | | | Make

public | Major | fairscheduler, yarn | Tao Jie | Tao Jie | | | Typo in NameNode UI | Trivial | namenode, ui | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix a typo in ReplicaUnderRecovery#setRecoveryID | Trivial | . | Yiqun Lin | Yiqun Lin | | | A couple of spelling errors in the docs | Trivial | documentation | Grant Sohn | Grant Sohn | | | Document the missing envvars commands | Minor | documentation | Yiqun Lin | Yiqun Lin | | | TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863 | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Document the missing options of NameNode bootstrap command | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Fix indefinite article in comments | Minor | . | LiXin Ge | LiXin Ge | | | Spelling errors in logging and exceptions for resource manager code | Trivial | resourcemanager | Grant Sohn | Grant Sohn | | | Fix unreleased resources and null dereferences | Major | . | Yufei Gu | Yufei Gu | | | TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindException | Major | . | Jason Lowe | Yiqun Lin | | | Improve moveApplicationQueues command line | Major | client | Sunil Govindan | Sunil Govindan | | | Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max | Major | . | Wangda Tan | Wangda Tan | | | Change the log level to DEBUG for recovering completed applications | Major | resourcemanager | Rohith Sharma K S | Daniel Templeton | | | ConcurrentModificationException during DataNode#refreshVolumes | Major | . | Jason Lowe | Manoj Govindassamy | | | Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Race condition in test case TestFetchFailure cause it to fail | Minor | test | Nemon Lou | Haibo Chen | | | Build failure due to errors of javadoc build in hadoop-azure | Major | fs/azure | Kai Sasaki | Kai Sasaki | | | Add description of -fs option in generic command usage | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Some modules have dependencies on hadoop-client jar removed by HADOOP-11804 | Blocker | build | Joe Pallas | Sean Busbey | | | Remove references to obsolete io.seqfile configuration variables | Minor | . | Ray Chiang | Ray Chiang | | | RM unable to start in secure setup | Blocker | . | Ajith S | Ajith S | | | Allow WebHDFS to reuse HTTP connections to NN | Major | hdfs | Zheng Shao | Zheng Shao | | | Invoke assembly plugin after building jars | Blocker | build | Allen Wittenauer | Andrew Wang | | | LICENSE/NOTICE are out of date for source artifacts | Blocker | common | Sean Busbey | Xiao Chen | | | Fix Several Unsafe Practices | Major | . | Yufei Gu | Yufei Gu | | | Document the missing metrics of DataNode Volume IO operations | Minor | documentation | Yiqun Lin | Yiqun Lin | | | JobImpl fails to handle preemption events on state COMMITTING | Major | . | Li Lu | Prabhu Joseph | | | Opportunistic containers minor fixes: API annotations and config parameter changes | Minor |

| Arun Suresh | Arun Suresh | | | Bump up release year to 2017 | Blocker | . | Junping Du | Junping Du | | | Log aggregation get failed when NM restart even with recovery | Blocker | . | Junping Du | Junping Du | | | Double wrapping over RandomAccessFile in LocalReplicaInPipeline#createStreams | Minor | hdfs | Hanisha Koneru | Hanisha Koneru | | | Misuse of format specifier in Preconditions.checkArgument | Trivial | . | Yongjun Zhang | Yuanbo Liu | | | Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement | Trivial | capacity scheduler | Ying Zhang | Ying Zhang | | | TimelineServer fails to start when some LevelDb state files are missing. | Critical | . | Ravi Prakash | Ravi Prakash | | | stop-yarn.sh is not able to gracefully stop node managers | Major | . | Weiwei Yang | Weiwei Yang | | | Revert changes of AbstractResourceRequest | Blocker | . | Wangda Tan | Wangda Tan | | | When killing app, RM releases app's resource before they are released by NM | Major | resourcemanager | Jun Gong | Jason Lowe | | | simple spelling errors in yarn test code | Trivial | test | Grant Sohn | vijay | | | Improvements to KMS logging to help debug authorization errors | Minor | kms | Tristan Stevens | Tristan Stevens | | | Remove LEASE\{SOFTLIMIT,HARDLIMIT}\PERIOD and unused import from HdfsServerConstants | Minor | . | Yiqun Lin | Yiqun Lin | | | MoveApplicationAcrossQueues does not check user permission on the target queue | Major | resourcemanager | Haibo Chen | Wilfred Spiegelenburg | | | Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto | Blocker | . | Sean Mackrory | Sean Mackrory | | | Fix compilation failure from missing hadoop-kms test jar | Blocker | build | Karthik Kambatla | Andrew Wang | | | RM unable to start in secure mode | Blocker | resourcemanager | Bibin A Chundatt | Ajith S | | | LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container | Critical | . | Wangda Tan | Wangda Tan | | | The default value of the setting dfs.disk.balancer.block.tolerance.percent is different | Trivial | diskbalancer | Yiqun Lin | Yiqun Lin | | | TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error | Major | fs/adl, test | John Zhuge | John Zhuge | | | Fix FileInputStream leak in loadLastPartialChunkChecksum | Major | datanode | Arpit Agarwal | Chen Liang | | | The rpc to portmap service for NFS has hardcoded timeout. | Major | . | Jitendra Nath Pandey | Mukul Kumar Singh | | | yarn.scheduler.minimum-allocation-\* descriptions are incorrect about behavior when a request is out of bounds | Minor | . | Bibin A Chundatt | Julia Sommer | | | Path globbing does not match newlines | Major | . | Eric Badger | Eric Badger | | | SNN can't checkpoint when rolling upgrade is not finalized | Critical | rolling upgrades | Xiaoyu Yao | Xiaoyu Yao | | | NameNode terminated after full GC thinking QJM is unresponsive. | Major | namenode, qjm | Konstantin Shvachko | Vinitha Reddy Gankidi | | | Change fsimage bool isStriped from boolean to an enum | Major | hdfs | Ewan Higgs | Ewan Higgs | | | Groups should be consistent in using default group mapping class | Minor | security | Yiqun Lin | Yiqun Lin | |

TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk | Minor | . | Yiqun Lin | Yiqun Lin | | | Storage class member storageDirs should be private to avoid unprotected access by derived classes | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Correctly reconstruct erasure coding file from FSImage | Critical | erasure-coding | Sammi Chen | Sammi Chen | | | Improve delegation token validity checking | Major | . | Akira Ajisaka | Akira Ajisaka | | | Fix opportunistic containers documentation | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | RegistrySecurity overrides zookeeper sasl system properties | Major | . | Billie Rinaldi | Billie Rinaldi | | | Missing hadoop-cloud-storage-project module in pom.xml | Critical | build | Akira Ajisaka | Akira Ajisaka | | | ATS client doesn't work with Jersey 2 on the classpath | Major | client, timelineserver | Steve Loughran | Weiwei Yang | | | Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files | Major | fs/adl, test | Vishwajeet Dusane | Vishwajeet Dusane | | | Fix some release build issues | Blocker | build | Andrew Wang | Andrew Wang | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | testAclCLI.xml setfacl test should expect mask r-x | Minor | . | John Zhuge | John Zhuge | | | Fix ReservationACLsTestBase | Major | . | Sean Po | Sean Po | | | Some tests using MiniDFSCluster errored complaining port in use | Minor | test | Kai Zheng | Masatake Iwasaki | | | Adding additional unit test for Trash (I) | Major | . | Xiaoyu Yao | Weiwei Yang | | | TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment | Minor | nodemanager, test | Takashi Ohnishi | Takashi Ohnishi | | | In TestFairScheduler, testcase should not create FairScheduler redundantly | Trivial | fairscheduler | Tao Jie | Tao Jie | | | TestAMRMTokens should use some random port | Major | test | Gera Shegalov | Takashi Ohnishi | | | TestSecureRMRegistryOperations fails with IBM\_JAVA | Major | . | pascal oliva | pascal oliva | | | Add timeout when starting JobHistoryServer in MiniMRYarnCluster | Minor | test | Andras Bokor | Andras Bokor | | | nativetask unit tests need to provide usable output; fix link errors during mvn test | Major | nativetask | Allen Wittenauer | Allen Wittenauer | | | Refine the assert messages in TestFSDirAttrOp | Minor | test | Akira Ajisaka | Jimmy Xiang | | | Add missing @Test annotation for TestSafeMode.testSafeModeUtils() | Trivial | namenode | Lukas Majercak | Lukas Majercak | | | Flaky test TestJobImpl.testKilledDuringKillAbort | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | TestFSNameSystemMBean should wait until JMX cache is cleared | Major | test | Akira Ajisaka | Erik Krogen | | | TestAMRMClient.setup() fails with ArrayOutOfBoundsException | Major | test | Daniel Templeton | Daniel Templeton | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | [YARN-3368] Initial commit of new YARN web UI | Major | . | Wangda Tan | Wangda Tan | | | [YARN-3368] Add nodes page | Major | yarn | Wangda Tan | Varun Saxena | | | [YARN-3368] Cleanup hardcoded configurations, such as RM/ATS addresses | Major |

| Wangda Tan | Sunil Govindan | | | [YARN-3368] Change urls in new YARN ui from camel casing to hyphens | Major | . | Varun Vasudev | Sunil Govindan | | | [YARN-3368] App attempt page is not loading when timeline server is not started | Major | . | Sunil Govindan | Sunil Govindan | | | [YARN-3368] Application and Container pages shows wrong values when RM is stopped | Major | . | Sunil Govindan | Sunil Govindan | | | [YARN-3368] Support hosting web UI framework inside YARN RM | Major | . | Wangda Tan | Sunil Govindan | | | [YARN-3368] Support for responsive navbar when window is resized | Major | . | Kai Sasaki | Kai Sasaki | | | [YARN-3368] Add Apache Hadoop logo in YarnUI home page | Major | webapp | Sunil Govindan | Kai Sasaki | | | [YARN-3368] Generic UI improvements | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | [YARN-3368] Cluster overview page improvements | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | [YARN-3368] Queues page improvements | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | [YARN-3368] Applications page improvements | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | [YARN-3368] Node details page improvements | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | [YARN-3368] Add resource usage for application by node managers | Major | . | Wangda Tan | Wangda Tan | | | [YARN-3368] Add a node heat chart map | Major | . | Wangda Tan | Wangda Tan | | | [YARN-3368] Introduce REFRESH button in various UI pages | Major | webapp | Sunil Govindan | Sreenath Somarajapuram | | | Build error due to preparing 3.0.0-alpha2 deployment | Major | yarn | Kai Sasaki | Kai Sasaki | | | Applications table overflows beyond the page boundary | Major | . | Harish Jaiprakash | Harish Jaiprakash | | | [YARN-3368] Fix YARN UI build pom.xml | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | Handle HADOOP\CLIENT\OPTS in a function | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | [YARN-3368] Fix wrong paths in .gitignore | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | Add an equivalent of hadoop\subcmd\opts for secure opts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | [YARN-3368] Add missing hidden files in webapp folder for deployment | Major | . | Sreenath Somarajapuram | Sreenath Somarajapuram | | | Change hadoop\subcommand\opts to use only uppercase | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Modify HDFS to use hadoop\subcommand\opts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Add a function to handle command\subcommand\OPTS | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Modify common to use hadoop\subcommand\opts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Modify YARN to use hadoop\subcommand\opts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Admins support for maintenance state | Major |

| Ming Ma | Ming Ma | | | modify mapred to use hadoop\subcommand\opts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | DiskBalancer: Add the getNodeList method in Command | Minor | balancer & mover | Yiqun Lin | Yiqun Lin | | | hadoop\subcommand\opts should print name not actual content during debug | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Documentation for HADOOP\subcommand\OPTS | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Expose UpdateResourceRequest API to allow AM to request for change in container properties | Major | . | Arun Suresh | Arun Suresh | | | Fix failing unit test in TestDockerContainerRuntime | Minor | nodemanager, yarn | Sidharta Seethana | Sidharta Seethana | | | Optimize IPC client protobuf decoding | Major | . | Daryn Sharp | Daryn Sharp | | | Store all queue-specific information in FSQueue | Major | fairscheduler | Yufei Gu | Yufei Gu | | | Allow resource localization while container is running | Major | . | Jian He | Jian He | | | Eliminate intermediate buffer for server-side PB encoding | Major | ipc | Daryn Sharp | Daryn Sharp | | | Refactor S3AFileSystem to support introduction of separate metadata repository and tests. | Major | fs/s3 | Chris Nauroth | Chris Nauroth | | | [YARN-3368] Fix create-release to be able to generate bits for the new yarn-ui | Major | yarn, yarn-ui-v2 | Wangda Tan | Wangda Tan | | | HDFS configuration should accept time units | Major | . | Yiqun Lin | Yiqun Lin | | | explicitly declare the Joda time version S3A depends on | Minor | build, fs/s3 | Steve Loughran | Steve Loughran | | | Use ByteBuffer in striping positional read | Major | erasure-coding | Kai Zheng | Sammi Chen | | | Change defaults in hdfs-site.xml to match timeunit type | Minor | datanode, namenode | Yiqun Lin | Yiqun Lin | | | DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI | Minor | balancer & mover | Anu Engineer | Manoj Govindassamy | | | DiskBalancer : reduce lock path in shutdown code | Major | balancer & mover | Anu Engineer | Yuanbo Liu | | | Client-side NM graceful decom is not triggered when jobs finish | Major | nodemanager | Robert Kanter | Robert Kanter | | | RPC client does no bounds checking of responses | Critical | ipc | Daryn Sharp | Daryn Sharp | | | improve section on troubleshooting s3a auth problems | Minor | documentation, fs/s3 | Steve Loughran | Steve Loughran | | | DiskBalancer does not execute multi-steps plan-redux | Major | balancer & mover | Anu Engineer | Anu Engineer | | | DiskBalancer: Report command support with multiple nodes | Major | balancer & mover | Yiqun Lin | Yiqun Lin | | | FBR processing may generate incorrect reportedBlock-blockGroup mapping | Blocker | erasure-coding | Jing Zhao | Jing Zhao | | | DiskBalancer: Execute CLI via Shell | Major | balancer & mover | Anu Engineer | Manoj Govindassamy | | | Override equals and hashCode to avoid connection leakage | Major | ipc | Xiaobing Zhou | Xiaobing Zhou | | | DiskBalancer: update documentation on how to report issues and debug | Minor | balancer & mover | Anu Engineer | Anu Engineer | | | Reduce runtime for append test | Minor | test | Gergely Novk | Gergely Novk | | | Core changes in NodeManager to support re-initialization of Containers with new launchContext | Major |

| Arun Suresh | Arun Suresh | | | s3:// should have been fully cut off from trunk | Major | documentation, fs/s3 | Mingliang Liu | Mingliang Liu | | | Changes in NodeManager to support Container rollback and commit | Major | . | Arun Suresh | Arun Suresh | | | Improve locks in AbstractCSQueue/LeafQueue/ParentQueue | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | DiskBalancerWorkItem should not import jackson relocated by htrace | Major | hdfs-client | Masatake Iwasaki | Manoj Govindassamy | | | Expose upgrade and restart API in ContainerManagementProtocol | Major | . | Arun Suresh | Arun Suresh | | | Remove unnecessary conf cache from DataNode | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | Improve locks in AppSchedulingInfo | Major | resourcemanager, scheduler | Wangda Tan | Varun Saxena | | | DiskBalancer: Complete the documents for the report command | Minor | documentation | Yiqun Lin | Yiqun Lin | | | DiskBalancer : Explore removing references to FsVolumeSpi | Major | balancer & mover | Anu Engineer | Manoj Govindassamy | | | Rename does not need to re-solve destination | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat' | Major | fs, shell, test | Mingliang Liu | Mingliang Liu | | | s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add a service for monitoring application life time out | Major | scheduler | nijel | Rohith Sharma K S | | | Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests | Major | resourcemanager | Arun Suresh | Konstantinos Karanasos | | | Create a CachingGetSpaceUsed implementation that uses df | Major | . | Elliott Clark | Elliott Clark | | | FSDirStatAndListingOp: stop passing path as string | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Expose priority in ReservationSystem submission APIs | Major | capacity scheduler, fairscheduler, resourcemanager | Sean Po | Sean Po | | | Cache path in InodesInPath | Major | hdfs | Daryn Sharp | Daryn Sharp | | | TestDFSShell.testStat fails intermittently | Major | test | Eric Badger | Eric Badger | | | [YARN-3368] Fix maven build to keep all generated or downloaded files in target folder | Major | . | Wangda Tan | Wangda Tan | | | S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. | Minor | fs/s3 | Chris Nauroth | Chris Nauroth | | | Remove rename/delete performance penalty when not using snapshots | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Move lock logging logic from FSNamesystem into FSNamesystemLock | Major | namenode | Erik Krogen | Erik Krogen | | | Refactor TestPBImplRecords so that we can reuse for testing protocol records in other YARN modules | Major |

| Subru Krishnan | Subru Krishnan | | | Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test | Major | test | Mingliang Liu | Mingliang Liu | | | Pass IIP for FSDirAttr methods | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Correctly report missing EC blocks in FSCK | Major | erasure-coding | Takanobu Asanuma | Takanobu Asanuma | | | s3a to handle delete("/", true) robustly | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Reduce performance penalty of encryption zones | Major | hdfs | Daryn Sharp | Daryn Sharp | | | hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. | Minor | fs/s3 | Chris Nauroth | Chris Nauroth | | | Pass IIP for FSDirDeleteOp methods | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Have an explicit KerberosAuthException for UGI to throw, text from public constants | Major | security | Steve Loughran | Xiao Chen | | | Optimize check for existence of parent directory | Major | hdfs | Daryn Sharp | Daryn Sharp | | | BlockManager#isInNewRack should consider decommissioning nodes | Major | erasure-coding, namenode | Jing Zhao | Jing Zhao | | | Refactor TestBalancerBandwidth | Major | balancer & mover, test | Brahma Reddy Battula | Brahma Reddy Battula | | | o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions | Minor | ha, test | Mingliang Liu | Mingliang Liu | | | Add unit test for HDFS command 'dfsadmin -getDatanodeInfo' | Major | fs, shell, test | Xiaobing Zhou | Xiaobing Zhou | | | Add unit test for HDFS command 'dfsadmin -printTopology' | Major | fs, shell, test | Xiaobing Zhou | Xiaobing Zhou | | | [YARN-3368] Launch new YARN UI under hadoop web app port | Major | . | Sunil Govindan | Sunil Govindan | | | DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacity | Minor | balancer & mover | Xiaobing Zhou | Xiaobing Zhou | | | Fix javac and checkstyle warnings in hadoop-auth package | Major | . | Kai Sasaki | Kai Sasaki | | | When there are unrecoverable ec block groups, Namenode Web UI shows "There are X missing blocks." but doesn't show the block names. | Major | erasure-coding | Takanobu Asanuma | Takanobu Asanuma | | | Change unit test for testing parsing "-source" parameter in Balancer CLI | Major | test | Mingliang Liu | Mingliang Liu | | | DiskBalancer: Print the full path to plan file | Minor | balancer & mover | Anu Engineer | Xiaobing Zhou | | | Retrospect yarn entity fields which are publishing in events info fields. | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | [YARN-3368] Move new YARN UI configuration to HADOOP\CONF\DIR | Major | . | Wangda Tan | Sunil Govindan | | | Correct typos in native erasure coding dump code | Trivial | erasure-coding, native | Lszl Bence Nagy | Lszl Bence Nagy | | | Adding additional unit tests for Trash (II) | Major | test | Xiaoyu Yao | Weiwei Yang | | | Block management for maintenance states | Major | . | Ming Ma | Ming Ma | | | Refactor erasure coders | Major | . | Rui Li | Kai Sasaki | | | [YARN-3368] Update UI2 documentation for new UI2 path | Major |

| Kai Sasaki | Kai Sasaki | | | Add unit tests for Trash with HDFS encryption zones | Major | encryption | Xiaoyu Yao | Hanisha Koneru | | | [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | ITestS3AFileContextStatistics.testStatistics() failing | Minor | fs/s3 | Steve Loughran | Pieter Reuse | | | Report erasure coding policy of EC files in Fsck | Major | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA | Major | test | Xiaobing Zhou | Xiaobing Zhou | | | Bad placement policy in FairScheduler causes the RM to crash | Major | fairscheduler | Ray Chiang | Ray Chiang | | | Refactor nodeUpdate across schedulers | Major | capacityscheduler, fairscheduler, scheduler | Ray Chiang | Ray Chiang | | | Add computation time metrics to datanode for ECWorker | Major | . | Li Bo | Sammi Chen | | | fsck -list-corruptfileblocks does not report corrupt EC files | Major | . | Wei-Chiu Chuang | Takanobu Asanuma | | | S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. | Minor | fs/s3 | Rajesh Balamohan | Chris Nauroth | | | s3a should use UGI.getCurrentUser.getShortname() for username | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Document S3A known limitations in file ownership and permission model. | Minor | fs/s3 | Chris Nauroth | Chris Nauroth | | | Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota' | Major | hdfs-client | Xiaobing Zhou | Xiaobing Zhou | | | Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode. | Major | datanode, fs | Virajith Jalaparti | Virajith Jalaparti | | | Native raw erasure coder in XOR codes | Major | io | Kai Zheng | Sammi Chen | | | Purge some superfluous/obsolete S3 FS tests that are slowing test runs down | Minor | fs/s3, test | Steve Loughran | Steve Loughran | | | DiskBalancer: support running multiple commands in single test | Major | balancer & mover | Xiaobing Zhou | Xiaobing Zhou | | | Replace old Hadoop logo with new one | Major | yarn-ui-v2 | Akira Ajisaka | Akhil PB | | | 'Master node' link under application tab is broken | Critical | . | Sumana Sathish | Akhil PB | | | Use different color for Undefined and Succeeded for Final State in applications page | Trivial | . | Yesha Vora | Akhil PB | | | [YARN-3368] Fix various alignment issues and broken breadcrumb link in Node page | Major | . | Sunil Govindan | Akhil PB | | | [YARN-3368] Document limits/notes of the new YARN UI | Major | . | Wangda Tan | Wangda Tan | | | [YARN-3368] Accessing applications and containers list from Node page is throwing few exceptions in console | Major | yarn-ui-v2 | Sunil Govindan | Akhil PB | | | Fix Opportunistic Allocation to set the correct value of Node Http Address | Major | resourcemanager | Arun Suresh | Arun Suresh | | | Split TestHBaseTimelineStorage into multiple test classes | Major | timelineserver | Varun Saxena | Varun Saxena | | | New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075 | Major | yarn-ui-v2 | Sunil Govindan | Sunil Govindan | | |

to use getLongBytes | Major | fs/s3 | Steve Loughran | Abhishek Modi | | | Trim configuration values in DockerLinuxContainerRuntime | Minor | nodemanager | Tianyin Xu | Tianyin Xu | | | TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing) | Major | datanode, test | Mingliang Liu | Mingliang Liu | | | Reduce number of path resolving methods | Major | namenode | Daryn Sharp | Daryn Sharp | | | TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently | Major | namenode, test | Hanisha Koneru | Yiqun Lin | | | DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Add space quota tests for heterogenous storages | Major | hdfs | Xiaobing Zhou | Xiaobing Zhou | | | Add additional unit test for DataNode startup behavior when volumes fail | Major | datanode, test | Mingliang Liu | Mingliang Liu | | | Submit plan request should throw exception if Datanode is in non-REGULAR status. | Major | datanode | Jitendra Nath Pandey | Xiaobing Zhou | | | Add unit test for extended Acls | Major | test | Chen Liang | Chen Liang | | | Enhance UI to show cluster resource utilization of various container Execution types | Blocker | resourcemanager | Sriram Rao | Konstantinos Karanasos | | | [YARN-3368] cleanup code base, integrate web UI related build to mvn, and fix licenses. | Major | . | Wangda Tan | Wangda Tan | | | Add unit test for NameNode failing to start when name dir is unwritable | Major | namenode, test | Mingliang Liu | Xiaobing Zhou | | | updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy | Critical | capacity scheduler | Bibin A Chundatt | Bibin A Chundatt | | | Add global scheduler interface definition and update CapacityScheduler to use it. | Major | capacityscheduler | Wangda Tan | Wangda Tan | | | Add validation to ensure default ports are unique in Configuration | Major | yarn | Konstantinos Karanasos | Konstantinos Karanasos | | | Add unit test for DFSAdmin -report command | Major | shell, test | Mingliang Liu | Xiaobing Zhou | | | Allow fetching exact reason as to why a submitted app is in ACCEPTED state in Fair Scheduler | Major | fairscheduler, resourcemanager | Naganarasimha G R | Yufei Gu | | | Application level node labels stats to be available in REST | Major | api, client, resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | Provide an API to update lifetime of an application. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Add more info to the msgs printed in AbstractDelegationTokenSecretManager for better supportability | Trivial | common, security | Yongjun Zhang | Yongjun Zhang | | | TestDFSAdmin#testReportCommand fails due to timed out | Minor | test | Yiqun Lin | Yiqun Lin | | | Support for parallel checking of StorageLocations on DataNode startup | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Add MutableRate metrics for FSNamesystemLock operations | Major | namenode | Erik Krogen | Erik Krogen | | | TestRBWBlockInvalidation#testRWRInvalidation fails intermittently | Major | namenode, test | Yiqun Lin | Yiqun Lin | | | Support for running async disk checks in DataNode | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics() | Major | fs | Brahma Reddy Battula | Brahma Reddy Battula | | | Enhance FS spec & tests to mandate

\>0 for non-empty files | Minor | fs, test | Steve Loughran | Steve Loughran | | | Document dfs.client.read.striped configuration in hdfs-default.xml | Minor | documentation, hdfs-client | Rakesh R | Rakesh R | | | Update DataNode to use StorageLocationChecker at startup | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | document object store use with fs shell and distcp | Major | documentation, fs, fs/s3 | Steve Loughran | Steve Loughran | | | Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611 | Major | . | Sunil Govindan | Sunil Govindan | | | regression: ITestS3AMiniYarnCluster failing | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Erasure Coding: Deprecate replication-related config keys | Major | erasure-coding | Rakesh R | Rakesh R | | | Add REST endpoints for updating application timeouts | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Add Application timeouts in Application report and CLI | Major | scheduler | nijel | Rohith Sharma K S | | | Support an XOR policy XOR-2-1-64k in HDFS | Major | erasure-coding | Sammi Chen | Sammi Chen | | | s3a rename: fail if dest file exists | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | Support for parallel checking of FsVolumes | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Bring back the feature to view chunks of files in the HDFS file browser | Major | ui | Ravi Prakash | Ravi Prakash | | | Separate QueueManager from Scheduler | Major | capacityscheduler | Xuan Gong | Xuan Gong | | | S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Fix a couple of the s3a statistic names to be consistent with the rest | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Improve Azure Data Lake contract tests. | Major | fs/adl | Chris Nauroth | Vishwajeet Dusane | | | The state of the parentQueue and its childQueues should be synchronized. | Major | capacity scheduler, resourcemanager | Xuan Gong | Xuan Gong | | | TestAddStripedBlockInFBR#testAddBlockInFullBlockReport fails frequently in trunk | Major | test | Yiqun Lin | Yiqun Lin | | | Support an erasure coding policy using RS 10 + 4 | Major | erasure-coding | Sammi Chen | Wei Zhou | | | Retrospect ApplicationReport#getApplicationTimeouts | Major | scheduler | Jian He | Rohith Sharma K S | | | Remove direct references of HBaseTimelineWriter/Reader in core ATS classes | Major | yarn | Haibo Chen | Haibo Chen | | | WebHDFS : Support get/set/unset StoragePolicy | Major | webhdfs | nijel | Surendra Singh Lilhore | | | Extract hbase-backend-exclusive utility methods from TimelineStorageUtil | Major | yarn | Haibo Chen | Haibo Chen | | | Simplify opportunistic container parameters and metrics | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Validate application update timeout request parameters | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful | Major | fs/s3 | Steve Loughran | Steve Loughran | | | CapacityScheduler: Add intra-queue preemption for app priority support | Major | capacityscheduler | Devaraj K | Sunil Govindan | | | Add bytes count metrics to datanode for ECWorker | Major |

| Li Bo | Sammi Chen | | | Shaded Hadoop client artifacts and minicluster | Major | build | Sean Busbey | Sean Busbey | | | Change min supported DN and NN versions back to 2.x | Critical | rolling upgrades | Andrew Wang | Andrew Wang | | | Yarn live log aggregation does not throw if command line arg is wrong | Major | log-aggregation | Prasanth Jayachandran | Xuan Gong | | | Render Application Timeout value in web UI | Major | scheduler | Rohith Sharma K S | Akhil PB | | | Update DataNode to use DatasetVolumeChecker | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Update hbase version to 1.2 (removes phoenix dependencies) | Major | . | Vrushali C | Vrushali C | | | Re-direction of a particular log file within in a container in NM UI does not redirect properly to Log Server ( history ) on container completion | Major | . | Hitesh Shah | Xuan Gong | | | Fail to launch SLSRunner due to NPE | Major | scheduler-load-simulator | Kai Sasaki | Kai Sasaki | | | Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes | Major | . | Arun Suresh | Arun Suresh | | | Add state-machine implementation for scheduler queues | Major | . | Xuan Gong | Xuan Gong | | | Document timeout interfaces CLI and REST APIs | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Create new DiskValidator class with metrics | Major | nodemanager | Ray Chiang | Yufei Gu | | | Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations | Major | capacity scheduler, scheduler | Naganarasimha G R | Naganarasimha G R | | | Unable to access logs for a running application if YARN\ACL\ENABLE is enabled | Major | . | Xuan Gong | Xuan Gong | | | Queue metrics of SLS capacity scheduler only activated after app submit to the queue | Major | capacity scheduler | Wangda Tan | Jonathan Hung | | | Update AppSchedulingInfo to use SchedulingPlacementSet | Major | . | Wangda Tan | Wangda Tan | | | RM fails to start during an upgrade - Failed to load/recover state (YarnException: Invalid application timeout, value=0 for type=LIFETIME) | Critical | resourcemanager | Gour Saha | Rohith Sharma K S | | | FlowRunEntity does not deserialize long values correctly | Major | timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | Support for application priority ACLs in queues of CapacityScheduler | Major | capacityscheduler | Sunil Govindan | Sunil Govindan | | | Add ability to unset and change directory EC policy | Major | erasure-coding | Andrew Wang | Sammi Chen | | | Update webUI/JMX to display maintenance state info | Major | . | Ming Ma | Manoj Govindassamy | | | TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped | Minor | test, yarn | Junping Du | Junping Du | | | S3A to support per-bucket configuration | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add a new web service to list the files on a container in AHSWebService | Major |

| Xuan Gong | Xuan Gong | | | Fix minor bugs in handling of local AMRMToken in AMRMProxy | Minor | federation | Botong Huang | Botong Huang | | | CapacityScheduler: Support deleting queues without requiring a RM restart | Major | capacity scheduler | Xuan Gong | Naganarasimha G R | | | Update fsck to display maintenance state info | Major | datanode, namenode | Manoj Govindassamy | Manoj Govindassamy | | | Propagate allowPreemptionFrom flag all the way down to the app | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers | Major | yarn | Haibo Chen | Haibo Chen | | | RM changes to support change of container ExecutionType | Major | . | Arun Suresh | Arun Suresh | | | Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes | Major | . | Wangda Tan | Wangda Tan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Refactor StripeReaders and use ECChunk version decode API | Major | . | Sammi Chen | Sammi Chen | | | Add support for striping files in benchmarking of TeraGen and TeraSort | Major | . | Sammi Chen | Sammi Chen | | | Update HDFS Erasure Coding doc to add how to use ISA-L based coder | Major | . | Sammi Chen | Sammi Chen | | | Expose nntop output as metrics | Major | namenode | Siddharth Wagle | Siddharth Wagle | | | Add tests for container-executor's is\feature\enabled function | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Provide config knobs to control enabling/disabling new/work in progress features in container-executor | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Add documents for native raw erasure coder in XOR codes | Major | documentation, erasure-coding | Sammi Chen | Sammi Chen | | | FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling | Major | fairscheduler, test | Karthik Kambatla | Kai Sasaki | | | Log ContainerRuntime initialization error in LinuxContainerExecutor | Trivial | nodemanager | Sidharta Seethana | Sidharta Seethana | | | Implement getTrashRoot() for ViewFileSystem | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Implement 'hadoop fs -df' command for ViewFileSystem | Major | . | Manoj Govindassamy | Manoj Govindassamy | | | Implement getFileChecksum(path, length) for ViewFileSystem | Major | viewfs | Manoj Govindassamy | Manoj Govindassamy | | | Typo in comment in ClientServiceDelegate | Trivial | . | Daniel Templeton | Alison Yu | | | hadoop-mapreduce-client-nativetask compilation broken on GCC-6.2.1 | Major | . | Ravi Prakash | Ravi Prakash | | | Implement getUsed() for ViewFileSystem | Major | viewfs | Manoj Govindassamy | Manoj Govindassamy | | | Add remoteBytesRead counter metrics for erasure coding reconstruction task | Major | . | Sammi Chen | Sammi Chen | | | Enforce a C standard for native container-executor | Major | nodemanager | Chris Douglas | Chris Douglas | | | Implement getLinkTarget for ViewFileSystem | Major | viewfs | Manoj Govindassamy | Manoj Govindassamy | | | Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers | Blocker | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Abstract implementation-specific details from the datanode | Major | datanode, fs |