Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.0.0-alpha1.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Remove the deprecated Syncable.sync() method | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Remove the deprecated Syncable.sync() method | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos. | Major | datanode, namenode | Konstantin Shvachko | Tao Luo | | | Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs | Major | . | Haohui Mai | Haohui Mai | | | Remove dead classes in hadoop-streaming | Major | tools | Haohui Mai | Haohui Mai | | | Remove 'dfs.support.append' flag from trunk code | Minor | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Move o.a.h.record to hadoop-streaming | Major | . | Haohui Mai | Haohui Mai | | | Shell script rewrite | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | VersionInfo output specifies subversion | Minor | build | Guo Ruijing | Tsuyoshi Ozawa | | | Fix typo 'programatically' in job.xml (and a few other places) | Trivial | . | Akira Ajisaka | Akira Ajisaka | | | Removed deprecated o.a.h.fs.permission.AccessControlException | Major | . | Haohui Mai | Li Lu | | | rework heap management vars | Major | scripts | Allen Wittenauer | John Smith | | | Derive heap size or mapreduce.\*.memory.mb automatically | Major | mr-am, task | Gera Shegalov | Gera Shegalov | | | LCE default banned user list should have yarn | Trivial | nodemanager | Allen Wittenauer | Varun Saxena | | | DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb | Major | distcp | Mithun Radhakrishnan | Ratandeep Ratti | | | Align the output of \`hadoop fs -du\` to be more Unix-like | Minor | . | Nick Dimiduk | Akira Ajisaka | | | namenode -rollingUpgrade downgrade may finalize a rolling upgrade | Major | namenode | Tsz Wo Nicholas Sze | Kai Sasaki | | | FsShell should report raw disk usage including replication factor | Major | fs | Alex Kozlov | Byron Wong | | | Exclude duplicate jars in hadoop package under different component's lib | Major | build | Vinayakumar B | Vinayakumar B | | | Should not upload partial logs for MR jobs or other "short-running' applications | Blocker | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | 'mapred job -list' command should show the job name as well | Minor | mrv2 | Harsh J | Avinash Kujur | | | FairScheduler handles "invalid" queue names inconsistently | Major | fairscheduler | zhihai xu | zhihai xu | | | WebHDFS should be always enabled | Major | . | Haohui Mai | Li Lu | | | Check and make checkpoint before stopping the NameNode | Major | namenode | Suresh Srinivas | Jing Zhao | | | Formalize the shell API | Blocker | documentation, scripts | Allen Wittenauer | Allen Wittenauer | | | Rework the changelog and releasenotes | Major | documentation | Allen Wittenauer | Allen Wittenauer | | | rewrite test-patch.sh | Major | build, test | Allen Wittenauer | Allen Wittenauer | | | Remove

| Major | native | Akira Ajisaka | Brahma Reddy Battula | | | Missing block is marked as corrupted block | Major | namenode | Ming Ma | Ming Ma | | | Non-HA rollback compatibility broken | Blocker | . | J.Andreina | J.Andreina | | | Avoid calling the partitioner when the numReduceTasks is 1. | Major | . | Ravi Teja Ch N V | Sunil Govindan | | | Remove unused NameNode startup option -finalize | Minor | namenode | Brahma Reddy Battula | Brahma Reddy Battula | | | DFS client API calls should check filesystem closed | Major | . | Rakesh R | Rakesh R | | | Remove DistCpV1 and Logalyzer | Major | tools/distcp | Allen Wittenauer | Brahma Reddy Battula | | | RPC Invoker relies on static ClientCache which has synchronized(this) blocks | Major | ipc, performance | Gopal V | Haohui Mai | | | Fair scheduler REST api returns a missing '[' bracket JSON for deep queue tree | Major | fairscheduler | Kenji Kikushima | Akira Ajisaka | | | Remove the deprecated FSConstants class | Major | . | Haohui Mai | Li Lu | | | Enable v2 FileOutputCommitter by default | Major | mrv2 | Gera Shegalov | Siqi Li | | | MAX\APP\ATTEMPTS\_ENV may no longer be a useful env var for a container | Major | . | Zhijie Shen | Darrell Taylor | | | The RM should handle the graceful shutdown of the NM. | Major | nodemanager, resourcemanager | Ravi Teja Ch N V | Devaraj K | | | RawLocalFileSystem#mkdir and create should honor umask | Major | fs | Chris Nauroth | Varun Saxena | | | Remove support for deprecated configuration key dfs.namenode.decommission.nodes.per.interval | Minor | namenode | Andrew Wang | Andrew Wang | | | Use slf4j instead of common-logging in hdfs-client | Major | build | Haohui Mai | Rakesh R | | | Support more than 2 NameNodes | Major | auto-failover, ha, namenode | Jesse Yates | Jesse Yates | | | Fix typo in JobHistoryEventHandler | Minor | . | Brahma Reddy Battula | Ray Chiang | | | Comparable type should be in FileStatus | Minor | fs | Yong Zhang | Yong Zhang | | | Update aws-sdk dependency to 1.10.6; move to aws-sdk-s3 | Major | fs/s3 | Thomas Demoor | Thomas Demoor | | | Remove deprecated BlockStorageLocation APIs | Major | . | Andrew Wang | Andrew Wang | | | Compact XAttrs to optimize memory footprint. | Major | namenode | Yi Liu | Yi Liu | | | Adding revision to data node jmx getVersion() method | Minor | . | Siqi Li | Siqi Li | | | Followup fixes after YARN-2019 regarding RM behavior when state-store error occurs | Major | . | Jian He | Jian He | | | RM should not issue delegation tokens in unsecure mode | Major | . | Jian He | Bibin A Chundatt | | | Trash messages should be handled by Logger instead of being delivered on System.out | Major | trash | Ashutosh Chauhan | Mingliang Liu | | | Correctly handle snapshot path for getContentSummary | Major | namenode | Jing Zhao | Jing Zhao | | | Show renewer information in DelegationTokenIdentifier#toString | Trivial | security | zhihai xu | zhihai xu | | | Fix posix\_spawn error on OS X | Major | util | Allen Wittenauer | Allen Wittenauer | | | Allow fsck display pending replica location information for being-written blocks | Major |

| Rui Gao | Rui Gao | | | Fix preferredBlockSize typo in OIV XML output | Trivial | tools | Nicole Pazmany | Nicole Pazmany | | | GlobPattern regex library has performance issues with wildcard characters | Major | fs | Matthew Paduano | Matthew Paduano | | | Rename/remove non-HADOOP\_\*, etc from the shell scripts | Blocker | scripts | Allen Wittenauer | Allen Wittenauer | | | Throw an Exception when fs.permissions.umask-mode is misconfigured | Major | conf | Chang Li | Chang Li | | | Fix use of generics within SortedMapWritable | Minor | . | Bertrand Dechoux | Bertrand Dechoux | | | DFS getEZForPath API on a non-existent file should throw FileNotFoundException | Major | encryption | Rakesh R | Rakesh R | | | Remove the TotalFiles metrics | Minor | . | Akira Ajisaka | Akira Ajisaka | | | RPC client does not time out by default | Critical | ipc | Wilfred Spiegelenburg | Masatake Iwasaki | | | Retire libwebhdfs | Major | webhdfs | Allen Wittenauer | Haohui Mai | | | Replace dev-support with wrappers to Yetus | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Change mapreduce.jobhistory.jhist.format default from json to binary | Minor | . | Ray Chiang | Ray Chiang | | | Make HDFS audit logging consistant | Major | . | Kihwal Lee | Kuhu Shukla | | | Fix undeclared/unused dependency to httpclient | Minor | build | Masatake Iwasaki | Masatake Iwasaki | | | pull shell code out of hadoop-dist | Major | build | Allen Wittenauer | Allen Wittenauer | | | Remove all of the CHANGES.txt files | Major | build | Allen Wittenauer | Andrew Wang | | | Rework hadoop-tools | Major | build | Allen Wittenauer | Allen Wittenauer | | | Remove FileUtil#copyMerge | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Change kms server port number which conflicts with HMaster port number | Critical | kms | Yufeng Jiang | Xiao Chen | | | Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO | Major | fairscheduler | Yufei Gu | Yufei Gu | | | HDFS should not default to ephemeral ports | Critical | datanode, hdfs-client, namenode | Arpit Agarwal | Xiaobing Zhou | | | Updated utility to create/modify token files | Major | . | Allen Wittenauer | Matthew Paduano | | | hadoop\add\classpath is not working in .hadooprc | Major | . | Akira Ajisaka | Allen Wittenauer | | | Remove usage of metrics v1 from hadoop-mapreduce | Blocker | . | Akira Ajisaka | Akira Ajisaka | | | Remove metrics v1 | Blocker | metrics | Akira Ajisaka | Akira Ajisaka | | | OfflineEditsViewer stats option should print 0 instead of null for the count of operations | Minor | . | Akira Ajisaka | Yiqun Lin | | | [JDK8] Set minimum version of Hadoop 3 to JDK 8 | Blocker | build | Robert Kanter | Robert Kanter | | | FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Remove hadoop-ant from hadoop-tools | Major | . | Chris Douglas | Chris Douglas | | | fix/rewrite create-release | Blocker | build | Allen Wittenauer | Allen Wittenauer | | | [JDK8] Update jersey version to latest 1.x release | Major | build | Timothy

Clair | Tsuyoshi Ozawa | | | replace slaves with workers | Major | scripts | John Smith | John Smith | | | Default permission for creating file should be 644 for WebHdfs/HttpFS | Minor | webhdfs | Juan Yu | Wellington Chevreuil | | | Remove bin/rcc script | Blocker | tools | Allen Wittenauer | Allen Wittenauer | | | Cut s3:// from trunk | Major | fs/s3 | Steve Loughran | Mingliang Liu | | | [JDK8] Update guice version to 4.0 | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Remove the long deprecated BlockReaderRemote | Major | hdfs | Kai Zheng | Kai Zheng | | | Millisecond timestamp for FsShell console log and MapReduce jobsummary log | Minor | . | John Zhuge | John Zhuge | | | remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects | Major | build | Matt Foley | Matt Foley | | | DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission | Minor | . | John Zhuge | John Zhuge | | | Hdfs dfs chmod should reset sticky bit permission when the bit is omitted in the octal mode | Minor | fs | Manoj Govindassamy | Manoj Govindassamy | | | Caller context should always be constructed by a builder | Minor | ipc | Mingliang Liu | Mingliang Liu | | | fsck should also report decommissioning replicas. | Major | namenode | Jitendra Nath Pandey | Xiaoyu Yao | | | Unsupported FS operations should throw UnsupportedOperationException | Major | fs | Daniel Templeton | Daniel Templeton | | | Remove unused TrashPolicy#getInstance and initialize code | Minor | . | Zhe Zhang | Yiqun Lin | | | Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus | Major | nodemanager | Yufei Gu | Yufei Gu | | | option to disable fsck dots | Minor | namenode | Allen Wittenauer | Mohammad Kamrul Islam | | | Extend NMStateStore to save queued container information | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Arun Suresh | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | inconsistent value type of the "type" field for LeafQueueInfo in response of RM REST API - cluster/scheduler | Major | webapp | Jayesh | Varun Vasudev | | | Revert HADOOP\PREFIX, go back to HADOOP\HOME | Major | scripts, tracing | Allen Wittenauer | Allen Wittenauer | | | [Umbrella] Dynamic subcommands for hadoop shell scripts | Critical | scripts | Allen Wittenauer | Allen Wittenauer | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Rehashing partitioner for better distribution | Major | . | Radim Kolar | Radim Kolar | | | CLASSPATH handling should be consolidated, debuggable | Major | . | Allen Wittenauer | Allen Wittenauer | | | Add a username check for hadoop sub-commands | Minor | scripts | Scott Chen | John Smith | | | Add support for

| Major | scripts | Allen Wittenauer | Allen Wittenauer | | | add FindClass main class to provide classpath checking of installations | Minor | util | Steve Loughran | Steve Loughran | | | Pluggable shell integration | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Shell command ls should include sort options | Minor | fs | Jonathan Allen | Jonathan Allen | | | Expose HadoopKerberosName as a hadoop subcommand | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Add --slaves shell option | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Add posixGroups support for LDAP groups mapping service | Major | . | Kai Zheng | Dapeng Sun | | | Signal congestion on the DataNode | Major | datanode | Haohui Mai | Haohui Mai | | | Add errors and warning metrics page to RM, NM web UI | Major | nodemanager, resourcemanager | Varun Vasudev | Varun Vasudev | | | Configuration to resolve environment variables via ${env.VARIABLE} references | Minor | conf, scripts | Steve Loughran | Kengo Seki | | | A block placement policy with best rack failure tolerance | Minor | namenode | Walter Su | Walter Su | | | Make setting up the build environment easier | Major | build | Niels Basjes | Niels Basjes | | | Add Task Attempt State API to MapReduce Application Master REST API | Minor | . | Ryu Kobayashi | Ryu Kobayashi | | | Add -C flag to make \`hadoop fs -ls\` print filenames only | Major | fs | Ryan Williams | Kengo Seki | | | Add a "Kill" link to Task Attempts page | Minor | applicationmaster | Ryu Kobayashi | Ryu Kobayashi | | | Validate XMLs if a relevant tool is available, when using scripts | Major | scripts | Harsh J | Kengo Seki | | | Generalize BlockInfo in preparation of merging HDFS-7285 into trunk and branch-2 | Major | namenode | Zhe Zhang | Zhe Zhang | | | Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks) | Major | . | Zhe Zhang | Zhe Zhang | | | Add SFTP FileSystem | Minor | fs | igo Goiri | ramtin | | | Implement GETCONTENTSUMMARY operation for WebImageViewer | Major | . | Jagadesh Kiran N | Jagadesh Kiran N | | | Support OAuth2 in WebHDFS | Major | webhdfs | Jakob Homan | Jakob Homan | | | Create a tool to combine aggregated logs into HAR files | Major | . | Robert Kanter | Robert Kanter | | | Create StatsD metrics2 sink | Minor | metrics | Dave Marion | Dave Marion | | | Erasure Coding Support inside HDFS | Major | . | Weihua Jiang | Zhe Zhang | | | metrics2 sink plugin for Apache Kafka | Major | metrics | Babak Behzad | Babak Behzad | | | Ability to fail AM attempts | Major | api | Jason Lowe | Rohith Sharma K S | | | Logging HDFS operation's caller context into audit logs | Major | . | Mingliang Liu | Mingliang Liu | | | allow/disallow snapshots via webhdfs | Major | webhdfs | Allen Wittenauer | Brahma Reddy Battula | | | Support CallerContext in YARN | Major | . | Wangda Tan | Wangda Tan | | | expose calculated paths | Major | . | Allen Wittenauer | Allen Wittenauer | | | Make JvmPauseMonitor an AbstractService | Major | . | Steve Loughran | Sunil Govindan | | | We should have a config to indicate the Timeline Service version | Major | timelineserver | Zhijie Shen | Xuan Gong | | | Add a option to skip newline on empty files with getMerge -nl | Minor |

| Jan Filipiak | Kanaka Kumar Avvaru | | | CPU resource monitoring in Windows | Minor | nodemanager | igo Goiri | igo Goiri | | | Add CSRF Filter for REST APIs to Hadoop Common | Major | security | Larry McCay | Larry McCay | | | Adding Append API support for WASB | Major | fs/azure | Dushyanth | Dushyanth | | | hadoop utilities need to support provided delegation tokens | Blocker | security | Allen Wittenauer | Heesoo Kim | | | Add an HDFS metrics sink | Major | metrics | Daniel Templeton | Daniel Templeton | | | Add Entry point for Kerberos health check | Minor | security | Steve Loughran | Steve Loughran | | | Support nested encryption zones | Major | encryption | Xiaoyu Yao | Zhe Zhang | | | Read s3a creds from a Credential Provider | Major | fs/s3 | Allen Wittenauer | Larry McCay | | | Integrate CSRF prevention filter in WebHDFS. | Major | datanode, namenode, webhdfs | Chris Nauroth | Chris Nauroth | | | OIV: add ReverseXML processor which reconstructs an fsimage from an XML file | Major | tools | Colin P. McCabe | Colin P. McCabe | | | DataNode Lifeline Protocol: an alternative protocol for reporting DataNode liveness | Major | datanode, namenode | Chris Nauroth | Chris Nauroth | | | Change ipc.Client to support asynchronous calls | Major | ipc | Tsz Wo Nicholas Sze | Xiaobing Zhou | | | Datanode command for evicting writers | Major | datanode | Kihwal Lee | Kihwal Lee | | | Add XFS Filter for UIs to Hadoop Common | Major | security | Larry McCay | Larry McCay | | | Add a new interface for retrieving FS and FC Statistics | Major | fs | Ram Venkatesh | Mingliang Liu | | | S3A: Add ability to plug in any AWSCredentialsProvider | Major | fs/s3 | Steven K. Wong | Steven K. Wong | | | hadoop daemonlog should support https and SPNEGO for Kerberized cluster | Major | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Support async call retry and failover | Major | io, ipc | Xiaobing Zhou | Tsz Wo Nicholas Sze | | | S3A to support Amazon STS temporary credentials | Minor | fs/s3 | Sean Mackrory | Sean Mackrory | | | Support Microsoft Azure Data Lake - as a file system in Hadoop | Major | fs/adl, tools | Vishwajeet Dusane | Vishwajeet Dusane | | | Re-balance disks within a Datanode | Major | datanode | Travis Crawford | Anu Engineer | | | YARN Timeline Service v.2: alpha 1 | Critical | timelineserver | Sangjin Lee | Sangjin Lee | | | [Umbrella] Make MapReduce work with Timeline Service Nextgen (YARN-2928) | Major | . | Vinod Kumar Vavilapalli | Sangjin Lee | | | support wildcard in libjars argument | Major | util | Sangjin Lee | Sangjin Lee | | | Allow long-running Balancer to login with keytab | Major | balancer & mover, security | Xiao Chen | Xiao Chen | | | Limit the number of resources a single map reduce job can submit for localization | Major | . | Chris Trezzo | Chris Trezzo | | | Add support for HDFS heterogeneous storage testing to TestDFSIO | Major |

| Wei Zhou | Wei Zhou | | | Allow pluggable audit loggers in KMS | Major | kms | Xiao Chen | Xiao Chen | | | Allow long-running Mover tool to login with keytab | Major | balancer & mover, security | Rakesh R | Rakesh R | | | Specifying node labels when submitting MR jobs | Major | job submission | Jian Fang | Naganarasimha G R | | | Support priorities across applications within the same queue | Major | api, resourcemanager | Arun C Murthy | Sunil Govindan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve checking of input for Api displayTasks() | Trivial | client | XieXianshan | XieXianshan | | | Configure hadoop-main pom to get rid of M2E plugin execution not covered | Major | build | Eric Charles | Eric Charles | | | Add capability to turn on security in unit tests. | Major | test | Jitendra Nath Pandey | Jaimin Jetly | | | Remove the use of the deprecated Syncable.sync() method | Minor | examples | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | test-patch should run tests with -fn to avoid masking test failures | Major | test | Robert Joseph Evans | Robert Joseph Evans | | | Remove two remaining references to "hadoop.native.lib" oldprop | Trivial | test | Harsh J | Harsh J | | | Streamline usage of bookkeeper journal manager | Major | ha | Ivan Kelly | Ivan Kelly | | | testpatch.sh should provide a simpler way to see which warnings changed | Major | scripts | Robert Joseph Evans | Robert Joseph Evans | | | Implement equals for journainfo class. | Minor | namenode | Hari Mankude | Hari Mankude | | | test-patch should stop immediately once it has found compilation errors | Major | build | Tsz Wo Nicholas Sze | Robert Joseph Evans | | | Writable javadocs don't carry default constructor | Trivial | documentation | Harsh J | Harsh J | | | fs -getmerge isn't guaranteed to work well over non-HDFS filesystems | Minor | fs | Harsh J | Harsh J | | | Remove the last dependency call from org.apache.hadoop.record package in MR. | Minor | client | Harsh J | Harsh J | | | The job submitter should make sure to validate jobs before creation of necessary files | Major | job submission, test | Denny Ye | Harsh J | | | Correct the default used in TestDFSClientRetries.busyTest() after HDFS-3462 | Minor | test | Harsh J | Harsh J | | | Test quotas with Long.Max\_Value | Minor | namenode | Sujay Rau | Sujay Rau | | | Check for cyclic dependencies in Jobcontrol job DAG | Major | mrv1 | madhukara phatak | madhukara phatak | | | test-patch.sh doesn't validate patches before building | Minor | build | Jack Dintruff | Jack Dintruff | | | Port StreamInputFormat to new Map Reduce API | Major | streaming | madhukara phatak | madhukara phatak | | | add the missed @Override to methods in Metric/Metric2 package | Minor | metrics | Brandon Li | Brandon Li | | | Exception in TestJettyHelper is incorrect | Minor | test | Jakob Homan | Eli Reisman | | | Unused Command.java fields NAME, DESCRIPTION and USAGE | Minor | shell | Jing Zhao | Jing Zhao | | | NameNode#main(...) can make use of

| Minor | namenode | Harsh J | Harsh J | | | SerializationFactory shouldn't throw a NullPointerException if the serializations list is empty | Minor | io | Harsh J | Sho Shimauchi | | | Colorize the test-patch output sent to JIRA | Trivial | build | Harsh J | Harsh J | | | hadoop-mapreduce-client-core fails compilation in Eclipse due to missing Avro-generated classes | Major | build | Chris Nauroth | Chris Nauroth | | | Addendum to HADOOP-8840: Add a coloring case for +0 results too. | Trivial | precommit | Harsh J | Harsh J | | | Use DFSTestUtil.createFile() for file creation and writing in test cases | Minor | test | Jing Zhao | Jing Zhao | | | Add more javadoc to metrics2 related classes | Minor | metrics | Suresh Srinivas | Suresh Srinivas | | | Provide an option in test-patch that can enable / disable compiling native code | Minor | build | Hemanth Yamijala | Chris Nauroth | | | Make arguments in TestDFSIO case insensitive | Trivial | test | Robert Kanter | Brandon Li | | | Improve documentation for rack awareness | Major | documentation | Jeff Hammerbacher | Adam Faris | | | FileContext#FSLinkResolver should be made static | Minor | . | Suresh Srinivas | Arpit Agarwal | | | Specify explicitly that the NN UI status total is talking of persistent objects on heap. | Trivial | namenode | Harsh J | Harsh J | | | Improve javadoc for FileContext | Minor | . | Andrew Wang | Andrew Wang | | | Port findHangingTest.sh from HBase to Hadoop | Minor | . | Andrew Wang | Andrew Wang | | | Move TestNetworkTopologyWithNodeGroup to Common | Minor | test | Luke Lu | Junping Du | | | log classpath and other key properties on child JVM start | Major | mrv1, mrv2 | Sangjin Lee | Sangjin Lee | | | Add the time of last heartbeat to dead server Web UI | Minor | . | Ted Yu | Ted Yu | | | Improve the javadoc of rpc code | Minor | . | Sanjay Radia | Sanjay Radia | | | Checkpoint Service: a library component to facilitate checkpoint of task state | Major | mrv2 | Carlo Curino | Carlo Curino | | | CheckpointAMPreemptionPolicy implements preemption in MR AM via checkpointing | Major | mr-am, mrv2 | Carlo Curino | Carlo Curino | | | comments typo fix for FsDatasetImpl.java | Trivial | . | yunjiong zhao | yunjiong zhao | | | Workaround for kerberos-related log errors upon running any hadoop command on OSX | Trivial | scripts | Jianbin Wei | Jianbin Wei | | | Typo in CredentialShell usage | Minor | security | Benoy Antony | Benoy Antony | | | mapred version is missing | Major | scripts | Allen Wittenauer | Akira Ajisaka | | | Task level native optimization | Major | task | Binglin Chang | Sean Zhong | | | yarn-env.sh cleanup | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | hadoop shell commands should print usage if not given a class | Major | scripts | Bruno Mah | Allen Wittenauer | | | Add a command that allows sysadmins to manually trigger full block reports from a DN | Major | datanode | Colin P. McCabe | Colin P. McCabe | | | hadoop-daemons.sh should just call hdfs directly | Major | scripts | Allen Wittenauer | Masatake Iwasaki | | | hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME | Minor | scripts | Masatake Iwasaki | Masatake Iwasaki | | | Replace "daemon" with better name in script subcommands | Major | scripts | Yongjun Zhang | Allen Wittenauer | | | Improve

to apply binary diffs | Major | . | Andrew Wang | Colin P. McCabe | | | Document hadoop properties expected to be set by the shell code in \*-env.sh | Major | documentation, scripts | Allen Wittenauer | Allen Wittenauer | | | yarn-daemons.sh should just call yarn directly | Major | scripts | Allen Wittenauer | Masatake Iwasaki | | | Improve yarn container command | Major | . | Jian He | Naganarasimha G R | | | Clean up test-patch.sh to disable "+1 contrib tests" | Major | build | Akira Ajisaka | Akira Ajisaka | | | start-yarn.sh/stop-yarn should give info | Major | scripts | Allen Wittenauer | Varun Saxena | | | Improve the concurrent access on FsVolumeList | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Missing HADOOP\CONF\DIR generates strange results | Major | scripts | Allen Wittenauer | Masatake Iwasaki | | | Improve log reporting during block report rpc failure | Minor | datanode | Charles Lamb | Charles Lamb | | | HDFS inotify should have the ability to determine what txid it has read up to | Major | hdfs-client | Colin P. McCabe | Colin P. McCabe | | | JMX metrics aren't accessible when NN is busy | Major | . | Ming Ma | Ming Ma | | | FastByteComparisons: do not use UNSAFE\_COMPARER on the SPARC architecture because it is slower there | Minor | io, performance, util | Suman Somasundar | Suman Somasundar | | | Deprecate shell vars | Major | scripts | Allen Wittenauer | John Smith | | | Rewrite sls/rumen to use new shell framework | Major | scripts, tools | Allen Wittenauer | John Smith | | | dfs -count -q should label output column | Trivial | . | Nigel Daley | Jonathan Allen | | | Allow compact property description in xml | Major | conf | Owen O'Malley | Kengo Seki | | | Configuration variable expansion regex expensive for long values | Major | conf | Dmitriy V. Ryaboy | Gera Shegalov | | | Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern | Minor | security | Harsh J | Harsh J | | | FSDataInputStream should support "unbuffer" | Major | . | Colin P. McCabe | Colin P. McCabe | | | deprecate sbin/hadoop-daemon.sh | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | deprecate sbin/yarn-daemon.sh | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | deprecate sbin/mr-jobhistory-daemon.sh | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Convert site documentation to markdown | Major | documentation | Allen Wittenauer | Masatake Iwasaki | | | Daemon log documentation is misleading | Major | documentation | Jagadesh Kiran N | Naganarasimha G R | | | Allow smart-apply-patch.sh to add new files in binary git patches | Major | . | Ravi Prakash | Ravi Prakash | | | Convert site documentation from apt to markdown (stragglers) | Major | documentation | Allen Wittenauer | Masatake Iwasaki | | | Clarify application states on the web UI | Major | . | Jian He | Jian He | | | Rewrite httpfs to use new shell framework | Major | scripts | Allen Wittenauer | John Smith | | | Convert site documentation from apt to markdown | Major | documentation | Allen Wittenauer | Gururaj Shetty | | | Fair Scheduler documentation improvements | Minor | fairscheduler | Rohit Agarwal | Rohit Agarwal | | |

JobSubmitter#copyAndConfigureFiles into it's own class | Minor | . | Chris Trezzo | Chris Trezzo | | | Add a "kill application" button to Resource Manager's Web UI | Minor | resourcemanager | Ryu Kobayashi | Ryu Kobayashi | | | Use org.junit.Assume to skip tests instead of return | Minor | test | Akira Ajisaka | Brahma Reddy Battula | | | Improve authentication failure WARN message to avoid user confusion | Major | ipc | Yongjun Zhang | Yongjun Zhang | | | Add a configuration to set ipc.Client's traffic class with IPTOS\LOWDELAY\|IPTOS\RELIABILITY | Major | ipc | Gopal V | Gopal V | | | Resource manager web service fields are not accessible | Trivial | resourcemanager | Krisztian Horvath | Krisztian Horvath | | | Provide a default value for AES/CTR/NoPadding CryptoCodec classes | Minor | . | Andrew Wang | Andrew Wang | | | Add main methods to JobConf and YarnConfiguration, for debug purposes | Major | client | Harsh J | Plamen Jeliazkov | | | Inconsistent configuration in property mapreduce.reduce.shuffle.merge.percent | Trivial | . | Dongwook Kwon | Ray Chiang | | | Ugly stacktrace when quota exceeds | Minor | hdfs-client | Rajit Saha | Harsh J | | | replace "mapreduce.job.credentials.binary" with MRJobConfig.MAPREDUCE\JOB\CREDENTIALS\_BINARY for better readability. | Trivial | mrv2 | zhihai xu | zhihai xu | | | MapTask.MapOutputBuffer#compare/swap should have @Override annotation | Trivial | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | TestRandomAlgorithm has an unused "import" statement | Trivial | contrib/gridmix | Amir Sanjar | Amir Sanjar | | | o.a.h.fs.FileSystem.Cache#remove should use a single hash map lookup | Minor | fs | Gera Shegalov | Brahma Reddy Battula | | | Time.NANOSECONDS\PER\MILLISECOND - use class-level final constant instead of method variable | Trivial | . | Ajith S | Ajith S | | | make initial sleeptime in locateFollowingBlock configurable for DFSClient. | Major | hdfs-client | zhihai xu | zhihai xu | | | Code clean up for LocatedBlock | Minor | . | Jing Zhao | Takanobu Asanuma | | | Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. | Trivial | jobhistoryserver | zhihai xu | zhihai xu | | | Add a more meaningful toString method to SampleStat and MutableStat | Minor | metrics | Karthik Kambatla | Karthik Kambatla | | | YARN RackResolver spams logs with messages at info level | Major | . | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core | Minor | client | Varun Saxena | Varun Saxena | | | Unnecessary condition test in RandomSampler | Minor | mrv2 | Jingguo Yao | Jingguo Yao | | | Deprecated methods in org.apache.hadoop.examples.Sort | Minor | examples | Chao Zhang | Chao Zhang | | | mockito's version in hadoop-nfs pom.xml shouldn't be specified | Minor | nfs | Kengo Seki | Kengo Seki | | | FairScheduler: Metric for latency to allocate first container for an application | Major | . | Ray Chiang | Ray Chiang | | | Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated | Trivial | datanode | nijel | nijel | | | Refactor DFSOutputStream separating DataStreamer out | Major | hdfs-client | Kai Zheng | Li Bo | | | Add LOG.isDebugEnabled() guard for some LOG.debug() | Major |

| Walter Su | Walter Su | | | Streaming "slowmatch" documentation | Trivial | contrib/streaming | Bo Adler | Harsh J | | | Scanning blocks from disk during rolling upgrade startup takes a lot of time if disks are busy | Major | datanode | Rushabh S Shah | Rushabh S Shah | | | hadoop\do\classpath\_subcommand throws a shellcheck warning | Minor | scripts | Allen Wittenauer | Chris Nauroth | | | [Fsshell] Remove bin/hadoop reference from GenericOptionsParser default help text | Minor | scripts | Brahma Reddy Battula | Brahma Reddy Battula | | | Use KeyProviderCryptoExtension#warmUpEncryptedKeys when creating an encryption zone | Trivial | encryption | Andrew Wang | Andrew Wang | | | Improve information on Top users for metrics in RollingWindowsManager and lower log level | Major | . | J.Andreina | J.Andreina | | | Add START\MSG/SHUTDOWN\MSG for ZKFC | Major | auto-failover, ha, tools | Liang Xie | Liang Xie | | | Add support for hardware crc of HDFS checksums on ARM aarch64 architecture | Minor | native | Edward Nevill | Edward Nevill | | | Rolling upgrade is restoring blocks from trash multiple times | Major | datanode | Nathan Roberts | Keisuke Ogiwara | | | EditLogTailer shouldn't log WARN when other node is in standby mode | Major | ha | Todd Lipcon | Todd Lipcon | | | Minor cleanup of BlockPoolManager#getAllNamenodeThreads | Minor | . | Arpit Agarwal | Arpit Agarwal | | | FairScheduler: Need to add more logging to investigate allocations | Minor | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | hdfs user guide should point to the common rack awareness doc | Major | documentation | Allen Wittenauer | Kai Sasaki | | | RM tests should use MockRM where possible | Major | resourcemanager, test | Karthik Kambatla | Karthik Kambatla | | | Debug log resources to be localized for a container | Trivial | nodemanager | Karthik Kambatla | Karthik Kambatla | | | Change logs for ContainerMonitorImpl's resourse monitoring from info to debug | Major | nodemanager | Anubhav Dhoot | Anubhav Dhoot | | | Display count of nodes blacklisted by apps in the web UI | Major | capacityscheduler, resourcemanager | Varun Vasudev | Varun Vasudev | | | Add LOG.isDebugEnabled() guard for some LOG.debug(..) | Major | . | Walter Su | Walter Su | | | Support client-side back off when the datanodes are congested | Major | hdfs-client | Haohui Mai | Haohui Mai | | | Change DataStreamer/DFSOutputStream/DFSPacket for convenience of subclassing | Minor | hdfs-client | Li Bo | Li Bo | | | Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager | Minor | namenode | Haohui Mai | Haohui Mai | | | Refactor RawLocalFileSystem#rename for improved testability. | Minor | fs, test | Chris Nauroth | Jean-Pierre Matsumoto | | | Reduce number of listStatus operation in distcp buildListing() | Minor | tools/distcp | Zoran Dimitrijevic | Zoran Dimitrijevic | | | Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth | Major | security | Larry McCay | Larry McCay | | | Allow dumping of Capacity Scheduler debug logs via web UI for a fixed time period | Major | capacityscheduler | Varun Vasudev | Varun Vasudev | | | Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided. | Trivial | namenode | Walter Su | Walter Su | | | Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible | Major |

| Zhe Zhang | Zhe Zhang | | | Initialize block report IDs with a random number | Minor | datanode | Andrew Wang | Andrew Wang | | | DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime | Minor | hdfs-client | Sean Busbey | Sean Busbey | | | Track and display capacity scheduler health metrics in web UI | Major | capacityscheduler | Varun Vasudev | Varun Vasudev | | | Correct mapred queue usage command | Minor | client | Brahma Reddy Battula | Brahma Reddy Battula | | | Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval | Minor | . | Akira Ajisaka | J.Andreina | | | Add a 'yarn top' tool to help understand cluster usage | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | ZKRMStateStore: Avoid setting watches that are not required | Minor | . | Jun Gong | Jun Gong | | | More accurate verification in SimulatedFSDataset: replace DEFAULT\_DATABYTE with patterned data | Major | . | Zhe Zhang | Zhe Zhang | | | Split TestLazyPersistFiles into multiple tests | Major | test | Arpit Agarwal | Arpit Agarwal | | | View the queue name to YARN Application page | Minor | . | Ryu Kobayashi | Ryu Kobayashi | | | Add start time and Elapsed in ApplicationAttemptReport and display the same in RMAttemptBlock WebUI | Major | api, webapp | Rohith Sharma K S | Rohith Sharma K S | | | Missing description of some methods and parameters in javadoc of FSDirDeleteOp | Minor | . | Yongjun Zhang | Brahma Reddy Battula | | | Refactoring of lazy persist storage cases | Major | test | Arpit Agarwal | Arpit Agarwal | | | Improve readability of deleted block check | Major | namenode | Daryn Sharp | Daryn Sharp | | | Implement listLocatedStatus for ViewFileSystem to speed up split calculation | Blocker | fs | Gera Shegalov | Gera Shegalov | | | Task Id of the failed task in diagnostics should link to the task page | Minor | jobhistoryserver | Siqi Li | Siqi Li | | | Speed-up distcp buildListing() using threadpool | Major | tools/distcp | Zoran Dimitrijevic | Zoran Dimitrijevic | | | Expose AM resource limit and usage in QueueMetrics | Major | . | Jian He | Rohith Sharma K S | | | Expose disk utilization percentage and bad local and log dir counts on NM via JMX | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | YARN admin should be able to remove individual application records from RMStateStore | Critical | resourcemanager, yarn | Wangda Tan | Rohith Sharma K S | | | Refactor NamenodeFsck#check method | Minor | namenode | Takanobu Asanuma | Takanobu Asanuma | | | Add errors and warnings page to ATS | Major | timelineserver | Varun Vasudev | Varun Vasudev | | | Record from/to snapshots in audit log for snapshot diff report | Minor | snapshots | J.Andreina | J.Andreina | | | Display count of running containers in the RM's Web UI | Minor | . | Ryu Kobayashi | Ryu Kobayashi | | | Print information of the build enviornment in test-patch.sh | Minor | scripts | Haohui Mai | Allen Wittenauer | | | Fix CommandFormat#commandFormat java doc annotation | Trivial | . | Xiaoyu Yao | J.Andreina | | | Remove obsolete entries from mapred-default.xml | Minor | . | Ray Chiang | Ray Chiang | | | Mover/Balancer should not schedule two replicas to the same DN | Minor | balancer & mover | Walter Su | Walter Su | | | Code Cleanup in DFSInputStream | Minor |

| Jing Zhao | Jing Zhao | | | DataStreamer cleanup and some minor improvement | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Remove buffer copy in BlockReader.skip | Trivial | . | Binglin Chang | Binglin Chang | | | Need document for storage type label of data node storage locations under dfs.datanode.data.dir | Major | documentation | Xiaoyu Yao | Xiaoyu Yao | | | Refactor FSDirStatAndListingOp | Major | . | Haohui Mai | Haohui Mai | | | add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. | Major | nodemanager | zhihai xu | zhihai xu | | | Add more detail to the documentation for the conf key "dfs.client.read.shortcircuit.streams.cache.size" | Minor | hdfs-client | Tsz Wo Nicholas Sze | Brahma Reddy Battula | | | Move health check script related functionality to hadoop-common | Blocker | . | Ming Ma | Varun Saxena | | | test-patch should allow configuration of default branch | Minor | . | Sean Busbey | Sean Busbey | | | Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Create unit test to automatically compare MR related classes and mapred-default.xml | Minor | . | Ray Chiang | Ray Chiang | | | releasedocmaker.py should use today's date instead of unreleased | Minor | build | Allen Wittenauer | Darrell Taylor | | | PublicLocalizer#addResource is too slow. | Critical | nodemanager | zhihai xu | zhihai xu | | | AM should explicity exit JVM after all services have stopped | Major | . | Jason Lowe | Eric Payne | | | Fsck should provide the info on mandatory option to be used along with "-blocks , -locations and -racks" | Trivial | documentation | J.Andreina | J.Andreina | | | Improper log message when blockreport interval compared with initial delay | Minor | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Rename JobImpl#username to reporterUserName | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Support different number of datanode directories in MiniDFSCluster. | Minor | test | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Drop YARN's overview document | Major | documentation | Allen Wittenauer | Brahma Reddy Battula | | | Make POM project names consistent | Minor | build | Rohith Sharma K S | Rohith Sharma K S | | | Log levels of certain MR logs can be changed to DEBUG | Major | . | Varun Saxena | Varun Saxena | | | "hadoop fs -text" does not give a useful text representation of MapWritable objects | Major | io | Steven K. Wong | Akira Ajisaka | | | More information for "yarn.resourcemanager.webapp.address" in yarn-default.xml | Trivial | documentation, resourcemanager | Nemon Lou | Bartosz ugowski | | | Add snapshot methods to FileContext. | Major | hdfs-client, snapshots | Chris Nauroth | Rakesh R | | | Update documentation about how to use HTrace with HDFS | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Optimize performance of DatanodeManager's node map | Critical | namenode | Daryn Sharp | Daryn Sharp | | | Update sbin commands and documentation to use new --slaves option | Blocker | documentation, scripts | Allen Wittenauer | Allen Wittenauer | | | Let NNBenchWithoutMR specify the replication factor for its test | Minor | client, test | Erik Paulson | Erik Paulson | |

JarFinder#getJar should delete the jar file upon destruction of the JVM | Major | util | Esteban Gutierrez | Jean-Baptiste Onofr | | | Document the Fair Scheduler REST API | Major | documentation, fairscheduler | Sandy Ryza | Kenji Kikushima | | | FairScheduler: Move tests related to max-runnable-apps from TestFairScheduler to TestAppRunnability | Major | . | Karthik Kambatla | nijel | | | Update document for applications REST API response examples | Minor | documentation | Kenji Kikushima | Brahma Reddy Battula | | | Simplify lease manager with INodeID | Major | . | Haohui Mai | Haohui Mai | | | Simplify quota calculations for snapshots and truncate | Major | . | Haohui Mai | Haohui Mai | | | Consolidate MockClocks | Major | . | Sandy Ryza | Sebastian Wong | | | Consolidate parameters of INode.CleanSubtree() into a parameter objects. | Major | . | Haohui Mai | Li Lu | | | Add cli option to test-patch to set the project-under-test | Minor | . | Sean Busbey | Sean Busbey | | | test-patch's issue matching regex should be configurable. | Major | . | Sean Busbey | Sean Busbey | | | Tasks are often killed before they exit on their own | Major | mr-am, mrv2 | Radim Kolar | Ming Ma | | | Remove unused variables in ContainersMonitorImpl and add debug log for overall resource usage by all containers | Trivial | nodemanager | Naganarasimha G R | Naganarasimha G R | | | Rename getBlockReplication to getPreferredBlockReplication | Major | . | Zhe Zhang | Zhe Zhang | | | TestContainerManagerSecurity should init and start Yarn cluster in setup instead of individual methods | Minor | test | Karthik Kambatla | nijel | | | Capture NN's thread dump when it fails over | Major | namenode | Ming Ma | Ming Ma | | | Compatibility doc to state that ATS v1 is a stable REST API | Major | documentation | Steve Loughran | Steve Loughran | | | Improve error message when hadoop archive output path already exists | Trivial | . | Stephen Chu | Yongjun Zhang | | | Dump java command line to \*.out file | Minor | . | Allen Wittenauer | Kengo Seki | | | ViewFileSystem should support snapshot methods. | Major | fs | Chris Nauroth | Rakesh R | | | Remove old webhdfs.xml and other outdated documentation stuff | Major | documentation | Akira Ajisaka | Brahma Reddy Battula | | | Enable Azure-Storage Client Side logging. | Major | tools | Dushyanth | Dushyanth | | | Allow selectively audit logging ops | Major | . | Kihwal Lee | Chen He | | | Refactor the error handling code in DataStreamer | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Move getAdditionalBlock() and related functionalities into a separate class | Major | . | Haohui Mai | Haohui Mai | | | Deprecate DistCpV1 and Logalyzer | Major |

| Brahma Reddy Battula | Brahma Reddy Battula | | | Add a metric for number of active leases | Major | namenode | Kihwal Lee | Rakesh R | | | Support file exclusion list in distcp | Minor | util | Senthil Subramanian | Steven Rand | | | Clean up RemoteException | Trivial | ipc | Sean Busbey | Sean Busbey | | | Implement a space balanced block placement policy | Minor | namenode | Liu Shaohui | Liu Shaohui | | | Replace uses of ThreadLocal\<Random\> with JDK7 ThreadLocalRandom | Major | . | Sean Busbey | Sean Busbey | | | Make jetty version configurable from the maven command line | Trivial | build, ld | sriharsha devineni | sriharsha devineni | | | Document the lease limits | Minor | . | Eli Collins | Mohammad Arshad | | | Add whitespaces between the classes for values in core-default.xml to fit better in browser | Minor | documentation | Chengwei Yang | Kanaka Kumar Avvaru | | | Improve the readability of site index of documentation | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | nodemanagers should obtain cpu/memory values from underlying OS | Major | nodemanager | Alejandro Abdelnur | Varun Vasudev | | | Record the time of calling in tracing span of IPC server | Minor | ipc | Masatake Iwasaki | Masatake Iwasaki | | | Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source | Minor | . | Karthik Kambatla | Darrell Taylor | | | Merge multiple TestWebAppUtils into o.a.h.yarn.webapp.util.TestWebAppUtils | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Remove hdfs dfs reference from file system shell documentation | Minor | documentation | Jonathan Allen | Kengo Seki | | | Bump the version of Apache HTrace to 3.2.0-incubating | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | RMServerUtils.validateResourceRequests should only obtain queue info once | Major | resourcemanager | Jason Lowe | Varun Saxena | | | Document dfs.namenode.service.handler.count in hdfs-site.xml | Major | documentation | Akira Ajisaka | J.Andreina | | | FairScheduler: Apps that have no resource demand should not participate scheduling | Major | fairscheduler | Xianyin Xin | Xianyin Xin | | | Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition | Minor | nodemanager | zhihai xu | zhihai xu | | | Display warning if defaultFs is not set when running fs commands. | Minor | fs | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI | Minor | webapp, yarn | Anthony Rojas | Anubhav Dhoot | | | Typo in trace enabled log in ExceptionHandler of WebHDFS | Trivial | webhdfs | Jakob Homan | Archana T | | | Add @VisibleForTesting annotation to {{BlockPoolSlice#selectReplicaToDelete}} | Trivial | . | Colin P. McCabe | Colin P. McCabe | | | Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. | Major | mrv2 | Eric Payne | Eric Payne | | | Make the visibility of DFSOutputStream#streamer member variable to private | Trivial | . | Rakesh R | Rakesh R | | | Clarify that dfs usage in dfsadmin -report output includes all block replicas. | Minor | documentation | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Pi job (QuasiMonteCarlo) should not try to read the results file if its job fails | Major | examples | Harsh J | Harsh J | | | FairScheduler: Trigger fairShare updates on node events | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | S3Credentials should support use of CredentialProvider | Major | fs/s3 | Sean Busbey | Sean Busbey | | | ShuffleHandler should be able to log shuffle connections | Major | . | Chang Li | Chang Li | | | Deprecate usage

NativeIO#link | Major | native | Andrew Wang | Andrew Wang | | | Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. | Major | namenode, rolling upgrades | Chris Nauroth | Chris Nauroth | | | Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager | Trivial | namenode | Xiaoyu Yao | Brahma Reddy Battula | | | Refactor ZKRMStateStore retry code with Apache Curator | Major | . | Jian He | Karthik Kambatla | | | Document hdfs class path options | Major | documentation | Xiaoyu Yao | Brahma Reddy Battula | | | Document yarn class path options | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | Document mapred class path options | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | Abort the balancer if an upgrade is in progress | Major | balancer & mover | Andrew Wang | Andrew Wang | | | Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Move test utilities for tracing from hadoop-hdfs to hadoop-common | Minor | tracing | Masatake Iwasaki | Masatake Iwasaki | | | Choose SSD over DISK in block placement | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Improve logs for LeafQueue#activateApplications() | Minor | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | Cleanup DFSOutputStream by removing unwanted changes | Minor | hdfs-client | Rakesh R | Rakesh R | | | Allow CORS related headers to passthrough in WebAppProxyServlet | Major | . | Prakash Ramachandran | Varun Saxena | | | Fix unused imports in BPServiceActor and BlockReportLeaseManager | Trivial | . | Colin P. McCabe | Colin P. McCabe | | | Allow appending to existing SequenceFiles | Major | io | Stephen Rose | Kanaka Kumar Avvaru | | | Merge Refactor of DFSOutputStream from HDFS-7285 branch | Major | . | Vinayakumar B | Vinayakumar B | | | Improve the commit failure messages in MRAppMaster recovery | Major | applicationmaster | Gera Shegalov | Brahma Reddy Battula | | | Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings | Minor | datanode, hdfs-client | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Task Attempt List entries should link to the task overview | Major | . | Siqi Li | Siqi Li | | | AM/Task log page should be able to link back to the job | Major | . | Siqi Li | Siqi Li | | | Scrub debug logging of tokens during resource localization. | Major | nodemanager | Chris Nauroth | Chris Nauroth | | | Queue name and user name should be printed on the job page | Major | applicationmaster | Siqi Li | Siqi Li | | | Option for HTTP port of NameNode by MiniDFSClusterManager | Minor | test | Kai Sasaki | Kai Sasaki | | | Add JMX metrics to TimelineDataManager | Major | timelineserver | Jason Lowe | Jason Lowe | | | Control http authentication cookie persistence via configuration | Major | security | Benoy Antony | H Lu | | | Implement GETXATTRS and LISTXATTRS operations for WebImageViewer | Major | . | Akira Ajisaka | Jagadesh Kiran N | | | Make reserved RBW space visible through JMX | Major |

| Kanaka Kumar Avvaru | Kanaka Kumar Avvaru | | | Use try with resources in DataStorage and Storage | Minor | datanode | Andrew Wang | Andrew Wang | | | Allow ditscp to accept bandwitdh in fraction MegaBytes | Major | tools/distcp | Raju Bairishetti | Raju Bairishetti | | | add a lint mode to releasedocmaker | Minor | build, documentation, yetus | Allen Wittenauer | ramtin | | | Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo | Major | . | Zhe Zhang | Zhe Zhang | | | Block scanner INFO message is spamming logs | Major | datanode | Yongjun Zhang | Yongjun Zhang | | | copy all of test-patch BINDIR prior to re-exec | Blocker | yetus | Allen Wittenauer | Allen Wittenauer | | | Add the last reporting reducer info for too many fetch failure diagnostics | Major | . | Chang Li | Chang Li | | | Improve error message in TestCryptoStreamsWithOpensslAesCtrCryptoCodec when OpenSSL is not installed | Trivial | test | Andrew Wang | Andrew Wang | | | FsShell mkdir -p makes an unnecessary check for the existence of the parent. | Minor | fs | Chris Nauroth | Chris Nauroth | | | Merge refactor of DFSInputStream from ErasureCoding branch | Major | . | Vinayakumar B | Vinayakumar B | | | Clarify automatic sync in FSEditLog#logEdit | Minor | . | Andrew Wang | Andrew Wang | | | Enable LocalFileSystem#setTimes to change atime | Minor | fs | Kazuho Fujii | Kazuho Fujii | | | NetworkTopology is not efficient adding/getting/removing nodes | Major | . | igo Goiri | igo Goiri | | | cleanup releasedocmaker | Major | yetus | Allen Wittenauer | Allen Wittenauer | | | Add annotation to package-info.java file to workaround MCOMPILER-205 | Trivial | . | Andrew Wang | Andrew Wang | | | Rename Touchz.java to Touch.java | Trivial | . | Andrew Wang | Andrew Wang | | | setSpaceQuota command should print the available storage type when input storage type is wrong | Major | hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Remove "public" and "abstract" modifiers in FsVolumeSpi and FsDatasetSpi | Trivial | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Support for incremental generation in the protoc plugin | Major | . | Andrew Wang | Andrew Wang | | | Move ResourceCalculatorPlugin from YARN to Common | Major | util | Chris Douglas | Chris Douglas | | | Collect network usage on the node | Major | . | Robert Grandl | Robert Grandl | | | Document missing properties in yarn-default.xml | Major | documentation | Ray Chiang | Ray Chiang | | | Fix typo InvalidStateTransitonException | Minor | api | Xiaoshuang LU | Brahma Reddy Battula | | | Collect disks usages on the node | Major | . | Robert Grandl | Robert Grandl | | | Optimize datanode writes for small writes and flushes | Critical | . | Kihwal Lee | Kihwal Lee | | | Upgrade Tomcat dependency to 6.0.44. | Major | build | Chris Nauroth | Chris Nauroth | | | YARN architecture document needs updating | Major | documentation | Allen Wittenauer | Brahma Reddy Battula | | | Consolidate the NodeManager and NodeManagerRestart documentation into one | Major | documentation | Allen Wittenauer | Masatake Iwasaki | | | When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient | Major |

| Ming Ma | Ming Ma | | | Auto-entries in hadoop\_usage | Minor | scripts | Allen Wittenauer | Allen Wittenauer | | | Mark org.apache.hadoop.security.token.Token as @InterfaceAudience.Public | Major | security | Steve Loughran | Brahma Reddy Battula | | | Fix UserGroupInformation.java to support 64-bit zLinux | Major | security | Adam Roberts | Akira Ajisaka | | | Parse 'HadoopArchive' commandline using cli Options. | Minor | . | Vinayakumar B | Vinayakumar B | | | rewrite resourcemanager restart doc to remove roadmap bits | Major | documentation | Allen Wittenauer | Gururaj Shetty | | | Fix MockRM/MockAM#waitForState sleep too long | Major | test | Karthik Kambatla | Tsuyoshi Ozawa | | | Add getStoragePolicy API to the FileSystem interface | Major | fs | Arpit Agarwal | Brahma Reddy Battula | | | Improve CallQueueManager#swapQueue to make queue elements drop nearly impossible. | Major | ipc, test | zhihai xu | zhihai xu | | | Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus | Minor | documentation, fs, test | Jakob Homan | J.Andreina | | | Utility to Dynamic port allocation | Major | test, util | Brahma Reddy Battula | Brahma Reddy Battula | | | Inotify : All events classes should implement toString() API. | Major | hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Improve HA Namenode Failover detection on the client | Major | hdfs-client | Arun Suresh | Arun Suresh | | | AbstractFileSystem Implementation for WebHdfs | Major | webhdfs | Santhosh G Nayak | Santhosh G Nayak | | | Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java | Major | . | Vinayakumar B | Vinayakumar B | | | Add SSD storagepolicy tests in TestBlockStoragePolicy#testDefaultPolicies | Major | . | Vinayakumar B | Vinayakumar B | | | Add unique YARN\SHELL\ID environment variable to DistributedShell | Major | applications/distributed-shell | Robert Kanter | Robert Kanter | | | Avoid cloning Resource in FSAppAttempt#updateDemand | Minor | fairscheduler | Hong Zhiguo | Hong Zhiguo | | | Improve visualization for the Datanode tab in the NN UI | Major | . | Haohui Mai | Haohui Mai | | | Explain message "Operation category X is not supported in state standby" | Minor | . | Gautam Gopalakrishnan | Gautam Gopalakrishnan | | | Hadoop Jar Error Should Be More Explanatory | Minor | . | Jesse Anderson | Josh Elser | | | pull argument parsing into a function | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Annotate the HTrace span created by FsShell with the command-line arguments passed by the user | Minor | tracing | Masatake Iwasaki | Masatake Iwasaki | | | Configurably turn off the saving of container info in Generic AHS | Major | timelineserver, yarn | Eric Payne | Eric Payne | | | Add startup timestamp to nodemanager UI | Minor | nodemanager | Hong Zhiguo | Hong Zhiguo | | | Skip unit tests based on maven profile rather than

| Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | DFS getStoragePolicy implementation using single RPC call | Major | hdfs-client | Arpit Agarwal | Surendra Singh Lilhore | | | Expose pending, running and reserved containers of a queue in REST api and yarn top | Major | capacityscheduler, fairscheduler, webapp | Anubhav Dhoot | Anubhav Dhoot | | | Add JvmPauseMonitor to ResourceManager and NodeManager | Major | nodemanager, resourcemanager | Robert Kanter | Robert Kanter | | | Add JvmPauseMonitor to Job History Server | Major | jobhistoryserver | Robert Kanter | Robert Kanter | | | Expose storage type and storage ID in BlockLocation | Major | . | Andrew Wang | Andrew Wang | | | Expose underlying LDAP exceptions in SaslPlainServer | Minor | security | Mike Yoder | Mike Yoder | | | Improve NetworkTopology#InnerNode#remove logic | Major | . | Yi Liu | Yi Liu | | | Multihoming docs should emphasize using hostnames in configurations | Major | documentation | Arpit Agarwal | Brahma Reddy Battula | | | Report node resource utilization in heartbeat | Major | nodemanager | igo Goiri | igo Goiri | | | Convert DatanodeDescriptor to use SLF4J logging | Trivial | . | Andrew Wang | Andrew Wang | | | NameNode Metrics : Add FSNameSystem lock Queue Length | Major | namenode | Anu Engineer | Anu Engineer | | | Add sorting and pagination in the datanode tab of the NN Web UI | Critical | namenode | Nathan Roberts | Haohui Mai | | | NameNode metrics logging | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | If ContainersMonitor is not enabled, only print related log info one time | Minor | nodemanager | Jun Gong | Jun Gong | | | Enable MaxInactiveInterval for hadoop http auth token | Major | security | Benoy Antony | H Lu | | | Support CreateFlag in WebHdfs | Major | webhdfs | Vinoth Sathappan | Jakob Homan | | | NameNode Metric : Add Editlog counters as a JMX metric | Major | namenode | Anu Engineer | Anu Engineer | | | Cleanup BlockInfoUnderConstruction from comments and tests | Minor | namenode | Zhe Zhang | Zhe Zhang | | | Fail-fast check in BlockPlacementPolicyDefault#chooseTarget | Major | . | Yi Liu | Yi Liu | | | Utilize Snapshot diff report to build diff copy list in distcp | Major | distcp, snapshots | Yufei Gu | Yufei Gu | | | Add pluggable interface for reading replicas in DFSClient | Major | hdfs-client | Colin P. McCabe | Colin P. McCabe | | | Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array | Minor | namenode | Zhe Zhang | Jing Zhao | | | dfsadmin -setBalancerBandwidth doesnot validate -ve value | Minor | balancer & mover | Rajit Saha | Gautam Gopalakrishnan | | | NameNode support for protected directories | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Improve choosing datanode storage for block placement | Major | namenode | Yi Liu | Yi Liu | | | Harden edit log reading code against out of memory errors | Major | . | Colin P. McCabe | Colin P. McCabe | | | Mark ViewFileSystemBaseTest and ViewFsBaseTest as abstract | Trivial | . | Andrew Wang | Andrew Wang | | | Move TestFileUtil's test resources to resources folder | Minor | . | Andrew Wang | Andrew Wang | | | Point hadoop-project/pom.xml java.security.krb5.conf within target folder | Minor | . | Andrew Wang | Andrew Wang | | | Improve fs -setrep error message for invalid replication factors | Major | namenode | Tsz Wo Nicholas Sze | Daniel Templeton | | | Trash documentation should describe its directory structure and configurations | Minor | documentation | Suman Sehgal | Weiwei Yang | | | Use a yellow elephant rather than a blue one in diagram | Minor |

| Andrew Wang | Andrew Wang | | | Add -safely flag to rm to prompt when deleting many files | Major | fs | Xiaoyu Yao | Xiaoyu Yao | | | YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat | Major | resourcemanager | Wangda Tan | Hong Zhiguo | | | Add "-direct" flag option for fs copy so that user can choose not to create ".\COPYING\" file | Major | fs | Chen He | J.Andreina | | | Adding informative message to sticky bit permission denied exception | Minor | security | Thejas M Nair | Xiaoyu Yao | | | Allow NN to startup if there are files having a lease but are not under construction | Minor | namenode | Tsz Wo Nicholas Sze | Jing Zhao | | | Add a metric to expose the timestamp of the last journal | Major | journal-node | Akira Ajisaka | Surendra Singh Lilhore | | | Add a command to get the balancer bandwidth | Major | balancer & mover | Akira Ajisaka | Rakesh R | | | Allow Aggregated Log readers to handle HAR files | Major | . | Robert Kanter | Robert Kanter | | | Convert docs in xdoc format to markdown | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Make RMHATestBase abstract so its not run when running all tests under that namespace | Minor | . | Anubhav Dhoot | Anubhav Dhoot | | | Better exception reporting in SaslPlainServer | Minor | security | Mike Yoder | Mike Yoder | | | Blacklisting support for scheduling AMs | Major | resourcemanager | Jason Lowe | Anubhav Dhoot | | | Make SO\RCVBUF and SO\SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning | Major | datanode | He Tianyi | He Tianyi | | | Include commas on # of files, blocks, total filesystem objects in NN Web UI | Minor | namenode | Daniel Templeton | Daniel Templeton | | | AccessControlList should avoid calling getGroupNames in isUserInList with empty groups. | Major | security | zhihai xu | zhihai xu | | | DataNode Metrics logging | Major | . | Kanaka Kumar Avvaru | Kanaka Kumar Avvaru | | | Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers | Minor | nodemanager | zhihai xu | zhihai xu | | | yarn logs -am should provide an option to fetch all the log files | Major | client, nodemanager | Varun Vasudev | Varun Vasudev | | | Change the log level in WebHdfsFileSystem.initialize() from INFO to DEBUG | Minor | webhdfs | Santhosh G Nayak | Santhosh G Nayak | | | Improve the assertion message in MockRM while failing after waiting for the state. | Trivial | . | nijel | nijel | | | Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob(). | Major | . | Junping Du | Junping Du | | | Disable caching for JarURLConnection to avoid sharing JarFile with other users when loading resource from URL in Configuration class. | Minor | conf | zhihai xu | zhihai xu | | | Fix inconsistency between log-level guards and statements | Minor | . | Jackie Chang | Jagadesh Kiran N | | | Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. | Major | nodemanager | zhihai xu | zhihai xu | | | RemoteBlockReader2#checkSuccess() shoud print error status | Trivial |

| Brandon Li | Xiao Chen | | | Improve error message for Haadmin when multiple name service IDs are configured | Major | tools | Anu Engineer | Anu Engineer | | | Pass genstamp to ReplicaAccessorBuilder | Major | hdfs-client | Colin P. McCabe | Colin P. McCabe | | | Display help if the command option to "hdfs dfs " is not valid | Minor | . | nijel | nijel | | | Enable parallel JUnit tests in pre-commit. | Major | build, scripts, test | Chris Nauroth | Chris Nauroth | | | Document distcp incremental copy | Major | distcp | Arpit Agarwal | Neelesh Srinivas Salian | | | Incorrect assert message in TestWriteToReplica#testWriteToTemporary | Trivial | test | Tony Wu | Tony Wu | | | Improve DataNode ReplicaMap memory footprint to save about 45% | Major | datanode | Yi Liu | Yi Liu | | | Make the lower and higher watermark in the DN Netty server configurable | Major | webhdfs | Xiaobing Zhou | Xiaobing Zhou | | | For better error recovery, check if the directory exists before using it for localization. | Major | nodemanager | zhihai xu | zhihai xu | | | Change scope of 'AccessTokenProvider.getAccessToken()' and 'CredentialBasedAccessTokenProvider.getCredential()' abstract methods to public | Major | webhdfs | Santhosh G Nayak | Santhosh G Nayak | | | Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common | Minor | documentation | Neelesh Srinivas Salian | Neelesh Srinivas Salian | | | Mover should print the exit status/reason on console like balancer tool. | Minor | balancer & mover | Archana T | Surendra Singh Lilhore | | | WASB Logging: Improve WASB Logging around deletes, reads and writes | Major | tools | Dushyanth | Dushyanth | | | UserGroupInformation doAs can throw misleading exception | Trivial | security | Aaron Dossett | Aaron Dossett | | | FileSystemRMStateStore use IOUtils#close instead of fs#close | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Use separate threshold configurations for disk-full detection and disk-not-full detection. | Critical | nodemanager | zhihai xu | zhihai xu | | | Add missing mapred job command options in mapreduce document | Major | documentation | nijel | nijel | | | org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter checking | Minor | . | Ray Chiang | Ray Chiang | | | Better handling of exceptions thrown during upgrade shutdown | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for better efficiency | Minor | . | Charlie Helin | Charlie Helin | | | HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array | Major | performance | Staffan Friberg | Staffan Friberg | | | Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap | Major | . | Yi Liu | Yi Liu | | | Enable parallel JUnit tests for HDFS Pre-commit | Major | test | Vinayakumar B | Vinayakumar B | | | Tracking methods that hold FSNamesytemLock for too long | Major | namenode | Jing Zhao | Mingliang Liu | | | Shell operations to fail with meaningful errors on windows if winutils.exe not found | Minor | util | Steve Loughran | Steve Loughran | | | Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor | Major | . | Sidharta Seethana | Sidharta Seethana | | | Log container-executor invocation details when exit code is non-zero | Minor | nodemanager | Sidharta Seethana | Sidharta Seethana | | | Update TestFileCreation#testLeaseExpireHardLimit() to avoid

DataNodeTestUtils#getFile() | Trivial | test | Tony Wu | Tony Wu | | | Make block corruption related tests FsDataset-agnostic. | Major | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Do not schedule corrupt blocks for replication | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | JWTRedirectAuthenticationHandler doesn't Retain Original Query String | Major | security | Larry McCay | Larry McCay | | | improve error message for "Absolute path required" in INode.java to contain the rejected path | Trivial | namenode | Marcell Szabo | Marcell Szabo | | | Refactor tests of libhdfs into a directory | Major | . | Haohui Mai | Haohui Mai | | | UserGroupInformation should not log at WARN level if no groups are found | Minor | security | Elliott Clark | Elliott Clark | | | Add overwrite option for 'get' shell command | Major | . | Keegan Witt | Jagadesh Kiran N | | | Add Precondition check to LocatedBlock#addCachedLoc | Major | namenode | Xiao Chen | Xiao Chen | | | Refactor TestWriteToReplica and TestFsDatasetImpl to avoid explicitly creating Files in tests code. | Major | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Fail fast rogue tasks that write too much to local disk | Major | task | Maysam Yabandeh | Maysam Yabandeh | | | Abstract BlockManager's rack policy into BlockPlacementPolicy | Major | . | Ming Ma | Brahma Reddy Battula | | | Ability to list /.reserved | Major | . | Andrew Wang | Xiao Chen | | | Document NFS gateway export point parameter | Trivial | documentation | Zhe Zhang | Xiao Chen | | | Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries | Major | tools | Gaurav Kanade | Gaurav Kanade | | | Deprecate metrics v1 | Blocker | metrics | Luke Lu | Akira Ajisaka | | | Fair scheduler should have ability to set MaxResourceDefault for each queue | Major | . | Siqi Li | Siqi Li | | | Minor cleanup of operations on FsVolumeList#volumes | Minor | . | Walter Su | Walter Su | | | dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby | Major | . | Gautam Gopalakrishnan | Zhe Zhang | | | Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile() | Trivial | test | Tony Wu | Tony Wu | | | Safemode should count and report orphaned blocks | Major | namenode | Todd Lipcon | Anu Engineer | | | Tests with 12345 as hard-coded port break jenkins | Blocker | . | Steve Loughran | Brahma Reddy Battula | | | Display resource usage as percentage of queue and cluster in the RM UI | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | Add more debug info to BlockInfoUnderConstruction#setGenerationStampAndVerifyReplicas | Major | namenode | Hu Liu, | Wei-Chiu Chuang | | | Make GenericTestUtils.assertExceptionContains robust | Minor | test | Steve Loughran | Steve Loughran | | | Fix TestInterDatanodeProtocol to be FsDataset-agnostic. | Minor | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Update the description about replica placement in HDFS Architecture documentation | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Make TestFileConcorruption independent to underlying FsDataset Implementation. | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Make SO\_SNDBUF size configurable at DFSClient side for hdfs write scenario | Major |

| Ming Ma | Mingliang Liu | | | Give ReplicationMonitor a readable thread name | Trivial | namenode | Staffan Friberg | Staffan Friberg | | | fuseConnect should be private to fuse\_connect.c | Trivial | fuse-dfs | Colin P. McCabe | Mingliang Liu | | | Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. | Major | fs/azure, test | Chris Nauroth | Chris Nauroth | | | Support optional offload of NameNode HA service health checks to a separate RPC server. | Major | ha, namenode | Chris Nauroth | Chris Nauroth | | | include nested stack trace in SaslRpcClient.getServerToken() | Minor | security | Steve Loughran | Steve Loughran | | | Consolidate block recovery related implementation into a single class | Minor | datanode | Walter Su | Walter Su | | | Integrate ReservationSystem with the RM failover mechanism | Major | capacityscheduler, fairscheduler, resourcemanager | Subru Krishnan | Subru Krishnan | | | Create a separate jar for hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | Refactor FS#getUsed() to use ContentSummary and add an API to fetch the total file length from a specific path | Minor | . | J.Andreina | J.Andreina | | | Expose size of NameNode directory as a metric | Minor | namenode | Zhe Zhang | Surendra Singh Lilhore | | | FairScheduler: Log skipping reservation messages at DEBUG level | Minor | fairscheduler | Arun Suresh | Arun Suresh | | | Randomize the DFSStripedOutputStreamWithFailure tests | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add schemas to Maven Assembly XMLs | Minor | . | Gabor Liptak | Gabor Liptak | | | Fix TestReplication to be FsDataset-agnostic. | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Add truncateMeta() and deleteMeta() to MiniDFSCluster | Minor | test | Tony Wu | Tony Wu | | | make re2j dependency consistent | Major | build | Allen Wittenauer | Matthew Paduano | | | Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use | Trivial | test | Tony Wu | Tony Wu | | | Add fetchReplica() to FsDatasetTestUtils to return FsDataset-agnostic Replica. | Minor | test | Tony Wu | Tony Wu | | | Make data directory count and storage raw capacity related tests FsDataset-agnostic | Minor | test | Tony Wu | Tony Wu | | | Improve validateSocketPathSecurity0 error message | Trivial | net | Casey Brotherton | Casey Brotherton | | | Make ByteArraryManager log message in one-line format | Minor | hdfs-client | Mingliang Liu | Mingliang Liu | | | WebHdfs : getContentSummary() should give quota for storage types | Major | webhdfs | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Use ctest to run tests for hadoop-hdfs-native-client | Minor | . | Haohui Mai | Haohui Mai | | | Make hadoop dockerfile usable by Yetus | Major | build | Allen Wittenauer | Allen Wittenauer | | | Capacity Scheduler: Rack Locality improvement | Major | capacityscheduler | Nathan Roberts | Nathan Roberts | | | Allow repeating job commit by extending OutputCommitter API | Critical | . | Nemon Lou | Junping Du | | | Add elapsed time for retired job in JobHistoryServer WebUI | Major | webapps | Yiqun Lin | Yiqun Lin | | | Change TestFileTruncate to use FsDatasetTestUtils to get block file size and genstamp. | Major |

| Lei (Eddy) Xu | Lei (Eddy) Xu | | | Update core-default.xml to describe posixGroups support | Minor | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable | Minor | client | Steve Loughran | Steve Loughran | | | Add build instruction for docker toolbox instead of boot2docker | Trivial | documentation | Kai Sasaki | Kai Sasaki | | | Decommissioned dead nodes should continue to be counted as dead after NN restart | Major | . | Ming Ma | Ming Ma | | | BytesWritable fails to support 2G chunks due to integer overflow | Major | . | Reynold Xin | Reynold Xin | | | Include status of closeAck into exception message in DataNode#run | Trivial | . | Xiao Chen | Xiao Chen | | | Switch DataNode.LOG to use slf4j | Minor | . | Walter Su | Walter Su | | | Review and improve HDFS trash documentation | Major | documentation | Harsh J | Madhu Kiran | | | Cleanup TestFilterFileSystem | Major | . | Suresh Srinivas | Suresh Srinivas | | | Add offset support to MurmurHash | Trivial | . | Sergey Shelukhin | Sergey Shelukhin | | | Improve log4j regex in testFindContainingJar | Trivial | . | Robert Rati | Robert Rati | | | Deprecate the TotalFiles metric | Major | . | Akira Ajisaka | Akira Ajisaka | | | Replace usage of ExactSizeInputStream with LimitInputStream. | Minor | . | Chris Nauroth | Walter Su | | | Improve BlockPlacementPolicyDefault's picking of excess replicas | Major | . | Ming Ma | Xiao Chen | | | Support for passing Job priority through Application Submission Context in Mapreduce Side | Major | client | Sunil Govindan | Sunil Govindan | | | Recommission a datanode with 500k blocks may pause NN for 30 seconds | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Separate configs for nodemanager to resourcemanager connection timeout and retries | Major | nodemanager | Chang Li | Chang Li | | | WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info | Major | webhdfs | Sumana Sathish | Xiaoyu Yao | | | FileContext and AbstractFileSystem should be annotated as a Stable interface. | Blocker | fs | Chris Nauroth | Chris Nauroth | | | Update the documentation and wrapper for fuse-dfs | Minor | fuse-dfs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block | Minor | namenode | Mingliang Liu | Mingliang Liu | | | MiniDFSCluster should change block generation stamp via FsDatasetTestUtils | Minor | test | Tony Wu | Tony Wu | | | Trash Support for deletion in HDFS encryption zone | Major | encryption | Xiaoyu Yao | Xiaoyu Yao | | | TestDiskError should get stored replicas through FsDatasetTestUtils. | Minor | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | TestPipelinesFailover should not fail when printing debug message | Major | . | Yongjun Zhang | John Zhuge | | | Tests should get the number of pending async delets via FsDatasetTestUtils | Minor | test | Tony Wu | Tony Wu | | | Improve handling of incomplete TimelineEntities | Minor | timelineserver | Steve Loughran | Varun Saxena | | | Add a config to disable the /logs endpoints | Major | security | Robert Kanter | Robert Kanter | | | The return type of

should be changed to INodeFile | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | concat() API does not give proper exception messages on ./reserved relative path | Major | namenode | Rakesh R | Rakesh R | | | add doc about timeline performance tool usage | Major | . | Chang Li | Chang Li | | | Detailed exception info is lost in reportTo method of ErrorReportAction and ReportBadBlockAction | Trivial | datanode | Yongjun Zhang | Yongjun Zhang | | | Cleanup namenode audit/log/exception messages | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Change TestDeleteBlockPool to not explicitly use File to check block pool existence. | Minor | test | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Remove unused Replica copyOnWrite code | Major | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | JobHistory cache issue | Blocker | . | Ryu Kobayashi | Kai Sasaki | | | Imrpove JavaDoc for getTrimmedStrings | Trivial | util | BELUGA BEHR | BELUGA BEHR | | | Reduce object allocation in PB conversion | Major | hdfs-client | Daryn Sharp | Daryn Sharp | | | Add a non-judgemental YARN app completion status | Major | . | Sergey Shelukhin | Rich Haase | | | Coalesce IBR processing in the NN | Major | namenode | Daryn Sharp | Daryn Sharp | | | Document types of permission checks performed for HDFS operations. | Major | documentation | Chris Nauroth | Chris Nauroth | | | describe dfs.ha.zkfc.port in hdfs-default.xml | Minor | . | Kanaka Kumar Avvaru | Kanaka Kumar Avvaru | | | Avoid creating temprorary strings in Block.toString() and getBlockName() | Minor | performance | Staffan Friberg | Staffan Friberg | | | HDFS Secure Mode Documentation updates | Major | documentation | Arpit Agarwal | Arpit Agarwal | | | Clean up some inappropriate imports | Major | . | Kai Zheng | Kai Zheng | | | TestLeaseRecoveryStriped file missing Apache License header and not well formatted | Minor | . | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Add -showDetails option to YARN Nodes CLI to print all nodes reports information | Major | client | Wangda Tan | Sunil Govindan | | | AsyncDispatcher.waitForDrained should be final | Trivial | yarn | Daniel Templeton | Daniel Templeton | | | Add NullGroupMapping | Major | . | Daniel Templeton | Daniel Templeton | | | Improve handling of container's stderr | Critical | . | Gera Shegalov | Naganarasimha G R | | | Remove trailing whitespaces from mapred-default.xml | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Update FileSystemShell documentation to mention the meaning of each columns of fs -du | Minor | documentation, fs | Daisuke Kobayashi | Daisuke Kobayashi | | | Remove Hard-Coded Values From FileSystem.java | Trivial | fs | BELUGA BEHR | BELUGA BEHR | | | All the log messages about rolling monitoring interval are shown with WARN level | Minor | log-aggregation, nodemanager | Takashi Ohnishi | Takashi Ohnishi | | | Implement RM leader election with curator | Major |

| Jian He | Jian He | | | DistCp minor refactoring and clean up | Minor | distcp | Kai Zheng | Kai Zheng | | | Label-related invalid resource request exception should be able to properly handled by application | Major | scheduler | Bibin A Chundatt | Bibin A Chundatt | | | Log the name of the fsimage being loaded for better supportability | Trivial | namenode | Yongjun Zhang | Yongjun Zhang | | | Job submission can take a long time during Cluster initialization | Major | performance | Kuhu Shukla | Kuhu Shukla | | | Add number of samples in last interval in snapshot of MutableStat | Minor | metrics | Vikram Srivastava | Vikram Srivastava | | | Use a Maven plugin to build the native code using CMake | Minor | build | Colin P. McCabe | Colin P. McCabe | | | Document dfs.cluster.administrators and dfs.permissions.superusergroup | Major | documentation | Arpit Agarwal | Xiaobing Zhou | | | Disable spurious checkstyle checks | Major | . | Andrew Wang | Andrew Wang | | | The build should fail if a -Dbundle option fails | Minor | . | Kai Zheng | Kai Zheng | | | MiniQJMHACluster should not use static port to avoid binding failure in unit test | Major | test | Brandon Li | Yongjun Zhang | | | HTrace: collect position/length information on read operations | Major | hdfs-client, tracing | Zhe Zhang | Zhe Zhang | | | FairScheduler should mention user requested queuename in error message when failed in queue ACL check | Trivial | fairscheduler | Tao Jie | Tao Jie | | | Add documentation for preemption supported in Capacity scheduler | Minor | capacity scheduler | Naganarasimha G R | Naganarasimha G R | | | The HTrace span for OpWriteBlock should record the maxWriteToDisk time | Major | datanode, tracing | Colin P. McCabe | Colin P. McCabe | | | "yarn application -kill" should take multiple application ids | Major | . | Tsuyoshi Ozawa | Sunil Govindan | | | Improve HA ResourceManager Failover detection on the client | Major | client, resourcemanager | Arun Suresh | Jian He | | | Expose the number of blocks pending deletion through dfsadmin report command | Major | hdfs-client, tools | Weiwei Yang | Weiwei Yang | | | Remove useless boxing/unboxing code | Minor | performance | Kousuke Saruta | Kousuke Saruta | | | Re-factor support for CPU as a resource using the new ResourceHandler mechanism | Critical | nodemanager | Sidharta Seethana | Varun Vasudev | | | Add hdfsStreamBuilder API to libhdfs to support defaultBlockSizes greater than 2 GB | Major | libhdfs | Colin P. McCabe | Colin P. McCabe | | | FairScheduler: Disallow preemption from a queue | Major | fairscheduler | Tao Jie | Tao Jie | | | Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp | Minor | namenode | Jing Zhao | Mingliang Liu | | | JobClient should be an AutoClosable | Major | . | Andr Kelpe | Haibo Chen | | | Code refactoring for HDFS-8578 | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | DirectoryScanner shouldn't abort the scan if one directory had an error | Major | datanode | Rakesh R | Rakesh R | | | Log more details in debug logs in BlockReceiver's constructor | Minor | . | Xiao Chen | Xiao Chen | | | Make RegisterNodeManagerRequestPBImpl thread-safe | Major | nodemanager | Karthik Kambatla | Karthik Kambatla | | | Improve DistCp Help and documentation | Minor | distcp | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove expensive 'BlocksMap#getStorages(Block b, final

state)' method | Major | namenode | Daryn Sharp | Daryn Sharp | | | Add additional logging to some NM state store operations | Minor | . | Sidharta Seethana | Sidharta Seethana | | | Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference | Major | . | Xiao Chen | Xiao Chen | | | TcpPeerServer should respect ipc.server.listen.queue.size | Major | . | Elliott Clark | Elliott Clark | | | Improve the performance and GC friendliness of NameNode startup and full block reports | Major | datanode, namenode, performance | Staffan Friberg | Staffan Friberg | | | Check storage ID uniqueness on datanode startup | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Document some newly added metrics | Major | . | Jian He | Jian He | | | Update the footer of Web UI to show year 2016 | Major | . | Xiao Chen | Xiao Chen | | | Add retry support to mapreduce CLI tool | Major | . | Varun Vasudev | Varun Vasudev | | | Refactor IBR code to a new class | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | NetworkTopologyWithNodeGroup.getNodeGroup() can loop infinitely for invalid 'loc' values | Minor | . | Kuhu Shukla | Kuhu Shukla | | | Extend CSRF Filter with UserAgent Checks | Major | security | Larry McCay | Larry McCay | | | Remove useless boxing/unboxing code | Minor | performance | Kousuke Saruta | Kousuke Saruta | | | RollingFileSystemSink should eagerly rotate directories | Critical | . | Daniel Templeton | Daniel Templeton | | | Reuse ObjectMapper instance in MapReduce | Minor | performance | Yiqun Lin | Yiqun Lin | | | OpensslAesCtrCryptoCodec should log which random number generator is used. | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Display application priority in yarn top | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Log uncaught exceptions/errors in various thread pools in YARN | Major | . | Sidharta Seethana | Sidharta Seethana | | | Increase default value of KMS maxHttpHeaderSize and make it configurable | Minor | . | Zhe Zhang | Zhe Zhang | | | Tests for RollingFileSystemSink | Major | test | Daniel Templeton | Daniel Templeton | | | OIV Delimited should show which files are ACL-enabled. | Trivial | tools | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Remove getaclstatus call for non-acl commands in getfacl. | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Reuse objectMapper instance in HDFS to improve the performance | Major | performance | Yiqun Lin | Yiqun Lin | | | Update encryption documentation to reflect nested EZs | Major | documentation, encryption | Zhe Zhang | Zhe Zhang | | | DFSClient and DFSOutputStream should set TCP\_NODELAY on sockets for DataTransferProtocol | Major | hdfs-client | Gary Helmling | Gary Helmling | | | Log Standby exceptions thrown by RequestHedgingProxyProvider at DEBUG Level | Minor | hdfs-client | igo Goiri | igo Goiri | | | AMRM client to log when AMRM token updated | Major | client | Steve Loughran | Prabhu Joseph | | | Annotate CanUnbuffer with @InterfaceAudience.Public | Major | . | Ted Yu | Ted Yu | | | Remove incorrect part of maxResources in FairScheduler documentation | Major | documentation | Ray Chiang | Ray Chiang | | | Skip object allocation

FSAppAttempt#getResourceUsage when possible | Major | . | Ming Ma | Ming Ma | | | Log uncaught exceptions/errors in various thread pools in mapreduce | Major | . | Sidharta Seethana | Sidharta Seethana | | | Enable TLS v1.1 and 1.2 | Major | security | Robert Kanter | Robert Kanter | | | Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases | Minor | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable | Major | namenode | James Kinley | Nathan Roberts | | | Design Document for Credential Provider API | Major | site | Larry McCay | Larry McCay | | | Add machine-readable output to mapred job -history command | Major | client | Robert Kanter | Robert Kanter | | | Print user when services are started | Trivial | . | Brandon Li | Wei-Chiu Chuang | | | Support additional compression levels for GzipCodec | Major | io | Ravi Mutyala | Ravi Mutyala | | | Missing default mapper type in TimelineServer performance test tool usage | Minor | timelineserver | Kai Sasaki | Kai Sasaki | | | Expose number of blocks per volume as a metric | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Support excluding weak Ciphers in HttpServer2 through ssl-server.xml | Critical | security | Vijay Singh | Vijay Singh | | | WASB to read credentials from a credential provider | Minor | fs/azure | Chris Nauroth | Larry McCay | | | On upgrade, Datanode should process all storage/data dirs in parallel | Critical | datanode | Raju Bairishetti | Vinayakumar B | | | StatisticsDataReferenceCleaner swallows interrupt exceptions | Minor | fs | Gregory Chanan | Gregory Chanan | | | mapred job -history command should be able to take Job ID | Major | client | Robert Kanter | Robert Kanter | | | Run FileSystem contract tests with hadoop-azure. | Major | fs/azure, test | Chris Nauroth | madhumita chakraborty | | | Log cipher suite negotiation more verbosely | Major | encryption | Wei-Chiu Chuang | Wei-Chiu Chuang | | | NM aggregation thread pool is not bound by limits | Critical | nodemanager | Haibo Chen | Haibo Chen | | | Document distcp options required for copying between encrypted locations | Major | distcp, documentation, encryption | Xiaoyu Yao | Xiaoyu Yao | | | Refactor the excessReplicateMap to a class | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Refactoring of checksum failure report related codes | Major | . | Kai Zheng | Kai Zheng | | | Collect network and disk usage on the node running Windows | Major | . | igo Goiri | igo Goiri | | | Allow DefaultContainerExecutor container log directory permissions to be configurable | Major | yarn | Ray Chiang | Ray Chiang | | | Skip unnecessary NN operations in log aggregation | Major | . | Ming Ma | Jun Gong | | | Update s3-related properties in

| Minor | fs/s3 | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Document webhdfs retry configuration keys introduced by HDFS-5219/HDFS-5122 | Major | documentation, webhdfs | Xiaoyu Yao | Xiaobing Zhou | | | Change DN to send block receipt IBRs in batches | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add capability to set JHS job cache to a task-based limit | Critical | jobhistoryserver | Ray Chiang | Ray Chiang | | | Rename variables in SchedulerNode to reduce ambiguity post YARN-1011 | Major | scheduler | Karthik Kambatla | igo Goiri | | | Refactor DFSClient#getFileChecksum and DataXceiver#blockChecksum | Major | . | Kai Zheng | Kai Zheng | | | There is no need to acquire CS lock when completing a container | Major | . | MENG DING | MENG DING | | | Change WASB documentation regarding page blob support | Minor | fs/azure | madhumita chakraborty | madhumita chakraborty | | | WebHdfs socket timeouts should be configurable | Major | fs, webhdfs | Austin Donnelly | Austin Donnelly | | | Disable hiding field style checks in class setters | Major | . | Kai Zheng | Kai Zheng | | | Add CLI command to clear storage policy from a path. | Major | tools | Chris Nauroth | Xiaobing Zhou | | | Write a new group mapping service guide | Major | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | In-page TOC of documentation should be automatically generated by doxia macro | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Update balancer/mover document about HDFS-6133 feature | Minor | . | Yongjun Zhang | Yongjun Zhang | | | Add yarn.app.mapreduce.am.log.level to mapred-default.xml | Trivial | documentation | Harsh J | Harsh J | | | TransferFsImage.receiveFile should account and log separate times for image download and fsync to disk | Minor | . | Wellington Chevreuil | Wellington Chevreuil | | | Remove spammy log spew when a datanode is restarted | Major | namenode | Elliott Clark | Brahma Reddy Battula | | | Add warning log when KMSClientProvider cannot create a connection to the KMS server | Minor | . | Xiao Chen | Xiao Chen | | | log classpath of ApplicationClassLoader at INFO level | Minor | util | Sangjin Lee | Sangjin Lee | | | Add heartbeatsTotal in Datanode metrics | Minor | datanode | Hua Liu | Hua Liu | | | Expand section "Data Encryption on HTTP" in SecureMode documentation | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Update Yetus to 0.2.0 | Blocker | build | Allen Wittenauer | Allen Wittenauer | | | Update changelog and release notes (2016-03-04) | Major | documentation | Allen Wittenauer | Allen Wittenauer | | | Clean up CHANGES.txt RAT exclusions from pom.xml files. | Trivial | build | Chris Nauroth | Chris Nauroth | | | Document the new OIV ReverseXML processor | Minor | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Clarify the "expected client Kerberos principal is null" authorization message | Minor | security | Stephen Chu | Stephen Chu | | | Add an HTrace span when refreshing the groups for a username | Major | . | Colin P. McCabe | Colin P. McCabe | | | Do not log StandbyException on NN, other minor logging fixes | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Add a helper library to maintain node state and allows common queries | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Move the test code in

to test | Minor | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Make HDFS commands guide up to date | Major | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Make scheduler error checking message more user friendly | Trivial | yarn | Ray Chiang | Ray Chiang | | | Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level | Major | . | Ming Ma | Ming Ma | | | \*ProcessTree classes have too many whitespace issues | Trivial | . | Karthik Kambatla | Gabor Liptak | | | Warmup NameNode EDEK caches in background thread | Major | encryption, namenode | Zhe Zhang | Xiao Chen | | | Use string constants for XML tags in OfflineImageReconstructor | Minor | . | Yiqun Lin | Yiqun Lin | | | /BUILDING example of zero-docs dist should skip javadocs | Trivial | build, documentation | Steve Loughran | Steve Loughran | | | Update documentation Hadoop Groups Mapping to add static group mapping, negative cache | Minor | documentation, security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | End to end unit test with mini YARN cluster for AMRMProxy Service | Major | nodemanager, resourcemanager | Kishore Chaliparambil | Giovanni Matteo Fumarola | | | Print fully qualified path in CommandWithDestination error messages | Minor | . | Andr Kelpe | John Zhuge | | | DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled | Trivial | applications/distributed-shell | Daniel Templeton | Matt LaMantia | | | [NNBench] Refactor nnbench as a Tool implementation. | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Exclude weak ciphers in SSLFactory through ssl-server.xml | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix missing documentation about rmadmin command regarding node labels | Minor | . | Kai Sasaki | Kai Sasaki | | | Allow RPC scheduler/callqueue backoff using response times | Major | ipc | Xiaoyu Yao | Xiaoyu Yao | | | ShutdownHookManager should have a timeout for each of the Registered shutdown hook | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | Remove dead code in TestDelegationTokenRenewer added in YARN-3055 | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | Deprecate FileUtil#copyMerge | Major | util | Brahma Reddy Battula | Brahma Reddy Battula | | | Ignore x-\* and response headers when copying an Amazon S3 object | Major | fs/s3 | Denis Jannot | Harsh J | | | Pagination support for AppAttempt page TotalOutstandingResource Requests table | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | NetUtils.wrapException to handle SocketException explicitly | Major | util | Steve Loughran | Steve Loughran | | | RPC timeout should not override IPC ping interval | Major | ipc | Masatake Iwasaki | Masatake Iwasaki | | | [NN UI] Last contact for Live Nodes should be relative time | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI | Major | . | Varun Vasudev | Varun Vasudev | | | Add GenericTestUtils.getTestDir method and use it for temporary directory in tests | Major | build, test | Steve Loughran | Steve Loughran | | | RPC Server and Client classes need InterfaceAudience and InterfaceStability annotations | Trivial | ipc | Brandon Li | Brandon Li | | | Unnecessary wait in Node Status Updater during reboot | Major |

| Eric Badger | Eric Badger | | | Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis | Minor | . | Ray Chiang | Ray Chiang | | | Improve documentation on KMS ACLs and delegation tokens | Major | . | Xiao Chen | Xiao Chen | | | Specify PositionedReadable, add contract tests, fix problems | Major | fs | Steve Loughran | Steve Loughran | | | Remove useless boxing/unboxing code | Minor | yarn | Kousuke Saruta | Kousuke Saruta | | | Improve validation of the configured number of tolerated failed volumes | Major | . | Yiqun Lin | Yiqun Lin | | | Change log message in LocalizedResource#handle() to DEBUG | Minor | . | Ray Chiang | Ray Chiang | | | Change "Metrics intern cache overflow" log level from WARN to INFO | Minor | metrics | Akira Ajisaka | Andras Bokor | | | Mark IPC.Client and IPC.Server as @Public, @Evolving | Minor | ipc | Xiaobing Zhou | Xiaobing Zhou | | | Allow using path style addressing for accessing the s3 endpoint | Minor | fs/s3 | Andrew Baptist | Stephen Montgomery | | | Document new dfsadmin command -evictWriters | Minor | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add block id when client got Unable to close file exception | Minor | . | Brahma Reddy Battula | Brahma Reddy Battula | | | getBlocks occupies FSLock and takes too long to complete | Major | balancer & mover, namenode | He Tianyi | He Tianyi | | | BlockPlacementPolicyDefault should use default replication considerload value | Trivial | . | Yiqun Lin | Yiqun Lin | | | Logging improvements in FSImageFormatProtobuf.Saver | Major | namenode | Konstantin Shvachko | Xiaobing Zhou | | | Support MetricsSource interface for DecayRpcScheduler Metrics | Major | . | Xiaoyu Yao | Xiaoyu Yao | | | Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives | Major | . | Xiaobing Zhou | Xiaobing Zhou | | | S3AFileSystem should configure Multipart Copy threshold and chunk size | Major | fs/s3 | Andrew Olson | Andrew Olson | | | Add missing Javadoc enries to Interns.java | Minor | metrics | Andras Bokor | Andras Bokor | | | Document the usage of distcp -diff option | Major | distcp, documentation | Akira Ajisaka | Takashi Ohnishi | | | FairScheduler: Move some logs to debug and check if debug logging is enabled | Major | fairscheduler | Sandy Ryza | Yufei Gu | | | Add documentation for configuration property ipc.maximum.data.length for controlling maximum RPC message size. | Major | documentation | Chris Nauroth | Mingliang Liu | | | Add Corrupt Blocks Information in Metasave Output | Major | . | Kuhu Shukla | Kuhu Shukla | | | RunJar.unJar() should write the last modified time found in the jar entry to the uncompressed file | Minor | util | Colin Evans | Andras Bokor | | | Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client | Minor | hdfs-client | Nicolas Liochon | Lei (Eddy) Xu | | | Increase default balance bandwidth and concurrent moves | Minor | balancer & mover | John Zhuge | John Zhuge | | | Limit the number of outstanding async calls | Major | ipc | Xiaobing Zhou | Xiaobing Zhou | | | Add container resource to RM audit log | Major | resourcemanager, scheduler | Nathan Roberts | Nathan Roberts | | | Support different values of

per storage type | Major | datanode | Pan Yuxuan | Brahma Reddy Battula | | | Clean up RunJar and related test class | Trivial | util | Andras Bokor | Andras Bokor | | | Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException | Minor | security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | add a subcommand for gridmix | Major | tools | Allen Wittenauer | Kai Sasaki | | | Allow ShuffleHandler readahead without drop-behind | Major | nodemanager | Nathan Roberts | Nathan Roberts | | | Document missing S3A and S3 properties | Minor | documentation, fs/s3, tools | Wei-Chiu Chuang | Wei-Chiu Chuang | | | hdfs erasurecode command should remove the redundant -usage option | Minor | erasure-coding | Wei-Chiu Chuang | Yuanbo Liu | | | FairScheduler: Display per-queue demand on the scheduler page | Minor | . | xupeng | xupeng | | | Enable aux services to have their own custom classpath/jar file | Major | . | Xuan Gong | Xuan Gong | | | More informative diagnostics when applications killed by a user | Major | resourcemanager | Jason Lowe | Eric Badger | | | Change project version from 3.0.0 to 3.0.0-alpha1 | Blocker | . | Andrew Wang | Andrew Wang | | | Fix hadoop-openstack undeclared and unused dependencies | Major | tools | Allen Wittenauer | Masatake Iwasaki | | | FileSystemShell doc should explain relative path | Critical | documentation | John Zhuge | John Zhuge | | | TestDistCpViewFs to include IOExceptions in test error reports | Minor | tools/distcp | Steve Loughran | Steve Loughran | | | Refactor RetryInvocationHandler | Minor | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | remove JDK7 from Dockerfile | Major | build | Allen Wittenauer | Allen Wittenauer | | | Improve DelegationTokenIdentifier.toString() for better logging | Major | . | Steve Loughran | Yongjun Zhang | | | Safely close resources in DFSTestUtil | Major | test | Mingliang Liu | Mingliang Liu | | | Add a way to download the job config from the mapred CLI | Major | client | Robert Kanter | Robert Kanter | | | Faster LDAP group name resolution with ActiveDirectory | Major | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | make ResourceTrackerService.nodeHeartbeat more concurrent | Critical | . | Hong Zhiguo | Hong Zhiguo | | | Improve error message from checkBlockLocalPathAccess | Minor | datanode | Tianyin Xu | Tianyin Xu | | | Support Future.get with timeout in ipc async calls | Major | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add a configuration to limit the number of map tasks allowed per job. | Major | job submission | zhihai xu | zhihai xu | | | In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. | Major | tools/distcp | Chris Nauroth | Chris Nauroth | | | Change CredentialShell to use CommandShell base class | Minor | . | Matthew Paduano | Matthew Paduano | | | Expose scheduling policy and max running apps over JMX for Yarn queues | Major | yarn | Yufei Gu | Yufei Gu | | | RollingFileSystemSink should have configurable roll interval | Major | . | Daniel Templeton | Daniel Templeton | | | Add support for OWASP's dependency-check | Minor | build, security | Mike Yoder | Mike Yoder | | | show 'blockScheduled' tooltip in datanodes

| Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Suppress checkstyle JavadocPackage check for test source | Minor | . | John Zhuge | John Zhuge | | | NM should not aggregate logs older than the retention policy | Major | log-aggregation, nodemanager | Haibo Chen | Haibo Chen | | | Upgrade Hadoop MiniKDC with Kerby | Major | test | Jiajia Li | Jiajia Li | | | Add doc for distcp -filters | Trivial | documentation | John Zhuge | John Zhuge | | | Upgrade to Apache Yetus 0.3.0 | Major | build, documentation, test | Allen Wittenauer | Kengo Seki | | | Add non-decayed call metrics for DecayRpcScheduler | Major | ipc, metrics | Xiaoyu Yao | Xiaoyu Yao | | | Add a metric to expose the timeout number of pending replication blocks | Major | . | Akira Ajisaka | Akira Ajisaka | | | Add ability to edit existing token file via dtutil -alias flag | Minor | security | Matthew Paduano | Matthew Paduano | | | Support timeouts in LDAP queries in LdapGroupsMapping. | Major | security | Chris Nauroth | Mingliang Liu | | | S3AFileSystem should read AWS credentials from environment variables | Minor | fs/s3 | Tobin Baker | Tobin Baker | | | Have AM trigger jstack on task attempts that timeout before killing them | Major | mr-am | Jason Lowe | Eric Payne | | | LocalDirAllocator should avoid holding locks while accessing the filesystem | Major | . | Jason Lowe | Jason Lowe | | | Follow on fixups after upgraded mini-kdc using Kerby | Major | test | Jiajia Li | Jiajia Li | | | Refactor UncompressedSplitLineReader.fillBuffer() | Major | . | Daniel Templeton | Daniel Templeton | | | Add support for nested groups in LdapGroupsMapping | Major | security | Gautam Gopalakrishnan | Esther Kundin | | | Upgrade Domains in HDFS | Major | . | Ming Ma | Ming Ma | | | Use GenericTestUtils.getTestDir method in tests for temporary directories | Major | build, test | Vinayakumar B | Vinayakumar B | | | YARN CLI for AM logs does not give any error message if entered invalid am value | Major | yarn | Sumana Sathish | Jian He | | | Add -w -r options in dfs -test command | Major | fs, scripts, tools | Weiwei Yang | Weiwei Yang | | | Add links to datanode web UI in namenode datanodes page | Major | namenode, ui | Weiwei Yang | Weiwei Yang | | | Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit | Major | . | sandflee | sandflee | | | The file localization process should allow for wildcards to reduce the application footprint in the state store | Critical | nodemanager | Daniel Templeton | Daniel Templeton | | | Cleanup javadoc for Path | Minor | documentation | Daniel Templeton | Daniel Templeton | | | The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store | Critical | distributed-cache | Daniel Templeton | Daniel Templeton | | | DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping | Critical | webhdfs | Daryn Sharp | Daryn Sharp | | |

should be legal to enforce shuffle to disk | Major | mrv2, task | Gera Shegalov | Gera Shegalov | | | Remove unused argument in TestRMWebServicesForCSWithPartitions#setupQueueConfiguration | Minor | test | Akira Ajisaka | Tao Jie | | | add rsync to Dockerfile so that precommit archive works | Trivial | build | Allen Wittenauer | Allen Wittenauer | | | cleanup the dockerfile | Major | . | Allen Wittenauer | Allen Wittenauer | | | AsyncCallHandler should use an event driven architecture to handle async calls | Major | io, ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Reload cached groups in background after expiry | Major | . | Stephen O'Donnell | Stephen O'Donnell | | | Improve DataNode web UI | Major | datanode, ui | Weiwei Yang | Weiwei Yang | | | Log message about input options in distcp lacks some items | Minor | tools/distcp | Takashi Ohnishi | Takashi Ohnishi | | | Use Date and Time API in KafkaSink | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Reuse ObjectMapper instance in CombinedHostsFileReader and CombinedHostsFileWriter | Major | hdfs-client | Yiqun Lin | Yiqun Lin | | | Update maven-enforcer-plugin version to 1.4.1 | Major | build | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Fix message when NodeManager runs into errors initializing the recovery directory | Major | yarn | Ray Chiang | Ray Chiang | | | TCP\_NODELAY not set before SASL handshake in data transfer pipeline | Major | datanode | Gary Helmling | Gary Helmling | | | Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc | Minor | documentation | Pan Yuxuan | Pan Yuxuan | | | Update WebHDFS documentation regarding CREATE and MKDIR default permissions | Minor | documentation, webhdfs | Wellington Chevreuil | Wellington Chevreuil | | | Remove deprecated StringUtils.getFormattedTimeWithDiff | Minor | . | Akira Ajisaka | Shen Yinjie | | | TestDistCpSystem should share MiniDFSCluster | Trivial | test | John Zhuge | John Zhuge | | | DataTransferProtocol#writeBlock missing some javadocs | Minor | datanode, hdfs | Yongjun Zhang | John Zhuge | | | passing file to -out for YARN log CLI doesnt give warning or error code | Major | . | Sumana Sathish | Xuan Gong | | | Appropriate use of generics in FairCallQueue | Major | ipc | Konstantin Shvachko | Jonathan Hung | | | Clean up ContainerExecutor JavaDoc | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Remove unused variables in TestFairCallQueue | Minor | test | Konstantin Shvachko | Ye Zhou | | | NMTimelinePublisher should use getMemorySize instead of getMemory | Major | timelineserver | Karthik Kambatla | Vrushali C | | | Log HDFS Balancer exit message to its own log | Minor | balancer & mover | Jiayi Zhou | Jiayi Zhou | | | Add configuration property to prevent JHS from loading jobs with a task count greater than X | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | ClusterNodeTracker: add method to get list of nodes matching a specific resourceName | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Fix the leftover L&N files in hadoop-build-tools/src/main/resources/META-INF/ | Minor | . | Xiao Chen | Tsuyoshi Ozawa | | | Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic, or better yet parameterized | Minor | resourcemanager, test | Daniel Templeton | Kai Sasaki | | | Add a link to HDFS disk balancer document in site.xml | Major | documentation | Akira Ajisaka | Yiqun Lin | | | Refactor JobResourceUploader#uploadFilesInternal | Minor |

| Chris Trezzo | Chris Trezzo | | | Clean up NNStorage and TestSaveNamespace | Minor | . | Andras Bokor | Andras Bokor | | | Make consistent operation name in AdminService | Minor | resourcemanager | Kai Sasaki | Kai Sasaki | | | Update release notes for 3.0.0-alpha1 | Blocker | documentation | Andrew Wang | Andrew Wang | | | Add validation of users input nodes-states options to nodes CLI | Major | . | Wei Yan | Wei Yan | | | MiniDFSCluster should implement AutoCloseable | Minor | test | John Zhuge | Andras Bokor | | | DataNode hot swap drives should disallow storage type changes. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Expose storage policy apis via HDFSAdmin interface | Major | . | Rakesh R | Rakesh R | | | ViewFileSystem should support storage policy related API | Major | fs, viewfs | Peter Shi | Peter Shi | | | Update dfsadmin command line help and HdfsQuotaAdminGuide | Minor | . | Wei-Chiu Chuang | Kai Sasaki | | | Report more accurate info about data corruption location | Major | datanode, hdfs | Yongjun Zhang | Yuanbo Liu | | | Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients | Major | fs/azure | Sivaguru Sankaridurg | Sivaguru Sankaridurg | | | Add a configuration option to enable in-progress edit log tailing | Minor | ha | Jiayi Zhou | Jiayi Zhou | | | Add namenode metric to measure time spent in generating EDEKs | Major | namenode | Hanisha Koneru | Hanisha Koneru | | | Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets | Minor | . | Vincent Poon | Vincent Poon | | | HA NameNode Web UI should show last checkpoint time | Minor | ui | John Zhuge | John Zhuge | | | Accurately compute the test execute time in DFSIO | Minor | benchmarks, performance, test | zhangminglei | zhangminglei | | | Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets | Minor | . | Vincent Poon | Vincent Poon | | | container-executor support for FreeBSD, NetBSD, and others if conf path is absolute | Major | nodemanager, security | Allen Wittenauer | Allen Wittenauer | | | Change container runtime type logging in DelegatingLinuxContainerRuntime to debug | Trivial | yarn | Shane Kumpf | Shane Kumpf | | | LoadBalancingKMSClientProvider#doOp should log IOException stacktrace | Trivial | kms | Wei-Chiu Chuang | Chen Liang | | | Use StandardCharsets instead of String overload in TextOutputFormat | Minor | . | Sahil Kang | Sahil Kang | | | Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets | Minor | . | Vincent Poon | Vincent Poon | | | Optimize UGI group lookups | Major | . | Daryn Sharp | Daryn Sharp | | | Add an AutoCloseableLock class | Major | . | Chen Liang | Chen Liang | | | Enhance logging for Cluster.java around InetSocketAddress | Minor | . | sarun singla | Vrushali C | | | Fix incomplete log info in ResourceLocalizationService | Trivial | . | Jun Gong | Jun Gong | | | TestBasicDiskValidator should not write data to /tmp | Minor | . | Lei (Eddy) Xu | Yufei Gu | | | Optimize RMAppAttempt#pullJustFinishedContainers | Major |

| sandflee | sandflee | | | Mention LoadBalancingKMSClientProvider in KMS HA documentation | Major | documentation, kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | ber-jira: Enhancements to NNThroughputBenchmark tool | Major | benchmarks, tools | Mingliang Liu | Mingliang Liu | | | BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt | Major | hdfs | Daryn Sharp | Kuhu Shukla | | | Add debug log message when splitting is not possible due to unsplittable compression | Minor | client, mrv1, mrv2 | Peter Vary | Peter Vary | | | Update Javadocs for LinuxContainerExecutor | Major | nodemanager | Daniel Templeton | Daniel Templeton | | | Replace FsDatasetImpl object lock with a separate lock object | Major | datanode | Chen Liang | Chen Liang | | | FairScheduler: Allow setting maxResources for ad hoc queues | Major | fairscheduler | Karthik Kambatla | Daniel Templeton | | | Improve SaslRpcClient failure logging | Major | security | Xiaobing Zhou | Xiaobing Zhou | | | Add Spark to CallerContext LimitedPrivate scope | Minor | ipc | Weiqing Yang | Weiqing Yang | | | Make block report size as a metric and add this metric to datanode web ui | Major | datanode, ui | Yuanbo Liu | Yuanbo Liu | | | Remove unused params in connection-related methods of Fetcher | Minor | . | Yiqun Lin | Yiqun Lin | | | yarn list command to support filtering by tags | Minor | client | Steve Loughran | Varun Saxena | | | Deprecate getInstance and initialize methods with Path in TrashPolicy | Minor | . | Yiqun Lin | Yiqun Lin | | | Add option to -du to calculate directory space usage excluding snapshots | Major | snapshots | Gautam Gopalakrishnan | Xiao Chen | | | Fix an error in ReaderStrategy#ByteBufferStrategy | Major | . | Sammi Chen | Sammi Chen | | | Make the threshold of reporting FSNamesystem lock contention configurable | Major | logging, namenode | Zhe Zhang | Erik Krogen | | | TestYarnCLI#testGetContainers should format according to CONTAINER\_PATTERN | Minor | client, test | Jonathan Hung | Jonathan Hung | | | add MR support to redact job conf properties | Major | mrv2 | Haibo Chen | Haibo Chen | | | RPC Metrics : Add the ability track and log slow RPCs | Major | ipc, metrics | Anu Engineer | Anu Engineer | | | Kerberos relogin interval in UserGroupInformation should be configurable | Major | security | Alejandro Abdelnur | Robert Kanter | | | Allow the directoryScanner to be rate-limited | Major | datanode | Nathan Roberts | Daniel Templeton | | | Log slow name resolutions | Major | . | Sidharta Seethana | Sidharta Seethana | | | Improve quota initialization performance | Major | . | Kihwal Lee | Kihwal Lee | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | TestOfflineEditsViewer is failing on trunk | Major | test | Aaron T. Myers | Uma Maheswara Rao G | | | TestBackupNode fails since HADOOP-7524 went in. | Major | test | Ivan Kelly | Ivan Kelly | | | Rat excludes has a typo for excluding editsStored files | Major | test | Aaron T. Myers | Aaron T. Myers | | | hudson findbugs not reporting properly | Major | mrv2 | Thomas Graves | Tom White | | | The namespace quota of root directory should not be

| Major | namenode | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | | | Commands using webhdfs and hftp print unnecessary debug information on the console with security enabled | Major | webhdfs | Arpit Gupta | Arpit Gupta | | | HttpServer wraps InterruptedExceptions by IOExceptions if interrupted in startup | Minor | util | Steve Loughran | Steve Loughran | | | add a test to verify that buildDTAuthority works for cases with no authority. | Major | test | John George | John George | | | TestJMXJsonServlet is failing | Major | test | Tom White | Tom White | | | MBeans shouldn't try to register when it fails to create MBeanName | Minor | metrics | Devaraj K | Devaraj K | | | Typo in fetchdt error message | Trivial | hdfs-client | Aaron T. Myers | AOE Takashi | | | Test file reference count is at least 3x actual value | Major | build | Jonathan Eagles | Jonathan Eagles | | | PowerPc Build error. | Major | build | Kumar Ravi | Kumar Ravi | | | jenkins complaining about 16 javadoc warnings | Major | build | Thomas Graves | Robert Joseph Evans | | | test-patch findbugs may fail if a dependent module is changed | Major | build | Tom White | Robert Joseph Evans | | | Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable | Trivial | util | Harsh J | Harsh J | | | FSNamesystem header comment says it's for the DN | Trivial | namenode | Aaron T. Myers | Harsh J | | | Incorrect class comments in a few tests | Trivial | test | Aaron T. Myers | Andy Isaacson | | | empty-configuration.xml fails xml validation | Minor | test | Radim Kolar | Radim Kolar | | | test-patch.sh gives out the wrong links for newPatchFindbugsWarnings | Minor | precommit | Colin P. McCabe | Colin P. McCabe | | | When the value of a configuration key is set to its unresolved form, it causes the IllegalStateException in Configuration.get() stating that substitution depth is too large. | Major | conf | Arvind Prabhakar | Arvind Prabhakar | | | MRBench allows Long-sized input-lines value but parses CLI argument as an Integer | Trivial | benchmarks | Harsh J | Harsh J | | | TestDFSClientRetries.busyTest() should restore default xceiver count in the config. | Major | test | Konstantin Shvachko | madhukara phatak | | | raid added javadoc warnings | Minor | build | Thomas Graves | Jason Lowe | | | dist tar build fails in hadoop-hdfs-raid project | Critical | build | Jason Lowe | Jason Lowe | | | test-patch.sh shows an incorrect link in Jekins builds | Major | build | Kihwal Lee | Kihwal Lee | | | Revert unused MiniDFSCluster constructor from HDFS-3049 | Blocker | . | Arun C Murthy | Arun C Murthy | | | test-patch.sh should not immediately exit when no tests are added or modified | Major | . | Colin P. McCabe | Colin P. McCabe | | | Should use equals() rather than == to compare String or Text in MD5MD5CRC32FileChecksum and TFileDumper | Minor | fs, io | Brandon Li | Brandon Li | | | Findbugs warning Configuration.dumpDeprecatedKeys() concatenates strings using + in a loop | Trivial | conf | Suresh Srinivas | Suresh Srinivas | | | Resource leak during failure scenario of closing of resources. | Minor | util | ramkrishna.s.vasudevan |

| | | NPE In getLocalPathForWrite ( path, conf ) when the required context item is not configured | Major | fs | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix the test-patch colorizer to cover all sorts of +1 lines. | Major | build | Harsh J | Harsh J | | | DBCount should not use ACCESS | Major | examples | Viji | Viji | | | Remove MR1 configs from mapred-default.xml | Major | documentation, mrv2 | Todd Lipcon | Todd Lipcon | | | test-patch's -1 on @author tag presence doesn't cause a -1 to the overall result | Trivial | build | Harsh J | Harsh J | | | Fix LocalRunner on trunk after MAPREDUCE-3223 broke it | Blocker | test | Harsh J | Harsh J | | | NameNode WebUI should allow sorting live datanode list by fields Block Pool Used, Block Pool Used(%) and Failed Volumes. | Major | namenode | Brahma Reddy Battula | Brahma Reddy Battula | | | test-patch should build the common native libs before running hdfs tests | Major | build | Eli Collins | Colin P. McCabe | | | Fix TotalOrderParitioner to work with non-WritableComparable key types | Trivial | client | Harsh J | Harsh J | | | dev-support/test-patch.sh is parsing modified files wrong | Major | build | Raja Aluri | Raja Aluri | | | Add examples to GlobExpander#expand method | Minor | fs | Suresh Srinivas | Suresh Srinivas | | | TestNameNodeMetrics.testCorruptBlock fails intermittently | Major | test | Uma Maheswara Rao G | Jing Zhao | | | TestUnderReplicatedBlocks may fail due to ReplicaAlreadyExistsException | Major | test | Eli Collins | Jing Zhao | | | TestHDFSCLI.testAll fails one test due to number format | Minor | test | Trevor Robinson | Trevor Robinson | | | TestDFVariations fails on Windows | Major | fs | Chris Nauroth | Chris Nauroth | | | Bug in test-patch.sh and precommit build process | Critical | build | Kihwal Lee | Kihwal Lee | | | InodeTree.java has redundant check for vName while throwing exception | Major | fs | Arup Malakar | Arup Malakar | | | fix test org.apache.hadoop.hdfs.server.datanode.TestStartSecureDataNode | Major | . | Ivan A. Veselovsky | Ivan A. Veselovsky | | | MiniDFSCluster-based tests fail on Windows due to failure to delete test namenode directory | Major | test | Chris Nauroth | Chris Nauroth | | | TestNameNodeMetrics#testCorruptBlock is flaky | Major | namenode | Andrew Wang | Andrew Wang | | | improve script hadoop-common-project/hadoop-common/src/test/resources/kdc/killKdc.sh | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | | | test-patch.sh fails during mvn eclipse:eclipse if patch adds a new module to the build | Major | build | Chris Nauroth | Chris Nauroth | | | Duplicate surefire plugin config in hadoop-common | Major | test | Andrey Klochkov | Andrey Klochkov | | | hadoop-maven-plugins version-info goal causes build failure when running with Clover | Major | build | Chris Nauroth | Chris Nauroth | | | Typo in javadoc for IdentityMapper class | Trivial | documentation | Adam Monsen | Adam Monsen | | | GzipCodec NPE upon reset with IBM JDK | Major | io | Luke Lu | Yu Li | | | TestDataTransferKeepalive fails when tests are executed in a certain order. | Major | test | Konstantin Shvachko | Andrew Wang | | | TestRPC fails with JDK7 | Major | . | Andrew Wang | Andrew Wang | |

TestSecurityUtil#testLocalHostNameForNullOrWild on systems where hostname contains capital letters | Trivial | security, test | Chris Nauroth | Chris Nauroth | | | S3n filesystem can move a directory under itself -and so lose data | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3 blockstore filesystem breaks part of the Filesystem contract | Major | fs/s3 | Steve Loughran | Steve Loughran | | | TestLocalFileSystem#testHasFileDescriptor leaks file handle | Major | fs, test | Chris Nauroth | Chris Nauroth | | | TestYarnVersionInfo failing if generated properties doesn't include an SVN URL | Minor | api | Steve Loughran | Steve Loughran | | | Update FSDirectory#inodeMap when replacing an INodeDirectory while setting quota | Minor | namenode | Jing Zhao | Jing Zhao | | | Refresh INodeMap in FSDirectory#reset() | Minor | namenode | Jing Zhao | Jing Zhao | | | Remove JournalService | Major | namenode | Suresh Srinivas | Arpit Agarwal | | | test-patch.sh should report build failure to JIRA | Major | . | Binglin Chang | Binglin Chang | | | Block Replication Policy Implementation May Skip Higher-Priority Blocks for Lower-Priority Blocks | Major | . | Derek Dagit | Derek Dagit | | | test-patch gives +1 despite build failure when running tests | Critical | . | Jason Lowe | Jason Lowe | | | The SLS simulator is not running the preemption policy for CapacityScheduler | Minor | . | Carlo Curino | Carlo Curino | | | Fix the inconsistency of layout version number of ADD\DATANODE\AND\STORAGE\UUIDS between trunk and branch-2 | Minor | namenode | Jing Zhao | Jing Zhao | | | Task pings are interpreted as task progress | Major | . | Jason Lowe | Jason Lowe | | | improve jenkins javadoc warnings from test-patch.sh | Major | . | Colin P. McCabe | Colin P. McCabe | | | Task.calculateOutputSize does not handle Windows files after MAPREDUCE-5196 | Major | client | Remus Rusanu | Remus Rusanu | | | UserProvider is not thread safe | Major | security | Benoy Antony | Benoy Antony | | | yarn application help doesn't work | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | shell escapes are incompatible with previous releases | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Stop violence in the \*\_HOME | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | shell scripts ignore JAVA\_HOME on OS X | Major | scripts | Lei (Eddy) Xu | Allen Wittenauer | | | hadoop\verify\secure\_prereq's results aren't checked in bin/hdfs | Critical | scripts | Allen Wittenauer | Allen Wittenauer | | | crypto subcommand is not sorted properly in hdfs's hadoop\_usage | Trivial | scripts | Allen Wittenauer | David Luo | | | yarn logs command gives error on trunk | Minor | scripts | Prakash Ramachandran | Akira Ajisaka | | | non-daemon pid files are missing | Blocker | scripts | Allen Wittenauer | Allen Wittenauer | | | User replaced functions get lost 2-3 levels deep (e.g., sbin) | Critical | scripts | Allen Wittenauer | Allen Wittenauer | | | Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild | Major | applicationmaster, client, task | Gera Shegalov | Gera Shegalov | | | balancer doesn't run as a daemon | Blocker | scripts | Allen Wittenauer | Allen Wittenauer | | | Jenkins build seems to be broken by changes in test-patch.sh | Blocker | . | Zhijie Shen | Steve Loughran | | | processIncrementalBlockReport performance degradation | Critical | namenode | Daryn Sharp | Eric Payne | |

DataNode#transferBlock should report blocks that don't exist using reportBadBlock | Major | datanode, namenode | Yongjun Zhang | Yongjun Zhang | | | Snapshot read can reveal future bytes for appended files. | Major | hdfs-client | Konstantin Shvachko | Tao Luo | | | Remove dead link to HFTP documentation from index.xml | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | TestSecurityUtil fails when run with JDK8 because of empty principal names | Minor | security, test | Stephen Chu | Stephen Chu | | | Fix UGI#reloginFromKeytab on Java 8 | Major | security | Tobi Vollebregt | Stephen Chu | | | Fix variable name mismatches in hadoop-functions.sh | Minor | scripts | Masatake Iwasaki | Masatake Iwasaki | | | slaves.sh and stop-all.sh are missing slashes | Trivial | scripts | Allen Wittenauer | Allen Wittenauer | | | Application recovery throw InvalidStateTransitonException: Invalid event: ATTEMPT\_KILLED at ACCEPTED | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | mapred hsadmin command missing from trunk | Major | scripts | Jason Lowe | Allen Wittenauer | | | NM fail to start with NPE during container recovery | Major | nodemanager | zhihai xu | zhihai xu | | | RM web UI: app page will crash if app is failed before any attempt has been created | Major | webapp | Zhijie Shen | Wangda Tan | | | Remove stale block invalidation work when DN re-registers with different UUID | Major | namenode | Zhe Zhang | Zhe Zhang | | | Minor typo in privileged pid/out/log names | Minor | . | Allen Wittenauer | Allen Wittenauer | | | Application recovery continuously fails with "Application with id already present. Cannot duplicate" | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | NameNode block deletion logging uses incorrect appender. | Minor | namenode | Chris Nauroth | Chris Nauroth | | | Prevent the Namenode's LeaseManager from looping forever in checkLeases | Critical | hdfs-client, namenode | Zesheng Wu | Ravi Prakash | | | CapacitySchedulerPage shows HTML tags for a queue's Active Users | Major | capacityscheduler | Jason Lowe | Jason Lowe | | | Fix deadlock in DomainSocketWatcher when the notification pipe is full | Major | . | yunjiong zhao | yunjiong zhao | | | AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted | Blocker | . | Jason Lowe | Varun Saxena | | | When ACL's are enabled, if RM switches then application can not be viewed from web. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | hadoop-daemons.sh throws 'host1: bash: host3: command not found...' | Major | scripts | Vinayakumar B | Vinayakumar B | | | Dead lock in "DelegationTokenRenewer" which blocks RM to execute any further apps | Blocker | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Overflow is not properly handled in caclulating final iv for AES CTR | Blocker | security | Haifeng Chen | Haifeng Chen | | | [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments | Minor | . | Andrew Purtell | Andrew Purtell | | | Fix SSLFactory truststore reloader thread leak in KMSClientProvider | Major | kms | Arun Suresh | Arun Suresh | | | Incorrect locking

FsVolumeList#checkDirs can hang datanodes | Critical | datanode | Noah Lorang | Noah Lorang | | | FSLeafQueue can throw ConcurrentModificationException | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Namenode restart after large deletions can cause slow processReport (due to logging) | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Potential deadlock in AsyncDispatcher when system.exit called in AsyncDispatcher#dispatch and AsyscDispatcher#serviceStop from shutdown hook | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Update the NameNode's Group Cache in the background when possible | Minor | . | Chris Li | Chris Li | | | Restore Rack Awareness documentation | Major | . | Allen Wittenauer | Allen Wittenauer | | | Reducers do not validate checksum of map outputs when fetching directly to disk | Major | mrv2 | Eric Payne | Eric Payne | | | Can't override HADOOP\IDENT\STRING | Trivial | . | Allen Wittenauer | Kengo Seki | | | RM prematurely cancels tokens for jobs that submit jobs (oozie) | Blocker | resourcemanager | Daryn Sharp | Jian He | | | change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration | Major | datanode, test | Liang Xie | Liang Xie | | | Datanode upgrade to BLOCKID\BASED\LAYOUT fails if duplicate block files are present in the same volume | Blocker | . | Kihwal Lee | Colin P. McCabe | | | Incorrect version check in RMStateStore | Major | . | Jian He | Rohith Sharma K S | | | FSLeafQueue app lists are accessed without required locks | Blocker | . | Karthik Kambatla | Karthik Kambatla | | | NPE thrown when RM restart after queue is STOPPED. There after RM can not recovery application's and remain in standby | Critical | resourcemanager, scheduler | Nishan Shetty | Rohith Sharma K S | | | ZKRMStateStore crashes due to session expiry | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | | | ConcurrentModificationException in CapacityScheduler's LeafQueue | Major | capacityscheduler, resourcemanager, scheduler | Jason Tufo | Rohith Sharma K S | | | Common needs updates for shell rewrite | Major | documentation | Allen Wittenauer | Allen Wittenauer | | | ResourceManager crashes with NPE while getting queue info | Critical | . | Jason Tufo | Varun Saxena | | | NM keeps sending already-sent completed containers to RM until containers are removed from context | Major | nodemanager | Chengbing Liu | Chengbing Liu | | | YARN documentation needs updating post-shell rewrite | Major | documentation | Allen Wittenauer | Allen Wittenauer | | | NameNode should prune dead storages from storageMap | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Datanode sometimes does not shutdown on receiving upgrade shutdown command | Major | . | Kihwal Lee | Eric Payne | | | test-patch says "-1 overall" even when all checks are +1 | Major | build | Jason Lowe | Jason Lowe | | | SecondaryNameNode need twice memory when calling reloadFromImageFile | Major | namenode | yunjiong zhao | yunjiong zhao | | | maximum-am-resource-percent could be respected for both LeafQueue/User when trying to activate

| Critical | resourcemanager | Wangda Tan | Craig Welch | | | HDFS documentation needs updating post-shell rewrite | Major | documentation | Allen Wittenauer | Allen Wittenauer | | | The size of header buffer of HttpServer is too small when HTTPS is enabled | Major | security | Benoy Antony | Benoy Antony | | | Fix removal of dynamically added DN volumes | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | hadoop-mapreduce-client-nativetask fails to build on ARM AARCH64 due to x86 asm statements | Minor | native | Edward Nevill | Edward Nevill | | | Fix NPE when namenode transition to active during startup by adding checkNNStartup() in NameNodeRpcServer | Major | auto-failover, ha | suja s | Vinayakumar B | | | Upgrade should generate a unique storage ID for each volume | Critical | . | Lars Francke | Arpit Agarwal | | | Use correct UGI when KMSClientProvider is called by a proxy user | Major | . | Arun Suresh | Arun Suresh | | | Various typos and improvements to HDFS Federation doc | Minor | documentation | Charles Lamb | Charles Lamb | | | API docs don't include HDFS | Critical | documentation | Eli Collins | Jolly Chen | | | The appearance of hadoop-hdfs-httpfs site docs is inconsistent | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | LocalizerRunner should give DIE action when all resources are localized | Major | nodemanager | Chengbing Liu | Chengbing Liu | | | "mvn package -Pdist,docs -DskipTests -Dtar" fails because of non-ascii characters | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | AMRMClientImpl does not update AMRM token properly | Blocker | client | Jason Lowe | Jason Lowe | | | MR AM does not survive RM restart if RM activated a new AMRM secret key | Blocker | mr-am | Jason Lowe | Jason Lowe | | | Edit log corruption due to delayed block removal again | Major | namenode | Yongjun Zhang | Yongjun Zhang | | | NFS: readdir/readdirplus return null directory attribute on failure | Major | nfs | Arpit Agarwal | Arpit Agarwal | | | Multiple mappers with DBInputFormat don't work because of reusing conections | Major | mrv2 | Kannan Rajah | Kannan Rajah | | | HDFS Quota guide has typos, incomplete command lines | Major | documentation | Allen Wittenauer | Brahma Reddy Battula | | | HDFS Design Doc references commands that no longer exist. | Major | documentation | Allen Wittenauer | Brahma Reddy Battula | | | httpfs shell code has hardcoded path to bash | Major | scripts | Dmitry Sivachenko | Dmitry Sivachenko | | | reset timer for liveness monitors after RM recovery | Major | resourcemanager | Jun Gong | Jun Gong | | | MR lib dir contains jdiff (which is gpl) | Critical | build | Eli Collins | Nemon Lou | | | Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. | Major | datanode | Chris Nauroth | Vinayakumar B | | | Job History Link in RM UI is redirecting to the URL which contains Job Id twice | Major | webapp | Devaraj K | Devaraj K | | | Add links to RackAwareness and InterfaceClassification to site index | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Fix Multithreaded correctness Warnings #org.apache.hadoop.fs.shell.Ls | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Remove SingleNodeSetup.md from trunk | Major | documentation | Allen Wittenauer | Akira Ajisaka | | | Fix syntax error in

| Trivial | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Remove the version and author information from distcp's README file | Major | documentation | Allen Wittenauer | Brahma Reddy Battula | | | Fix formatting in Tracing.md | Trivial | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Fix various typos in ClusterSetup.md | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | dfs count -v should be added to quota documentation | Major | documentation | Allen Wittenauer | Akira Ajisaka | | | Wrong command mentioned in HDFSHighAvailabilityWithQJM documentation | Minor | documentation | Mohammad Arshad | Mohammad Arshad | | | TestHighRamJob fails due to the change in MAPREDUCE-5785 | Major | contrib/gridmix, mrv2 | Masatake Iwasaki | Masatake Iwasaki | | | RPC Server Reader thread can't shutdown if RPCCallQueue is full | Major | . | Ming Ma | Ming Ma | | | Post-2.6 namenode may not start up with an image containing inodes created with an old release. | Blocker | . | Kihwal Lee | Rushabh S Shah | | | RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. | Critical | net | Liang Xie | Chris Nauroth | | | Connection timeouts to nodemanagers are retried at multiple levels | Blocker | . | Jason Lowe | Jason Lowe | | | secondary filter matches entites which do not have the key being filtered for. | Major | timelineserver | Prakash Ramachandran | Zhijie Shen | | | Active NN and standby NN have different live nodes | Major | datanode | Ming Ma | Ming Ma | | | Minor fixes to command manual & SLA doc | Major | documentation | Allen Wittenauer | Allen Wittenauer | | | fix zkfc hung issue due to not catching exception in a corner case | Major | ha | Liang Xie | Liang Xie | | | WebAppProxy does not support a final tracking url which has query fragments and params | Major | . | Hitesh Shah | Jian He | | | bash location hard-coded in shell scripts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | TestJobConf#testNegativeValueForTaskVmem failures | Major | test | Gera Shegalov | Varun Saxena | | | Update KerberosTestUtils for hadoop-auth tests when using IBM Java | Major | security | sangamesh | sangamesh | | | TestBlocksWithNotEnoughRacks#testCorruptBlockRereplicatedAcrossRacks fails using IBM java | Major | test | Tian Hong Wang | Ayappan | | | NameNodeEditLogRoller can keep printing "Swallowing exception" message | Critical | . | Jing Zhao | Jing Zhao | | | RMNodeImpl#ReconnectNodeTransition should send scheduler events in sequential order | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | FairScheduler: Changing queueMaxRunningApps interferes with pending jobs | Critical | . | Siqi Li | Siqi Li | | | Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client | Critical | resourcemanager | zhihai xu | zhihai xu | | | oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static | Critical | io | Sean Busbey | Sean Busbey | | | Datanode should not trust the generation stamp provided by client | Critical | datanode | vitthal (Suhas) Gogate | Tsz Wo Nicholas Sze | | | Timeline renew delegation token fails when RM user's TGT is expired | Critical | . | Jonathan Eagles | Zhijie Shen | | | shellprofiles should require

extension | Major | scripts | Allen Wittenauer | Brahma Reddy Battula | | | TimelineClient kerberos authentication failure uses wrong login context. | Major | . | Jonathan Eagles | Daryn Sharp | | | hadoop-daemons.sh bw compat broke with --slaves change | Major | scripts | Vinayakumar B | Vinayakumar B | | | DataNode does not release the volume lock when adding a volume fails. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | git should ignore .DS\_Store files on Mac OS X | Major | . | Abin Shahab | Abin Shahab | | | \*Compressor\_deflateBytesDirect passes instance instead of jclass to GetStaticObjectField | Minor | native | Eric Abbott | Hui Zheng | | | Timelineserver applies the ACL rules after applying the limit on the number of records | Major | . | Prakash Ramachandran | Chang Li | | | The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error | Major | . | Colin P. McCabe | Colin P. McCabe | | | MiniDFSCluster shoud close all open FileSystems when shutdown() | Critical | test | haosdent | Rakesh R | | | Confusing log generated by CapacityScheduler | Minor | capacityscheduler | Hitesh Shah | Varun Saxena | | | CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | | | AM-Used Resource for leafqueue is wrongly populated if AM ResourceRequest is less than minimumAllocation | Major | scheduler | Rohith Sharma K S | Rohith Sharma K S | | | FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. | Major | resourcemanager | zhihai xu | zhihai xu | | | Print usage for TeraSort job. | Trivial | examples | Rohith Sharma K S | Rohith Sharma K S | | | AppMaster tracking URL is broken in HA | Major | webapp | Anubhav Dhoot | Anubhav Dhoot | | | inotify unable fetch pre-upgrade edit log segments once upgrade starts | Major | . | Zhe Zhang | Zhe Zhang | | | Edit log corruption can happen if append fails with a quota violation | Blocker | namenode | Kihwal Lee | Jing Zhao | | | Update action param from "start" to "prepare" in rolling upgrade javadoc | Trivial | . | J.Andreina | J.Andreina | | | Fix javadoc in Terasort | Trivial | . | Albert Chu | Albert Chu | | | commitBlockSynchronization() does not remove locations | Blocker | namenode | Konstantin Shvachko | Yi Liu | | | Missing NullPointer check in AppSchedulingInfo causes RM to die | Blocker | resourcemanager | Giovanni Matteo Fumarola | Brahma Reddy Battula | | | Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path | Major | . | Xuan Gong | Xuan Gong | | | NullPointerException caused by job history server addr not resolvable | Minor | applicationmaster | Peng Zhang | Peng Zhang | |

MapFileOutputFormat#getReaders bug with invisible files/folders | Minor | mrv2 | Maysam Yabandeh | Maysam Yabandeh | | | Progress report log is incredibly excessive in application master | Major | applicationmaster | Jian Fang | Varun Saxena | | | Getting application(s) goes wrong when app finishes before starting the attempt | Critical | timelineserver | Zhijie Shen | Zhijie Shen | | | NullPointerException in BlockSender | Blocker | datanode | Tsz Wo Nicholas Sze | Brahma Reddy Battula | | | The full block report should prune zombie storages even if they're not empty | Critical | . | Lei (Eddy) Xu | Colin P. McCabe | | | When configuring "dfs.namenode.safemode.threshold-pct" to a value greater or equal to 1 there is mismatch in the UI report | Minor | . | J.Andreina | J.Andreina | | | AdminService should use "warn" instead of "info" to log exception when operation fails | Major | resourcemanager | Wangda Tan | Li Lu | | | yarn rmadmin should skip -failover | Minor | resourcemanager | J.Andreina | J.Andreina | | | DistCp throws NPE when the target directory is root. | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | [JDK 8] Build Failure due to unreported exceptions in RPCUtil | Major | . | Robert Kanter | Robert Kanter | | | haadmin command usage #HDFSHighAvailabilityWithQJM.html | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | IBR delete ack should not be delayed | Major | namenode | Daryn Sharp | Daryn Sharp | | | Fix typo of javadoc in DistCp | Trivial | . | Chen He | Brahma Reddy Battula | | | Remove an extra parameter described in Javadoc of TockenCache | Trivial | . | Chen He | Brahma Reddy Battula | | | TransactionsSinceLastCheckpoint can be negative on SBNs | Major | namenode | Harsh J | Gautam Gopalakrishnan | | | favoring decommissioning node for replication can cause a block to stay underreplicated for long periods | Major | namenode | Nathan Roberts | Nathan Roberts | | | Website refers to /trash directory | Major | documentation | Mike Drob | Brahma Reddy Battula | | | storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState() | Major | . | Ted Yu | Brahma Reddy Battula | | | The first non-existing xattr should also throw IOException | Minor | . | zhouyingchao | zhouyingchao | | | BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed | Critical | namenode | Akira Ajisaka | Akira Ajisaka | | | NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed | Minor | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors | Major | . | Rakesh R | Rakesh R | | | Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk | Minor | . | Colin P. McCabe | Colin P. McCabe | | | Read timeout when retrying a fetch error can be fatal to a reducer | Blocker | . | Jason Lowe | Jason Lowe | | | Non-AM containers can be counted towards amResourceUsage of a Fair Scheduler queue | Critical | fairscheduler | Rohit Agarwal | zhihai xu | | | releasedocmaker.py needs to put ASF headers on output | Major | build | Allen Wittenauer | Allen Wittenauer | | | Clean up some test methods in TestCodec.java | Major | test | Akira Ajisaka | Brahma Reddy Battula | | | Counters / AbstractCounters have constant references not declared final | Major | . | Gera Shegalov | Brahma Reddy Battula | | | AM container to be allocated Appattempt AM container shown as null | Trivial | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time | Major | . | zhouyingchao | zhouyingchao | | |

fails Intermittently with error message:Invalid AMRMToken | Major | test | zhihai xu | zhihai xu | | | dfs.datanode.du.reserved is not considered while computing available space | Major | datanode | Brahma Reddy Battula | Brahma Reddy Battula | | | NPE when NodeManager.serviceInit fails and stopRecoveryStore called | Minor | nodemanager | Bibin A Chundatt | Bibin A Chundatt | | | Fix failiure of TestLog4jWarningErrorMetricsAppender | Blocker | . | Li Lu | Varun Vasudev | | | Reserved RBW space is not released if client terminates while writing block | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Allow better control of getContentSummary | Major | . | Kihwal Lee | Kihwal Lee | | | MiniYarnCluster should turn on timeline service if configured to do so | Major | . | Mit Desai | Mit Desai | | | Incorrect "nodes in service" metrics caused all writes to fail | Major | . | Ming Ma | Ming Ma | | | DatanodeMetrics#blocksReplicated will get incremented early and even for failed transfers | Major | datanode | Vinayakumar B | Vinayakumar B | | | Use LinkedHashMap to preserve order of resource requests | Major | nodemanager | zhihai xu | zhihai xu | | | Change "DFSInputStream has been closed already" message to debug log level | Minor | hdfs-client | Charles Lamb | Charles Lamb | | | The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer | Blocker | security | Yi Liu | Daryn Sharp | | | ACLStatus and XAttributes not properly presented to INodeAttributesProvider before returning to client | Major | namenode | Arun Suresh | Arun Suresh | | | Job#getTrackingURL should consistently return a proper URL | Minor | . | Ray Chiang | Ray Chiang | | | Split getAdditionalBlock() into two methods. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Two fsimage\rollback\\* files are created which are not deleted after rollback. | Critical | . | J.Andreina | J.Andreina | | | TestPipeApplication fails on trunk | Major | test | Varun Vasudev | Varun Vasudev | | | Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column | Major | resourcemanager, webapp | Jason Lowe | Jason Lowe | | | DistributedFIleSystem should not look for keyProvider in cache if Encryption is disabled | Minor | hdfs-client | Arun Suresh | Arun Suresh | | | NPE thrown when invalid FSImage filename given for "hdfs oiv\_legacy" cmd | Minor | tools | Archana T | Surendra Singh Lilhore | | | Fix typos in hadoop-project/pom.xml and TestAccessControlList | Minor | . | Chen He | Brahma Reddy Battula | | | Some of UserMetricsInfo metrics are incorrectly set to root queue metrics | Major | webapp | Rohit Agarwal | Rohit Agarwal | | | Possible leak in DelegationTokenRenewer#allTokens | Major | . | Jian He | Rohith Sharma K S | | | WebApplication proxy documentation is incomplete | Minor | resourcemanager | Bibin A Chundatt | Naganarasimha G R | | | HttpServerFunctionalTest#prepareTestWebapp should create web app directory if it does not exist. | Minor | . | Rohith Sharma K S | Rohith Sharma K S | | | Abort NameNode and DataNode startup if security is enabled but block access token is not enabled. | Minor | datanode, namenode, security | Chris Nauroth | Vijay Bhat | | | NullPointerException when topology script is

| Major | namenode | Anu Engineer | Anu Engineer | | | RMContext inactiveNodes should have NodeId as map key | Major | resourcemanager | Chengbing Liu | Chengbing Liu | | | Fix URIs in documention of YARN web service REST APIs | Minor | documentation, resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | NameNode Failover during HA upgrade can cause DataNode to finalize upgrade | Blocker | ha | Jing Zhao | Jing Zhao | | | Always use snapshot path as source when invalid snapshot names are used for diff based distcp | Minor | distcp | Sushmitha Sreenivasan | Jing Zhao | | | Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN | Critical | documentation | J.Andreina | J.Andreina | | | The footer of the Web UI "Hadoop, 2014" is old | Major | . | Akira Ajisaka | Brahma Reddy Battula | | | DistributedFileSystem encryption zone commands should resolve relative paths | Major | . | Rakesh R | Rakesh R | | | Task list sort by task id broken | Minor | . | Siqi Li | Siqi Li | | | YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp | Major | security | Harsh J | Yongjun Zhang | | | Error Message points to wrong parent directory in case of path component name length error | Major | namenode | Anu Engineer | Anu Engineer | | | RM fails to come up with error "Failed to load/recover state" when mem settings are changed | Critical | yarn | Sumana Sathish | Jian He | | | NPE in MiniDFSCluster teardown | Major | test | Steve Loughran | Brahma Reddy Battula | | | NPE thrown at DataNode shutdown when HTTP server was not able to create | Minor | datanode | Archana T | Surendra Singh Lilhore | | | ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy | Major | client | Jason Lowe | Jason Lowe | | | MR2 can't run local jobs with -libjars command options which is a regression from MR1 | Critical | mrv2 | zhihai xu | zhihai xu | | | DFSClient#getServerDefaults returns null within 1 hour of system start | Blocker | . | Xiaoyu Yao | Xiaoyu Yao | | | Provide each Replica details in fsck | Major | . | Ming Ma | J.Andreina | | | Using monotonicNow for block report scheduling causes test failures on recently restarted systems | Blocker | datanode | Arpit Agarwal | Arpit Agarwal | | | DelegationTokenAuthenticationFilter must pass ipaddress instead of hostname to ProxyUsers#authorize() | Major | . | Anubhav Dhoot | Anubhav Dhoot | | | Confusing log generated by FairScheduler | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Set job classloader on uber-job's LocalContainerLauncher event thread | Major | mr-am | Sangjin Lee | Sangjin Lee | | | TestCertificateUtil.testCorruptPEM failing on Jenkins JDK8 | Major | build, security | Steve Loughran | Larry McCay | | | PseudoAuthenticationHandler fails with httpcomponents v4.4 | Major | . | Eugene Koifman | Eugene Koifman | | | Fix typo in Task Attempt API's URL in documentations | Minor | documentation | Ryu Kobayashi | Ryu Kobayashi | | | Incorrect arguments to sizeof in DomainSocket.c | Major | native | Malcolm Kavalsky | Malcolm Kavalsky | | | Invalid user logins trigger large backtraces in server log | Major | . | Chang Li | Chang Li | | |

rewrite addendum patch | Major | build | Anu Engineer | Allen Wittenauer | | | JWTRedirectAuthenticationHandler breaks java8 javadocs | Major | build | Steve Loughran | Larry McCay | | | Allow HA NN to start in secure mode with wildcard address configured | Major | ha, namenode, security | Aaron T. Myers | Aaron T. Myers | | | DistributedShell uses the wrong user to put timeline data | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | | | Mover should not schedule two replicas to the same DN storage | Major | balancer & mover | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Regression: s3n read failure recovery broken | Major | fs/s3 | Takenori Sato | Takenori Sato | | | killing ContainerLocalizer action doesn't take effect when private localizer receives FETCH\_FAILURE status. | Minor | nodemanager | zhihai xu | zhihai xu | | | DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm | Major | . | Eric Payne | Colin P. McCabe | | | Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode | Blocker | caching | Gopal V | Colin P. McCabe | | | During block recovery for truncate Log new Block Id in case of copy-on-truncate is true. | Major | datanode | Vinayakumar B | Vinayakumar B | | | StackTrace displayed at client while QuotaByStorageType exceeds | Major | hdfs-client | J.Andreina | J.Andreina | | | Remove unsupported 'hdfs namenode -rollingUpgrade downgrade' from document | Minor | documentation | J.Andreina | J.Andreina | | | Fix byte to integer casting in SimulatedFSDataset#simulatedByte | Minor | . | Zhe Zhang | Zhe Zhang | | | Previous AM's container complete message couldn't pass to current am if am restarted and rm changed | Critical | resourcemanager | sandflee | sandflee | | | Refactor code to make it more readable, minor maybePrintStats bug | Trivial | tools/distcp | Zoran Dimitrijevic | Zoran Dimitrijevic | | | Fix typo capabililty | Trivial | applications/distributed-shell | Gabor Liptak | Gabor Liptak | | | NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked | Major | nodemanager | Brahma Reddy Battula | Brahma Reddy Battula | | | DataNode UUID is always null in the JMX counter | Major | datanode | Anu Engineer | Anu Engineer | | | TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297 | Major | . | Siqi Li | Siqi Li | | | synthetic load generator docs give incorrect/incomplete commands | Major | documentation | Allen Wittenauer | Brahma Reddy Battula | | | Fix the typos in hadoop-hdfs-httpfs | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Race condition in LocalizerRunner kills localizer before localizing all resources | Critical | nodemanager | zhihai xu | zhihai xu | | | Incorrect path mentioned in document for accessing script files | Major | documentation | J.Andreina | J.Andreina | | | JobHistoryServer should not fail when encountering a missing directory | Major | jobhistoryserver | Craig Welch | Craig Welch | | | ATS throws exception on trying to filter results without otherinfo. | Critical | timelineserver | Sreenath Somarajapuram | Zhijie Shen | | | CommandFormat#parse() should not parse option as value of option | Blocker |

| Peter Shi | Peter Shi | | | [JDK8] AuthenticationFilter, CertificateUtil, SignerSecretProviders, KeyAuthorizationKeyProvider Javadoc issues | Major | build | Robert Kanter | Robert Kanter | | | Uber jobs fail to update AMRM token when it rolls over | Blocker | mr-am | Jason Lowe | Jason Lowe | | | Missing datanode counters when using Metrics2 sink interface | Major | datanode | Anu Engineer | Anu Engineer | | | Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler | Blocker | . | Eric Payne | Eric Payne | | | JobConf.parseMaximumHeapSizeMB() fails to parse value greater than 2GB expressed in bytes | Major | . | Hao Xia | Hao Xia | | | FSNamesystem#Delete() should not call logSync() when holding the lock | Blocker | namenode | Jing Zhao | Haohui Mai | | | FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies | Critical | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime | Blocker | . | Yesha Vora | Haohui Mai | | | RM web ui for dumping scheduler logs should be for admins only | Blocker | resourcemanager, security | Varun Vasudev | Varun Vasudev | | | Secondary NN Web UI shows wrong date for Last Checkpoint | Major | namenode | Charles Lamb | Charles Lamb | | | Test: Fix launchAM in MockRM to wait for attempt to be scheduled | Major | yarn | Anubhav Dhoot | Anubhav Dhoot | | | Job history file is not flushed correctly because isTimerActive flag is not set true when flushTimerTask is scheduled. | Critical | mrv2 | zhihai xu | zhihai xu | | | Fix findbugs warnings in hadoop-sls | Major | tools | Akira Ajisaka | Brahma Reddy Battula | | | Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly | Major | . | Jian He | Jian He | | | OsSecureRandom should lazily fill its reservoir | Major | security | Arun Suresh | Arun Suresh | | | increase readability and reliability of checkstyle, shellcheck, and whitespace reports | Minor | . | Naganarasimha G R | Allen Wittenauer | | | Move conditional in fmt\_time from dfs-dust.js to status.html | Minor | namenode | Charles Lamb | Charles Lamb | | | Fix unit test failures and findbugs warning caused by HDFS-8283 | Major | . | Jing Zhao | Jing Zhao | | | LazyPersistFileScrubber should be disabled if scrubber interval configured zero | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Make checkstyle runnable from root project | Major | build, test | Gera Shegalov | Gera Shegalov | | | DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace | Critical | . | Billie Rinaldi | Colin P. McCabe | | | LAZY\_PERSIST file gets deleted after NameNode restart. | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream | Major | resourcemanager | Gera Shegalov | zhihai xu | | | HarFs incorrectly declared as requiring an authority | Critical | fs | Gera Shegalov | Brahma Reddy Battula | | | Add failIfNoTests=false to hadoop-build-tools pom | Major | test | Gera Shegalov | Gera Shegalov | | | Documentation fix for when CRLA is enabled for MRAppMaster logs | Trivial | documentation | Rohit Agarwal | Rohit Agarwal | | | Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. | Major | . | Xu Yang | Xu Yang | | | Cross-site scripting vulnerability in TextView.java | Major | webapp | Ted Yu | Kenji Kikushima | | |

outputs unnecessary "null" strings | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Fix typo in property org.apache.hadoop.mapreduce.lib.chain.Chain.REDUCER\INPUT\VALUE\_CLASS | Minor | . | Ray Chiang | Ray Chiang | | | SnappyCodec#checkNativeCodeLoaded may unexpectedly fail when native code is not loaded | Major | . | Zhijie Shen | Zhijie Shen | | | Logging of resource recovery on NM restart has redundancies | Minor | nodemanager | Jason Lowe | Eric Payne | | | WebHDFS calls before namesystem initialization can cause NullPointerException. | Minor | webhdfs | Chris Nauroth | Chris Nauroth | | | Reduce cannot use more than 2G memory for the final merge | Major | mrv2 | stanley shi | Gera Shegalov | | | IllegalArgumentException due to missing job submit time | Major | jobhistoryserver | zhihai xu | zhihai xu | | | NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner | Minor | nodemanager | Devaraj K | Devaraj K | | | Adding test cases of retrying requests about ZKRMStateStore | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | [JDK8] TestCombineFileInputFormat failed on JDK8 | Minor | . | Wei Yan | Akira Ajisaka | | | ZKFailoverController does not log Exception when doRun raises errors | Major | ha | Tianyin Xu | Tianyin Xu | | | TestStringUtils#testLowerAndUpperStrings failed on MAC due to a JVM bug | Minor | . | Ming Ma | Ming Ma | | | RM Web UI shows -1 running containers for completed apps | Trivial | webapp | Rohith Sharma K S | Rohith Sharma K S | | | hadoop fs -rmr gives wrong advice | Major | . | Allen Wittenauer | Juliet Hougland | | | Handle URISyntaxException in ResourceLocalizationService | Major | nodemanager | Chengbing Liu | Brahma Reddy Battula | | | Progress bars in Web UI always at 100% due to non-US locale | Major | webapp | Johannes Simon | Akira Ajisaka | | | HDFS INotify: the destination field of RenameOp should always end with the file name | Major | . | Colin P. McCabe | Colin P. McCabe | | | test-patch.sh should work with ${BASEDIR}/patchprocess setups | Blocker | . | Allen Wittenauer | Allen Wittenauer | | | Modify NNThroughputBenchmark to be able to operate on a remote NameNode | Major | . | Colin P. McCabe | Charles Lamb | | | setStoragePolicy with folder behavior is different after cluster restart | Major | . | Peter Shi | Surendra Singh Lilhore | | | test-patch.sh mv does wrong math | Major | . | Allen Wittenauer | Allen Wittenauer | | | Extra configuration key used in TraceUtils should respect prefix | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | NPE in WebAppProxyServlet | Major | . | Jian He | Jian He | | | checkLease should throw FileNotFoundException when file does not exist | Major | namenode | Konstantin Shvachko | Rakesh R | | | Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion. | Critical | resourcemanager | zhihai xu | zhihai xu | | | DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. | Major | datanode | Chris Nauroth | Lei (Eddy) Xu | | | Misspelling of threshold in log4j.properties for tests | Minor | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Misspelling of threshold in

for tests | Minor | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Misspelling of threshold in log4j.properties for tests in hadoop-common-project | Minor | conf, test | Kenji Kikushima | Brahma Reddy Battula | | | Misspelling of threshold in log4j.properties for tests | Minor | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Cleanup ResourceManagerAdministrationProtocol interface audience | Major | client, resourcemanager | Wangda Tan | Naganarasimha G R | | | Incremental BlockReport will dramatically slow down the startup of a namenode | Major | . | Hui Zheng | Walter Su | | | Dockerfile references a removed image | Major | . | Allen Wittenauer | Allen Wittenauer | | | [Log mesage correction] : MIssing space in Diagnostics message | Trivial | . | nijel | nijel | | | CacheDirectives and CachePool operations should throw RetriableException in safemode | Major | . | Haohui Mai | Haohui Mai | | | CheckAccess in WebHDFS silently accepts malformed FsActions parameters | Minor | webhdfs | Jake Low | Walter Su | | | Fix wrong MockLocalizerStatus#equals implementation | Minor | nodemanager | Hong Zhiguo | Hong Zhiguo | | | Correct typos in WritingYarnApplications.md | Minor | documentation | Sandeep Khurana | Gabor Liptak | | | Misspelling of threshold in log4j.properties for tests in hadoop-tools | Minor | . | Brahma Reddy Battula | Gabor Liptak | | | Namenode rollingUpgrade option is incorrect in document | Major | documentation | J.Andreina | J.Andreina | | | haadmin prints out stale help messages | Minor | hdfs-client | Ajith S | Ajith S | | | Fix typos in RMNodeLabelsManager | Trivial | resourcemanager | Junping Du | Sunil Govindan | | | Update replication count to live rep count in fsck report | Minor | . | J.Andreina | J.Andreina | | | FSImage may be left unclosed in BootstrapStandby#doRun() | Minor | ha | Ted Yu | Sanghyun Yun | | | Audit log not present while refreshing Service ACLs | Minor | resourcemanager | Varun Saxena | Varun Saxena | | | Remove .xml and documentation references to dfs.webhdfs.enabled | Minor | . | Ray Chiang | Ray Chiang | | | RM and AH web UI display DOCTYPE wrongly | Major | webapp | Rohith Sharma K S | Rohith Sharma K S | | | HDFS Federation : Command mentioned to add a NN to existing federated cluster is wrong | Minor | documentation | Ajith S | Ajith S | | | Remove usage of "dfsadmin -upgradeProgress " from document which is no longer supported | Major | documentation | J.Andreina | J.Andreina | | | Remove usage of "-setStoragePolicy" and "-getStoragePolicy" using dfsadmin cmd (as it is not been supported) | Major | documentation | J.Andreina | J.Andreina | | | Provide information on snapshotDiff for supporting the comparison between snapshot and current status | Major | documentation | J.Andreina |

| | | Make POM project names consistent | Minor | build | Rohith Sharma K S | Rohith Sharma K S | | | Datanode log is generating at root directory in security mode | Minor | datanode, scripts | Surendra Singh Lilhore | Surendra Singh Lilhore | | | SnappyDecompressor's Logger class name is wrong | Major | conf | Surendra Singh Lilhore | Surendra Singh Lilhore | | | DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery | Major | hdfs-client | Brahma Reddy Battula | Uma Maheswara Rao G | | | Default value for maximum nodemanager connect wait time is too high | Major | . | Jason Lowe | Naganarasimha G R | | | Rolling upgrade readiness is not updated in jmx until query command is issued. | Critical | . | Kihwal Lee | Brahma Reddy Battula | | | AM container link is broken (on a killed application, at least) | Major | . | Sergey Shelukhin | Naganarasimha G R | | | libwebhdfs build fails during link due to unresolved external symbols. | Major | native | Chris Nauroth | Chris Nauroth | | | NFS configuration nfs.dump.dir not working | Major | nfs | Ajith S | Ajith S | | | Fix NFS documentation of nfs.wtmax | Minor | documentation, nfs | Ajith S | Ajith S | | | Corrections in winutils/chmod.c | Trivial | bin | Ren Nyffenegger | Ren Nyffenegger | | | listLocatedStatus(path, filter) is not redefined in FilterFs | Major | fs | Hairong Kuang | Hairong Kuang | | | Duplicate job.getOutputValueGroupingComparator() in ReduceTask | Major | . | Binglin Chang | Binglin Chang | | | DataStreamer.transfer() should timeout the socket InputStream. | Major | hdfs-client | Esteban Gutierrez | Esteban Gutierrez | | | Add check for null BlockCollection pointers in BlockInfoContiguous structures | Major | namenode | Chengbing Liu | Chengbing Liu | | | The example code of org.apache.hadoop.util.Tool is incorrect | Major | util | hellojinjie | hellojinjie | | | LineRecordReader should not seek into non-splittable, compressed streams. | Major | task | Niels Basjes | Niels Basjes | | | Add links to SLGUserGuide to site index | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Standby namenode doesn't process DELETED\_BLOCK if the add block request is in edit log. | Major | . | Rushabh S Shah | Rushabh S Shah | | | Unify the default value for yarn.scheduler.capacity.node-locality-delay in code and default xml file | Trivial | capacityscheduler | nijel | nijel | | | Documentation about when checkpoints are run is out of date | Major | documentation | Misty Linville | Misty Linville | | | removeApplication in ZKRMStateStore should also disable watch. | Minor | resourcemanager | zhihai xu | zhihai xu | | | Nodemanager can fail to delete local logs if log aggregation fails | Major | log-aggregation, nodemanager | Jason Lowe | Rohith Sharma K S | | | Fix RM Web UI configuration for some properties | Minor | resourcemanager | Ray Chiang | Ray Chiang | | | TestFileTruncate is failing intermittently | Major | test | Rakesh R | Rakesh R | | | Fix Javadoc typos in hadoop-openstack module | Trivial | documentation | Chen He | Yanjun Wang | | | ResourceLocalizer started without any jvm memory control | Major | nodemanager | stanley shi | Masatake Iwasaki | | | RM HA setup, "Cluster" tab links populated with AM hostname instead of RM | Minor | . | Aroop Maliakkal | yunjiong zhao | | | Divide by zero error in MR AM when calculating available containers | Major | mr-am | Anubhav Dhoot | Anubhav Dhoot | | | FairScheduler: Trim whitespaces when using username for queuename | Major | fairscheduler | zhihai xu | zhihai xu | | | Remove namenode -finalize option from document | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Fix the javadoc of DelegationTokenSecretManager in projects of yarn,

| Minor | documentation | Akira Ajisaka | Gabor Liptak | | | Remove description about Java 6 from docs | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Test-patch check for @author tags incorrectly flags removal of @author tags | Major | . | Sean Busbey | Kengo Seki | | | test-patch should give better info about failures to handle dev-support updates without resetrepo option | Minor | . | Sean Busbey | Sean Busbey | | | test-patch should return early from determine-issue when run in jenkins mode. | Minor | . | Sean Busbey | Sean Busbey | | | 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop | Critical | datanode | Vinayakumar B | Rushabh S Shah | | | Interaction between reservations and userlimit can result in significant ULF violation | Major | capacityscheduler | Thomas Graves | Thomas Graves | | | Java Compilation Error in TestHdfsConfigFields.java | Major | . | Mohammad Arshad | Mohammad Arshad | | | TestMapreduceConfigFields is placed in wrong dir, introducing compile error | Major | . | Vinayakumar B | Mohammad Arshad | | | NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host | Critical | . | Junping Du | Junping Du | | | NodeID is always printed as "null" in node manager initialization log. | Major | . | nijel | nijel | | | JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases | Major | jobhistoryserver, mrv2 | Craig Welch | Craig Welch | | | mapreduce.terasort.final.sync configuration in TeraSort doesn't work | Trivial | examples | Takuya Fukudome | Takuya Fukudome | | | Prevent multiple balancers from running simultaneously | Critical | balancer & mover | Rakesh R | Rakesh R | | | TestTraceAdmin fails | Major | . | Kihwal Lee | Masatake Iwasaki | | | Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. | Critical | scripts | Chris Nauroth | Chris Nauroth | | | Always call addStoredBlock on blocks which have been shifted from one storage to another | Major | . | Colin P. McCabe | Colin P. McCabe | | | Remove confusing comment in Path#isAbsolute() | Trivial | fs | Suresh Srinivas | Suresh Srinivas | | | Make getFileChecksum fail for blocks under construction | Critical | . | Kihwal Lee | J.Andreina | | | Duplicate String.format in YarnOutputFiles.getSpillFileForWrite | Minor | . | Konstantin Weitz | Konstantin Weitz | | | check if sysconf is implemented before using it | Major | nodemanager | Radim Kolar | Radim Kolar | | | Fix test failure in TestHdfsConfigFields for spanreceiver properties | Major |

| Ray Chiang | Ray Chiang | | | HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE\_FAILED state | Minor | jobhistoryserver | zhihai xu | zhihai xu | | | RM still allocates containers to an app in the FINISHING state | Major | scheduler | Thomas Graves | Chang Li | | | ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster | Major | resourcemanager, webapp | Weiwei Yang | Weiwei Yang | | | Fix typo in the document for hadoop fs -find | Trivial | documentation | Akira Ajisaka | Kengo Seki | | | Fix the test case for copying to non-existent dir in TestFsShellCopy | Minor | fs | Kousuke Saruta | Kousuke Saruta | | | Fix the test failures in HTTPFS: In some tests setReplication called after fs close. | Major | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Storage policy APIs must be exposed via the FileSystem interface | Major | hdfs-client | Arpit Agarwal | Arpit Agarwal | | | Fix a typo in NamenodeFsck | Minor | namenode | Tsz Wo Nicholas Sze | Takanobu Asanuma | | | SecondaryNameNode not terminating properly on runtime exceptions | Major | namenode | Rakesh R | Rakesh R | | | Fix UT TestRMFailover.testRMWebAppRedirect | Critical | resourcemanager, webapp | Weiwei Yang | Weiwei Yang | | | Pending block replication can get stuck using older genstamp | Major | namenode | Nathan Roberts | Nathan Roberts | | | Ensure ZkDelegationTokenSecretManager namespace znodes get created with ACLs | Major | security | Gregory Chanan | Gregory Chanan | | | Metrics documentation for FSNamesystem misspells PendingDataNodeMessageCount. | Trivial | documentation | Chris Nauroth | Anu Engineer | | | Distributed shell app master becomes unresponsive sometimes | Major | applications/distributed-shell | Varun Vasudev | Varun Vasudev | | | Fix findbugs warnings in yarn-server-resourcemanager | Minor | resourcemanager | Akira Ajisaka | Vinod Kumar Vavilapalli | | | yarn cmd says "could not find main class 'queue'" in windows | Blocker | yarn | Sumana Sathish | Varun Saxena | | | ContainerLogsPage web UI should not have meta-refresh | Major | yarn | Xuan Gong | Xuan Gong | | | Fix dead link for TimelineServer REST API | Minor | documentation | Akira Ajisaka | Jagadesh Kiran N | | | Applications are getting stuck some times in case of retry policy forever | Major | client | Raju Bairishetti | Raju Bairishetti | | | Move startFile() and related operations into FSDirWriteFileOp | Major | . | Haohui Mai | Haohui Mai | | | DFSClient probe for encryption testing interprets empty URI property for "enabled" | Blocker | encryption | Steve Loughran | Steve Loughran | | | FairScheduler: RM quits when node removal races with continousscheduling on the same node | Critical | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | Typo in FileSystem:: listStatusIterator | Trivial | fs | Jakob Homan | Arthur Vigil | | | hadoop-config.cmd displays a wrong error message | Minor | scripts | Kengo Seki | Kengo Seki | | | Fix a typo in the cluster setup doc | Trivial | . | Kihwal Lee | Yanjun Wang | | | Port conflict log for data node server is not sufficient | Minor | datanode | Mohammad Shahid Khan | Mohammad Shahid Khan | | | WintuilsProcessStubExecutor.startStreamReader leaks streams | Trivial | nodemanager | Steve Loughran | Lars Francke | | | maven doesn't clean all the site files | Minor | documentation | Allen Wittenauer | ramtin | | | Fix "undefined reference to dlopen" error when compiling libhadooppipes | Major | build, native, tools | Xianyin Xin | Xianyin Xin | | | Isolating the error of generating a single app report when getting all apps from generic history service | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | | | RM Web UI queue filter doesn't work | Blocker | . | Wangda Tan | Wangda Tan | | | filtering on UI sticks even if I move away from the page | Major | webapp | Sangjin Lee | Jian He | | | NPE in

when Token is constructed using null identifier | Minor | security | Vlad Rozov | Kanaka Kumar Avvaru | | | ThreadLocal initialization in several classes is not thread safe | Critical | io | Sean Busbey | Sean Busbey | | | On Windows localized resources are not moved to the front of the classpath when they should be | Major | yarn | Craig Welch | Craig Welch | | | CLIMiniCluster instructions fail with MiniYarnCluster ClassNotFoundException | Minor | documentation | Steve Loughran | Darrell Taylor | | | hdfs crypto class not found in Windows | Critical | scripts | Sumana Sathish | Anu Engineer | | | test-patch breaks with reexec in certain situations | Critical | . | Allen Wittenauer | Sean Busbey | | | Need to clearly document primaryFilter and otherInfo value type | Critical | timelineserver | Zhijie Shen | Zhijie Shen | | | xargs -P is not portable | Critical | scripts | Allen Wittenauer | Kengo Seki | | | HADOOP\USER\CLASSPATH\_FIRST works the opposite of what it is supposed to do | Major | scripts | Sangjin Lee | Sangjin Lee | | | libhdfs hdfsListDirectory must set errno to 0 on success | Major | native | Juan Yu | Masatake Iwasaki | | | Avoid stuck threads if there is an error in DomainSocketWatcher that stops the thread | Major | . | zhouyingchao | zhouyingchao | | | WASB should configure client side socket timeout in storage client blob request options | Major | tools | Ivan Mitic | Ivan Mitic | | | Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop | Blocker | security | Mike Yoder | Larry McCay | | | Add block info to DFSInputStream' WARN message when it adds node to deadNodes | Minor | . | Ming Ma | Mohammad Arshad | | | Users may see TrashPolicy if hdfs dfs -rm is run | Major | . | Allen Wittenauer | J.Andreina | | | Avoid retry cache collision when Standby NameNode loading edits | Critical | namenode | Carrey Zhan | Ming Ma | | | hadoop-dist dist-layout-stitching.sh does not work with dash | Major | build | Andrew Wang | Andrew Wang | | | App submission via REST API is broken in secure mode due to Timeline DT service address is empty | Blocker | resourcemanager, timelineserver | Zhijie Shen | Zhijie Shen | | | remove dependency of zookeeper for hadoop-client | Major | . | Vinayakumar B | Vinayakumar B | | | Fix wrong classname in example configuration of hadoop-auth documentation | Trivial | documentation | Masatake Iwasaki | Masatake Iwasaki | | | fsck "-storagepolicies , -blockId ,-replicaDetails " options are missed out in usage and from documentation | Major | documentation | J.Andreina | J.Andreina | | | DN startup may cause severe data loss | Blocker | datanode | Daryn Sharp | Daryn Sharp | | | Improve synchronization of 'streamer' reference in DFSOutputStream | Major | hdfs-client | Rakesh R | Rakesh R | | | Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant | Minor | namenode | Andrew Wang | Andrew Wang | | | test-patch.sh isn't re-executed even if smart-apply-patch.sh is modified | Major | test | Kengo Seki | Kengo Seki | | | create() always retried with hardcoded timeout when file already exists with open lease | Major | hdfs-client | Andrey Stepachev | J.Andreina | | | fsimage loading progress should update inode, delegation token and cache pool

| Minor | namenode | tongshiquan | Surendra Singh Lilhore | | | Remove usage information on unsupported operation "fsck -showprogress" from branch-2 | Major | documentation | J.Andreina | J.Andreina | | | Purger should remove stale fsimage ckpt files | Minor | namenode | suja s | J.Andreina | | | TestAHSWebServices fails after YARN-3467 | Major | . | Zhijie Shen | Sunil Govindan | | | NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled | Critical | . | Peng Zhang | Rohith Sharma K S | | | Distributed Cache File visibility should check permission of full path | Major | . | Chang Li | Chang Li | | | FairScheduler: CME on FSParentQueue#getQueueUserAclInfo | Critical | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | We should make a copy of configuration when init MiniYARNCluster with multiple RMs | Major | . | Chun Chen | Chun Chen | | | Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" | Major | . | Arup Malakar | Wilfred Spiegelenburg | | | smart-apply-patch.sh fails if the patch edits CR+LF files and is created by 'git diff --no-prefix' | Minor | build | Akira Ajisaka | Kengo Seki | | | update BUILDING.txt to include python for 'mvn site' in windows | Major | . | Vinayakumar B | Vinayakumar B | | | Don't escape HTML links in Diagnostics in JHS job overview | Major | . | Siqi Li | Siqi Li | | | Fix dead links to DistCp and Hadoop Archives pages. | Minor | documentation, site | Kazuho Fujii | Kazuho Fujii | | | Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | CapacityScheduler should forbid moving LeafQueue from one parent to another | Blocker | . | Wangda Tan | Wangda Tan | | | smart-apply-patch wrongly assumes that git is infallible | Major | test | Allen Wittenauer | Kengo Seki | | | Tolerate JDK-8047340-related exceptions in Shell#isSetSidAvailable preventing class init | Major | . | Gera Shegalov | Tsuyoshi Ozawa | | | Fix DominantRC#compare() does not work as expected if cluster resource is empty | Blocker | resourcemanager | Bibin A Chundatt | Rohith Sharma K S | | | JHS sorting on state column not working in webUi | Minor | jobhistoryserver | Bibin A Chundatt | zhihai xu | | | Serialize the recently added Task#encryptedSpillKey field at the end | Minor | . | Arun Suresh | Arun Suresh | | | Use DirectoryStream in DiskChecker#checkDirs to detect errors when listing a directory | Major | util | zhihai xu | zhihai xu | | | Change heavily recorded NN logs from INFO to DEBUG level | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | | | FairScheduler: potential livelock due to maxAMShare limitation and container reservation | Critical | fairscheduler | zhihai xu | zhihai xu | | | Hdfs doesnt have class 'debug' in windows | Major | scripts | Sumana Sathish | Anu Engineer | | | Should use equals when compare Resource in RMNodeImpl#ReconnectNodeTransition | Minor | resourcemanager | zhihai xu | zhihai xu | | | TestLocalDirsHandlerService should delete the created test directory logDir2 | Minor | test | David Moore | David Moore | | | TestDatanodeLayoutUpgrade fails on Windows. | Major | test | Chris Nauroth | Chris Nauroth | | | Fix Yarn resourcemanger CLI usage | Major |

| Brahma Reddy Battula | Brahma Reddy Battula | | | RPC client should not retry for InvalidToken exceptions | Critical | ipc | Daryn Sharp | Varun Saxena | | | Fix hdfs CLI usage message for namenode and zkfc | Major | . | Xiaoyu Yao | Brahma Reddy Battula | | | Azure FileSystem PageBlobInputStream does not return -1 on EOF | Major | tools | Ivan Mitic | Ivan Mitic | | | TestClusterId#testFormatWithEmptyClusterIdOption is failing | Major | . | Rakesh R | Rakesh R | | | RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() | Major | fs | Kristofer Tomasette | Kanaka Kumar Avvaru | | | JobHistory doesn't support fully-functional search | Critical | jobhistoryserver | Siqi Li | Siqi Li | | | Fix BaileyBorweinPlouffe CLI usage message | Trivial | . | Brahma Reddy Battula | Brahma Reddy Battula | | | IPC client downgrades all exception types to IOE, breaks callers trying to use them | Critical | . | Steve Loughran | Brahma Reddy Battula | | | Support for Resource as an argument during submitApp call in MockRM test class | Minor | resourcemanager | Sunil Govindan | Sunil Govindan | | | in Shell.java#runCommand() rethrow InterruptedException as InterruptedIOException | Minor | . | Lavkesh Lahngir | Lavkesh Lahngir | | | HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands | Major | documentation | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Document that NFS gateway does not work with rpcbind on SLES 11 | Major | documentation | Arpit Agarwal | Arpit Agarwal | | | DN always uses HTTP/localhost@REALM principals in SPNEGO | Blocker | . | Haohui Mai | Haohui Mai | | | TestRMEmbeddedElector fails because of ambiguous LOG reference | Major | test | Chengbing Liu | Chengbing Liu | | | Calculation of effective layout version mishandles comparison to current layout version in storage. | Major | namenode | Chris Nauroth | Chris Nauroth | | | TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute | Blocker | datanode | Yongjun Zhang | Yongjun Zhang | | | TestCommitBlockSynchronization fails in branch-2.7 | Major | test | Arpit Agarwal | Arpit Agarwal | | | TestFileCorruption doesn't work as expected | Major | test | Walter Su | Walter Su | | | Limiting LDAP search conflicts with posixGroup addition | Blocker | security | Patrick White | Patrick White | | | The default retry policy does not handle RetriableException correctly | Critical | ipc | Arpit Agarwal | Arpit Agarwal | | | TestPipeApplication fails by NullPointerException | Major | test | Akira Ajisaka | Brahma Reddy Battula | | | Lease recovery should return true if the lease can be released and the file can be closed | Major | namenode | J.Andreina | J.Andreina | | | SafeModeException never get unwrapped | Major | . | Haohui Mai | Haohui Mai | | | org.apache.hadoop.fs.shell.TestCount fails | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Block corruption can happen during pipeline recovery | Blocker | datanode | Peng Zhang | Kihwal Lee | | | Minicluster throws NPE on shutdown | Major | . | Mike Drob | Surendra Singh Lilhore | | | AM proxy filter can not get RM webapp address from yarn.resourcemanager.hostname.rm-id | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | Fix hdfs datanode CLI usage message | Major |

| Xiaoyu Yao | Brahma Reddy Battula | | | Incomplete Cache Mechanism in CredentialProvider API | Major | security | Larry McCay | Larry McCay | | | Fix WindowsResourceCalculatorPlugin.getCpuFrequency() returning always -1 | Minor | . | Georg Berendt | J.Andreina | | | Both RM are on standBy state when kerberos user not in yarn.admin.acl | Critical | resourcemanager | Bibin A Chundatt | Varun Saxena | | | Separate safemode related operations in GetBlockLocations() | Minor | . | Haohui Mai | Haohui Mai | | | Correct HTTP method in WebHDFS document | Major | documentation | Akira Ajisaka | Brahma Reddy Battula | | | The logger reports total input paths but it is referring to input files | Trivial | . | Andi Chirita Amdocs | Bibin A Chundatt | | | Fix two minor nits in member variable properties of YarnConfiguration | Trivial | yarn | Ray Chiang | Ray Chiang | | | ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission | Major | . | Robert Kanter | Bibin A Chundatt | | | Two RMNodes for the same NodeId are used in RM sometimes after NM is reconnected. | Major | resourcemanager | zhihai xu | zhihai xu | | | Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys | Minor | datanode | Ray Chiang | Ray Chiang | | | Reserved RBW space is not released if creation of RBW File fails | Blocker | . | Kanaka Kumar Avvaru | Kanaka Kumar Avvaru | | | NullPointerException in App Attempts page | Major | . | Siqi Li | Siqi Li | | | Small refactoring of DelegationTokenAuthenticationFilter to allow code sharing | Minor | security | Yongjun Zhang | Yongjun Zhang | | | Accessing httpfs via webhdfs doesn't work from a jar with kerberos | Major | security, webhdfs | Yongjun Zhang | Yongjun Zhang | | | Fix typo in the usage of NNBench | Trivial | documentation | Akira Ajisaka | Jagadesh Kiran N | | | Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them | Critical | . | Zhe Zhang | Zhe Zhang | | | org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well | Critical | . | Kris Geusebroek | Akira Ajisaka | | | WebHDFS getHomeDirectory behavior does not match specification | Major | webhdfs | Jakob Homan | Kanaka Kumar Avvaru | | | NMProxy should retry on NMNotYetReadyException | Critical | . | Karthik Kambatla | Robert Kanter | | | hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml | Minor | resourcemanager | Vamsee Yarlagadda | Vamsee Yarlagadda | | | Update FileOutputCommitter.FILEOUTPUTCOMMITTER\ALGORITHM\VERSION\_DEFAULT to match mapred-default.xml | Minor | mrv2 | Ray Chiang | Ray Chiang | | | Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings in secure cluster | Critical |

| Zhang Wei | Varun Saxena | | | Multiple shuffle transfer fails because input is closed too early | Blocker | task | Akira Ajisaka | Brahma Reddy Battula | | | Failed to launch new attempts because ApplicationMasterLauncher's threads all hang | Major | resourcemanager | Jun Gong | Jun Gong | | | Resource Localization fails on a cluster due to existing cache directories | Critical | nodemanager | Ranga Swamy | Brahma Reddy Battula | | | usedResource from rootQueue metrics may get stale data for FS scheduler after recovering the container | Major | fairscheduler, test | Rohith Sharma K S | zhihai xu | | | MetricsSystemImpl fails to show backtrace when an error occurs | Major | . | Jason Lowe | Jason Lowe | | | Prune cached replicas from DatanodeDescriptor state on replica invalidation | Major | caching | Andrew Wang | Andrew Wang | | | Race condition in ResourceTrackerService leads to wrong diagnostics messages | Major | resourcemanager | Chengbing Liu | Chengbing Liu | | | SerializedException should also try to instantiate internal exception with the default constructor | Major | . | Lavkesh Lahngir | Lavkesh Lahngir | | | TestLocalJobSubmission is failing with unknown host | Major | test | Jason Lowe | zhihai xu | | | Fix replication check in DFSTestUtils#waitForReplication | Trivial | test | Andrew Wang | Andrew Wang | | | Error handling in snappy decompressor throws invalid exceptions | Major | io, native | Todd Lipcon | Matt Foley | | | NM fails to read files from full disks which can lead to container logs being lost and other issues | Blocker | log-aggregation, nodemanager | Varun Saxena | Varun Saxena | | | Preserve compatibility of ClientProtocol#rollingUpgrade after finalization | Critical | rolling upgrades | Andrew Wang | Andrew Wang | | | LeafQueue doesn't print user properly for application add | Minor | capacityscheduler | Devaraj K | Varun Saxena | | | BlockScanner is incorrectly disabled by default | Blocker | datanode | Andrew Wang | Arpit Agarwal | | | rmadmin -transitionToActive should check the state of non-target node | Major | resourcemanager | Masatake Iwasaki | Masatake Iwasaki | | | Dead Datanode is allocated for write when client is from deadnode | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | hadoop fs -expunge does not work for federated namespace | Major | . | Vrushali C | J.Andreina | | | Update missing command option for fetchdt | Major | documentation | J.Andreina | J.Andreina | | | ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. | Major | . | Junping Du | Raju Bairishetti | | | StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB | Major | tools | Duo Xu | Duo Xu | | | SerializedException should also handle java.lang.Error | Major | . | Lavkesh Lahngir | Lavkesh Lahngir | | | Remove the duplicate usage message from Dfsck.java | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | long running apps may have a huge number of StatisticsData instances under FileSystem | Critical | fs | Sangjin Lee | Sangjin Lee | | | Update HDFS usage with missing options | Minor | . | J.Andreina | J.Andreina | | | FileSystem#getUsed() returns the file length only from root '/' | Major | . | tongshiquan | J.Andreina | | | NPE thrown if unable to fetch token from Namenode | Major | . | J.Andreina | J.Andreina | | | ArrayIndexOutOfBoundsException with empty environment variables | Major | yarn | Joe Ferner | zhihai xu | | | globStatus() should always return a sorted list of files | Minor | . | Felix Borchers | Colin P. McCabe | | | Add HTrace support for FsShell | Major | . | Colin P. McCabe | Colin P. McCabe | | | Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces | Major | . | Ray Chiang | Ray Chiang | |

JobResourceUpdater#compareFs() doesn't handle HA namespaces | Major | mrv2 | Thomas Graves | Ray Chiang | | | Fix unrecommended syntax usages in hadoop/hdfs/yarn script for cygwin in branch-2 | Major | scripts | Li Lu | Li Lu | | | Fix TestMove and TestFsShellReturnCode failed to get command name using reflection. | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Fix mismatch in default values for yarn.scheduler.maximum-allocation-vcores property | Minor | . | Ray Chiang | Ray Chiang | | | AbstractYarnScheduler.createReleaseCache may try to clean a null attempt | Major | scheduler | nijel | nijel | | | Fix typo in datanode startup options in HDFSCommands.html | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Interrupted Exception in LocalContainerLauncher should be logged in warn/info level | Major | . | Chang Li | Chang Li | | | MRApp should not shutdown LogManager during shutdown | Major | test | Chang Li | Chang Li | | | Several NPEs when deleting local files on NM recovery | Major | nodemanager | Karthik Kambatla | Varun Saxena | | | speedup TestMover | Major | test | Walter Su | Walter Su | | | Shorten overly-long htrace span names for server | Major | tracing | Colin P. McCabe | Colin P. McCabe | | | FSSchedulerNode#reserveResource() doesn't print Application Id properly in log | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | Prevent processing preemption events on the main RM dispatcher | Major | resourcemanager, scheduler | Jason Lowe | Varun Saxena | | | NetworkTopology#add calls NetworkTopology#toString always | Major | . | igo Goiri | igo Goiri | | | Avoid retrying to recover lease on a file which does not exist | Major | . | J.Andreina | J.Andreina | | | AggregatedLogFormat should close aclScanner and ownerScanner after create them. | Minor | nodemanager | zhihai xu | zhihai xu | | | ShuffleHandler passes wrong "base" parameter to getMapOutputInfo if mapId is not in the cache. | Major | mrv2, nodemanager | zhihai xu | zhihai xu | | | ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times | Major | ha | zhihai xu | zhihai xu | | | WebHdfsFileSystem#getXAttr(Path p, final String name) doesn't work if namespace is in capitals | Major | webhdfs | Jagadesh Kiran N | Kanaka Kumar Avvaru | | | javadocs of TimelineAuthenticationFilterInitializer give wrong prefix for auth options | Minor | timelineserver | Steve Loughran | Bibin A Chundatt | | | Potential NPE from Configuration#loadProperty with allowNullValueProperties set. | Minor | conf | zhihai xu | zhihai xu | | | A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial | Minor | . | Pei Ma | Tsuyoshi Ozawa | | | NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | [JDK8] 'mvn site' fails | Major | api, site | Akira Ajisaka | Brahma Reddy Battula | | | releasedocmaker drops missing component and assignee entries | Blocker | yetus | Allen Wittenauer | Allen Wittenauer | | | Make TestFileTruncate more reliable | Minor | . | Rakesh R | Rakesh R | | | FileContext.java # fixRelativePart should check for not null for a more informative exception | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Add tracing to FileSystem#createFileSystem and Globber#glob | Major | . | Colin P. McCabe | Colin

McCabe | | | TestCryptoStreamsWithOpensslAesCtrCryptoCodec should be skipped in non-native profile | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | TestShuffleHandler#testGetMapOutputInfo is failing | Major | test | Devaraj K | zhihai xu | | | Fix testTruncateWithDataNodesRestartImmediately occasionally failed | Minor | . | Walter Su | Walter Su | | | ApplicationMaster link is broken in RM WebUI when appstate is NEW | Minor | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | Fix findbugs warning in BlockManager.java | Minor | . | Akira Ajisaka | Brahma Reddy Battula | | | calling fetchdt without a --renewer argument throws NPE | Major | security | Todd Lipcon | Vinayakumar B | | | Remove setBlocks API from INodeFile and misc code cleanup | Major | namenode | Zhe Zhang | Zhe Zhang | | | Too much of preemption activity causing continuos killing of containers across queues | Critical | capacityscheduler | Sunil Govindan | Sunil Govindan | | | getResourceCalculatorPlugin for the default should intercept all exceptions | Major | . | Gera Shegalov | Gera Shegalov | | | RM startup should fail for wrong CS xml NodeLabel capacity configuration | Critical | capacityscheduler | Bibin A Chundatt | Bibin A Chundatt | | | Fix findbugs warning in RMContainerAllocator.reduceNodeLabelExpression | Major | . | Ray Chiang | Brahma Reddy Battula | | | Bzip2Factory is not thread safe | Major | io | Jason Lowe | Brahma Reddy Battula | | | hdfs dfsclient newConnectedPeer has no write timeout | Major | fuse-dfs, hdfs-client | zhangshilong | Xiaoyu Yao | | | ByteBufferReadable doesn't declare @InterfaceAudience and @InterfaceStability | Minor | fs | Steve Loughran | Brahma Reddy Battula | | | TestBlockReportRateLimiting#testLeaseExpiration can deadlock | Major | test | Arpit Agarwal | Arpit Agarwal | | | FileInputStream in JenkinsHash#main() is never closed | Minor | . | Ted Yu | Chen He | | | Fix tests requiring native library to be skipped in non-native profile | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Update the documentation of Disk Checker based on YARN-90 | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | RawLocalFileSystem.listStatus() returns null for UNIX pipefile | Critical | . | Haohui Mai | Kanaka Kumar Avvaru | | | ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level | Blocker | yarn | Ajith S | Ajith S | | | Fair Scheduler: Parts of preemption logic uses DefaultResourceCalculator even in DRF mode causing thrashing | Major | fairscheduler | Ashwin Shankar | Arun Suresh | | | Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED | Critical | capacityscheduler, fairscheduler, resourcemanager | Peng Zhang | Peng Zhang | | | Application History Server UI NPEs when accessing apps run after RM restart | Major | timelineserver | Eric Payne | Eric Payne | | | hadoop-openstack junit & mockito dependencies should be "provided" | Minor | build, fs/swift | Steve Loughran | Ted Yu | | | KMSClientProvider uses equalsIgnoreCase("application/json") | Major | kms | Steve Loughran | Brahma Reddy Battula | | | ProtobufRpcEngine.invoke() should use Exception.toString() over getMessage in logging/span events | Minor | ipc | Steve Loughran | Varun Saxena | | | releasedocmaker.py doesn't work behind a proxy | Major | yetus | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Update docs for mSNN | Minor |

| Jesse Yates | Jesse Yates | | | Enforce maximum number of ACL entries separately per access and default. | Major | namenode | Vinayakumar B | Vinayakumar B | | | hdfs --daemon stop namenode corrupts logs | Major | scripts | John Smith | John Smith | | | scmadmin help message correction | Minor | yarn | Bibin A Chundatt | Bibin A Chundatt | | | Uber mode's log message is missing a vcore reason | Minor | . | Steven K. Wong | Steven K. Wong | | | Enable swift distcp to secure HDFS | Major | fs/swift | Chen He | Chen He | | | Few FSNamesystem metrics are not documented in the Metrics page | Major | documentation | Rakesh R | Rakesh R | | | AsyncDispatcher can hang while stopping if it is configured for draining events on stop | Critical | . | Varun Saxena | Varun Saxena | | | Avoid updating quota usage while loading edits | Major | . | Jing Zhao | Jing Zhao | | | Sasl message with MD5 challenge text shouldn't be LOG out even in debug level. | Critical | ipc, security | Junping Du | Junping Du | | | Hadoop archives command should use configurable replication factor when closing | Major | . | Zhe Zhang | Bibin A Chundatt | | | StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB | Major | fs/azure, tools | Duo Xu | Duo Xu | | | SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources | Major | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml fails in trunk | Major | . | Varun Saxena | Varun Saxena | | | Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore | Critical | . | Junping Du | Jian He | | | WebHdfsFileSystem creates too many connections for pread | Major | webhdfs | Jing Zhao | Jing Zhao | | | Proportional Preemption policy should try to avoid sending duplicate PREEMPT\_CONTAINER event to scheduler | Major | capacityscheduler | Sunil Govindan | Sunil Govindan | | | Protobuf layout of yarn\security\token causes errors in other protos that include it | Major | . | Anubhav Dhoot | Anubhav Dhoot | | | Scheduler page does not render RGBA color combinations in IE11 | Minor | . | Jagadesh Kiran N | Mohammad Shahid Khan | | | Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared | Major | . | Zhe Zhang | Zhe Zhang | | | Fetch the application report from the AHS if the RM does not know about it | Major | . | Mit Desai | Mit Desai | | | FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500 | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. | Critical | nodemanager | zhihai xu | zhihai xu | | | Recent changes to application priority management break reservation system from YARN-1051 | Major | resourcemanager | Carlo Curino | Carlo Curino | | | TestYarnConfigurationFields should be moved to hadoop-yarn-api module | Major | . | Varun Saxena | Varun Saxena | | | Correct assertions in TestDFSInotifyEventInputStream

| Minor | test | Surendra Singh Lilhore | Surendra Singh Lilhore | | | TestDistributedFileSystem is failing in trunk | Major | test | Arpit Agarwal | Xiaoyu Yao | | | Cgroups cease to work in RHEL7 | Critical | nodemanager | Wei Yan | Wei Yan | | | RM Web UI queue filter is not working | Major | yarn | Mohammad Shahid Khan | Mohammad Shahid Khan | | | References to misspelled REMAINING\_QUATA in FileSystemShell.md | Minor | documentation | Gera Shegalov | Gabor Liptak | | | container-executor parsing of container-executor.cfg broken in trunk and branch-2 | Blocker | nodemanager | Varun Vasudev | Varun Vasudev | | | FsShell must load SpanReceiverHost to support tracing | Major | tracing | Masatake Iwasaki | Masatake Iwasaki | | | Better to exclude decommissioned nodes for namenode NodeUsage JMX | Major | . | Ming Ma | J.Andreina | | | 4-digit octal umask permissions throws a parse error | Major | fs | Jason Lowe | Chang Li | | | NPEs' while stopping service after exception during CommonNodeLabelsManager#start | Trivial | resourcemanager | Varun Saxena | Varun Saxena | | | AddNodeLabel on duplicate label addition shows success | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected | Critical | resourcemanager | Rohith Sharma K S | Bibin A Chundatt | | | BlockStateChange logs are too noisy | Major | namenode | Arpit Agarwal | Chang Li | | | AbstractContractAppendTest#testRenameFileBeingAppended misses rename operation. | Major | test | zhihai xu | zhihai xu | | | change TestHDFSContractAppend to not override testRenameFileBeingAppended method. | Major | test | zhihai xu | zhihai xu | | | VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks | Major | datanode | Colin P. McCabe | Colin P. McCabe | | | TestHDFSCLI does not cleanup the test directory | Minor | test | Akira Ajisaka | Masatake Iwasaki | | | Remove direct download link from BUILDING.txt | Minor | documentation | Caleb Severn | Caleb Severn | | | Fix native compilation on Windows after HADOOP-7824 | Blocker | . | Vinayakumar B | Vinayakumar B | | | Make CapacityScheduler to easier extend application allocation logic | Major | . | Wangda Tan | Wangda Tan | | | Applications using FileContext fail with the default file system configured to be wasb/s3/etc. | Blocker | fs | Chris Nauroth | Chris Nauroth | | | MetricsSinkAdapter hangs when being stopped | Critical | . | Jian He | Brahma Reddy Battula | | | Document encrypted spills | Major | security | Allen Wittenauer | Bibin A Chundatt | | | Make LeaseManager#countPath O(1) | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails | Major | . | Walter Su | Walter Su | | | Fix excessive loggings in CapacityScheduler | Major | . | Jian He | Jian He | | | Typo in docs: Rumtime -\> Runtime | Trivial | documentation, webhdfs | Jakob Homan | Gabor Liptak | | | RM hangs on draining events | Major | . | Jian He | Jian He | | | FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests | Major | . | Wangda Tan | Wangda Tan | | | Quota by storage type usage incorrectly initialized upon namenode restart | Major | namenode | Kihwal Lee | Xiaoyu Yao | | | Need translate

to FileNotFoundException to avoid regression | Critical | fs | zhihai xu | zhihai xu | | | Completed container whose app is finished is not removed from NMStateStore | Major | . | Jun Gong | Jun Gong | | | ClientRMService getApplications has high scheduler lock contention | Major | resourcemanager | Jason Lowe | Jason Lowe | | | am container complete msg ack to NM once RM receive it | Major | resourcemanager | sandflee | sandflee | | | typos in rpcmetrics.java | Trivial | ipc | Anu Engineer | Anu Engineer | | | Typo in dfshealth.html - "Decomissioning" | Trivial | . | nijel | nijel | | | Mappers get rescheduled on node transition even after all reducers are completed | Major | applicationmaster | Sangjin Lee | Sangjin Lee | | | HDFS concat should keep srcs order | Blocker | . | Yong Zhang | Yong Zhang | | | AM may fail instead of retrying if RM shuts down during the allocate call | Critical | . | Anubhav Dhoot | Anubhav Dhoot | | | DiskChecker should not traverse the entire tree | Major | . | Chang Li | Chang Li | | | HDFS architecture documentation of version 2.x is outdated about append write support | Major | documentation | Hong Dai Thanh | Ajith S | | | Memory leak in ResourceManager with SIMPLE mode | Critical | resourcemanager | mujunchao | mujunchao | | | AppBlock page key update and diagnostics value null on recovery | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | TestAppendSnapshotTruncate may fail with IOException: Failed to replace a bad datanode | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Enable optimized block reports | Major | . | Rushabh S Shah | Daryn Sharp | | | Applications fail on NM restart on some linux distro because NM container recovery declares AM container as LOST | Critical | . | Anubhav Dhoot | Anubhav Dhoot | | | The remaining space check in BlockPlacementPolicyDefault is flawed | Critical | . | Kihwal Lee | Kihwal Lee | | | Link the native\mini\dfs test library with libdl, since IBM Java requires it | Major | build | Ayappan | Ayappan | | | HDFS fsck reports under construction blocks as "CORRUPT" | Major | tools | Sudhir Prakash | Jing Zhao | | | MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute | Major | documentation | Ivan Mitic | Dustin Cote | | | getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface instead | Major | scheduler | Varun Saxena | Varun Saxena | | | Fix mismatch parameter name in javadocs of AuthToken#setMaxInactives | Trivial | . | Xiaoyu Yao | Xiaoyu Yao | | | Update hyperlink to rack awareness page in HDFS Architecture documentation | Trivial | documentation | Masatake Iwasaki | Masatake Iwasaki | | | RMNode transitioned from RUNNING to REBOOTED because its response id had not been reset synchronously | Major | resourcemanager | Jun Gong | Jun Gong | | | Block report lease may leak if the 2nd full block report comes when NN is still in safemode | Minor | . | Jing Zhao | Jing Zhao | | | NPE thrown in NameNode when try to get "TotalSyncCount" metric before editLogStream initialization | Major |

| Surendra Singh Lilhore | Surendra Singh Lilhore | | | Delay in checkpointing Trash can leave trash for 2 intervals before deleting | Trivial | trash | Casey Brotherton | Casey Brotherton | | | Add a unit test for INotify functionality across a layout version upgrade | Major | namenode | Zhe Zhang | Zhe Zhang | | | DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager | Minor | test | Walter Su | Walter Su | | | Surpress findbug warnings of o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager in hdfs-client | Major | . | Haohui Mai | Mingliang Liu | | | Set hadoop.tmp.dir and hadoop.log.dir in pom | Major | . | Charlie Helin | Charlie Helin | | | Clean up findbugs warnings for HDFS-8823 and HDFS-8932 | Major | namenode | Anu Engineer | Anu Engineer | | | Fix incorrect sign extension of xattr length in HDFS-8900 | Critical | . | Haohui Mai | Colin P. McCabe | | | NPE getting application report with a null appId | Minor | client | Steve Loughran | Weiwei Yang | | | NPE when intermediate encrypt enabled for LocalRunner | Major | . | Bibin A Chundatt | zhihai xu | | | NameNode refresh doesn't remove DataNodes that are no longer in the allowed list | Major | datanode, namenode | Daniel Templeton | Daniel Templeton | | | Increase some default timeouts / retries for S3a connector | Major | fs/s3 | Sean Mackrory | Sean Mackrory | | | hadoop fs -getmerge doc is wrong | Major | documentation | Daniel Templeton | Jagadesh Kiran N | | | BufferedOutputStream in FileUtil#unpackEntries() should be closed in finally block | Minor | util | Ted Yu | Kiran Kumar M R | | | Flaw in registration bookeeping can make DN die on reconnect | Critical | . | Kihwal Lee | Kihwal Lee | | | Time and Date format need to be in sync in NameNode UI page | Minor | . | Archana T | Surendra Singh Lilhore | | | Unused ApplicationACLsManager in ContainerManagerImpl | Trivial | . | Naganarasimha G R | Naganarasimha G R | | | ForkJoin thread pool leaks | Major | . | Kihwal Lee | Kihwal Lee | | | ByteRangeInputStream used in webhdfs does not override available() | Minor | webhdfs | Shradha Revankar | Shradha Revankar | | | Incorrect reference to nodeFile in RumenToSLSConverter error message | Minor | . | Ted Yu | Wei Yan | | | Interrupted exception can occur when Client#stop is called | Minor | . | Oleg Zhurakousky | Kuhu Shukla | | | Send metrics logs to NullAppender by default | Major | logging | Arpit Agarwal | Arpit Agarwal | | | When validating the edit log, do not read at or beyond the file offset that is being written | Major | journal-node, namenode | Zhe Zhang | Zhe Zhang | | | Test(S)WebHdfsFileContextMainOperations failing on branch-2 | Major | webhdfs | Jakob Homan | Chris Nauroth | | | RM WebServices missing scheme for appattempts logLinks | Major | . | Jonathan Eagles | Jonathan Eagles | | | Capacity Scheduler headroom for DRF is wrong | Major | capacityscheduler | Chang Li | Chang Li | | | Stack trace is missing when error occurs in client protocol provider's constructor | Major | client | Chang Li | Chang Li | | | Resource localization on a bad disk causes subsequent containers failure | Major | . | Lavkesh Lahngir | Lavkesh Lahngir | | | Typos in capacity scheduler

| Trivial | documentation | Kai Sasaki | Kai Sasaki | | | App local logs are leaked if log aggregation fails to initialize for the app | Major | log-aggregation, nodemanager | Jason Lowe | Jason Lowe | | | Fix components' version information in the web page 'About the Cluster' | Minor | util | Jun Gong | Jun Gong | | | dfsadmin -metasave prints "NaN" for cache used% | Major | . | Archana T | Brahma Reddy Battula | | | introduce a new config specifically for safe mode block count | Major | . | Chang Li | Chang Li | | | ContentSummary on / skips further counts on yielding lock | Minor | namenode | tongshiquan | J.Andreina | | | Initialize file system-wide quota once on transitioning to active | Major | ha, namenode | Daryn Sharp | Kihwal Lee | | | ShuffleHandler can possibly exhaust nodemanager file descriptors | Major | mrv2, nodemanager | Nathan Roberts | Kuhu Shukla | | | NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Reduce loglevel of ContainerManagementProtocolProxy to Debug | Minor | . | Anubhav Dhoot | Anubhav Dhoot | | | MetricsSystemImpl creates MetricsSourceAdapter with wrong time unit parameter. | Major | metrics | zhihai xu | zhihai xu | | | Update document for the Storage policy name | Minor | documentation | J.Andreina | J.Andreina | | | In BlockPlacementPolicyWithNodeGroup#chooseLocalStorage , random node is selected eventhough fallbackToLocalRack is true. | Major | . | J.Andreina | J.Andreina | | | Test failing: hadoop.ipc.TestSaslRPC | Critical | security, test | Steve Loughran | Steve Loughran | | | [JDK8] Fix javadoc errors caused by incorrect or illegal tags | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | TestNameNodeMetricsLogger failing -port in use | Critical | test | Steve Loughran | Steve Loughran | | | Consolidate validateLog and scanLog in FJM#EditLogFile | Major | journal-node, namenode | Zhe Zhang | Zhe Zhang | | | Fix findbugs errors in hadoop-yarn-server-common module | Major | . | MENG DING | MENG DING | | | o.a.h.hdfs.server.datanode.fsdataset.impl.TestLazyWriter is failing in trunk | Critical | . | Haohui Mai | Surendra Singh Lilhore | | | MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks | Major | mr-am | Jason Lowe | Naganarasimha G R | | | Description of hdfs expunge command is confusing | Major | documentation, trash | Weiwei Yang | Weiwei Yang | | | getPendingResourceRequestForAttempt is present in AbstractYarnScheduler should be present in YarnScheduler interface instead | Minor | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Fix random failures in TestJMXGet | Critical | test | J.Andreina | J.Andreina | | | Fix failures in TestLazyPersistLockedMemory#testReleaseOnEviction | Critical | test | J.Andreina | J.Andreina | | | Jersey tests failing with Port in Use -again | Critical | build, test | Steve Loughran | Brahma Reddy Battula | | | TestWebDelegationToken failing with port in use | Major | test | Steve Loughran | Mingliang Liu | | | LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. | Critical | mrv2 | zhihai xu | zhihai xu | | | AM could potentially allocate a reduce container to a map attempt | Major | mr-am | Jason Lowe | Chang Li | | | Task attempts that fail from the ASSIGNED state can disappear | Major | mr-am | Jason Lowe | Chang Li | | |

should be able to specify a retry interval | Major | . | Wangda Tan | Sunil Govindan | | | FairScheduler: ContinuousSchedulingThread can fail to shutdown | Critical | fairscheduler | zhihai xu | zhihai xu | | | DN should reserve disk space for a full block when creating tmp files | Major | datanode | Arpit Agarwal | Kanaka Kumar Avvaru | | | NameNode does not check for inode type before traversing down a path | Trivial | namenode | Harsh J | Xiao Chen | | | TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails | Major | test | zhihai xu | zhihai xu | | | NPE on RMActiveServices#serviceStop when store is null | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | RM should respect retry-interval when uses RetryPolicies.RETRY\_FOREVER | Critical | . | Wangda Tan | Sunil Govindan | | | MoveApplicationAcrossQueuesResponse should be an abstract class | Minor | resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | Doc updation for commands in HDFS Federation | Minor | documentation | J.Andreina | J.Andreina | | | Deprecate NameNodeMXBean#getNNStarted in branch2 and remove from trunk | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | WebAppProxyServlet should not redirect to RM page if AHS is enabled | Major | . | Mit Desai | Mit Desai | | | Reset RawLocalFileSystem.useDeprecatedFileStatus in TestLocalFileSystem | Trivial | test | Chris Nauroth | Chris Nauroth | | | TestWebHdfsFileContextMainOperations and TestSWebHdfsFileContextMainOperations fail due to invalid HDFS path on Windows. | Trivial | test | Chris Nauroth | Chris Nauroth | | | NM crash with NPE when LogAggregationService#stopContainer called for absent container | Critical | log-aggregation, nodemanager | Bibin A Chundatt | Bibin A Chundatt | | | Running applications information changes such as movequeue is not published to TimeLine server | Critical | resourcemanager, timelineserver | Sunil Govindan | Sunil Govindan | | | Log full path instead of inodeId in DFSClient#closeAllFilesBeingWritten() | Major | hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. | Major | client, security | zhihai xu | zhihai xu | | | Allow SecurityUtil to lookup alternate hostnames | Major | net, security | Arpit Agarwal | Arpit Agarwal | | | LocalDirAllocator should not throw NPE with empty string configuration. | Minor | fs | zhihai xu | zhihai xu | | | ApplicationHistoryServer reverses the order of the filters it gets | Major | timelineserver | Mit Desai | Mit Desai | | | Copying from the root to a subdirectory should be forbidden | Minor | fs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | archive-logs tool may miss applications | Major | . | Robert Kanter | Robert Kanter | | | Prevent NN's unrecoverable death spiral after full GC | Critical | namenode | Daryn Sharp | Daryn Sharp | | | ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF | Major | hdfs-client | Colin P. McCabe | Colin P. McCabe | | | update htrace version to 4.0.1 | Major | tracing | Colin P. McCabe | Colin P. McCabe | | | ConcurrentModificationException in FairSchedulerQueueInfo | Major | . | Anubhav Dhoot | Anubhav Dhoot | | | Transfer failure during pipeline recovery causes permanent write failures | Critical |

| Kihwal Lee | Kihwal Lee | | | Fix the setting of visibleLength in ExternalBlockReader | Major | hdfs-client | Colin P. McCabe | Colin P. McCabe | | | AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | TestRPC#testRPCServerShutdown did not produce the desired thread states before shutting down | Minor | . | Xiao Chen | Xiao Chen | | | Nfs silently drops overlapping write requests and causes data copying to fail | Major | nfs | Yongjun Zhang | Yongjun Zhang | | | AMLauncher does not retry on failures when talking to NM | Critical | resourcemanager | Anubhav Dhoot | Anubhav Dhoot | | | Thread leak in Datanode#refreshVolumes | Major | datanode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Large number of queues choke fair scheduler | Major | fairscheduler | Johan Gustavsson | Johan Gustavsson | | | Clean up some htrace integration issues | Major | tracing | Colin P. McCabe | Colin P. McCabe | | | Fix findbugs warnings in FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread | Critical | . | Yi Liu | Yi Liu | | | DFSUtil.getNsServiceRpcUris() can return too many entries in a non-HA, non-federated cluster | Major | . | Daniel Templeton | Daniel Templeton | | | TestTextCommand: use mkdirs rather than mkdir to create test directory | Major | test | Colin P. McCabe | Colin P. McCabe | | | Permission issue when running archive-logs tool as different users | Major | . | Robert Kanter | Robert Kanter | | | Fix wrong value of JOB\_FINISHED event in JobHistoryEventHandler | Major | . | Shinichi Yamashita | Shinichi Yamashita | | | Incorrect null check in SwiftRestClient#buildException() | Minor | . | Ted Yu | Kanaka Kumar Avvaru | | | getLocalPathForWrite should throw IOException for invalid paths | Major | fs | Brahma Reddy Battula | Brahma Reddy Battula | | | MR job hanged forever because all resources are taken up by reducers and the last map attempt never get resource to run | Critical | applicationmaster | Bob.zhao | Xianyin Xin | | | Fix null tracer in ErasureCodingWorker | Critical | erasure-coding | Rakesh R | Rakesh R | | | Typo in Hdfs.java. NoSuchElementException is misspelled | Trivial | documentation | Catherine Palmer | Catherine Palmer | | | HDFS Balancer does not respect dfs.client.use.datanode.hostname | Major | balancer & mover | Yongjun Zhang | Casey Brotherton | | | ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException | Major | nodemanager | Jason Lowe | zhihai xu | | | Fix incorrect references the usages of the DN in dfshealth.js | Minor | . | Chang Li | Chang Li | | | [JDK8] Max Non Heap Memory default changed between JDK7 and 8 | Major | . | Travis Thompson | Tsuyoshi Ozawa | | | Fix kill command behavior under some Linux distributions. | Critical | . | Wangda Tan | Wangda Tan | | | Fix tracing documention reflecting the update to htrace-4 | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Resync NM nodelabels with RM periodically for distributed nodelabels | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Improve EOF error message | Minor | hdfs-client | Harsh J | Jagadesh Kiran N | | | archive-logs tool should use HADOOP\PREFIX instead of HADOOP\HOME | Major |

| Robert Kanter | Robert Kanter | | | RMStateStore FENCED state doesn't work due to updateFencedState called by stateMachine.doTransition | Critical | resourcemanager | zhihai xu | zhihai xu | | | Fix TestWebHdfsContentLength | Major | . | Tsuyoshi Ozawa | Masatake Iwasaki | | | Slow datanode I/O can cause a wrong node to be marked bad | Critical | . | Kihwal Lee | Kihwal Lee | | | Cleanup the findbugs and other issues after HDFS EC merged to trunk. | Critical | erasure-coding | Yi Liu | Uma Maheswara Rao G | | | Incorrect javadoc in WritableUtils.java | Minor | documentation | Martin Petricek | Jagadesh Kiran N | | | TestDirectoryScanner#testThrottling often fails. | Minor | test | Yi Liu | Daniel Templeton | | | DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded | Major | datanode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | cTime is 0 in VERSION file for newly formatted NameNode. | Minor | namenode | Chris Nauroth | Xiao Chen | | | DatanodeDescriptor#PendingReplicationWithoutTargets is wrongly calculated | Major | . | Jing Zhao | Mingliang Liu | | | Fix broken findbugsExcludeFile.xml for hadoop-hdfs-client module | Critical | build | Mingliang Liu | Mingliang Liu | | | FairScheduler PrimaryGroup does not handle empty groups returned for a user | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | Preempt reducers after a configurable timeout irrespective of headroom | Critical | . | mai shurong | Karthik Kambatla | | | Separating Configuration object for namenode(s) in MiniDFSCluster | Major | . | Siqi Li | Siqi Li | | | DistributedFileSystem listCorruptFileBlocks API should resolve relative path | Major | hdfs-client | Rakesh R | Rakesh R | | | Add hadoop-hdfs-client as a dependency of hadoop-hdfs-native-client | Major | . | Haohui Mai | Mingliang Liu | | | AMBlacklist does not work for minicluster | Major | resourcemanager | Jun Gong | Jun Gong | | | Suppress the RAT warnings in hdfs-native-client module | Minor | . | Haohui Mai | Haohui Mai | | | TestFileTruncate fails intermittently with BindException | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | container-executor overuses PATH\_MAX | Major | nodemanager | Allen Wittenauer | Sidharta Seethana | | | Delayed rolling upgrade finalization can cause heartbeat expiration and write failures | Critical | . | Kihwal Lee | Walter Su | | | MiniKMS should use random ports for Jetty server by default | Major | . | Mingliang Liu | Mingliang Liu | | | TestDNS and TestNetUtils failing if no network | Minor | test | Steve Loughran | Steve Loughran | | | Upgrade jsch lib to jsch-0.1.51 to avoid problems running on java7 | Major | build | Johannes Zillmann | Tsuyoshi Ozawa | | | hdfs-native-client build getting errors when built with cmake 2.6 | Minor | hdfs-client | Eric Payne | Eric Payne | | | WebUI fails to display block IDs that are larger than 2^53 - 1 | Minor | webhdfs | Walter Su | Haohui Mai | | | Fix null pointer error in Globber when FS was not constructed via FileSystem#createFileSystem | Major | tracing | stack | Colin P. McCabe | | | Blocks in newly completed files are considered under-replicated too quickly | Major | namenode | Todd Lipcon | Masatake Iwasaki | | | NPE

AppSchedulingInfo#isRequestLabelChanged | Major | resourcemanager, scheduler | Brahma Reddy Battula | Brahma Reddy Battula | | | Deleting pid file after stop is causing the daemons to keep restarting | Minor | . | Siqi Li | Siqi Li | | | Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath() | Critical | util | Steve Loughran | Steve Loughran | | | Reading small file (\< 512 bytes) that is open for append fails due to incorrect checksum | Blocker | . | Bogdan Raducanu | Jing Zhao | | | ProtocMojo does not log the reason for a protoc compilation failure. | Minor | build | Chris Nauroth | Chris Nauroth | | | SPNEGO auth does not work with CNAMEs in JDK8 | Blocker | security | Daryn Sharp | Daryn Sharp | | | MiniYARNCluster should propagate reason for AHS not starting | Major | test | Steve Loughran | Steve Loughran | | | TestLogAggregationService.testLogAggregationServiceWithInterval failing | Critical | test | Steve Loughran | Bibin A Chundatt | | | Proxy user verification NPEs if remote host is unresolvable | Critical | ipc, security | Daryn Sharp | Benoy Antony | | | Maintain wrapped SASL ordering for postponed IPC responses | Critical | ipc | Daryn Sharp | Daryn Sharp | | | NPE at TestDataNodeVolumeFailureToleration#tearDown | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Interrupted client may try to fail-over and retry | Major | ipc | Kihwal Lee | Kihwal Lee | | | bash unit tests are failing | Major | . | Allen Wittenauer | Allen Wittenauer | | | Limit application resource reservation on nodes for non-node/rack specific requests | Major | fairscheduler | Arun Suresh | Arun Suresh | | | Disabling atime may fail clients like distCp | Major | . | Kihwal Lee | Kihwal Lee | | | TestShortCircuitLocalRead should not leave socket after unit test | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | TestRPC.testRPCInterruptedSimple fails intermittently | Major | test | Steve Loughran | Kihwal Lee | | | Docs for archive-logs tool | Major | documentation | Robert Kanter | Robert Kanter | | | ssl-server.xml causes NullPointer | Minor | datanode, security | Evert Lammerts | Xiao Chen | | | Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent | Trivial | datanode | Yi Liu | Yi Liu | | | Set SO\_KEEPALIVE on shuffle connections | Major | mrv2, nodemanager | Nathan Roberts | Chang Li | | | Fix intermittent test failure of TestBlockManager.testBlocksAreNotUnderreplicatedInSingleRack | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | ACLs on root directory may be lost after NN restart | Critical | namenode | Xiao Chen | Xiao Chen | | | RM crashes with NPE if leaf queue becomes parent queue during restart | Major | capacityscheduler, resourcemanager | Jason Lowe | Varun Saxena | | | Improving log message when SequenceFile reader throws EOFException on zero-length file | Major | . | Chu Tong | Zhe Zhang | | | YARN fair scheduler vcores with decimal values | Minor | fairscheduler | Prabhu Joseph | Jun Gong | | | Single File Rename Throws Incorrectly In Potential Race Condition Scenarios | Major | tools | Gaurav Kanade | Gaurav Kanade | | | HttpFs does not parse ACL syntax correctly for operation REMOVEACLENTRIES | Major | fs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | CORS support for ResourceManager REST API | Major | . | Prakash Ramachandran | Varun Vasudev | | | Slow delegation token renewal can severely prolong RM recovery | Major | resourcemanager | Jason Lowe | Sunil Govindan | |

DFSClient#callAppend() is not backward compatible for slightly older NameNodes | Blocker | . | Tony Wu | Tony Wu | | | HDFS clients can't construct HdfsConfiguration instances | Major | . | Steve Loughran | Mingliang Liu | | | [JDK8] Fix javadoc errors caused by wrong reference and illegal tag | Blocker | build, documentation | Akira Ajisaka | Akira Ajisaka | | | TestDistributedShell failing with bind exception | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | DistributedShell Log.info is not friendly | Major | applications/distributed-shell | Xiaowei Wang | Xiaowei Wang | | | NPE while listing app attempt | Major | . | Varun Saxena | nijel | | | Findbugs warnings in hadoop-yarn-server-nodemanager project | Minor | nodemanager | Varun Saxena | Varun Saxena | | | Dockerfile lacks initial 'apt-get update' | Trivial | build | Akihiro Suda | Akihiro Suda | | | condition for AM blacklisting is too narrow | Major | resourcemanager | Sangjin Lee | Sangjin Lee | | | fuse\_dfs chown crashes when uid is passed as -1 | Minor | . | Wei-Chiu Chuang | Colin P. McCabe | | | fsck command should not print exception trace when file not found | Major | . | Jagadesh Kiran N | Jagadesh Kiran N | | | Delayed heartbeat processing causes storm of subsequent heartbeats | Major | datanode | Chris Nauroth | Arpit Agarwal | | | Fix racing condition of TestNodeStatusUpdaterForLabels | Critical | test | Steve Loughran | Naganarasimha G R | | | [JDK8] Fix javadoc errors caused by wrong tags | Blocker | build, documentation | Akira Ajisaka | Akira Ajisaka | | | Mockito dependency is missing in hadoop-kafka module | Major | test | Kai Zheng | Kai Zheng | | | SLS not able start due to NPE in SchedulerApplicationAttempt#getResourceUsageReport | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | NPE during handling of SASL setup if problem with SASL resolver class | Minor | ipc | Steve Loughran | Steve Loughran | | | fetchdt stores the token based on token kind instead of token service | Major | . | Heesoo Kim | Heesoo Kim | | | StorageException complaining " no lease ID" during HBase distributed log splitting | Major | tools | Duo Xu | Duo Xu | | | fsck doesn't list correct file path when Bad Replicas/Blocks are in a snapshot | Major | snapshots | Xiao Chen | Xiao Chen | | | WebHDFS truncate throws NullPointerException if newLength is not provided | Minor | webhdfs | Karthik Palaniappan | Jagadesh Kiran N | | | TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | hadoop-azure tests should avoid creating a metrics configuration file in the module root directory. | Minor | fs/azure, test | Chris Nauroth | Chris Nauroth | | | Decomissioned capacity should not be considered for configured/used capacity | Major | . | Kuhu Shukla | Kuhu Shukla | | | Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group | Major | . | J.Andreina | J.Andreina | | | Duplicate declaration of ApplicationId in RMAppManager#submitApplication method | Trivial | resourcemanager | Kai Sasaki | Kai Sasaki | | | NodeManager restart should keep retrying to register to RM while connection exception happens during RM failed

| Critical | nodemanager | Junping Du | Junping Du | | | Update Application priority in AM side from AM-RM heartbeat | Major | applicationmaster | Sunil Govindan | Sunil Govindan | | | Fix Precondition failures from NameNodeEditLogRoller while saving namespace | Major | . | Andrew Wang | Andrew Wang | | | Race condition in MiniMRYarnCluster when getting history server address | Major | . | Jian He | Jian He | | | TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188 | Major | . | Varun Saxena | Varun Saxena | | | Memory leak for HistoryFileManager.getJobSummary() | Critical | jobhistoryserver | Junping Du | Junping Du | | | DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic | Major | distcp | Kuhu Shukla | Kuhu Shukla | | | Fix test failure of TestMiniMRClientCluster.testRestart | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Introduce FileNotFoundException in WASB for read and seek API | Major | tools | Dushyanth | Dushyanth | | | delete fails with exception when lease is held on blob | Blocker | fs/azure | Gaurav Kanade | Gaurav Kanade | | | TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size | Minor | test | Zhe Zhang | Zhe Zhang | | | Possible NullPointerException in BlockManager if no excess replica can be chosen | Major | . | Ming Ma | Ming Ma | | | TestDNS fails on Windows after HADOOP-12437. | Major | net | Chris Nauroth | Chris Nauroth | | | Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188 | Major | . | MENG DING | MENG DING | | | Make DataStreamer#block thread safe and verify genStamp in commitBlock | Critical | . | Chang Li | Chang Li | | | RM fail with noAuth error if switched from failover mode to non-failover mode | Major | resourcemanager | Jian He | Varun Saxena | | | checkNNStartup() need to be called when fsck calls FSNamesystem.getSnapshottableDirs() | Major | namenode | Yongjun Zhang | Xiao Chen | | | when setnetgrent returns 0 in linux, exception should be thrown | Major | . | Chang Li | Chang Li | | | NN UI renders icons of decommissioned DN incorrectly | Critical | . | Archana T | Surendra Singh Lilhore | | | TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. | Major | fs/azure, test | Chris Nauroth | Gaurav Kanade | | | Storage type usage isn't updated properly after file deletion | Major | . | Ming Ma | Ming Ma | | | hadoop-hdfs-client tests do not write logs. | Minor | test | Chris Nauroth | Chris Nauroth | | | ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom() | Major | namenode | Kihwal Lee | Kihwal Lee | | | Remove dead code in DataStorage.recoverTransitionRead | Major | datanode | Duo Zhang | Duo Zhang | | | Mismatched free() / delete / delete [] in HadoopPipes | Major | pipes | Charles Earl | Joe Mudd | | | SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh | Major | pipes | Simone Leo | Simone Leo | | | TestWebHdfsContentLength intermittently hangs and fails due to TCP conversation mismatch between client and server. | Minor | test | Chris Nauroth | Chris Nauroth | |

DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs | Minor | namenode | Ted Yu | Tsz Wo Nicholas Sze | | | considerLoad factor can be improved | Major | . | Kuhu Shukla | Kuhu Shukla | | | Missing sanity check for block size during block recovery | Major | datanode | Tony Wu | Tony Wu | | | branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing. | Critical | hdfs-client | Chris Nauroth | Mingliang Liu | | | [JDK8] Fix javadoc error caused by illegal tag | Blocker | build, documentation | Akira Ajisaka | Akira Ajisaka | | | Warn message about httpshuffle in NM logs | Major | . | Sandy Ryza | Akira Ajisaka | | | TestByteArrayManager#testByteArrayManager fails | Major | . | Kihwal Lee | Tsz Wo Nicholas Sze | | | NPE is thrown if an IOException is thrown in NameNode constructor | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | AOP unit tests are not getting compiled or run | Minor | test | Giridharan Kesavan | Haohui Mai | | | Fix findbugs warnings in BlockRecoveryWorker | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Race condition in JMX cache update | Major | . | Tony Wu | Tony Wu | | | Unnecessary DNS resolution attempts when creating NameNodeProxies | Major | ha, performance | Xiao Chen | Xiao Chen | | | Fix sprintf warnings in {{DomainSocket.c}} introduced by HADOOP-12344 | Major | native | Colin P. McCabe | Mingliang Liu | | | Fix findbugs warnings in hdfs-nfs/WriteCtx | Major | nfs | Mingliang Liu | Mingliang Liu | | | Fix typo of property name in yarn-default.xml | Major | documentation | Anthony Rojas | Anthony Rojas | | | testDetermineCacheVisibilities of TestClientDistributedCacheManager is broken | Major | . | Chang Li | Chang Li | | | Total files and directories on jmx and web UI on standby is uninitialized | Blocker | . | Kihwal Lee | Kihwal Lee | | | TestMRTimelineEventHandling fails | Major | test | Sangjin Lee | Sangjin Lee | | | Resource manager fails with Null pointer exception | Major | yarn | Yesha Vora | Jian He | | | Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp | Major | documentation | Mohammad Arshad | Mohammad Arshad | | | Public resource localization fails with NPE | Blocker | nodemanager | Jason Lowe | Jason Lowe | | | getContentSummary() on standby should throw StandbyException | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | TestNodeCount#testNodeCount timed out | Major | test | Wei-Chiu Chuang | Masatake Iwasaki | | | Fix typo for readChecksum() LOG.warn in BlockSender.java | Trivial | . | Enrique Flores | Enrique Flores | | | TestRollingUpgradeRollback fails on branch-2. | Blocker | . | Chris Nauroth | Brahma Reddy Battula | | | DistributedFileSystem#concat fails if the target path is relative. | Major | hdfs-client | Kazuho Fujii | Kazuho Fujii | | | ApplicationHistoryServer binds to default port 8188 in MiniYARNCluster | Critical | timelineserver | Hitesh Shah | Vinod Kumar Vavilapalli | | | RM capacity scheduler UI rounds user limit factor | Major | capacityscheduler | Chang Li | Chang Li | | | Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler | Major | capacityscheduler | Eric Payne | Eric Payne | | | Disabling HDFS client socket cache causes logging message printed to console for CLI

| Trivial | hdfs-client | Chris Nauroth | Chris Nauroth | | | Add TLSv1.1,TLSv1.2 to KMS, HttpFS, SSLFactory | Critical | kms | Robert Kanter | Vijay Singh | | | Respect user-defined JAVA\LIBRARY\PATH in Windows Hadoop scripts | Minor | scripts | Radhey Shah | Radhey Shah | | | Fix intermittent test failure of TestZKSignerSecretProvider | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | Fix wrong use of BytesWritable in FSEditLogOp#RenameOp | Minor | namenode | Yi Liu | Yi Liu | | | Remove redundant test dependencies in Hadoop Archives | Minor | . | Varun Saxena | Varun Saxena | | | Shutdown metrics when stopping JournalNode | Major | . | zhouyingchao | zhouyingchao | | | Increase the timeout of TestZKFailoverController | Major | test | Rajat Jain | Steve Loughran | | | Add cookie flags for logs and static contexts | Major | . | nijel | nijel | | | Decommissioning node does not have Last Contact value in the UI | Major | . | Archana T | Surendra Singh Lilhore | | | NPE in JvmPauseMonitor when calling stop() before start() | Critical | . | Rohith Sharma K S | Gabor Liptak | | | Fix intermittent failure of TestDNFencing.testQueueingWithAppend | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | TestBlockRecovery#testRBWReplicas is failing intermittently | Major | . | Rakesh R | Rakesh R | | | Bump up commons-collections version to 3.2.2 to address a security flaw | Blocker | build, security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations | Critical | resourcemanager | Varun Vasudev | Varun Vasudev | | | create constant MAX\_CAPACITY in RetryCache rather than hard-coding 16 in RetryCache constructor | Minor | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Fix intermittent failure of TestEditLogTailer | Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | Fix findbugs warnings in hadoop-yarn-common | Minor | . | Varun Saxena | Sunil Govindan | | | Fix typo in FairScheduler log message | Minor | fairscheduler | Xin Wang | Xin Wang | | | TestReplaceDatanodeOnFailure fails occasionally | Major | test | Arpit Agarwal | Wei-Chiu Chuang | | | Webhdfs client leaks active NameNode connections | Major | webhdfs | Bob Hansen | Xiaobing Zhou | | | FSNamesystem should construct FSPermissionChecker only if permission is enabled | Major | . | David Bryson | Gabor Liptak | | | "Total megabyte-seconds" in job counters is slightly misleading | Minor | . | Nathan Roberts | Nathan Roberts | | | FileSystemNodeLabelStore should check for root dir existence on startup | Major | resourcemanager | Jason Lowe | Kuhu Shukla | | | dfs.datanode.data.dir does not handle spaces between storageType and URI correctly | Major | datanode | Anu Engineer | Anu Engineer | | | hdfs and nfs builds broken on -missing compile-time dependency on netty | Major | nfs | Konstantin Boudnik | Tom Zeng | | | Replace '\\u2b05' with '\<-' in rendering job configuration | Minor | jobhistoryserver | Akira Ajisaka | Gabor Liptak | | | TestMRAppMaster fails on trunk | Major | . | Varun Saxena | Junping Du | | | add XML namespace declarations for some hadoop/tools modules | Minor | build, tools | Xin Wang | Xin Wang | | |

fails intermittently | Major | test | Tsuyoshi Ozawa | Varun Saxena | | | Some tests in mapreduce-client-app are writing outside of target | Blocker | build | Allen Wittenauer | Akira Ajisaka | | | hadoop-hdfs-native-client fails test build on Windows after transition to ctest. | Blocker | build, test | Chris Nauroth | Chris Nauroth | | | Clean up depreated umasks and related unit tests | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestFileTruncate fails with BindException | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix data race accessing writeLockHeldTimeStamp in FSNamesystem | Major | namenode | Mingliang Liu | Mingliang Liu | | | multibyte delimiters with LineRecordReader cause duplicate records | Major | mrv1, mrv2 | Dustin Cote | Wilfred Spiegelenburg | | | archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor | Major | . | Robert Kanter | Robert Kanter | | | Partial group resolution failure should not result in user lockout | Minor | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Rollingupgrade finalization is not backward compatible | Blocker | . | Kihwal Lee | Kihwal Lee | | | TestIPC.testIpcWithReaderQueuing may fail | Major | ipc | Tsz Wo Nicholas Sze | Xiao Chen | | | Encryption zone on root not loaded from fsimage after NN restart | Critical | . | Xiao Chen | Xiao Chen | | | deleteSnapshot throws NPE when snapshotname is null | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Fix intermittent failure of TestDecayRpcScheduler | Minor | ipc, test | Masatake Iwasaki | Masatake Iwasaki | | | TestBPOfferService#testBasicFunctionalitytest fails intermittently | Major | datanode, hdfs-client | Yongjun Zhang | Wei-Chiu Chuang | | | Yarn recover functionality causes the cluster running slowly and the cluster usage rate is far below 100 | Major | resourcemanager | NING DING | NING DING | | | DFSClient deadlock when close file and failed to renew lease | Blocker | hdfs-client | DENG FEI | Brahma Reddy Battula | | | Replace DSA with RSA for SSH key type in SingleCluster.md | Minor | documentation | Alexander Veit | Mingliang Liu | | | NodeManager still reports negative running containers | Major | nodemanager | Robert Kanter | Robert Kanter | | | Remove waitForLoadingFSImage since checkNNStartup has ensured image loaded and namenode started. | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | ApplicationCreatedEvent event time resets after RM restart/failover | Critical | . | Xuan Gong | Naganarasimha G R | | | Generic AHS sometimes doesn't show started, node, or logs on App page | Major | . | Eric Payne | Eric Payne | | | Fix deadlock in RMAppImpl | Blocker | . | Yesha Vora | Jian He | | | SPNEGO authentication request to non-default realm gets default realm name inserted in target server principal | Major | security | Matt Foley | Matt Foley | | | Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM | Major | nodemanager | Junping Du | Junping Du | | | Remove dead code in RmAppImpl.RMAppRecoveredTransition | Trivial | resourcemanager | Daniel Templeton | Daniel Templeton | | | NPE in TestSequenceFile | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | NodeManager Disk Checker parameter documentation is not correct | Minor | documentation, nodemanager | Takashi Ohnishi | Weiwei Yang | |

TestMetricsSystemImpl#testQSize occasionally fail | Major | test | Wei-Chiu Chuang | Masatake Iwasaki | | | Some coding improvement inSecondaryNameNode#main | Major | namenode | Yongjun Zhang | Xiao Chen | | | TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed | Major | hdfs-client, test | Steve Loughran | Wei-Chiu Chuang | | | Newly completed blocks in IBR should not be considered under-replicated too quickly | Major | namenode | Jing Zhao | Mingliang Liu | | | AM Resource Limit per partition can be updated to ResourceUsage as well | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | UnsatisfiedLinkError while checking ISA-L in checknative command. | Minor | native | Kai Sasaki | Kai Sasaki | | | (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period | Critical | . | Junping Du | Junping Du | | | TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Clarify NMContainerStatus#toString method. | Major | . | Jian He | Jian He | | | FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time | Major | fairscheduler | Yiqun Lin | Yiqun Lin | | | truncate file fails with data dirs on multiple disks | Major | datanode | Bogdan Raducanu | Plamen Jeliazkov | | | Set SO\_KEEPALIVE on DN server sockets | Major | datanode | Nathan Roberts | Kanaka Kumar Avvaru | | | update releasedocmaker commands | Blocker | . | Allen Wittenauer | Allen Wittenauer | | | Redundant nodeLocalityDelay log in LeafQueue | Trivial | capacityscheduler | Jason Lowe | Eric Payne | | | NPE when submit Unmanaged application | Critical | . | Naganarasimha G R | Naganarasimha G R | | | Minor typos, grammar, and case sensitivity cleanup in HdfsPermissionsGuide.md's | Trivial | documentation | Travis Campbell | Travis Campbell | | | TestDistributedFileSystem.testLocatedFileStatusStorageIdsTypes is flaky due to race condition | Minor | fs, test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add preemption status to yarn queue -status for capacity scheduler | Minor | capacity scheduler, yarn | Eric Payne | Eric Payne | | | Fix NPE in MiniKMS.start() | Minor | kms, test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | NPE when MiniDFSCluster#shutdown is invoked on uninitialized reference | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Prevent DataNode log spam if a client connects on the data transfer port but sends no data. | Major | datanode | Chris Nauroth | Chris Nauroth | | | seen\_txid in the shared edits directory is modified during bootstrapping | Major | ha, namenode | Kihwal Lee | Kihwal Lee | | | Fix ASF Licence warnings in Jenkins reports | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | After choosing favored nodes, choosing nodes for remaining replicas should go through BlockPlacementPolicy | Major | . | J.Andreina | J.Andreina | | | Add /usr/lib default for ISAL library and fix missing .c file in CMakeLists | Major | . | Kai Zheng | Kai Zheng | | | Invariant assumption in TestQuorumJournalManager.shutdown() is wrong | Major | test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | MiniKdc throws "address in use" BindException | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Prevent ServiceLoader failure init for unused FileSystems | Major | fs | igo Goiri | igo Goiri | | | Clarify confusing sentence in MapReduce tutorial document | Minor | documentation | chris snow | Kai Sasaki | | | HDFS Architecture documentation needs to be

| Major | documentation | Chris Nauroth | Masatake Iwasaki | | | NM to nodelabel mapping going wrong after RM restart | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | TestComputeInvalidateWork#testDatanodeReRegistration failed due to unexpected number of invalidate blocks. | Major | datanode, namenode, test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. | Major | fairscheduler | Tao Jie | Tao Jie | | | Block files which have been hardlinked should be duplicated before the DataNode appends to the them | Major | datanode | Colin P. McCabe | Colin P. McCabe | | | TestBackupNode always binds to port 50070, which can cause bind failures. | Major | test | Chris Nauroth | Xiao Chen | | | Exception on RM scheduler page loading with labels | Minor | . | Bibin A Chundatt | Mohammad Shahid Khan | | | TestAMRestart#testAMBlacklistPreventsRestartOnSameNode assumes CapacityScheduler | Major | . | Anubhav Dhoot | Anubhav Dhoot | | | JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999 | Major | webapps | Devaraj K | Mohammad Shahid Khan | | | BaseReplicationPolicyTest should update data node stats after adding a data node | Minor | datanode, test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | WebHdfsFileSystem should retry reads according to the configured retry policy. | Major | webhdfs | Eric Payne | Eric Payne | | | KMS connection failures should trigger TGT renewal | Major | security | Zhe Zhang | Zhe Zhang | | | MR AM should print host of failed tasks. | Major | . | Wangda Tan | Mohammad Shahid Khan | | | Make RM and Timeline-server REST APIs more consistent | Major | . | Wangda Tan | Wangda Tan | | | TestTaskLog outputs a log under directory other than target/test-dir | Major | test | Akira Ajisaka | Akira Ajisaka | | | NaN in Queue percentage for cluster apps page | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | TestSeveralNameNodes fails occasionally | Major | test | Kihwal Lee | Masatake Iwasaki | | | Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak | Major | resourcemanager | Alejandro Abdelnur | Rohith Sharma K S | | | Fix TestKMS#testKMSRestart\* failure | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix exception message in WASB when connecting with anonymous credential | Major | tools | Dushyanth | Dushyanth | | | Cleanup AppSchedulingInfo | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Queue acl can be checked at app submission | Major | . | Jian He | Jian He | | | Fix SLS startup failure caused by NPE | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | LocalizedResource trigger a NPE Cause the NodeManager exit | Major | . | tangshangwen | tangshangwen | | | TestJobHistoryEventHandler.testTimelineEventHandling fails on trunk because of NPE | Major | . | Tsuyoshi Ozawa | Naganarasimha G R | | | Datanode may deadlock while handling a bad volume | Blocker | . | Kihwal Lee | Walter Su | | | Clear javadoc and check style issues around DomainSocket | Trivial |

| Kai Zheng | Kai Zheng | | | Exception may be swallowed in KMSClientProvider | Major | kms | Yongjun Zhang | Yongjun Zhang | | | Add links to failed volumes to explorer.html in HDFS Web UI | Minor | . | Archana T | Archana T | | | MR AM unable to load native library without MR\AM\ADMIN\USER\ENV set | Critical | mr-am | Sangjin Lee | Sangjin Lee | | | S3 filesystem operations stopped working correctly | Major | tools | Matthew Paduano | Matthew Paduano | | | ResourceManager crash due to scheduling opportunity overflow | Critical | resourcemanager | Jason Lowe | Jason Lowe | | | Fix variable name typo in DFSConfigKeys | Trivial | . | Ray Chiang | Ray Chiang | | | Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy | Critical | . | Gaurav Kanade | Gaurav Kanade | | | do not check replication if the block is under construction | Critical | . | Phil Yang | Phil Yang | | | SimulatedFSDataset sometimes can not find blockpool for the correct namenode | Major | datanode, test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix description about retention period in usage of expunge command | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | TestHttpServer.testBindAddress bind port range is wider than expected | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestFind.processArguments occasionally fails | Major | test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestRPC.testClientBackOff failing | Major | test | Steve Loughran | Xiao Chen | | | TestNetworkedJob fails consistently due to delegation token changes on RM. | Major | test | Rohith Sharma K S | Akira Ajisaka | | | Reduce client failures during datanode restart | Major | . | Kihwal Lee | Kihwal Lee | | | Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk | Major | test | Mingliang Liu | Tony Wu | | | Handle empty rename pending metadata file during atomic rename in redo path | Critical | fs/azure | madhumita chakraborty | madhumita chakraborty | | | TestCompressorDecompressor failing without stack traces | Critical | test | Steve Loughran | John Zhuge | | | Hadoop AuthToken refuses to work without a maxinactive attribute in issued token | Blocker | security | Steve Loughran | Benoy Antony | | | Introduce FileNotFoundException for WASB FileSystem API | Major | tools | Dushyanth | Dushyanth | | | Illegal progress value warnings in map tasks | Major | mrv2, task | Todd Lipcon | Binglin Chang | | | Inconsistent Logging in BootstrapStandby | Minor | ha | BELUGA BEHR | Xiaobing Zhou | | | NPE in distcp when ssl configuration file does not exist in class path. | Major | distcp | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Disable browsing the static directory in HttpServer2 | Major | security | Robert Kanter | Robert Kanter | | | javadoc failing on java 8 | Blocker | build | Steve Loughran | Steve Loughran | | | Nodemanager connection errors are retried at multiple levels | Major | nodemanager | Jason Lowe | Chang Li | | | Use SO\_REUSEADDR to avoid getting "Address already in use" when using kerberos and attempting to bind to any port on the local IP address | Major | net | Colin P. McCabe | Colin P. McCabe | |

TestSymlinkLocalFSFileContext#testSetTimesSymlinkToDir occasionally fail | Major | test | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Make app id/name available to the yarn authorizer provider for better auditing | Major | . | Jian He | Jian He | | | Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions | Minor | nodemanager | Karthik Kambatla | Sunil Govindan | | | Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes | Trivial | distcp | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove the redundant symbol in yarn rmadmin help msg | Trivial | . | Yiqun Lin | Yiqun Lin | | | Remove unused import in TestCompressorDecompressor.java | Minor | . | John Zhuge | John Zhuge | | | Fix typo in Job#setUseNewAPI | Trivial | . | Kai Sasaki | Kai Sasaki | | | FairScheduler headroom calculation should exclude nodes in the blacklist | Major | fairscheduler | zhihai xu | zhihai xu | | | TestStartup.testImageChecksum is broken by HDFS-9569's message change | Trivial | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp times out occasionally | Major | test | Jason Lowe | Sangjin Lee | | | AHS writer thread leak makes RM crash while RM is recovering | Major | resourcemanager | sandflee | sandflee | | | MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | "yarn.am.blacklisting.enabled" and "yarn.am.blacklisting.disable-failure-threshold" should be app specific rather than a setting for whole YARN cluster | Critical | applications | Junping Du | Sunil Govindan | | | DistCp worker threads are not terminated after jobs are done. | Major | distcp | Wei-Chiu Chuang | Wei-Chiu Chuang | | | DataNode start slowly due to the initial DU command operations | Major | . | Yiqun Lin | Yiqun Lin | | | Fix some cmake plugin and native build warnings | Minor | native | Colin P. McCabe | Colin P. McCabe | | | Fix hadoop-mapreduce-client-nativetask unit test which fails because it is not able to open the "glibc bug spill" file. | Major | native | Colin P. McCabe | Colin P. McCabe | | | QueueMetrics pending cores and memory metrics wrong | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities | Major | timelineserver | Li Lu | Li Lu | | | Fix two AM containers get allocated when AM restart | Critical | . | Yesha Vora | Vinod Kumar Vavilapalli | | | Update example configuration of block state change log in log4j.properties | Minor | logging | Masatake Iwasaki | Masatake Iwasaki | | | When sizeBasedWeight enabled for FairOrderingPolicy in CapacityScheduler, Sometimes lead to situation where all queue resources consumed by AMs only | Major | capacity scheduler, capacityscheduler | Karam Singh | Wangda Tan | | | Fix computing CPU usage statistics on Windows | Major | util | Yunqi Zhang | igo Goiri | | | Deadlock in DN.FsDatasetImpl between moveBlockAcrossStorage and createRbw | Major | datanode | ade | ade | | | NN should start JVM pause monitor before loading fsimage | Critical | . | John Zhuge | John Zhuge | | | Make leader elector and zk store share the same curator client | Major | . | Jian He | Jian He | | | Fix scheduler load simulator to support multi-layer network location | Major | . | Ming Ma | Ming Ma | | | Fix intermittent failure of

| Minor | test | Masatake Iwasaki | Masatake Iwasaki | | | set replication for empty file failed when set storage policy | Major | namenode | DENG FEI | DENG FEI | | | Handle failure of registering shutdownhook by ShutdownHookManager in static block | Minor | fs | Abhishek Agarwal | Abhishek Agarwal | | | webhdfs client side exceptions don't provide enough details | Major | webhdfs | Eric Payne | Eric Payne | | | Redundant code statement in WritingYarnApplications | Minor | documentation | Kai Sasaki | Kai Sasaki | | | Remove unnecessary oro package from dependency management section | Minor | build | Todd Lipcon | Sean Busbey | | | Reservations continue looking for one app causes other apps to starve | Blocker | capacityscheduler | Jason Lowe | Jason Lowe | | | Incorrect usage of config parameters in token manager of KMS | Major | security | Tianyin Xu | Mingliang Liu | | | RM startup failure when AM attempts greater than max-attempts | Critical | . | Bibin A Chundatt | Bibin A Chundatt | | | Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks | Minor | namenode | Masatake Iwasaki | Masatake Iwasaki | | | Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml | Major | documentation | Dong Zhen | Kai Sasaki | | | Spelling mistake in the help message of "yarn applicationattempt" command | Trivial | client | Manjunath Ballur | Weiwei Yang | | | Fix a typo "aplication" in HttpFS document | Trivial | documentation | Weiwei Yang | Weiwei Yang | | | Hadoop streaming -mapper and -reducer options are wrongly documented as required | Major | documentation | DeepakVohra | Kengo Seki | | | RM might fail to restart when recovering apps whose attempts are missing | Critical | . | Jun Gong | Jun Gong | | | Create API and command-line argument to get quota and quota usage without detailed content summary | Major | fs | Joep Rottinghuis | Ming Ma | | | Invalid event: RESOURCE\FAILED at CONTAINER\CLEANEDUP\AFTER\KILL | Major | nodemanager | tangshangwen | tangshangwen | | | Remove unused GetContainerStatus proto | Minor | . | Chang Li | Chang Li | | | FinishAppEvent is leaked in leveldb if no app's container running on this node | Major | nodemanager | sandflee | sandflee | | | JobHistoryEventHandler should not swallow timeline response | Trivial | . | Li Lu | Li Lu | | | ClientProtocol.addBlock is not idempotent after HDFS-8071 | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Fix git environment check during test-patch | Major | . | Ray Chiang | Allen Wittenauer | | | Fix rumen and scheduler load simulator handle killed tasks properly | Major | . | Ming Ma | Ming Ma | | | maven install triggers bats test | Major | build | Allen Wittenauer | Allen Wittenauer | | | core-default.xml misspells hadoop.workaround.non.threadsafe.getpwuid | Minor | . | Ray Chiang | Ray Chiang | | | Allow a file to be closed with COMMITTED but not yet COMPLETE blocks. | Major | namenode | Jitendra Nath Pandey | Tsz Wo Nicholas Sze | | | Fix findbugs warnings in OutputCommitter and FileOutputCommitter | Major | . | Akira Ajisaka | Akira Ajisaka | | | Streaming documentation contains a stray '%'

| Trivial | documentation | Chris Nauroth | Chris Nauroth | | | potential deadlock of CapacityScheduler between decrease container and assign containers | Major | capacityscheduler | sandflee | MENG DING | | | Fail to create jobhistory file if there are some multibyte characters in the job name | Major | jobhistoryserver | Akira Ajisaka | Kousuke Saruta | | | RMAppAttemptImpl#createApplicationAttemptReport throws IllegalArgumentException | Major | resourcemanager | yarntime | Bibin A Chundatt | | | LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy | Major | capacity scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Redirect RM page to AHS page when AHS turned on and RM page is not available | Major | . | Chang Li | Chang Li | | | EditLogTailerThread to Active Namenode RPC should timeout | Critical | ha, namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | YarnClientProtocolProvider leaking the YarnClient thread. | Major | . | Xuan Gong | Xuan Gong | | | Fix some misuse of %n in VolumeScanner#printStats | Minor | datanode | Xiaoyu Yao | Xiaoyu Yao | | | DN may deadlock when hot-swapping under load | Major | . | Xiao Chen | Xiao Chen | | | Decommisioned Nodes not listed in Web UI | Minor | resourcemanager | Bibin A Chundatt | Kuhu Shukla | | | FSImage may get corrupted after deleting snapshot | Major | namenode | Stanislav Antic | Yongjun Zhang | | | HAUtil#getConfForOtherNodes should unset independent generic keys before initialize | Major | namenode | DENG FEI | DENG FEI | | | FSNamesystem.initAuditLoggers() doesn't trim classnames | Minor | fs | Steve Loughran | Mingliang Liu | | | Findbug compilation fails for 'Kafka Library support' | Major | . | Rohith Sharma K S | Akira Ajisaka | | | Memory Leak in JobClient#submitJobInternal() | Major | . | Xuan Gong | Xuan Gong | | | Fix typo in defaultFS warning message | Trivial | . | Andrew Wang | Andrew Wang | | | Jobs that did not start are shown as starting in 1969 in the JHS web UI | Major | jobhistoryserver | Daniel Templeton | Haibo Chen | | | DatanodeStorage.isValidStorageId() is broken | Critical | hdfs-client | Kihwal Lee | Mingliang Liu | | | Use a reasonable limit in DFSTestUtil.waitForMetric() | Major | test | Kihwal Lee | Chang Li | | | Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent | Major | . | Xuan Gong | Xuan Gong | | | incremental maven build is not really incremental | Minor | build | Sangjin Lee | Sangjin Lee | | | container-executor fails to remove directory tree when chmod required | Major | nodemanager | Colin P. McCabe | Colin P. McCabe | | | When addExpectedReplicasToPending is called twice, pendingReplications should avoid duplication | Minor | . | Walter Su | Walter Su | | | Fix logging statements in resource manager's Application class | Trivial | . | Sidharta Seethana | Sidharta Seethana | | | Storage ID update does not happen when there is a layout change | Major | datanode | Kihwal Lee | Tsz Wo Nicholas Sze | | | Degraded performance in WebHDFS listing as it does not reuse ObjectMapper | Critical | performance | Akira Ajisaka | Akira Ajisaka | | | The default value of "hadoop.workaround.non.threadsafe.getpwuid" is different between core-default.xml and NativeIO.java | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Rebalancer sleeps too long between iterations | Blocker | balancer & mover | Adrian Bridgett | Mingliang Liu | |

DataXceiver#copyBlock should return if block is pinned | Major | datanode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | HBase classes fail to load with client/job classloader enabled | Major | util | Sangjin Lee | Sangjin Lee | | | Fix typos in DFSAdmin command line and documentation | Trivial | hdfs-client | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Example usage is not correct in Transparent Encryption document | Major | documentation | Takashi Ohnishi | Takashi Ohnishi | | | Permanent write failures may happen to slow writers during datanode rolling upgrades | Critical | . | Kihwal Lee | Walter Su | | | WebHDFS AuthFilter cannot be configured with custom AltKerberos auth handler | Major | webhdfs | Ryan Sasson | Ryan Sasson | | | TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value | Minor | test | Kuhu Shukla | Kuhu Shukla | | | Distributed shell breaks under strong security | Major | applications/distributed-shell, security | Daniel Templeton | Daniel Templeton | | | TestUserGroupInformation#testGetServerSideGroups fails in chroot | Minor | security, test | Eric Badger | Eric Badger | | | Incompatible tag renumbering in HeartbeatResponseProto | Blocker | rolling upgrades | Andrew Wang | Andrew Wang | | | KMS does not log detailed stack trace for unexpected errors. | Major | kms | Chris Nauroth | Chris Nauroth | | | TestKMS#testKMSProvider intermittently fails during 'test rollover draining' | Major | . | Xiao Chen | Xiao Chen | | | HDFS Balancer should exit with a proper message if upgrade is not finalized | Major | . | Xiaobing Zhou | Xiaobing Zhou | | | ReconfigurableBase should update the cached configuration | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | During WASB atomic rename handle crash when one directory has been renamed but not file under it. | Critical | fs/azure | madhumita chakraborty | madhumita chakraborty | | | Fix intermittent test failure of TestCopyPreserveFlag | Major | test | Tsuyoshi Ozawa | Masatake Iwasaki | | | "hadoop key" command usage is not documented | Major | documentation | Akira Ajisaka | Xiao Chen | | | TestBlockScanner#testVolumeIteratorWithCaching fails intermittently | Major | test | Mingliang Liu | Akira Ajisaka | | | BlockPlacementPolicyWithNodeGroup should override verifyBlockPlacement | Major | . | Junping Du | Xiaobing Zhou | | | FileSystem#listLocatedStatus causes unnecessary RPC calls | Major | fs, fs/s3 | Ryan Blue | Ryan Blue | | | SNNs stop uploading FSImage to ANN once isPrimaryCheckPointer changed to false. | Major | ha | Guocui Mi | Guocui Mi | | | Fix typo in mapreduce tutorial | Trivial | . | John Michael Luy | John Michael Luy | | | KMS SPNEGO sequence does not work with WEBHDFS | Major | kms, security | Xiaoyu Yao | Xiaoyu Yao | | | Move o.a.h.fs.Hdfs to hadoop-hdfs-client | Blocker | . | Haohui Mai | Vinayakumar B | | | Disk IO imbalance in HDFS with heterogeneous storages | Major | . | Wei Zhou | Wei Zhou | | | Reimplement getCurrentTrashDir to remove incompatibility | Blocker | . | Zhe Zhang | Zhe Zhang | | | FairScheduler: Cleanup preemptContainer to be more readable | Trivial | fairscheduler | Karthik Kambatla | Kai Sasaki | | | Yarn node label CLI should parse "=" correctly when trying to remove all labels on a node | Major | . | Wangda Tan | Naganarasimha G R | | | Fix the usage of 'hdfs namenode' command | Minor |

| Akira Ajisaka | Manjunath Ballur | | | Add -source flag to balancer usage message | Trivial | balancer & mover, documentation | Chris Trezzo | Chris Trezzo | | | DistCp doesn't print value for several arguments including -numListstatusThreads | Minor | distcp | Yongjun Zhang | Wei-Chiu Chuang | | | Testcase Failure : TestFileInputFormat.testSplitLocationInfo | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Reduce verbosity of processReport logging | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | skip StandbyException log for getCorruptFiles() | Minor | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Remove the extra char (\>) from SecureContainer.md | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | refreshNodesGracefully() should send recommission event to active RMNodes only | Minor | graceful | Kuhu Shukla | Kuhu Shukla | | | dfs.datanode.balance.bandwidthPerSec should accept friendly size units | Minor | balancer & mover | Yiqun Lin | Yiqun Lin | | | NMWebServices produces incorrect JSON for containers | Critical | . | Varun Saxena | Varun Saxena | | | Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException | Critical | . | Sergey Shelukhin | Junping Du | | | movetoqueue option does not documented in 'YARN Commands' | Major | documentation | Takashi Ohnishi | Takashi Ohnishi | | | Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption | Major | fairscheduler | Karthik Kambatla | Kai Sasaki | | | TestCacheDirectives#testExceedsCapacity is flaky | Major | . | Wei-Chiu Chuang | Xiao Chen | | | Out of band heartbeats are sent only on container kill and possibly too early | Major | nodemanager | Jason Lowe | Ming Ma | | | Correct path creation in getTrashRoot to handle root dir | Blocker | encryption | Zhe Zhang | Zhe Zhang | | | AsyncDispatcher logs redundant event queue sizes | Major | . | Jason Lowe | Jason Lowe | | | Some of the bin/hadoop subcommands are not available on Windows | Major | scripts | Kengo Seki | Kengo Seki | | | KerberosAuthenticator#doSpnegoSequence use incorrect class to determine isKeyTab in JDK8 | Major | security | Xiaoyu Yao | Xiaoyu Yao | | | SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | When task logs are not available, port 8041 is referenced instead of port 8042 | Major | yarn | Haibo Chen | Haibo Chen | | | RollingFileSystemSink can throw an NPE on non-secure clusters | Major | . | Daniel Templeton | Daniel Templeton | | | NodesListManager$UnknownNodeId ClassCastException | Critical | resourcemanager | Jason Lowe | Kuhu Shukla | | | TestSymlinkLocalFSFileSystem fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | LocalFS/FSOutputSummer NPEs in constructor if bytes per checksum set to 0 | Minor | fs | Steve Loughran | Mingliang Liu | | | Credential Provider Recursive Dependencies | Major | . | Larry McCay | Larry McCay | | | Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | container-executor should not follow symlinks in recursive\unlink\children | Blocker | . | Bibin A Chundatt | Colin P. McCabe | | | RetryPolicies (other than FailoverOnNetworkExceptionRetry) should put on retry failed reason or the log from RMProxy's retry could be very

| Critical | auto-failover | Junping Du | Junping Du | | | ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport | Major | timelineserver | Li Lu | Li Lu | | | TestCompositeService should cover all enumerations of adding a service to a parent service | Major | . | Karthik Kambatla | Ray Chiang | | | S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders | Major | fs/s3 | Larry McCay | Larry McCay | | | Remove unused imports from DFSUtil | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | block ID-based DN storage layout can be very slow for datanode on ext4 | Blocker | datanode | Nathan Roberts | Chris Trezzo | | | TestDatanodeRegistration fails occasionally | Major | test | Kihwal Lee | Kihwal Lee | | | DistributedFileSystem#getTrashRoot returns incorrect path for encryption zones | Critical | . | Andrew Wang | Andrew Wang | | | TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently | Major | test | Mingliang Liu | Xiao Chen | | | Name node throws NPE when setPermission is called on a path that does not exist | Critical | namenode | David Yan | Brahma Reddy Battula | | | Configuration properties for hedged read is broken | Blocker | . | Akira Ajisaka | Akira Ajisaka | | | Fix typo admininistration in CommandsManual.md | Minor | documentation | Akira Ajisaka | John Zhuge | | | DistCp documentation is out-of-dated | Major | . | Haohui Mai | Daisuke Kobayashi | | | Fix dead link to NativeLibraries.html in CommandsManual.md | Minor | documentation | Akira Ajisaka | Brahma Reddy Battula | | | Fix formatting in ServiceLevelAuth.md | Trivial | documentation | Akira Ajisaka | Brahma Reddy Battula | | | TestMRApp occasionally fails | Major | mrv2, test | Jason Lowe | Haibo Chen | | | NPE when trying to rename a directory in Windows Azure Storage FileSystem | Blocker | . | Robert Yokota | Robert Yokota | | | RMApps Page crashes with NPE | Major | webapp | Bibin A Chundatt | Bibin A Chundatt | | | NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler | Major | fairscheduler | Sangjin Lee | Sangjin Lee | | | Add CSRF filter support in YARN | Major | nodemanager, resourcemanager, webapp | Jonathan Maron | Jonathan Maron | | | Too many signal to container failure in case of LCE | Major | . | Bibin A Chundatt | Sidharta Seethana | | | proxy redirect to history server uses wrong URL | Major | webapp | Jason Lowe | Eric Badger | | | TestBlockReplacement fails intermittently in trunk | Major | test | Yiqun Lin | Yiqun Lin | | | NMs failing on DelegatingLinuxContainerRuntime init with LCE on | Blocker | . | Vinod Kumar Vavilapalli | Sidharta Seethana | | | Streamer threads leak if failure happens when closing DFSOutputStream | Major | hdfs-client | Yiqun Lin | Yiqun Lin | | | TestStandbyCheckpoints#testNonPrimarySBNUploadFSImage waitForCheckpoint incorrectly | Minor | test | Yiqun Lin | Yiqun Lin | | | Application submission fails when submitted queue is not available in scheduler xml | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | SSLFactory#createSSLSocketFactory exception message is wrong | Trivial | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix deadlinks in

| Major | documentation | Akira Ajisaka | Gabor Liptak | | | IPC Server should allow suppressing exception logging by type, not log 'server too busy' messages | Major | ipc | Arpit Agarwal | Arpit Agarwal | | | AuthenticatedURL should convert a 404/Not Found into an FileNotFoundException. | Minor | io, security | Steve Loughran | Steve Loughran | | | ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's | Major | tools | Colin P. McCabe | Colin P. McCabe | | | ReverseXML should be capitalized in oiv usage message | Minor | tools | Colin P. McCabe | Colin P. McCabe | | | External distribution stitching scripts do not work correctly on Windows. | Blocker | build | Chris Nauroth | Chris Nauroth | | | Download File from UI broken after pagination | Blocker | namenode | Brahma Reddy Battula | Brahma Reddy Battula | | | Block#toString should not output information from derived classes | Minor | . | Colin P. McCabe | Colin P. McCabe | | | SystemClock API broken in 2.9.0 | Major | . | Siddharth Seth | Siddharth Seth | | | testCheckpointCancellationDuringUpload occasionally fails | Major | test | Kihwal Lee | Yiqun Lin | | | JobStatus#getFailureInfo should not output diagnostic information when the job is running | Blocker | test | Rohith Sharma K S | Akira Ajisaka | | | RollingFileSystemSink doesn't work on secure clusters | Critical | . | Daniel Templeton | Daniel Templeton | | | Shell to disable bash and setsid support when running under JVM security manager | Major | security | Costin Leau | Costin Leau | | | Deadlock in AbstractService.getConfig() | Major | yarn | Steve Loughran | Steve Loughran | | | Typo in DataXceiverServer | Trivial | datanode | Michael Han | Michael Han | | | TestWordStats outputs logs under directories other than target/test-dir | Major | test | Akira Ajisaka | Gabor Liptak | | | TestFairScheduler#testContinuousScheduling fails intermittently | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Long living DataXceiver threads cause volume shutdown to block. | Critical | datanode | Rushabh S Shah | Rushabh S Shah | | | dfs.namenode.edits.dir.required missing from hdfs-default.xml | Major | documentation, namenode | Todd Lipcon | Mark Yang | | | Intel ISA-L libraries should be added to the Dockerfile | Blocker | io | Allen Wittenauer | Kai Zheng | | | SbNN Web UI shows #Under replicated blocks and #pending deletion blocks | Major | ha | Juan Yu | Wei-Chiu Chuang | | | MiniYARNCluster.start() returns before cluster is completely started | Major | test | Rohith Sharma K S | Eric Badger | | | [NNBench] Lease mismatch error when running with multiple mappers | Critical | benchmarks | Brahma Reddy Battula | Bibin A Chundatt | | | JobHistoryServer CLI throws NullPointerException with job ids that do not exist | Minor | jobhistoryserver | Li Lu | Kai Sasaki | | | PacketResponder#toString should include the downstreams for PacketResponderType.HAS\DOWNSTREAM\IN\_PIPELINE | Minor | datanode | Joe Pallas | Joe Pallas | | | Test failure : TestMRJobsWithProfiler | Major | . | Rohith Sharma K S | Eric Badger | | | [NNBench] OP\_DELETE operation isn't working after MAPREDUCE-6363 | Blocker | . | J.Andreina | J.Andreina | | | fuse\_dfs segfaults if uid cannot be resolved to a username | Major | fuse-dfs | John Thiltges | John Thiltges | | | ResourceManager web redirects in HA mode drops query parameters | Major | . | Varun Vasudev | Varun Vasudev | | |

isn't valid xml | Blocker | fairscheduler, test | Allen Wittenauer | Yufei Gu | | | TaskID default constructor results in NPE on toString() | Major | mrv2 | Daniel Templeton | Daniel Templeton | | | KMS key names are incorrectly encoded when creating key | Major | . | Xiao Chen | Xiao Chen | | | PhantomReference for filesystem statistics can trigger OOM | Major | . | Jason Lowe | Sangjin Lee | | | Remove MRv1 terms from HttpAuthentication.md | Major | documentation | Akira Ajisaka | Brahma Reddy Battula | | | Hedged read might overwrite user's buf | Major | . | zhouyingchao | zhouyingchao | | | Fix a typo (STRICT\_IE6) in Encrypted Shuffle | Trivial | documentation | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Add a way to change hadoop.security.token.service.use\_ip | Major | security | Robert Kanter | Robert Kanter | | | Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled | Minor | nodemanager | Jason Lowe | Jun Gong | | | Clear ASF Warnings on test data files | Minor | . | Vinayakumar B | Vinayakumar B | | | "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | AHS Security login should be in serviceInit() instead of serviceStart() | Major | timelineserver | Junping Du | Junping Du | | | TestFsDatasetCache failing intermittently due to timeout | Major | test | Yiqun Lin | Yiqun Lin | | | Reason for failing ipc.FairCallQueue contruction should be thrown | Minor | . | Archana T | Ajith S | | | TestHDFSCLI fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Track Reserved resources in ResourceUsage and QueueCapacities | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Lz4Compressor#getLibraryName returns the wrong version number | Trivial | native | John Zhuge | Colin P. McCabe | | | blocksScheduled count should be decremented for abandoned blocks | Major | namenode | Vinayakumar B | Vinayakumar B | | | peerFromSocketAndKey performs SASL exchange before setting connection timeouts | Major | . | Colin P. McCabe | Colin P. McCabe | | | Add .json to the rat exclusions | Blocker | build | Ming Ma | Ming Ma | | | JavaDocs for SignerSecretProvider are out-of-date in AuthenticationFilter | Major | documentation | Robert Kanter | Gabor Liptak | | | Clarify the behavior of timeline service config properties | Major | . | Mit Desai | Naganarasimha G R | | | Fix bugs in the initialization of the ISA-L library JNI bindings | Major | native | Kai Zheng | Kai Zheng | | | Javadoc comment is broken for Resources.multiplyByAndAddTo() | Trivial | . | Daniel Templeton | Daniel Templeton | | | Fix TestRefreshCallQueue failure. | Major | . | Brahma Reddy Battula | Xiaoyu Yao | | | yarn web services should convert parse failures of appId, appAttemptId and containerId to 400 | Minor | webapp | Steve Loughran | Bibin A Chundatt | | | TestDecommissioningStatus.testDecommissionStatus occasionally fails | Major | namenode | Wei-Chiu Chuang | Yiqun Lin | | | UI Hosting Configuration in TimelineServer doc is broken | Critical | documentation | Akira Ajisaka | Akira Ajisaka | | | Permanent write failures can happen if pipeline recoveries occur for the first packet | Critical |

| Kihwal Lee | Kihwal Lee | | | Calling stopWriter() with FSDatasetImpl lock held may block other threads | Major | . | zhouyingchao | Colin P. McCabe | | | IBR accumulate more objects when SNN was down for sometime. | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE | Major | . | Rohith Sharma K S | Sunil Govindan | | | RM Nodes list page takes too much time to load | Major | webapp | Bibin A Chundatt | Bibin A Chundatt | | | Fsshell mv fails if port usage doesn't match in src and destination paths | Major | . | Kuhu Shukla | Kuhu Shukla | | | Fix some intermittent test failures in TestRMAdminService | Blocker | . | Junping Du | Brahma Reddy Battula | | | TestNMProxy.tesNMProxyRPCRetry fails. | Minor | . | Tibor Kiss | Tibor Kiss | | | Fix typo in YARN Secure Containers documentation | Trivial | documentation, yarn | Takashi Ohnishi | Takashi Ohnishi | | | Fix typos in documentation of Capacity Scheduler. | Minor | documentation | Takashi Ohnishi | Takashi Ohnishi | | | TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports | Major | . | Eric Badger | Eric Badger | | | Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node | Critical | capacity scheduler | Sunil Govindan | Sunil Govindan | | | Capture container start/finish time in container metrics | Major | . | Jian He | Jian He | | | fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits | Blocker | build | Allen Wittenauer | Allen Wittenauer | | | TestJobListCache#testEviction sometimes fails on Windows with timeout | Minor | test | Gergely Novk | Gergely Novk | | | Namenode safemode not coming out during failover | Major | namenode | Brahma Reddy Battula | Brahma Reddy Battula | | | hdfs fsck -move passes invalid length value when creating BlockReader | Major | . | Shengjun Xin | Xiao Chen | | | MR usage counters use the resources requested instead of the resources allocated | Major | . | Haibo Chen | Haibo Chen | | | Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose | Major | datanode | Colin P. McCabe | Colin P. McCabe | | | FileContextMainOperationsBaseTest.testListStatusThrowsExceptionForNonExistentFile() doesnt run | Minor | test | Steve Loughran | Kai Sasaki | | | AM may not receive the container complete msg when it restarts | Major | . | sandflee | sandflee | | | AM should retry map attempts if the reduce task encounters commpression related errors. | Major | . | Rushabh S Shah | Rushabh S Shah | | | TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler | Major | test | Karthik Kambatla | Bibin A Chundatt | | | MiniYarnCluster should not request transitionToActive to RM on non-HA environment | Major | test | Akira Ajisaka | Eric Badger | | | YARN WebApp ignores the configuration passed to it for keystore settings | Major | . | Sergey Shelukhin | Sergey Shelukhin | | | AbstractMapWritable.readFields throws ClassNotFoundException with custom writables | Blocker | io | Nadeem Douba | Nadeem Douba | | | Change ShutdownHookManger complete shutdown log from INFO to DEBUG | Minor | . | Xiaoyu Yao | Xiaoyu Yao | | | dataTables\_wrapper change min height | Minor |

| Bibin A Chundatt | Bibin A Chundatt | | | PositionedReadable test testReadFullyZeroByteFile failing in HDFS | Major | test | Steve Loughran | Steve Loughran | | | Extra bytes are getting released from reservedSpace for append | Critical | . | Brahma Reddy Battula | Brahma Reddy Battula | | | Document missing properties in hdfs-default.xml | Major | documentation | Ray Chiang | Ray Chiang | | | NM applicationpage cause internal error 500 | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Remove duplicate logSync() and log message in FSN#enterSafemode() | Minor | . | Vinayakumar B | Vinayakumar B | | | Http server vulnerable to clickjacking | Major | . | Haibo Chen | Haibo Chen | | | Deadlock in NMClientImpl | Critical | . | Sumana Sathish | Jian He | | | TestBlockReplacement#testThrottler doesn't work as expected | Minor | test | Yiqun Lin | Yiqun Lin | | | TestJMXGet:testNameNode() fails | Minor | test | Andras Bokor | Gergely Novk | | | The VolumeScanner should warn about replica files which are misplaced | Major | datanode | Colin P. McCabe | Colin P. McCabe | | | distcp -diff relative path exception | Major | distcp | John Zhuge | Takashi Ohnishi | | | NM recovery race can lead to container not cleaned up | Major | nodemanager | Nathan Roberts | sandflee | | | Some tests in org.apache.hadoop.fs.shell.find occasionally time out | Major | test | Akira Ajisaka | Takashi Ohnishi | | | Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator | Critical | . | Xuan Gong | Xuan Gong | | | o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | yarn node -list -all failed if RM start with decommissioned node | Major | . | sandflee | sandflee | | | Distributed shell AM failed due to ClientHandlerException thrown by jersey | Critical | . | Sumana Sathish | Junping Du | | | Reserved Resource for QueueMetrics needs to be handled correctly in few cases | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | TestShortCircuitLocalRead failing | Major | test | Steve Loughran | Steve Loughran | | | TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly | Major | test | Yiqun Lin | Yiqun Lin | | | getFailureInfo not returning any failure info | Major | . | Eric Badger | Eric Badger | | | OEV tool fails to read edit xml file if OP\UPDATE\BLOCKS has no BLOCK tag | Minor | tools | Wan Chang | Wan Chang | | | SafeModeMonitor should not leave safe mode if name system is starting active service | Major | namenode | Mingliang Liu | Mingliang Liu | | | TestDirectoryScanner#testThrottling occasionally time out after 300 seconds | Minor | datanode | Wei-Chiu Chuang | Yiqun Lin | | | TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing | Major | test | Rakesh R | Rakesh R | | | Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. | Major | namenode | Chris Nauroth | Chris Nauroth | | | JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.) | Major | jobhistoryserver | Junping Du | Junping Du | |

TestYarnClient#testSubmitIncorrectQueue fails with FairScheduler | Major | test | Yufei Gu | Yufei Gu | | | MRBench: setting the baseDir parameter has no effect | Minor | benchmarks | Michael Noll | Wilfred Spiegelenburg | | | Missing NullPointer check in ContainerLaunchContextPBImpl causes RM to die | Major | resourcemanager | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola | | | DistCp throws NPE when source is root | Major | distcp | Yongjun Zhang | John Zhuge | | | Restore lost leveldbjni LICENSE and NOTICE changes | Major | . | Andrew Wang | Andrew Wang | | | Add LICENSE.txt entries for bundled javascript dependencies | Major | . | Andrew Wang | Andrew Wang | | | Balancer should not try to pair storages with different types | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga) | Minor | balancer & mover | Amit Anand | Amit Anand | | | LazyPersistFileScrubber should still sleep if there are errors in the clear progress | Major | . | Phil Yang | Phil Yang | | | ChecksumFileSystem mishandles crc file permissions | Major | fs | Daryn Sharp | Daryn Sharp | | | WebHdfsFileSystem#runWithRetry should display original stack trace on error | Major | test | Kihwal Lee | Wei-Chiu Chuang | | | Property "io.compression.codec.lzo.class" does not work with other value besides default | Major | io | fang fang chen | Yuanbo Liu | | | TestJMXGet hides the real error in case of test failure | Minor | test | Andras Bokor | Andras Bokor | | | TestDFSShell#testMoveWithTargetPortEmpty fails | Major | test | Takanobu Asanuma | Takanobu Asanuma | | | ContainerMetrics drops records | Major | resourcemanager | Daniel Templeton | Daniel Templeton | | | TestFifoScheduler.testResourceOverCommit fails | Major | scheduler, test | Akihiro Suda | Akihiro Suda | | | Bad initialisation of StringBuffer in RequestHedgingProxyProvider.java | Minor | ha | Max Schaefer | Yiqun Lin | | | Distcp need to enforce the order of snapshot names passed to -diff | Major | distcp | Yongjun Zhang | Yiqun Lin | | | JVM and UGI metrics disappear after RM is once transitioned to standby mode | Major | . | Li Lu | Li Lu | | | Handle special characters in passwords in KMS startup script | Major | kms | Xiao Chen | Xiao Chen | | | Add retry for SocketTimeoutException in TimelineClient | Critical | . | Xuan Gong | Xuan Gong | | | BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. | Major | . | Kuhu Shukla | Kuhu Shukla | | | LeveldbRMStateStore database can grow substantially leading to long recovery times | Major | resourcemanager | Jason Lowe | Jason Lowe | | | NMLeveldbStateStoreService database can grow substantially leading to longer recovery times | Major | nodemanager | Jason Lowe | Jason Lowe | | | Fix findbugs warnings in hadoop-tools module | Major | tools | Akira Ajisaka | Akira Ajisaka | | | TestFsDatasetImpl#testCleanShutdownOfVolume often fails | Major | datanode, test | Wei-Chiu Chuang | Rushabh S Shah | | | Mover$Processor#chooseTarget() always chooses the first matching target storage group | Critical | balancer & mover | Mingliang Liu | Mingliang Liu | | | TestTeraSort fails on Windows | Minor | test | Tibor Kiss | Tibor Kiss | | | Namenode report bad block method doesn't log the bad block or datanode. | Minor | namenode | Rushabh S Shah | Rushabh S Shah | | | yetus-wrapper should fail sooner when download fails | Minor | yetus | Steven K. Wong | Steven

Wong | | | WindowsGetSpaceUsed constructor should be public | Major | . | Vinayakumar B | Vinayakumar B | | | Include hadoop-pipes examples in the release tarball | Blocker | pipes | Allen Wittenauer | Kai Sasaki | | | Fix hadoop-hdfs-native-client compilation on Windows | Blocker | build | Brahma Reddy Battula | Brahma Reddy Battula | | | DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash | Major | . | Namit Maheshwari | Xiaoyu Yao | | | Refresh time in SysInfoWindows is in nanoseconds | Major | util | igo Goiri | igo Goiri | | | ProcfsBasedProcessTree doesn't track daemonized processes | Major | nodemanager | Nathan Roberts | Nathan Roberts | | | distcp should not ignore the ignoreFailures option | Critical | tools/distcp | Gera Shegalov | Mingliang Liu | | | Rack failures may result in NN terminate | Major | . | Xiao Chen | Xiao Chen | | | LocalContainerAllocator doesn't specify resource of the containers allocated. | Major | mr-am | Haibo Chen | Haibo Chen | | | TestJobImpl.testUnusableNode failed | Major | mrv2 | Haibo Chen | Haibo Chen | | | Removing nodes from include and exclude lists will not remove them from decommissioned nodes list | Major | . | Kuhu Shukla | Kuhu Shukla | | | LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. | Critical | log-aggregation | Junping Du | Junping Du | | | Handle special characters in passwords in httpfs.sh | Major | . | Xiao Chen | Xiao Chen | | | Dynamic LogLevel setting page should accept case-insensitive log level string | Major | . | Junping Du | Junping Du | | | Trash directory in an encryption zone should be pre-created with correct permissions | Major | encryption | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Job hangs as ask is not updated after ramping down of all reducers | Blocker | applicationmaster | Varun Saxena | Varun Saxena | | | TestHFlush failing intermittently | Major | test | Aaron T. Myers | Yiqun Lin | | | getApplicationReport call may raise NPE for removed queues | Critical | . | Sumana Sathish | Jian He | | | MapReduce job can infinitely increase number of reducer resource requests | Blocker | . | Wangda Tan | Wangda Tan | | | AHS error 500 due to NPE when container start event is missing | Major | timelineserver | Jason Lowe | Varun Saxena | | | Increase DecompressorStream skip buffer size | Major | . | Yongjun Zhang | John Zhuge | | | typo in HDFSHighAvailabilityWithQJM doc. | Minor | documentation | Takashi Ohnishi | Takashi Ohnishi | | | DelegationTokenRenewer#skipTokenRenewal may throw NPE | Major | . | Jian He | Jian He | | | Change nodelabel rest API invalid reponse status to 400 | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume | Major | test | Rushabh S Shah | Rushabh S Shah | | | High contention on scanning of user directory under immediate\_done in Job History Server | Critical | jobhistoryserver | Haibo Chen | Haibo Chen | | | Fix IOUtils#cleanup

IOUtils#closeStream javadoc | Trivial | io | Wei-Chiu Chuang | Wei-Chiu Chuang | | | getAvailablePhysicalMemorySize can be inaccurate on linux | Major | nodemanager | Nathan Roberts | Nathan Roberts | | | Remove synchronized input streams from Writable deserialization | Major | io | Gopal V | Gopal V | | | FS Contract tests don't report FS initialization errors well | Minor | test | Steve Loughran | Steve Loughran | | | RM needs to send update event with YarnApplicationState as Running to ATS/AHS | Critical | . | Xuan Gong | Xuan Gong | | | Jets3tNativeS3FileSystemContractTest does not run. | Minor | test | Chris Nauroth | Chris Nauroth | | | Process hangs in LocatedFileStatusFetcher if FileSystem.get throws | Major | mrv2 | Ryan Blue | Ryan Blue | | | Change log message "token.Token: Cannot find class for token kind kms-dt" to debug | Major | kms, security | Yi Liu | Yi Liu | | | multibyte delimiters with compressed input files generate duplicate records | Major | mrv1, mrv2 | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | The number of javadocs warnings is limited to 100 | Critical | . | Li Lu | Gergely Novk | | | MR job got hanged forever when one NM unstable for some time | Critical | applicationmaster, resourcemanager | Bob.zhao | Varun Saxena | | | Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk | Major | hdfs | Yongjun Zhang | Yiqun Lin | | | hadoop credential commands non-obviously use password of "none" | Major | security | Mike Yoder | Mike Yoder | | | Nodemanager log handlers fail to send finished/failed events in some cases | Critical | . | Junping Du | Junping Du | | | RedundantEditLogInputStream#LOG is set to wrong class | Minor | . | John Zhuge | John Zhuge | | | Addendum for HDFS-9579: to handle the case when client machine can't resolve network path | Major | . | Ming Ma | Ming Ma | | | Cannot create space quota of zero | Major | fs | Takashi Ohnishi | Takashi Ohnishi | | | ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit | Critical | . | Bibin A Chundatt | Ajith S | | | Reuse pre-computed filestatus in Distcp-CopyMapper | Minor | tools/distcp | Rajesh Balamohan | Rajesh Balamohan | | | DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. | Major | hdfs-client | Surendra Singh Lilhore | Surendra Singh Lilhore | | | application master log can not be available when clicking jobhistory's am logs link | Critical | jobhistoryserver | chenyukang | Haibo Chen | | | Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | Fix formatting of CacheAdmin command usage help text | Major | caching | Yiqun Lin | Yiqun Lin | | | job history server can fail on startup when NameNode is in start phase | Major | jobhistoryserver | Haibo Chen | Haibo Chen | | | Fix potential NPE in Metrics2 source for DecayRpcScheduler | Major | ipc | Xiaoyu Yao | Xiaoyu Yao | | | Distcp should ignore -delete option if -diff option is provided instead of exiting | Major | distcp | Mingliang Liu | Mingliang Liu | | | DataStreamer DataNode exclusion log message should be warning | Minor | hdfs-client | John Zhuge | John Zhuge | | | DataNode may format directory and lose blocks if current/VERSION is missing | Major | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils | Major | test | Josh Elser | Josh Elser | | | Unable to append to a SequenceFile with Compression.NONE. | Critical |

| Gervais Mickal | Vinayakumar B | | | Follow-on improvements to hadoop credential commands | Major | security | Mike Yoder | Mike Yoder | | | Resource Manager web ui issue when sorting application by id (with application having id \> 9999) | Major | resourcemanager | LINTE | Varun Saxena | | | TestContainerMetrics fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | Native tests fail on OS X, because DYLD\LIBRARY\PATH is not defined to include libhadoop.dylib. | Minor | build | Chris Nauroth | Chris Nauroth | | | update apache httpclient version to 4.5.2; httpcore to 4.4.4 | Major | build | Artem Aliev | Artem Aliev | | | The YarnApplicationState is always running in ATS no matter the application is running or finishes. | Blocker | . | Xuan Gong | Xuan Gong | | | YARN web interfaces lack XFS protection | Major | nodemanager, resourcemanager, timelineserver | Jonathan Maron | Jonathan Maron | | | S3A proxy tests fail after httpclient/httpcore upgrade. | Major | fs/s3 | Chris Nauroth | Steve Loughran | | | Fix Documentation for Yarn Capacity Scheduler on Resource Calculator | Minor | . | Jo Desmet | Takashi Ohnishi | | | DatanodeLifelineProtocol not able to use under security cluster | Blocker | . | gu-chi | Chris Nauroth | | | When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. | Minor | ha, namenode | KWON BYUNGCHANG | Chris Nauroth | | | Update setOwner doc in HdfsPermissionsGuide | Minor | documentation | John Zhuge | John Zhuge | | | Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set | Minor | applicationmaster | Maysam Yabandeh | Kai Sasaki | | | Excessive log warnings for directory permission issue on NM recovery. | Major | . | Jian He | Jian He | | | With NM recovery enabled, restarting NM multiple times results in AM restart | Critical | yarn | Sumana Sathish | Junping Du | | | max apps per user can be larger than max per queue | Minor | capacityscheduler, resourcemanager | Jason Lowe | Eric Badger | | | FSAppAttempt demand calculation considers demands at multiple locality levels different | Major | fairscheduler | zhihai xu | zhihai xu | | | Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery | Critical | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | Balancer does not work with the HDFS-6376 HA setup | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers | Major | . | Arun Suresh | Arun Suresh | | | Distributed shell AM fails when extra container arrives during finishing | Major | . | Sumana Sathish | Wangda Tan | | | Fix warning - procfs stat file is not in the expected format | Major | . | Jon Bringhurst | Ravindra Kumar Naik | | | container-executor should only kill process groups | Major | nodemanager | Jun Gong | Jun Gong | | | FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy | Major | fairscheduler | Karthik Kambatla | Yufei Gu | | | TestRollingFileSystemSinkWithHdfs needs some cleanup | Critical | . | Daniel Templeton | Daniel Templeton | | | javadoc:javadoc goal fails on hadoop-yarn-client | Major |

| Andras Bokor | Andras Bokor | | | HDFS should not expose path info that user has no permission to see. | Major | fs, security | Kevin Cox | Yuanbo Liu | | | Exclude generated protobuf sources from YARN Javadoc build | Critical | . | Subru Krishnan | Subru Krishnan | | | Handle ClassCastException on AuthenticationException in LoadBalancingKMSClientProvider | Major | kms | Miklos Szurap | Wei-Chiu Chuang | | | TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called | Major | test | Sangjin Lee | Mingliang Liu | | | TraceAdmin should support Kerberized cluster | Major | tracing | Wei-Chiu Chuang | Wei-Chiu Chuang | | | TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail | Major | . | Wei-Chiu Chuang | Masatake Iwasaki | | | TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Grep job in Single Cluster document fails | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | DFSAdmin#SetQuotaCommand's help msg is not correct | Minor | documentation | Yiqun Lin | Yiqun Lin | | | Yarn Application log Aggreagation fails due to NM can not get correct HDFS delegation token | Major | yarn | Yesha Vora | Jian He | | | Implement TokenRenewer to renew and cancel delegation tokens in KMS | Major | kms, security | Xiao Chen | Xiao Chen | | | HTTPFS server should correctly impersonate as end user to open file | Major | httpfs | Xiao Chen | Xiao Chen | | | Fix findbugs warning in FSEditLog.java | Major | . | Akira Ajisaka | Akira Ajisaka | | | getFileEncryptionInfo should return quickly for non-encrypted cluster | Major | encryption, namenode | Zhe Zhang | Zhe Zhang | | | HDFS read ends up ignoring an interrupt | Major | . | Siddharth Seth | Jing Zhao | | | RegistrySecurity includes id:pass in exception text if considered invalid | Minor | client, security | Steve Loughran | Steve Loughran | | | A large number of expired leases can make namenode unresponsive and cause failover | Major | namenode | Nicolas Fraison | Nicolas Fraison | | | Hadoop client displays confusing error message | Major | client | Mohammad Kamrul Islam | Gera Shegalov | | | ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats | Major | nodemanager | Sunil Govindan | Sunil Govindan | | | DFSInputStream should set thread's interrupt status after catching InterruptException from sleep | Major | . | Jing Zhao | Jing Zhao | | | RetryInvocationHandler need wrap InterruptedException in IOException when call Thread.sleep | Major | ipc | zhihai xu | zhihai xu | | | Small Documentation bug with AuthenticatedURL in hadoop-auth | Minor | documentation | Tom Ellis | Tom Ellis | | | TestRollingFileSystemSink.testSetInitialFlushTime() fails intermittently | Minor | test | Daniel Templeton | Daniel Templeton | | | Add -q option to Ls to print ? instead of non-printable characters | Major | . | John Zhuge | John Zhuge | | | Fix bug when warming up EDEK cache of more than one encryption zone | Major | encryption, namenode | Xiao Chen | Xiao Chen | | | BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks | Critical | . | Haibo Chen | Kai Sasaki | | | GenericOptionsParser is not thread-safe because commons-cli OptionBuilder is not thread-safe | Minor | . | hongbin ma | hongbin ma | |

o.a.h.ipc.Server#Server should honor handlerCount when queueSizePerHandler is specified in consturctor | Minor | ipc | Xiaoyu Yao | Kai Sasaki | | | Fix up some misc create-release issues | Blocker | build | Allen Wittenauer | Allen Wittenauer | | | OIV's ReverseXML processor should support ACLs | Major | tools | Colin P. McCabe | Surendra Singh Lilhore | | | Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap | Major | caching | Xiao Chen | Xiao Chen | | | Fix missing log files issue in rolling log aggregation. | Major | . | Siddharth Seth | Xuan Gong | | | Typo in RollingUpgrade docs | Major | documentation | Arpit Agarwal | Yiqun Lin | | | "s3:" URLs break when Secret Key contains a slash, even if encoded | Minor | fs/s3 | Stuart Sierra | Steve Loughran | | | TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky | Major | fs, hdfs-client | Wei-Chiu Chuang | Wei-Chiu Chuang | | | KMSClientProvider should check and renew tgt when doing delegation token operations. | Major | kms | Xiao Chen | Xiao Chen | | | Authenticate to Azure Data Lake using client ID and keys | Major | fs/adl | Atul Sikaria | Atul Sikaria | | | DecayRpcScheduler MXBean should only report decayed CallVolumeSummary | Major | ipc | Namit Maheshwari | Xiaoyu Yao | | | Windows distro build fails on dist-copynativelibs. | Blocker | build | Chris Nauroth | Chris Nauroth | | | Fix FSLeafQueue#getFairShare() for queues with zero fairshare | Major | . | Yufei Gu | Yufei Gu | | | ReconfigurationProtocol not covered by HDFSPolicyProvider. | Major | namenode | Chris Nauroth | Arpit Agarwal | | | NMWebAppFilter web redirects drop query parameters | Major | . | Varun Vasudev | Varun Vasudev | | | org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly | Critical | util | binde | binde | | | CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 | Major | caching | Yiqun Lin | Yiqun Lin | | | TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains '+'. | Minor | fs/s3, test | Chris Nauroth | Chris Nauroth | | | Cache MapOutputLocations in ShuffleHandler | Major | . | Siddharth Seth | Junping Du | | | Wrong exit code while trying to get app logs using regex via CLI | Critical | yarn | Sumana Sathish | Xuan Gong | | | test\native\mini\_dfs fails by NoClassDefFoundError | Major | native, test | Akira Ajisaka | Akira Ajisaka | | | Unable to loadFSEdits due to a failure in readCachePoolInfo | Critical | caching, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Javadoc for CLI#listEvents() contains no-existent param | Minor | client, documentation | Shen Yinjie | Shen Yinjie | | | WebHdfs logging is missing on Datanode | Critical | . | Kihwal Lee | Kihwal Lee | | | Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll

| Critical | auto-failover | XingFeng Shen | XingFeng Shen | | | HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe | Major | jobhistoryserver | zhangyubiao | zhangyubiao | | | Authenticate with Kerberos credentials when renewing KMS delegation token | Major | kms | Xiao Chen | Xiao Chen | | | Enforce Kerberos authentication for required ops in DelegationTokenAuthenticator | Blocker | kms, security | Xiao Chen | Xiao Chen | | | Counters page display all task neverthless of task type( Map or Reduce) | Minor | jobhistoryserver | Rohith Sharma K S | Kai Sasaki | | | KerberosName static initialization gets default realm, which is unneeded in non-secure deployment. | Major | security | Chris Nauroth | Dmytro Kabakchei | | | hdfs dfs -setfacl error message is misleading | Minor | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | MockNodes.newNodes creates one more node per rack than requested | Major | test | Karthik Kambatla | Karthik Kambatla | | | Pad hostname correctly in CredentialsSys.java | Critical | nfs | Pradeep Nayak Udupi Kadbet | Pradeep Nayak Udupi Kadbet | | | Fix typos in CapacityScheduler documentation | Trivial | documentation | Ray Chiang | Ray Chiang | | | Remove 'package-info.java' from 'test\\java\\org\\apache\\hadoop\\fs\\shell\\' to remove eclipse compile error | Trivial | . | Vinayakumar B | Vinayakumar B | | | Fix TestOfflineEditsViewer#testGenerated | Blocker | test | Xiaoyu Yao | Surendra Singh Lilhore | | | Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params | Minor | documentation, hdfs | Shen Yinjie | Shen Yinjie | | | Add RPC port info in RM web service's response when getting app status | Major | . | Jun Gong | Jun Gong | | | ISAL download is breaking the Dockerfile | Blocker | build | Allen Wittenauer | Allen Wittenauer | | | Pending on synchronized method DirectoryCollection#checkDirs can hang NM's NodeStatusUpdater | Critical | nodemanager | Junping Du | Junping Du | | | NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl | Major | nodemanager | Karam Singh | Junping Du | | | Pass remote ip address down to YarnAuthorizationProvider | Major | . | Jian He | Jian He | | | Additional fix to LICENSE and NOTICE | Blocker | build | Xiao Chen | Xiao Chen | | | Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning | Major | test | Rakesh R | Rakesh R | | | HADOOP\CLASSPATH vs HADOOP\USER\_CLASSPATH | Blocker | scripts | Allen Wittenauer | Allen Wittenauer | | | Fix arguments check in documentation for WordCount v2.0 | Minor | documentation | niccolo becchi | niccolo becchi | | | ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore | Major | timelineserver | Karam Singh | Li Lu | | | the decommissioning Node should keep alive if NM restart | Major | . | sandflee | sandflee | | | Verify LICENSE.txt and NOTICE.txt | Blocker | build | Allen Wittenauer | Xiao Chen | | | Make X-FRAME-OPTIONS configurable in HttpServer2 | Major | net, security | Anu Engineer | Anu Engineer | | | TestBalancer failing intermittently because of not reseting UserGroupInformation completely | Major | test | Yiqun Lin | Yiqun Lin | | | VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks | Major | datanode | Wei-Chiu Chuang | Yiqun Lin | | | Fix OOM issue in DistributedShell. AM failed with "java.lang.OutOfMemoryError: GC overhead limit exceeded" | Major | . | Sumana Sathish | Jian He | | | Add missing dependency in setting maven-remote-resource-plugin to fix builds | Major | build | Akira Ajisaka | Sean Busbey | | | Solve miscellaneous issues caused by YARN-4844 | Blocker | . | Wangda Tan | Wangda Tan | | | TestDistributedShell.checkTimelineV2() may fail for concurrent tests | Major | timelineserver | Sangjin Lee | Sangjin Lee | | |

may fail for concurrent tests | Major | test | Sangjin Lee | Sangjin Lee | | | HDFS web interfaces lack configs for X-FRAME-OPTIONS protection | Major | datanode, namenode | Anu Engineer | Anu Engineer | | | Fix Lint Warnings in YARN Common | Major | yarn | Daniel Templeton | Daniel Templeton | | | TestCLI#testGetJob fails occasionally | Major | test | Jason Lowe | Haibo Chen | | | FileContext#umask is not initialized properly | Minor | . | John Zhuge | John Zhuge | | | ResourceManager can leak delegation tokens when they are shared across apps | Critical | resourcemanager | Jason Lowe | Jason Lowe | | | Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures | Major | . | Rakesh R | Rakesh R | | | Fix a race condition in MetricsSourceAdapter.updateJmxCache | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail | Major | . | Jason Lowe | sandflee | | | FileSystemTimelineReader/Writer uses unix-specific default storage path | Minor | . | Sangjin Lee | Sangjin Lee | | | timelineservice modules have indirect dependencies on mapreduce artifacts | Minor | timelineserver | Sangjin Lee | Sangjin Lee | | | Balancer doesn't work with IPFailoverProxyProvider | Major | balancer & mover, ha | Zhe Zhang | Zhe Zhang | | | PendingReconstructionBlocks.size() should be synchronized | Major | . | Eric Badger | Eric Badger | | | TestDFSClientSocketSize buffer size tests are flaky | Major | . | Aaron Fabbri | Aaron Fabbri | | | TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently | Minor | test, timelineserver | Sangjin Lee | Vrushali C | | | Fix findbugs for nodemanager & checkstyle warnings in nodemanager.ContainerExecutor | Major | nodemanager | Vrushali C | Vrushali C | | | MapReduce JerseyTest tests failing with "java.net.BindException: Address already in use" | Critical | test | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | Create framework for configurable disk checkers | Major | util | Yufei Gu | Yufei Gu | | | Handle queue names consistently in FairScheduler | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Fix typos in HDFSDiskbalancer.md | Trivial | documentation | Akira Ajisaka | Yiqun Lin | | | RawLocalFileSystem does not react on changing umask | Major | fs | Andras Bokor | Andras Bokor | | | Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes | Major | util | zhengbing li | Kai Sasaki | | | Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently | Major | test | Rohith Sharma K S | Xuan Gong | | | Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot | Major | hdfs, namenode | Yongjun Zhang | Yiqun Lin | | | Conflicting default ports in DelegateToFileSystem | Major | fs | Kevin Hogeland | Kai Sasaki | | | Fix SSLFactory truststore reloader thread leak in TimelineClientImpl | Blocker | timelineserver, yarn | Thomas Friedrich | Weiwei Yang | | | Race condition in RollingLevelDBTimelineStore#getAndSetStartTime() | Critical | timelineserver | Sumana Sathish | Li Lu | | | Document HA support in the HDFS HdfsDesign.md | Major | documentation | Ravindra Babu | Lars Francke | | | Fix intermittent test failure of TestGangliaMetrics | Major | . | Tsuyoshi Ozawa | Masatake Iwasaki | | | TestAclCommands.testSetfaclValidations fail | Minor | test | linbao111 | John Zhuge | | | [Azure Data Lake] Correction needed in

support in documentation | Major | fs/adl | Vishwajeet Dusane | Vishwajeet Dusane | | | [Azure Data Lake] OAuth2 configuration should be default set to true when AdlFileSystem is configured. | Major | fs/adl | Vishwajeet Dusane | Vishwajeet Dusane | | | TestS3ATemporaryCredentials.testSTS error when using IAM credentials | Major | fs/s3 | Steven K. Wong | Steven K. Wong | | | S3AFileSystem: Consider reusing filestatus in delete() and mkdirs() | Minor | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Increase timeout on TestDFSIO tests | Major | . | Eric Badger | Eric Badger | | | BPServiceActor may run into a tight loop for sending block report when hitting IOException | Major | datanode | Jing Zhao | Chen Liang | | | Fix typo in HdfsRollingUpgrade.md | Trivial | documentation | Yiqun Lin | Yiqun Lin | | | ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process | Major | . | Sergey Shelukhin | Sergey Shelukhin | | | Use JUnit assumptions to skip platform-specific tests | Major | test | Arpit Agarwal | Gergely Novk | | | Fixing minor Scheduler test case failures | Major | . | Subru Krishnan | Subru Krishnan | | | TimeLineReader daemon start should allow to pass its own reader opts | Major | scripts, timelinereader | Rohith Sharma K S | Rohith Sharma K S | | | TestHDFSCLI fails | Major | test | Akira Ajisaka | Kai Sasaki | | | Lock already held by another process while LevelDB cache store creation for dag | Critical | timelineserver | Karam Singh | Li Lu | | | Use AHSClient in YarnClient when TimelineServer is running | Major | . | Xuan Gong | Xuan Gong | | | TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM | Major | timelineserver | Karam Singh | Rohith Sharma K S | | | Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API | Major | . | Subru Krishnan | Ellen Hui | | | Failed to Update HDFS Delegation Token for long running application in HA mode | Major | fs, ha, security | Liangliang Gu | Liangliang Gu | | | Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also) | Major | . | Zhiyuan Yang | Zhiyuan Yang | | | KMS clients should use KMS Delegation Tokens from current UGI. | Critical | kms | Xiao Chen | Xiao Chen | | | FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException | Major | . | Yiqun Lin | Yiqun Lin | | | fix some container-executor portability issues | Blocker | nodemanager, security | Allen Wittenauer | Allen Wittenauer | | | FileContext does not react on changing umask via configuration | Major | . | Yufei Gu | Akira Ajisaka | | | Fix failing unit tests in TestLinuxContainerExecutorWithMocks | Major | nodemanager | Yufei Gu | Yufei Gu | | | Single shuffle to memory must not exceed Integer#MAX\_VALUE | Major | mrv2 | Haibo Chen | Haibo Chen | | | TestCheckpoint.testCheckpoint fails intermittently on branch2 | Major | . | Mit Desai | Eric Badger | | | hadoop-azure runs several test cases repeatedly, causing unnecessarily long running

| Minor | fs/azure, test | Chris Nauroth | Chris Nauroth | | | A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list | Minor | balancer & mover | Hairong Kuang | Mit Desai | | | CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster | Major | capacity scheduler | Kuhu Shukla | Kuhu Shukla | | | TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently | Major | . | Eric Badger | Eric Badger | | | In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved with NN lock protection | Major | namenode | Rui Gao | Rui Gao | | | Increase timeout of TestAmFilter.testFilter | Minor | . | Eric Badger | Eric Badger | | | A bug causes OutOfIndex error in BlockListAsLongs | Minor | . | Weiwei Yang | Weiwei Yang | | | Add quoting to Shell class | Major | . | Owen O'Malley | Owen O'Malley | | | TestMRCJCFileOutputCommitter fails intermittently | Major | test | Brahma Reddy Battula | Akira Ajisaka | | | KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default | Minor | kms | Anthony Young-Garner | Anthony Young-Garner | | | In Balancer, the target task should be removed when its size \< 0. | Minor | balancer & mover | Yiqun Lin | Yiqun Lin | | | Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Shell#getSignalKillCommand should use the bash builtin on Linux | Major | . | Arpit Agarwal | Arpit Agarwal | | | BlockManager#createLocatedBlocks may return blocks on failed storages | Major | hdfs | Daryn Sharp | Kuhu Shukla | | | NPE when applying AvailableSpaceBlockPlacementPolicy | Major | namenode | Guangbin Zhu | Guangbin Zhu | | | Some recovered apps are put into default queue when RM HA | Major | . | Jun Gong | Jun Gong | | | LdapGroupsMapping getPassward shouldn't return null when IOException throws | Major | security | Zhaohao Liang | Wei-Chiu Chuang | | | NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI | Major | . | Brahma Reddy Battula | Brahma Reddy Battula | | | TestReloadingX509TrustManager is flaky | Major | . | Ratandeep Ratti | Mingliang Liu | | | AzureNativeFileSystem rename/delete performance improvements | Major | fs/azure | Subramanyam Pattipaka | Subramanyam Pattipaka | | | Remove hardcoded absolute path for shell executable | Major | util | Arpit Agarwal | Chen Liang | | | DataNode should not auto-format block pool directory if VERSION is missing | Major | datanode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | EditLogTailer should call rollEdits with a timeout | Major | ha, namenode | Todd Lipcon | Lei (Eddy) Xu | | | Fix race between TestMetricsSystemImpl and TestGangliaMetrics | Minor | test | Masatake Iwasaki | Chen Liang | | | Typo mistake in AMRMClient#getRegisteredTimeineClient API | Trivial | . | Rohith Sharma K S | Naganarasimha G R | | | TestHSAdminServer.testRefreshSuperUserGroups is failing | Minor | test | Kihwal Lee | Kihwal Lee | | | Tracing in IPC Server is broken | Major | . | Wei-Chiu Chuang | Daryn Sharp | | | Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure | Major | test | Rakesh R | Rakesh R | | | JMXJsonServlet is vulnerable to TRACE | Minor |

| Haibo Chen | Haibo Chen | | | CredentialProviderFactory fails at class loading from libhdfs (JNI) | Major | fs/s3 | Aaron Fabbri | Aaron Fabbri | | | Schedule a block for scanning if its metadata file is corrupt | Major | datanode | Rushabh S Shah | Rushabh S Shah | | | RM does not audit log kill request for active applications | Major | resourcemanager | Jason Lowe | Vrushali C | | | Namenode should use loginUser(hdfs) to generateEncryptedKey | Major | encryption, namenode | Xiaoyu Yao | Xiaoyu Yao | | | NPE in KeyProvider.rollNewVersion | Minor | . | Colm O hEigeartaigh | Colm O hEigeartaigh | | | Fix intermittent test failure of TestDataNodeErasureCodingMetrics | Major | . | Rakesh R | Rakesh R | | | Balancer should handle fs.defaultFS trailing slash in HA | Major | balancer & mover | LINTE | John Zhuge | | | Document LdapGroupsMapping keystore password properties | Minor | security | Wei-Chiu Chuang | Yuanbo Liu | | | FSDirectory#verifyMaxDirItems does not log path name | Minor | namenode | Wei-Chiu Chuang | Wei-Chiu Chuang | | | For Queue AccessControlException client retries multiple times on both RM | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Not existed application reported as ACCEPTED state by YarnClientImpl | Critical | yarn | Yesha Vora | Junping Du | | | Random Failure TestCapacityScheduler#testCSQueueBlocked | Major | test | Bibin A Chundatt | Bibin A Chundatt | | | Garbage snapshot records lingering forever | Critical | . | Kihwal Lee | Kihwal Lee | | | TestCapacityScheduler#testKillAllAppsInQueue fails randomly | Major | . | Varun Saxena | sandflee | | | testConf.xml ls comparators in wrong order | Trivial | fs | John Zhuge | Vrushali C | | | KMS should reload whitelist and default key ACLs when hot-reloading | Major | kms | Xiao Chen | Xiao Chen | | | Test failed for TestAggregatedLogFormat on trunk | Major | . | Junping Du | Jun Gong | | | GenericTestUtils$LogCapturer is flaky | Major | test, util | Mingliang Liu | Mingliang Liu | | | Yarn running container log fetching causes OutOfMemoryError | Major | log-aggregation | Prasanth Jayachandran | Xuan Gong | | | ReconfigurableBase can log sensitive information | Major | security | Sean Mackrory | Sean Mackrory | | | ReservedSpace is not cleared for abandoned Blocks | Critical | datanode | Fei Hui | Brahma Reddy Battula | | | Fix Javadoc typos in org.apache.hadoop.fs.FileSystem | Trivial | documentation | Chen He | Andras Bokor | | | Correctly revoke file leases when closing files | Major | hdfs-client | Yiqun Lin | Yiqun Lin | | | BlockSender should not synchronize on the dataset object | Major | datanode | Arpit Agarwal | Chen Liang | | | Java 1.7 support for org.apache.hadoop.fs.azure testcases | Minor | fs/azure | Tibor Kiss | Tibor Kiss | | | ReloadingX509TrustManager should keep reloading in case of exception | Critical | security | Mingliang Liu | Mingliang Liu | | | Open files can leak permanently due to inconsistent lease update | Critical | . | Kihwal Lee | Kihwal Lee | | | Display blacklisted nodes for AM container in the RM web UI | Major | resourcemanager, webapp | Naganarasimha G R | Naganarasimha G R | | | JMX AM Used metrics for queue wrong when app submited to nodelabel partition | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Shuffle server listen queue is too small | Major | mrv2 | Jason Lowe | Jason Lowe | | | User facing aspects of 'AM blacklisting' feature need fixing | Critical | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | Add jdiff support to MapReduce | Blocker |

| Li Lu | Li Lu | | | Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit | Major | capacityscheduler | Nathan Roberts | Nathan Roberts | | | Disable AM blacklisting by default to mitigate situations that application get hanged | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Fix hadoop-common to generate jdiff | Blocker | . | Wangda Tan | Wangda Tan | | | Fix INodeFile#getBlocks to not return null | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | ControlledJob#toString failed with NPE when job status is not successfully updated | Major | . | Weiwei Yang | Weiwei Yang | | | Point JDiff base version for HDFS from 2.6.0 to 2.7.2 | Blocker | . | Wangda Tan | Wangda Tan | | | Hadoop KMS should load old delegation tokens from Zookeeper on startup | Major | kms | Alex Ivanov | Xiao Chen | | | Add detailed logging in KMS for the authentication failure of proxy user | Minor | kms | Suraj Acharya | Suraj Acharya | | | Trash does not descent into child directories to check for permissions | Critical | fs, security | Eric Yang | Weiwei Yang | | | The option '-maxSize' and '-step' fail in OfflineImageViewer | Major | tools | Yiqun Lin | Yiqun Lin | | | mvn eclipse:eclipse generates .gitignore'able files | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | fix wrong command in CredentialProviderAPI.md | Trivial | documentation | Yuanbo Liu | Yuanbo Liu | | | Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure | Major | mrv2 | Peter Vary | Peter Vary | | | TestSlive fails after a common change | Major | . | Kihwal Lee | Daniel Templeton | | | Intermittent test failure of TestAMRMClient#testAMRMClientWithContainerResourceChange | Major | . | Varun Saxena | Bibin A Chundatt | | | Fix typo in hadoop\connect\to\_hosts error message | Trivial | scripts | Albert Chu | Albert Chu | | | User cannot set empty HADOOP\SSH\OPTS environment variable option | Minor | scripts | Albert Chu | Albert Chu | | | TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout | Major | test | Xiaoyu Yao | Yiqun Lin | | | Return container's ip and host from NM ContainerStatus call | Major | . | Jian He | Jian He | | | TestFSNamesystem.testFSLockGetWaiterCount fails intermittently in jenkins | Minor | test | Anu Engineer | Masatake Iwasaki | | | NPE listing wildcard directory in containerLaunch | Blocker | nodemanager | Haibo Chen | Daniel Templeton | | | Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184 | Blocker | . | Andrew Wang | Manoj Govindassamy | | | Add a unit test for HDFS-4660 | Major | datanode, hdfs | Yongjun Zhang | Vinayakumar B | | | RetryInvocationHandler logs all remote exceptions | Blocker | ipc | Jason Lowe | Jason Lowe | | | HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections | Major | . | Min Shen | Min Shen | | |

failed with NPE | Major | mrv2 | Haibo Chen | Haibo Chen | | | Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes | Minor | fs | Aaron Fabbri | Aaron Fabbri | | | Throw helpful exception when DNS entry for JournalNode cannot be resolved | Trivial | ha, journal-node, namenode | Damien Hardy | John Zhuge | | | Registering/unregistering container metrics triggered by ContainerEvent and ContainersMonitorEvent are conflict which cause uncaught exception in ContainerMonitorImpl | Blocker | . | Junping Du | Junping Du | | | In kms-site.xml configuration "hadoop.security.keystore.JavaKeyStoreProvider.password" should be updated with new name | Minor | security | huangyitian | Surendra Singh Lilhore | | | Increase default value of httpfs maxHttpHeaderSize | Minor | hdfs | Nicolae Popa | Nicolae Popa | | | RunJar adds the content of the jar twice to the classpath | Major | util | Sangjin Lee | Yuanbo Liu | | | Remove classpath conflicts of netty-all jar in hadoop-hdfs-client | Minor | test | Vinayakumar B | Vinayakumar B | | | ResourceManager REST API missing descriptions for what's returned when using Fair Scheduler | Minor | documentation, fairscheduler, resourcemanager | Grant Sohn | | | | StringBuilder created and appended even if logging is disabled | Major | namenode | Staffan Friberg | Staffan Friberg | | | ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | launchTime may be negative | Major | jobhistoryserver, mrv2 | Allen Wittenauer | zhihai xu | | | TestTaskAttempt fails in trunk | Major | test | Tsuyoshi Ozawa | Haibo Chen | | | RM intermittently crashed with NPE while handling APP\ATTEMPT\REMOVED event when async-scheduling enabled in CapacityScheduler | Major | resourcemanager | Karam Singh | sandflee | | | While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple | Critical | security | Bolke de Bruin | Bolke de Bruin | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Some new examples and test cases for them. | Minor | examples | Plamen Jeliazkov | Plamen Jeliazkov | | | hdfs tests for HADOOP-8014 | Major | test | John George | John George | | | TestConfiguration currently has no tests for direct setter methods | Major | test | Harsh J | madhukara phatak | | | Add a test that ensure AccessControlExceptions contain a full path | Major | namenode | Eli Collins | Stephen Chu | | | Add a plaintext fs -text test-case | Major | fs | Harsh J | Akira Ajisaka | | | jenkins patchprocess links are broken | Major | scripts | Colin P. McCabe | Arpit Agarwal | | | TestJavaSerialization fails with getting incorrect MR job result | Minor | test | sam liu | sam liu | | | TestZKFailoverControllerStress#testExpireBackAndForth occasionally fails | Major | test | Ted Yu | Ming Ma | | | TestDockerContainerExecutor should pull a single image and not the entire centos repository | Minor | test | Ravindra Kumar Naik | Ravindra Kumar Naik | | | Cleanup TestApplicationClientProtocolOnHA | Trivial | test | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Remove DRFA.MaxBackupIndex config from log4j.properties | Minor | . | Akira Ajisaka | Abhiraj Butala | | | Remove redundant definitions in log4j.properties | Minor | test | Abhiraj Butala | Abhiraj Butala | | | TestFairScheduler.testContinuousScheduling fails Intermittently | Major | scheduler | Tsuyoshi Ozawa | zhihai xu | | | fix race conditions and add URL support to smart-apply-patch.sh | Major | test | Allen Wittenauer | Raymie Stata | | | TestDiskspaceQuotaUpdate#testAppendOverTypeQuota is failing | Major | test | Anu Engineer | Xiaoyu Yao | | |

javac result is wildly wrong | Major | build, test | Allen Wittenauer | Kengo Seki | | | test-patch.sh goes into an infinite loop on non-maven builds | Critical | test | Allen Wittenauer | Allen Wittenauer | | | TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk | Minor | . | Xuan Gong | Rohith Sharma K S | | | [JDK 8] TestClientRMService.testGetLabelsToNodes fails | Major | test | Robert Kanter | Robert Kanter | | | Create unit test to automatically compare HDFS related classes and hdfs-default.xml | Minor | . | Ray Chiang | Ray Chiang | | | test-patch.sh should use 'file' command for patch determinism | Major | . | Allen Wittenauer | Sean Busbey | | | TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup | Minor | test | zhihai xu | zhihai xu | | | Files written by TestHostsFiles and TestNameNodeMXBean are causing Release Audit Warnings. | Minor | test | Ruth Wisniewski | Ruth Wisniewski | | | test-patch.sh should pull the real findbugs version | Minor | test | Allen Wittenauer | Kengo Seki | | | add option to test-patch to avoid relocating patch process directory | Minor | . | Sean Busbey | Sean Busbey | | | Add user-provided plugins to test-patch | Major | . | Sean Busbey | Sean Busbey | | | cannot use --java-home in test-patch | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | TestJobCounters should use new properties instead of JobConf.MAPRED\TASK\JAVA\_OPTS | Minor | test | sam liu | sam liu | | | shellcheck plugin displays a wrong version potentially | Trivial | build | Kengo Seki | Kengo Seki | | | test-patch should only report on newly introduced findbugs warnings. | Major | . | Sean Busbey | Sean Busbey | | | test-patch in offline mode should tell maven to be in offline mode | Major | . | Sean Busbey | Sean Busbey | | | determine-flaky-tests needs a summary mode | Minor | . | Allen Wittenauer | Yufei Gu | | | TestRMRestart#testRMRestartGetApplicationList sometime fails in trunk | Minor | . | Ted Yu | zhihai xu | | | Fix TestNodeManagerHardwareUtils fails on Mac | Minor | nodemanager | Varun Vasudev | Varun Vasudev | | | TestReplication#testReplicationWhenBlockCorruption is not valid after HDFS-6482 | Minor | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | unit tests for the shell scripts | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | TestApplicationPriority.testApplicationPriorityAllocation fails intermittently | Major | . | Zhijie Shen | Sunil Govindan | | | Add more unit tests for FsShell getmerge | Major | test | XieXianshan | Daniel Templeton | | | UtilTest#isCygwin method appears to be unused | Trivial | contrib/streaming, test | Chris Nauroth | Neelesh Srinivas Salian | | | MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated | Major | timelineserver | Mit Desai | Brahma Reddy Battula | | | Add a thorough test of the full KMS code path | Critical | security, test | Daniel Templeton | Daniel Templeton | | | Extend full test of KMS ACLs | Major | test | Daniel Templeton | Daniel Templeton | | | Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows | Major | test | Xiaoyu Yao | Xiaoyu Yao | | | Some tests should always reset sysout and syserr | Minor | test | Xiao Chen | Xiao Chen | | | Upgrade JUnit3 TestCase to JUnit 4 in

package | Trivial | test | Dustin Cote | Dustin Cote | | | Pretty-format the output for DFSIO | Major | . | Kai Zheng | Kai Zheng | | | Tests in TestDFSAdminWithHA intermittently fail with EOFException | Major | test | Xiao Chen | Xiao Chen | | | Add tests for PB RPC in case version mismatch of client and server | Major | ipc | Junping Du | Junping Du | | | TestDirectoryScanner.testThrottle() is still a little flakey | Major | balancer & mover, test | Daniel Templeton | Daniel Templeton | | | fix the TestDFSUpgradeWithHA due to BindException | Major | test | Liang Xie | Xiao Chen | | | TestBlockReplacement#testBlockReplacement fails occasionally | Minor | test | Xiao Chen | Xiao Chen | | | Add Tests for S3FileSystem Contract | Major | tools | Matthew Paduano | Matthew Paduano | | | TestTimedOutTestsListener#testThreadDumpAndDeadlocks sometimes times out | Major | . | Xiao Chen | Xiao Chen | | | TestValueQueue#testgetAtMostPolicyALL fails intermittently | Major | . | Xiao Chen | Xiao Chen | | | Remove dead code related to SimulatedFSDataset in tests | Minor | test | Akira Ajisaka | Brahma Reddy Battula | | | TestResourceManager#testResourceAllocation() fails when using FairScheduler | Major | fairscheduler, test | Ray Chiang | Yufei Gu | | | Allow reseting KerberosName in unit tests | Minor | . | Xiao Chen | Xiao Chen | | | TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup | Minor | nodemanager | Daniel Templeton | Daniel Templeton | | | Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared | Minor | . | Harsh J | Harsh J | | | Create unit test to automatically compare Common related classes and core-default.xml | Minor | . | Ray Chiang | Ray Chiang | | | Add automatic search of default Configuration variables to TestConfigurationFieldsBase | Major | test | Ray Chiang | Ray Chiang | | | Add Glob unit test for special characters | Minor | fs | Harsh J | Harsh J | | | Test timeout is happening for TestRMWebServicesNodes | Major | test | Bibin A Chundatt | Bibin A Chundatt | | | Run checkstyle on test source files | Minor | . | John Zhuge | John Zhuge | | | TestFifoScheduler.testResourceOverCommit race condition | Major | test | Eric Badger | Eric Badger | | | TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X | Major | test | Allen Wittenauer | Kai Sasaki | | | Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8 | Minor | . | Bibin A Chundatt | Bibin A Chundatt | | | TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure | Major | test | Bibin A Chundatt | Bibin A Chundatt | | | TestRMWebServices#testDumpingSchedulerLogs fails randomly | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Tests fails with localizer port bind exception. | Major | test | Brahma Reddy Battula | Brahma Reddy Battula | | | Remove redundant TestMiniDFSCluster.testDualClusters | Trivial | test | John Zhuge | Jiayi Zhou | | | TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently | Major | test | Rohith Sharma K S | Ajith S | | | Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with

enabled | Blocker | test | Rohith Sharma K S | Rohith Sharma K S | | | [Azure Data Lake] Support for contract test and unit test cases | Major | fs/adl, test, tools | Vishwajeet Dusane | Vishwajeet Dusane | | | fix coverage org.apache.hadoop.net | Major | . | Aleksey Gorshkov | Ivan A. Veselovsky | | | Add custom JUnit4 test runner with configurable timeout | Major | test | Steve Loughran | Steve Loughran | | | TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry random failure | Major | . | Bibin A Chundatt | sandflee | | | TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider fails intermittently. | Minor | . | sandflee | Jun Gong | | | TestRMRestart#testQueueMetricsOnRMRestart random failure | Major | . | sandflee | sandflee | | | testAMRestartNotLostContainerCompleteMsg may fail | Minor | . | sandflee | sandflee | | | Wrong Javadoc tag in MiniYarnCluster | Major | documentation | Andras Bokor | Andras Bokor | | | TestJobListCache.testAddExisting failed intermittently in slow VM testbed | Minor | . | Junping Du | Junping Du | | | TestRMDelegationTokens fails intermittently | Major | test | Rohith Sharma K S | Jason Lowe | | | TestActiveStandbyElector fails occasionally in trunk | Minor | . | Ted Yu | Eric Badger | | | Enhance TestKMSAudit | Minor | kms | Xiao Chen | Xiao Chen | | | TestContinuousScheduling#testSortedNodes fails intermittently | Minor | . | sandflee | Yufei Gu | | | TestResourceTrackerOnHA crashes | Major | test | Jason Lowe | Ajith S | | | TestSubmitApplicationWithRMHA is failing sporadically during precommit builds | Major | test | Jason Lowe | Vrushali C | | | TestNodeBlacklistingOnAMFailures fails on trunk | Major | test | Varun Saxena | Sunil Govindan | | | TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts | Minor | . | Xiao Chen | Xiao Chen | | | TestYarnClient#testReservationDelete fails | Major | . | Rohith Sharma K S | Sean Po | | | Clean up bad exception catching practices in TestYarnClient | Major | . | Sean Po | Sean Po | | | TestRawLocalFileSystemContract is needed | Major | fs | Matt Foley | Andras Bokor | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add states for journal synchronization in journal daemon | Major | ha, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Refactor BackupImage and FSEditLog | Major | ha, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Remove the deprecated DistributedFileSystem.DiskStatus and the related methods | Major | hdfs-client | Tsz Wo Nicholas Sze | Arpit Gupta | | | Add stricter tests to FileSystemContractTestBase | Major | test | Steve Loughran | Steve Loughran | | | Basic AM changes to support preemption requests (per YARN-45) | Major | mr-am, mrv2 | Carlo Curino | Carlo Curino | | | Possible NPE in KillAMPreemptionPolicy related to ProportionalCapacityPreemptionPolicy | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Add LazyPersist flag to INodeFile, save it in FsImage and edit logs | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Updated editsStored and

to bump layout version and add LazyPersist flag | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | | | native-task: fix gtest build on macosx | Trivial | task | Binglin Chang | Binglin Chang | | | Allow data migration tool to run as a daemon | Minor | balancer & mover, scripts | Benoy Antony | Benoy Antony | | | Make adding a new data directory to the DataNode an atomic operation and improve error handling | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | ApplicationHistoryManager is expected to return a sorted list of apps/attempts/containers | Major | timelineserver | Robert Kanter | Robert Kanter | | | LeveldbTimelineStore does not handle db exceptions properly | Major | . | Jason Lowe | Varun Saxena | | | CapacityScheduler should be notified when labels on nodes changed | Major | . | Wangda Tan | Wangda Tan | | | Bootstrap TimelineServer Next Gen Module | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Create common ResourceUsage class to track labeled resource usages in Capacity Scheduler | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | [Collector wireup] Set up timeline collector with basic request serving structure and lifecycle | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | NM dies because of the failure of resource localization | Major | nodemanager | Wang Hao | Varun Saxena | | | Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | Create common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues | Major | capacityscheduler | Wangda Tan | Wangda Tan | | | Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | | | Fix TestHDFSCLI for quota with storage type | Minor | test | Xiaoyu Yao | Xiaoyu Yao | | | Capacity Scheduler LeafQueue/ParentQueue should use QueueCapacities to track capacities-by-label | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | [Data Model] create overall data objects of TS next gen | Major | timelineserver | Sangjin Lee | Zhijie Shen | | | [Data Mode] Implement client API to put generic entities | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | [Collector implementation] the REST server (web server) for per-node collector does not work if it runs inside node manager | Major | timelineserver | Sangjin Lee | Li Lu | | | [Event producers] Change distributed shell to use new timeline service | Major | timelineserver | Zhijie Shen | Junping Du | | | [Source organization] Refactor timeline collector according to new code organization | Major | timelineserver | Li Lu | Li Lu | | | [Storage implementation] Create backing storage write interface and a POC only file based storage implementation | Major | timelineserver | Vrushali C | Vrushali C | | | Synchronize RM and Generic History Service Web-UIs | Major | . | Zhijie Shen | Xuan Gong | | | ContainerReport should have nodeHttpAddress | Major | . | Zhijie Shen | Xuan Gong | | | Make CryptoOutputStream behave like DFSOutputStream wrt synchronization | Critical | fs | Sean Busbey | Sean Busbey | | | Sort by Application id, AppAttempt & ContainerID doesn't work in ATS / RM web ui | Minor | timelineserver | Jeff Zhang | Naganarasimha G R | | | [Collector wireup] Implement timeline app-level collector service discovery | Major | timelineserver | Sangjin Lee | Junping Du | | | TestTimelineServiceClientIntegration fails | Minor | timelineserver | Sangjin Lee | Sangjin Lee | | | Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track

| Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | | | Add non-exclusive node label API to RMAdmin protocol and NodeLabelsManager | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | Separate class DataStreamer out of DFSOutputStream | Major | hdfs-client | Li Bo | Li Bo | | | [Collector wireup] Implement RM starting its timeline collector | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | Implement mkdirs in the HDFS Web UI | Major | ui | Ravi Prakash | Ravi Prakash | | | [Data Model] Make putEntities operation be aware of the app's context | Major | timelineserver | Sangjin Lee | Zhijie Shen | | | Allow admin specify labels from each NM (Distributed configuration) | Major | resourcemanager | Wangda Tan | Naganarasimha G R | | | Collector's web server should randomly bind an available port | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Update the POM to create a separate hdfs-client jar | Major | build | Haohui Mai | Haohui Mai | | | Add support for using the 'tc' tool via container-executor | Major | nodemanager | Sidharta Seethana | Sidharta Seethana | | | Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows | Minor | test | Xiaoyu Yao | Xiaoyu Yao | | | [Event Producers] NM TimelineClient container metrics posting to new timeline service. | Major | nodemanager | Junping Du | Junping Du | | | Few issues in ApplicationHistory web ui | Minor | applications, timelineserver | Bibin A Chundatt | Naganarasimha G R | | | Annotation client implementation as private | Major | hdfs-client | Tsz Wo Nicholas Sze | Takuya Fukudome | | | Separate the client retry conf from DFSConfigKeys | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Separate JSON related routines used by WebHdfsFileSystem to a package local class | Minor | hdfs-client | Haohui Mai | Haohui Mai | | | Move CorruptFileBlockIterator to the hdfs.client.impl package | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Move o.a.h.hdfs.web.resources.\* to the client jars | Minor | build | Haohui Mai | Haohui Mai | | | Separate webhdfs retry configuration keys from DFSConfigKeys | Minor | hdfs-client | Haohui Mai | Haohui Mai | | | Clearly define flow ID/ flow run / flow version in API and storage | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | NM need to notify the log aggregation status to RM through Node heartbeat | Major | . | Xuan Gong | Xuan Gong | | | Refactor DFSClient.Conf to a standalone class | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Move

into BlockTokenIdentifier | Minor | security | Haohui Mai | Haohui Mai | | | Separate the client failover conf from DFSConfigKeys | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Separate the client write conf from DFSConfigKeys | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Improve YARN log command to get AMContainer logs as well as running containers logs | Major | log-aggregation | Xuan Gong | Xuan Gong | | | Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM | Major | nodemanager | Sidharta Seethana | Sidharta Seethana | | | Support reporting per storage type quota and usage with hadoop/hdfs shell | Major | datanode, namenode | Xiaoyu Yao | Peter Shi | | | CapacityScheduler side changes to support non-exclusive node labels | Major | capacityscheduler | Wangda Tan | Wangda Tan | | | Create Initial OrderingPolicy Framework and FifoOrderingPolicy | Major | scheduler | Craig Welch | Craig Welch | | | Support RESTful API for getLabelsToNodes | Minor | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Container should contains node-labels asked by original ResourceRequests | Major | api, capacityscheduler, nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | Separate the client read conf from DFSConfigKeys | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Move GRANDFATHER\GENERATION\STAMP and GRANDFATER\INODE\ID to hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | Related Web UI, CLI changes on exposing client API to check log aggregation status | Major | . | Xuan Gong | Xuan Gong | | | Queue sorting in CapacityScheduler should consider node label | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | | | CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue | Critical | capacityscheduler | Jason Lowe | Jason Lowe | | | getTransferredContainers can be a bottleneck during AM registration | Major | scheduler | Jason Lowe | Sunil Govindan | | | Move LocatedBlocks and related classes to hdfs-client | Major | build, hdfs-client | Haohui Mai | Haohui Mai | | | Integrate OrderingPolicy Framework with CapacityScheduler | Major | capacityscheduler | Craig Welch | Craig Welch | | | [Event producers] Implement MapReduce AM writing MR events/counters to v2 ATS | Major | . | Sangjin Lee | Junping Du | | | Separate client related routines in HAUtil into a new class | Major | build, hdfs-client | Haohui Mai | Haohui Mai | | | New parameter or CLI for decommissioning node gracefully in RMAdmin CLI | Major | graceful | Junping Du | Devaraj K | | | Move classes that used by ClientProtocol into hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | Typos in hadoop-common java docs | Minor | . | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Outbound network bandwidth : classify/shape traffic originating from YARN containers | Major | . | Sidharta Seethana | Sidharta Seethana | | | [RM HA] Rest api endpoints doing redirect incorrectly | Major | resourcemanager | bc Wong | Xuan Gong | | | Implement a FairOrderingPolicy | Major | scheduler | Craig Welch | Craig Welch | | | Node label attributes (like exclusivity) should settable via addToClusterNodeLabels but shouldn't be changeable at runtime | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | Disable symlinks in trunk | Major | . | Andrew Wang | Andrew Wang | | | Reuse TimelineCollectorManager for RM | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Respect labels in preemption policy of capacity scheduler for inter-queue preemption | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Sub resources of timeline entity needs to be passed to a separate endpoint. | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled | Major | resourcemanager | Wangda Tan | Naganarasimha G R | | | convert load test driver to timeline service v.2 | Major |

| Sangjin Lee | Sangjin Lee | | | AM logs link missing in the RM UI for a completed app | Blocker | . | Hitesh Shah | Xuan Gong | | | NodeManager: Add cgroups support for disk I/O isolation | Major | . | Wei Yan | Varun Vasudev | | | Move LeaseRenewer to the hdfs.client.impl package | Minor | hdfs-client | Tsz Wo Nicholas Sze | Takanobu Asanuma | | | Improve the error message when attempting manual failover with auto-failover enabled | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Separate HdfsConstants into the client and the server side class | Major | hdfs-client | Haohui Mai | Haohui Mai | | | Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows | Minor | test | Xiaoyu Yao | Xiaoyu Yao | | | Move all protocol classes used by ClientProtocol to hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | Consolidate data model change according to the backend implementation | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Move HdfsServerConstants#IO\FILE\BUFFER\SIZE and SMALL\BUFFER\_SIZE to the users | Major | . | Haohui Mai | Li Lu | | | Fix TestCLI.testAll 'help: help for find' on Windows | Minor | test | Xiaoyu Yao | Kiran Kumar M R | | | Fix the format issue of the new RM web UI and AHS web UI after YARN-3272 / YARN-3262 | Major | resourcemanager | Xuan Gong | Xuan Gong | | | unit tests failures and issues found from findbug from earlier ATS checkins | Minor | timelineserver | Sangjin Lee | Naganarasimha G R | | | Add Rolling Time To Lives Level DB Plugin Capabilities | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | | | Don't fail RM if queue's configured labels are not existed in cluster-node-labels | Major | resourcemanager | Rohith Sharma K S | Wangda Tan | | | Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer | Minor | nodemanager | Omkar Vinit Joshi | Varun Saxena | | | Add label-type and Improve "DEFAULT\_PARTITION" in Node Labels Page | Major | webapp | Naganarasimha G R | Naganarasimha G R | | | Distinguish shutdown during supervision vs. shutdown for rolling upgrade | Major | nodemanager | Jason Lowe | Jason Lowe | | | [Storage implementation] Exploiting the option of using Phoenix to access HBase backend | Major | timelineserver | Zhijie Shen | Li Lu | | | Add miniHBase cluster and Phoenix support to ATS v2 unit tests | Major | timelineserver | Li Lu | Li Lu | | | TestMRTimelineEventHandling and TestApplication are broken | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | CommonNodeLabelsManager should support NodeLabel instead of string label name when getting node-to-label/label-to-label mappings | Minor | resourcemanager | Sunil Govindan | Sunil Govindan | | | add a mode to replay MR job history files to the timeline service | Major | . | Sangjin Lee | Sangjin Lee | | | Node's Log Aggregation Report with SUCCEED should not cached in RMApps | Critical | log-aggregation | Junping Du | Xuan Gong | | | Eliminate retries

TestFileCreation#testOverwriteOpenForWrite | Major | test | Arpit Agarwal | Arpit Agarwal | | | Writes to RAM DISK reserve locked memory for block files | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Add version info on timeline service / generic history web UI and REST API | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String | Blocker | api, client, resourcemanager | Wangda Tan | Naganarasimha G R | | | Support of NodeLabel object instead of plain String in YarnClient side. | Major | client | Sunil Govindan | Sunil Govindan | | | Move load labels from storage from serviceInit to serviceStart to make it works with RM HA case. | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Change ContainerExecutor's primary lifecycle methods to use a more extensible mechanism for passing information. | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | [Storage implementation] explore & create the native HBase schema for writes | Critical | timelineserver | Sangjin Lee | Vrushali C | | | Remove unnecessary throttling in TestDatanodeDeath | Major | test | Arpit Agarwal | Arpit Agarwal | | | Timeline service v2 load generator needs to write event id | Major | . | Li Lu | Li Lu | | | Ordering policy should be allowed to reorder an application when demand changes | Major | capacityscheduler | Craig Welch | Craig Welch | | | CapacityScheduler should trim default\node\label\_expression | Critical | api, client, resourcemanager | Wangda Tan | Sunil Govindan | | | RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Deprecate -directlyAccessNodeLabelStore in RMAdminCLI | Major | api, client, resourcemanager | Wangda Tan | Naganarasimha G R | | | Rename BlockInfoContiguous to BlockInfo | Major | . | Zhe Zhang | Zhe Zhang | | | ATS Web Performance issue at load time when large number of jobs | Major | resourcemanager, webapp, yarn | Xuan Gong | Xuan Gong | | | Fix TestHBaseTimelineWriterImpl unit test failure by fixing it's test data | Major | timelineserver | Vrushali C | Vrushali C | | | build is broken on YARN-2928 branch due to possible dependency cycle | Blocker | timelineserver | Sangjin Lee | Li Lu | | | Node-label-expression should be included by ResourceRequestPBImpl.toString | Minor | api | Xianyin Xin | Xianyin Xin | | | Fixed the typo with the configuration name: APPLICATION\HISTORY\PREFIX\MAX\APPS | Major | resourcemanager, webapp, yarn | Xuan Gong | Xuan Gong | | | Application (Attempt and Container) Not Found in AHS results in Internal Server Error (500) | Major | timelineserver | Jonathan Eagles | Mit Desai | | | Subclass BlockInfo to represent contiguous blocks | Major | namenode | Zhe Zhang | Zhe Zhang | | | Refactor and fix null casting in some map cast for TimelineEntity (old and new) and fix findbug warnings | Major | timelineserver | Junping Du | Junping Du | | | add more diags about app retry limits on AM failures | Minor | resourcemanager | Steve Loughran | Steve Loughran | | | ATS Web UI breaks because of YARN-3467 | Blocker | resourcemanager, webapp, yarn | Xuan Gong | Xuan Gong | | | AHS API and other AHS changes to handle tags for completed MR jobs | Major | . | Karthik Kambatla | Xuan Gong | | | Expose a client API to allow clients to figure if log aggregation is complete | Major |

| Arun C Murthy | Xuan Gong | | | loading applications by filtering appstartedTime period for ATS Web UI | Major | resourcemanager, webapp, yarn | Xuan Gong | Xuan Gong | | | The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages | Major | . | Colin P. McCabe | Colin P. McCabe | | | [Event producers] Implement RM writing app lifecycle events to ATS | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | Mover should exit with NO\MOVE\BLOCK if no block can be moved | Major | balancer & mover | Tsz Wo Nicholas Sze | Surendra Singh Lilhore | | | Documentation of ResourceManager HA should explain configurations about listen addresses | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Exclude jdk.tools from hbase-client and hbase-testing-util | Major | timelineserver | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Fix TestFSImage#testZeroBlockSize on Windows | Major | datanode, test | Xiaoyu Yao | Xiaoyu Yao | | | Feature documentation for HDFS-6581 | Major | documentation | Arpit Agarwal | Arpit Agarwal | | | Move ClientProtocol to the hdfs-client | Major | build | Haohui Mai | Takanobu Asanuma | | | Output AclEntry in PBImageXmlWriter | Minor | tools | Akira Ajisaka | Surendra Singh Lilhore | | | Generalize native HBase writer for additional tables | Minor | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | Support return structured NodeLabel objects in REST API | Major | api, client, resourcemanager | Wangda Tan | Sunil Govindan | | | Eviction should key off used locked memory instead of ram disk free space | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Test case failures in TestDistributedShell and some issue fixes related to ATSV2 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Make hadoop-hdfs-project Native code -Wall-clean | Major | native | Alan Burlison | Alan Burlison | | | Consolidate all of the cmake extensions in one directory | Major | . | Allen Wittenauer | Alan Burlison | | | Provide JMX metrics per storage type | Major | . | Benoy Antony | Benoy Antony | | | Migrate Hadoop Pipes native build to new CMake framework | Major | build | Alan Burlison | Alan Burlison | | | Make hadoop-common-project Native code -Wall-clean | Major | native | Alan Burlison | Alan Burlison | | | Consolidate truncate() related implementation in a single class | Major | . | Haohui Mai | Rakesh R | | | Migrate HDFS native build to new CMake framework | Major | build | Alan Burlison | Alan Burlison | | | Migrate MAPREDUCE nativetask build to new CMake framework | Major | build | Alan Burlison | Alan Burlison | | | Migrate YARN native build to new CMake framework | Major | build | Alan Burlison | Alan Burlison | | | Add avro binary support for jhist files | Major | jobhistoryserver | Ray Chiang | Ray Chiang | | | [Storage abstraction] Create backing storage read interface for ATS readers | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Clean up the checkstyle warinings about ClientProtocol | Major |

| Takanobu Asanuma | Takanobu Asanuma | | | Move protobuf files that define the client-sever protocols to hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | [Data Serving] Set up ATS reader with basic request serving structure and lifecycle | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Report NM aggregated container resource utilization in heartbeat | Major | nodemanager | Arun C Murthy | igo Goiri | | | Reduce storage footprint for ReservationAllocation | Major | capacityscheduler, fairscheduler, resourcemanager | Anubhav Dhoot | Anubhav Dhoot | | | add equals and hashCode to TimelineEntity and other classes in the data model | Major | timelineserver | Sangjin Lee | Li Lu | | | Cache runningApps in RMNode for getting running apps on given NodeId | Major | nodemanager, resourcemanager | Junping Du | Junping Du | | | [Collector wireup] We need an assured way to determine if a container is an AM container on NM | Major | nodemanager, timelineserver | Zhijie Shen | Giovanni Matteo Fumarola | | | Mover should exit with NO\MOVE\PROGRESS if there is no move progress | Minor | balancer & mover | Tsz Wo Nicholas Sze | Surendra Singh Lilhore | | | Inotify: Support event for OP\_TRUNCATE | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | | | AM-NM protocol changes to support container resizing | Major | api | Wangda Tan (No longer used) | MENG DING | | | Fix FIONREAD #include on Solaris | Minor | net | Alan Burlison | Alan Burlison | | | FileSystemNodeLabelsStore should make sure edit log file closed when exception is thrown | Major | api, client, resourcemanager | Dian Fu | Dian Fu | | | Make hadoop-yarn-project Native code -Wall-clean | Major | build | Alan Burlison | Alan Burlison | | | Improve CorruptReplicasMap#corruptReplicasMap | Major | . | Yi Liu | Yi Liu | | | Display information per tier on the Namenode UI | Major | . | Benoy Antony | Benoy Antony | | | Support for Application priority : Changes in RM and Capacity Scheduler | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Add a metric for number of encryption zones | Major | encryption | Rakesh R | Rakesh R | | | ContainerManager implementation to support container resizing | Major | nodemanager | Wangda Tan | MENG DING | | | Consolidate append() related implementation into a single class | Major | namenode | Rakesh R | Rakesh R | | | Improve InvalidateBlocks#node2blocks | Major | . | Yi Liu | Yi Liu | | | Remove unused Linux-specific constants in NativeIO | Major | native | Martin Walsh | Martin Walsh | | | Clean up the import statements in ClientProtocol | Minor | . | Takanobu Asanuma | Takanobu Asanuma | | | Allow jobs to be submitted to reservation that is active but does not have any allocations | Major | capacityscheduler, fairscheduler, resourcemanager | Subru Krishnan | Subru Krishnan | | | hadoop-common's

is redundant and can be removed | Minor | native | Alan Burlison | Alan Burlison | | | LowCost: A Cost-Based Placement Agent for YARN Reservations | Major | capacityscheduler, resourcemanager | Ishai Menache | Jonathan Yaniv | | | ensure timely flush of timeline writes | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Add docker container support to container-executor | Major | yarn | Sidharta Seethana | Abin Shahab | | | Add docker container runtime support to LinuxContainterExecutor | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Bugs in HBaseTimelineWriterImpl | Major | timelineserver | Zhijie Shen | Vrushali C | | | ContainerImpl changes to support container resizing | Major | nodemanager | MENG DING | MENG DING | | | DN should ignore lazyPersist hint if the writer is not local | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | When RM is catching up with node updates then it should not expire acquired containers | Major | resourcemanager | Bikas Saha | Xuan Gong | | | Speed up Task processing loop in HsTasksBlock#render() | Major | jobhistoryserver | Ray Chiang | Ray Chiang | | | NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values | Major | native | Dmytro Shteflyuk | Martin Walsh | | | ApplicationReport should be able to tell whether the Application is AM managed or not. | Major | api | Spandan Dutta | Rohith Sharma K S | | | Change to use the AM flag in ContainerContext determine AM container | Major | timelineserver | Zhijie Shen | Sunil Govindan | | | container-executor should print output of docker logs if the docker container exits with non-0 exit status | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | Add RMStateStore apis to store and load accepted reservations for failover | Major | capacityscheduler, fairscheduler, resourcemanager | Subru Krishnan | Anubhav Dhoot | | | Make ContainersMonitor can support change monitoring size of an allocated container in NM side | Major | nodemanager | Wangda Tan | MENG DING | | | Rethink event column key issue | Major | timelineserver | Zhijie Shen | Vrushali C | | | Refactor the reservation system test cases to use parameterized base test | Major | capacityscheduler, fairscheduler | Subru Krishnan | Subru Krishnan | | | Display Application Priority in RM Web UI | Major | webapp | Sunil Govindan | Sunil Govindan | | | [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend | Major | timelineserver | Sangjin Lee | Zhijie Shen | | | pendingApplications in LeafQueue should also use OrderingPolicy | Major | capacityscheduler | Sunil Govindan | Sunil Govindan | | | Support for changing Application priority during runtime | Major | capacityscheduler, resourcemanager | Sunil Govindan | Sunil Govindan | | | Archival Storage: getStoragePolicy should not need superuser privilege | Major | balancer & mover, namenode | Hui Zheng | Brahma Reddy Battula | | | Move WebHdfsFileSystem into hadoop-hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | split the application table from the entity table | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Publish Application Priority to TimelineServer | Major | timelineserver | Sunil Govindan | Sunil Govindan | | | Do not use small blocks for balancing the cluster | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Collect memory/cpu usage on the node | Major | nodemanager, resourcemanager | igo Goiri | igo Goiri | | | Convert BlockInfoUnderConstruction as a feature | Major | namenode | Zhe Zhang | Jing Zhao | | | BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory | Major | . | Yi Liu | Yi Liu | | | Refactor timelineservice.storage to add support to online and offline aggregation writers | Major | timelineserver | Li Lu | Li Lu | |

BlockManager#excessReplicateMap should use a HashMap | Major | namenode | Yi Liu | Yi Liu | | | HDFS Balancer should consider remaining storage % when checking for under-utilized machines | Major | balancer & mover | Gopal V | Tsz Wo Nicholas Sze | | | [Event producers] Implement NM writing container lifecycle events to ATS | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | Balancer may not move blocks efficiently in some cases | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Deal with byte representations of Longs in writer code | Major | timelineserver | Vrushali C | Sangjin Lee | | | Move DfsClientConf to hdfs-client | Major | build | Haohui Mai | Mingliang Liu | | | build is broken at TestHBaseTimelineWriterImpl.java | Blocker | timelineserver | Sangjin Lee | Sangjin Lee | | | Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup | Major | nodemanager | Naganarasimha G R | Naganarasimha G R | | | RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing | Major | nodemanager | Wangda Tan | MENG DING | | | ContainerManager recovery for container resizing | Major | nodemanager | MENG DING | MENG DING | | | REST API implementation for getting raw entities in TimelineReader | Major | timelineserver | Varun Saxena | Varun Saxena | | | Move replication factor into individual blocks | Major | . | Haohui Mai | Haohui Mai | | | NM should provide a way for AM to tell it not to aggregate logs. | Major | log-aggregation, nodemanager | Robert Joseph Evans | Ming Ma | | | Move ShortCircuitShm to hdfs-client | Major | build | Mingliang Liu | Mingliang Liu | | | Use GenericTestUtils to set log levels in TestPread and TestReplaceDatanodeOnFailure | Major | build | Mingliang Liu | Mingliang Liu | | | Miscellaneous issues in NodeManager project | Minor | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Support user cli interface in for Application Priority | Major | client, resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Move the shortcircuit package to hdfs-client | Major | build | Mingliang Liu | Mingliang Liu | | | Store INodeId instead of the INodeFile object in BlockInfoContiguous | Major | . | Haohui Mai | Haohui Mai | | | Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf | Major | build | Mingliang Liu | Mingliang Liu | | | Support admin cli interface in for Application Priority | Major | resourcemanager | Sunil Govindan | Rohith Sharma K S | | | Move BlockReaderLocal to hdfs-client | Major | build | Mingliang Liu | Mingliang Liu | | | Remove unnecessary block replacement in INodeFile | Major | namenode | Jing Zhao | Jing Zhao | | | Move RemoteBlockReader to hdfs-client module | Major | build | Mingliang Liu | Mingliang Liu | | | RM HA UI redirection needs to be fixed when both RMs are in standby mode | Major | resourcemanager | Xuan Gong | Xuan Gong | | | Container shouldn't be killed when node's label updated. | Major | capacity scheduler | Wangda Tan | Wangda Tan | | | Add documentation for node labels feature | Major | documentation | Gururaj Shetty | Wangda Tan | | | Both RM in active state

Admin#transitionToActive failure from refeshAll() | Critical | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | Allow admin to specify which blockpools the balancer should run on | Major | balancer & mover | Chris Trezzo | Chris Trezzo | | | RM should print alert messages if Zookeeper and Resourcemanager gets connection issue | Critical | yarn | Yesha Vora | Xuan Gong | | | REST api support for Application Priority | Major | webapp | Sunil Govindan | Naganarasimha G R | | | Move o.a.h.hdfs.net/\*Peer classes to hdfs-client | Major | build | Mingliang Liu | Mingliang Liu | | | Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to hadoop-hdfs-client module | Major | build | Mingliang Liu | Mingliang Liu | | | Move replication queues related methods in FSNamesystem to BlockManager | Major | . | Haohui Mai | Haohui Mai | | | Proxying all AM-RM communications | Major | nodemanager, resourcemanager | Carlo Curino | Kishore Chaliparambil | | | LinuxContainerExecutor loses info when forwarding ResourceHandlerException | Trivial | nodemanager | Steve Loughran | Bibin A Chundatt | | | Add a "skip existing table" mode for timeline schema creator | Major | timelineserver | Li Lu | Li Lu | | | Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client | Major | build | Haohui Mai | Mingliang Liu | | | Replace NameNode.DEFAULT\PORT with HdfsClientConfigKeys.DFS\NAMENODE\RPC\PORT\_DEFAULT config key | Major | build | Mingliang Liu | Mingliang Liu | | | CapacityScheduler side changes to support increase/decrease container resource. | Major | resourcemanager, scheduler | Wangda Tan | Wangda Tan | | | Add Admin API for supporting node resource configuration in command line | Critical | client, graceful | Junping Du | igo Goiri | | | Balancer#Parameters class could use a builder pattern | Minor | balancer & mover | Chris Trezzo | Chris Trezzo | | | Get-queue-mapping should be a common interface of YarnScheduler | Major | scheduler | Wangda Tan | Tan, Wangda | | | Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API | Major | . | Naganarasimha G R | Naganarasimha G R | | | Allow files / directories to be deleted from the NameNode UI | Major | ui | Ravi Prakash | Ravi Prakash | | | Implement chmod in the HDFS Web UI | Major | ui | Ravi Prakash | Ravi Prakash | | | Render cluster Max Priority in scheduler metrics in RM web UI | Minor | resourcemanager, webapp | Rohith Sharma K S | Rohith Sharma K S | | | Populate flow run data in the flow\_run & flow activity tables | Major | timelineserver | Vrushali C | Vrushali C | | | Move NameNode.getAddress() and NameNode.getUri() to hadoop-hdfs-client | Major | hdfs-client | Mingliang Liu | Mingliang Liu | | | Resolve findbugs/javac warnings in YARN-1197 branch | Major | api, nodemanager, resourcemanager | Wangda Tan | Wangda Tan | | | RMNode State Transition Update with DECOMMISSIONING state | Major | graceful, resourcemanager | Junping Du | Junping Du | | | Remove deprecated NameNode.getUri() static helper method | Major | . | Mingliang Liu | Mingliang Liu | | | Add upgrade domain to DatanodeInfo | Major | . | Ming Ma | Ming Ma | | | Move hdfs-client protobuf convert methods from PBHelper to PBHelperClient | Major | . | Mingliang Liu | Mingliang Liu | | | Separate client and server side methods of o.a.h.hdfs.NameNodeProxies | Major | . | Mingliang Liu | Mingliang Liu | | | Keep server related definition in hdfs.proto on server side | Major | . | Yi Liu | Mingliang Liu | | | Move config keys used by hdfs-client to HdfsClientConfigKeys | Major |

| Mingliang Liu | Mingliang Liu | | | recover broken rebase during precommit | Critical | yetus | Allen Wittenauer | Allen Wittenauer | | | Consolidate encryption zone related implementation into a single class | Major | . | Haohui Mai | Rakesh R | | | Move LEASE\{SOFTLIMIT,HARDLIMIT}\PERIOD constants from HdfsServerConstants to HdfsConstants | Major | . | Mingliang Liu | Mingliang Liu | | | [reader REST API] implement support for querying for flows and flow runs | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Listing an empty s3a root directory throws FileNotFound. | Minor | fs/s3 | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Move DFSIn/OutputStream and related classes to hadoop-hdfs-client | Major | build | Haohui Mai | Mingliang Liu | | | Move DistributedFileSystem to hadoop-hdfs-client | Major | build | Yi Liu | Mingliang Liu | | | Runtime Application Priority change should not throw exception for applications at finishing states | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client | Major | build | Haohui Mai | Mingliang Liu | | | Remove guards when calling LOG.debug() and LOG.trace() in client package | Major | build | Mingliang Liu | Mingliang Liu | | | Add request/response logging & timing for each REST endpoint call | Minor | timelineserver | Vrushali C | Varun Saxena | | | HBase reader throws NPE if Get returns no rows | Major | timelineserver | Varun Saxena | Varun Saxena | | | Refactor TestReplicationPolicy to test different block placement policies | Major | . | Ming Ma | Ming Ma | | | CLI and core support for signal container functionality | Major | api | Ming Ma | Ming Ma | | | [OEV-Doc] : Document does not mention about "-f" and "-r" options | Major | . | nijel | nijel | | | OEV should treat .XML files as XML even when the file name extension is uppercase | Major | . | nijel | nijel | | | Implement support for querying single app and all apps for a flow run | Blocker | timelineserver | Varun Saxena | Varun Saxena | | | Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers | Major | erasure-coding | Jing Zhao | Jing Zhao | | | RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request | Major | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | [storage implementation] app id as string in row keys can cause incorrect ordering | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Inconsistent default value of dfs.datanode.stripedread.buffer.size | Minor | erasure-coding | Walter Su | Walter Su | | | Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client | Major | . | Haohui Mai | Haohui Mai | | | [OIV] : return value of the command is not correct if invalid value specified in "-p (processor)" option | Major | . | nijel | nijel | | | Erasure coding: Add apache license header in TestFileStatusWithECPolicy.java | Major | . | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Add InterfaceAudience annotation to the erasure coding classes | Major | . | Rakesh R | Rakesh R | | | Support NodeLabelsProvider at Resource Manager side | Major |

| Dian Fu | Dian Fu | | | Increasing container resource while there is no headroom left will cause ResourceManager to crash | Critical | resourcemanager | MENG DING | MENG DING | | | Provide BlockPlacementPolicy that supports upgrade domain | Major | . | Ming Ma | Ming Ma | | | [OIV-Doc] : Missing details of "delimited" for processor options | Major | . | nijel | nijel | | | Erasure Coding: Allow concat striped files if they have the same ErasureCodingPolicy | Major | . | Walter Su | Walter Su | | | Update pom.xml in other modules to depend on hdfs-client instead of hdfs | Major | build | Haohui Mai | Mingliang Liu | | | container-executor does not clean up docker operation command files. | Minor | . | Sidharta Seethana | Sidharta Seethana | | | Code cleanup for DatanodeDescriptor and HeartbeatManager | Minor | namenode | Jing Zhao | Jing Zhao | | | Add support for controlling capabilities for docker containers | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | [OEV and OIV] : Unnecessary parsing for mandatory arguements if "-h" option is specified as the only option | Major | . | nijel | nijel | | | Support additional queries for ATSv2 Web UI | Major | . | Varun Saxena | Varun Saxena | | | Replace guava Cache with ConcurrentHashMap for caching Connection in ipc Client | Major | conf, io, ipc | Walter Su | Walter Su | | | CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API | Major | api, client, resourcemanager | Naganarasimha G R | Naganarasimha G R | | | AM need to be notified with priority in AllocateResponse | Major | resourcemanager | Sunil Govindan | Sunil Govindan | | | Tool to measure the performance of the timeline server | Major | timelineserver | Jonathan Eagles | Chang Li | | | Allow whitelisted users to run privileged docker containers. | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Add additional logging to container launch implementations in container-executor | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Make ReservationSystem persist state using RMStateStore reservation APIs | Major | resourcemanager | Anubhav Dhoot | Anubhav Dhoot | | | Host framework UIs in YARN for use with the ATS | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | | | Add reservation system recovery to RM recovery process | Major | capacityscheduler, fairscheduler, resourcemanager | Subru Krishnan | Subru Krishnan | | | Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit | Major | resourcemanager | Xuan Gong | Xuan Gong | | | [reader implementation] support flow activity queries based on time | Minor | timelineserver | Sangjin Lee | Varun Saxena | | | Refactor the SystemMetricPublisher in RM to better support newer events | Major | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Store user in app to flow table | Major | timelineserver | Varun Saxena | Varun Saxena | | | Add support for recovery of reserved apps running under dynamic queues | Major | capacityscheduler, resourcemanager | Subru Krishnan | Subru Krishnan | | | Use POSIX nftw(3) instead of fts(3) | Major | . | Malcolm Kavalsky | Alan Burlison | | | Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup | Major | nodemanager | Naganarasimha G R | Naganarasimha G R | | | Add HdfsClientConfigKeys class

TestHdfsConfigFields#configurationClasses | Major | build | Mingliang Liu | Mingliang Liu | | | Max-AM-Resource-Percentage should respect node labels | Critical | resourcemanager | Wangda Tan | Sunil Govindan | | | [JDK8] Fix a failure of compiling common by javadoc | Major | . | Tsuyoshi Ozawa | Akira Ajisaka | | | util/Timer.cc completely misunderstands \POSIX\CPUTIME | Major | build | Alan Burlison | Alan Burlison | | | Make hadoop-mapreduce-client Native code -Wall-clean | Major | build | Alan Burlison | Alan Burlison | | | Not all platforms have d\_type in struct dirent | Major | build | Alan Burlison | Alan Burlison | | | MapReduce client assumes the world is x86 | Major | client | Alan Burlison | Alan Burlison | | | Adjust inputs order for the decode API in raw erasure coder | Major | . | Kai Zheng | Kai Zheng | | | Move client side unit test to hadoop-hdfs-client | Major | build | Haohui Mai | Haohui Mai | | | Initialize output buffers with ZERO bytes in erasure coder | Major | . | Kai Zheng | Kai Zheng | | | Erasure Coding: Skip encoding the data cells if all the parity data streamers are failed for the current block group | Minor | . | Rakesh R | Rakesh R | | | Empty caller context considered invalid | Major | . | Mingliang Liu | Mingliang Liu | | | Indicate preference not to affect input buffers during coding in erasure coder | Major | . | Kai Zheng | Kai Zheng | | | Erasure Coding: add tests for taking snapshots on EC files | Major | erasure-coding | Jing Zhao | Rakesh R | | | Wait previous ErasureCodingWork to finish before schedule another one | Major | . | Walter Su | Walter Su | | | TestAuditLogger#testAuditLoggerWithCallContext assumes Unix line endings, fails on Windows. | Minor | test | Chris Nauroth | Chris Nauroth | | | Erasure Coding: create dummy raw coder to isolate performance issues in testing | Major | . | Rui Li | Rui Li | | | Fix HDFS Balancer to honor upgrade domain policy | Major |

| Ming Ma | Ming Ma | | | Introduce Intel ISA-L erasure coding library for native erasure encoding support | Major | io | Kai Zheng | Kai Zheng | | | S3a to use thread pool that blocks clients | Major | fs/s3 | Thomas Demoor | Thomas Demoor | | | Make NNThroughputBenchmark$BlockReportStats support more than 10 datanodes | Major | test | Mingliang Liu | Mingliang Liu | | | Make NMClient support change container resources | Major | nodemanager | Wangda Tan (No longer used) | MENG DING | | | yarn rmadmin -updateNodeResource doesn't work | Critical | graceful, resourcemanager | Sushmitha Sreenivasan | Junping Du | | | Erasure coding: a comprehensive I/O throughput benchmark tool | Major | erasure-coding, test | Kai Zheng | Rui Li | | | Make AMRMClient support send increase container request and get increased/decreased containers | Major | resourcemanager | Wangda Tan (No longer used) | MENG DING | | | Fix namenodeUri parameter parsing in NNThroughputBenchmark | Major | test | Mingliang Liu | Mingliang Liu | | | NNThroughputBenchmark replication test NPE with -namenode option | Major | benchmarks | Xiaoyu Yao | Mingliang Liu | | | Remove update reservation state api from state store as its not used by ReservationSystem | Major | capacityscheduler, fairscheduler, resourcemanager | Anubhav Dhoot | Sean Po | | | Using BytesWritable's getLength() and getBytes() instead of get() and getSize() | Major | . | Tsuyoshi Ozawa | Akira Ajisaka | | | Change the way metric values are stored in HBase Storage | Major | timelineserver | Varun Saxena | Varun Saxena | | | Add efficient merge operation to RLESparseResourceAllocation | Major | resourcemanager | Carlo Curino | Carlo Curino | | | Erasure Coding: DFS GetErasureCodingPolicy API on a non-existent file should be handled properly | Minor | erasure-coding | Rakesh R | Rakesh R | | | Include X-editable for slick contenteditable fields in the web UI | Major | ui | Ravi Prakash | Ravi Prakash | | | Plumb resource-utilization info in node heartbeat through to the scheduler | Major | resourcemanager, scheduler | Karthik Kambatla | igo Goiri | | | Solaris does not support RLIMIT\_MEMLOCK as in Linux | Major | . | Malcolm Kavalsky | Alan Burlison | | | updateNodeResource CLI should not accept negative values for resource | Major | graceful, resourcemanager | Sushmitha Sreenivasan | Junping Du | | | TestPipelinesFailover assumes Linux ifconfig | Minor | test | Alan Burlison | John Zhuge | | | TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch | Major | . | Varun Saxena | Varun Saxena | | | Support for fetching specific configs and metrics based on prefixes | Major | timelineserver | Varun Saxena | Varun Saxena | | | ResourceUtilization should be a part of NodeInfo REST API | Major | . | Wangda Tan | Sunil Govindan | | | Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default | Minor | test | Mingliang Liu | Mingliang Liu | | | NNThroughputBenchmark$BlockReportStats should not send empty block reports | Major | test | Mingliang Liu | Mingliang Liu | | | Support node label store in non-appendable file system | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | Support reconfiguring dfs.datanode.balance.max.concurrent.moves without DN restart | Major | datanode | Xiaobing Zhou | Xiaobing Zhou | | | Move the safemode block count into BlockManager | Major | namenode | Haohui Mai | Mingliang Liu | | | REST API for submit/update/delete Reservations | Major | resourcemanager | Carlo Curino | Carlo Curino | | | Refactor reconfiguration of ClientDatanodeProtocol for reusability | Major | . | Xiaobing Zhou | Xiaobing Zhou | | | ensure the timeline service v.2 is disabled cleanly and has no impact when it's turned off | Critical | timelineserver | Sangjin Lee | Sangjin Lee | | | Improve relationship between SharingPolicy and ReservationAgent | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Carlo Curino | | | TestTimelineAuthenticationFilter and TestYarnConfigurationFields fail | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message | Major | capacity scheduler, resourcemanager | Sumit Nigam | Naganarasimha G R | | | Add container launch related debug information to container logs when a container fails | Major | nodemanager | Varun Vasudev | Varun Vasudev | | | Code cleanup for DatanodeManager | Major | namenode | Jing Zhao | Jing Zhao | | | [Bug fix] RM fails to start when SMP is enabled | Major | timelineserver | Li Lu | Li Lu | | | Unify the term flowId and flowName in timeline v2 codebase | Major | timelineserver | Li Lu | Zhan Zhang | | | ResourceUtilization should be a part of yarn node CLI | Major |

| Wangda Tan | Sunil Govindan | | | Deadlock due to synchronised get Methods in AbstractCSQueue | Minor | capacity scheduler, resourcemanager | Naganarasimha G R | Naganarasimha G R | | | cgroups based memory monitoring for containers | Major | nodemanager | Karthik Kambatla | Varun Vasudev | | | Erasure coding: friendly log information for write operations with some failed streamers | Major | erasure-coding | Li Bo | Li Bo | | | UI changes for decommissioning node | Major | graceful | Junping Du | Sunil Govindan | | | Retrospect update ApplicationPriority API return type | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | TestDistributedShell fails for V2 scenarios | Major | timelineserver | Sangjin Lee | Naganarasimha G R | | | Erasure Coding: Lease recovery for striped file | Major | erasure-coding | Walter Su | Walter Su | | | Use byte array for internal block indices in a striped block | Major | erasure-coding | Jing Zhao | Jing Zhao | | | RM only get back addresses of Collectors that NM needs to know. | Critical | resourcemanager, timelineserver | Junping Du | Junping Du | | | New put APIs in TimelineClient for ats v1.5 | Major | timelineserver | Xuan Gong | Xuan Gong | | | Add an OPPORTUNISTIC ExecutionType | Major | nodemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Recovery may get very slow with lots of services with lots of app-attempts | Major | resourcemanager | Jun Gong | Jun Gong | | | Document ApplicationPriority feature | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Support changing ownership, group and replication in HDFS Web UI | Major | ui | Ravi Prakash | Ravi Prakash | | | Input buffer position after encode/decode not consistent between different kinds of buffers | Minor | . | Rui Li | Rui Li | | | Move code that tracks blocks with future generation stamps to BlockManagerSafeMode | Major | namenode | Mingliang Liu | Mingliang Liu | | | Killing a container that is localizing can orphan resources in the DOWNLOADING state | Major | nodemanager | Jason Lowe | Varun Saxena | |

TestResourceLocalizationService#testFailedDirsResourceRelease fails intermittently | Major | test | Varun Saxena | Varun Saxena | | | some tests in TestContainerLanch fails on non-english locale environment | Minor | nodemanager, test | Takashi Ohnishi | Takashi Ohnishi | | | Allow ResourceRequests to specify ExecutionType of a request ask | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Retrospect app-priority in pendingOrderingPolicy during recovering applications | Major | api, resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Some of the NM events are not getting published due race condition when AM container finishes in NM | Major | nodemanager, timelineserver | Naganarasimha G R | Naganarasimha G R | | | getListing wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory | Critical | erasure-coding | Sushmitha Sreenivasan | Jing Zhao | | | Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy | Major | capacity scheduler | Rohith Sharma K S | Rohith Sharma K S | | | Enable YARN to use S3A | Major | fs/s3 | Thomas Demoor | Pieter Reuse | | | Add cgroups support for docker containers | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | Provide new timeline plugin storage to support fine-grained entity caching | Major | timelineserver | Li Lu | Li Lu | | | AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics | Major | webapp | Sunil Govindan | Sunil Govindan | | | Make SystemClock singleton so AppSchedulingInfo could use it | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=\* | Major | resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Implement the Hitchhiker erasure coding algorithm | Major | . | Zhe Zhang | jack liuquan | | | Move BlockIdManager from FSNamesystem to BlockManager | Major | namenode | Jing Zhao | Jing Zhao | | | TestDistributedShell fails for v2 test cases after modifications for 1.5 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | NNThroughputBenchmark.BlockReportStats should handle NotReplicatedYetException on adding block | Major | test | Masatake Iwasaki | Masatake Iwasaki | | | Directories that are mounted in docker containers need to be more restrictive/container-specific | Major | yarn | Sidharta Seethana | Sidharta Seethana | | | TestAMRMClientOnRMRestart fails on trunk | Major | client, test | Takashi Ohnishi | Takashi Ohnishi | | | ErasureCodingWorker may fail when recovering data blocks with length less than the first internal block | Critical | erasure-coding | Takuya Fukudome | Jing Zhao | | | TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority fails occasionally | Major | test | Jason Lowe | Sunil Govindan | | | TestClientRMService#testGetClusterNodes fails occasionally | Major | test | Jason Lowe | Takashi Ohnishi | | | Add command line option to ask NameNode reload configuration. | Major | namenode | Xiaobing Zhou | Xiaobing Zhou | | | o.a.h.hdfs.TestLeaseRecovery2 fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | TestRMAppTransitions.testAppRunningKill and testAppKilledKilled fail on trunk | Major | resourcemanager, test | Takashi Ohnishi | Takashi Ohnishi | | | correctly set createdTime and remove modifiedTime when publishing entities | Major | timelineserver | Varun Saxena | Varun Saxena | | | Support fetching entities by UID and change the REST interface to conform to current REST APIs' in YARN | Major | timelineserver | Varun Saxena | Varun Saxena | | | TestRMRestart fails and findbugs issue in YARN-2928 branch | Major | timelineserver | Varun Saxena | Varun Saxena | | | TestRMRestart.testRMRestartAfterPreemption fails intermittently in trunk | Major | test | Rohith Sharma K S | Bibin A Chundatt | | | Container recovery is broken with delegating container runtime | Critical | yarn | Sidharta Seethana | Sidharta Seethana | | | New levelDB cache storage for timeline v1.5 | Major | . | Li Lu | Li Lu | | | TestNodeStatusUpdater.testStopReentrant fails + JUnit misusage | Minor | nodemanager | Akihiro Suda | Akihiro Suda | | | TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt fails occasionally | Major | test | Jason Lowe | Sunil Govindan | | | Parallel optimization

DFSStripedOutputStream#flushAllInternals( ) | Minor | hdfs-client | Rui Gao | Rui Gao | | | Add "list" API to reservation system | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Sean Po | | | Add Documentation for Distributed and Delegated-Centralized Node Labels feature | Major | api, client, resourcemanager | Naganarasimha G R | Naganarasimha G R | | | Erasure Coding: Rename BlockECRecoveryCommand to BlockECReconstructionCommand | Major | erasure-coding | Rakesh R | Rakesh R | | | Erasure coding: some EC tests are missing timeout | Minor | erasure-coding, test | Zhe Zhang | Rui Gao | | | Erasure Coding: allow to use multiple EC policies in striping related tests | Major | . | Rui Li | Rui Li | | | Implement another Reed-Solomon coder in pure Java | Major | . | Kai Zheng | Kai Zheng | | | Replace -namenode option with -fs for NNThroughputBenchmark | Major | test | Konstantin Shvachko | Mingliang Liu | | | Refactor reader API for better extensibility | Major | timelineserver | Varun Saxena | Varun Saxena | | | Make use of DeleteObjects optional | Major | fs/s3 | Thomas Demoor | Thomas Demoor | | | RM Admin CLI for refreshNodesResources throws NPE when nothing is configured | Critical | client | Naganarasimha G R | Naganarasimha G R | | | Improve diagnostics/use of envvar/sysprop credential propagation | Minor | security | Steve Loughran | Steve Loughran | | | Fix javadoc and checkstyle issues in timelineservice code | Major | timelineserver | Varun Saxena | Varun Saxena | | | Replace starting a separate thread for post entity with event loop in TimelineClient | Major | timelineserver | Junping Du | Naganarasimha G R | | | Erasure Coding : Rename BlockRecoveryWork to BlockReconstructionWork | Major | namenode | Rakesh R | Rakesh R | | | Roll back container resource allocation after resource increase token expires | Major | api, nodemanager, resourcemanager | MENG DING | MENG DING | | | Correctly update DataNode's scheduled block size when writing small EC file | Major | namenode | Namit Maheshwari | Jing Zhao | | | Erasure Coding: allow to use multiple EC policies in striping related tests [Part 2] | Major | erasure-coding, test | Rui Li | Rui Li | | | Create separate ACLs for Reservation create/update/delete/list ops | Major | capacityscheduler, fairscheduler, resourcemanager | Subru Krishnan | Sean Po | | | TestYarnCLI#testGetContainers failing in CN locale | Major | yarn | Bibin A Chundatt | Bibin A Chundatt | | | Streamer threads may leak if failure happens when closing the striped outputstream | Critical | hdfs-client | Namit Maheshwari | Jing Zhao | | | TestBlockManagerSafeMode#testCheckSafeMode fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | Remove dependency on commons-httpclient for TestHttpServerLogs | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Remove references to hftp in ViewFs documentation in trunk | Minor | documentation, fs | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Erasure Coding: Improve few exception handling logic of ErasureCodingWorker | Minor | erasure-coding | Rakesh R | Rakesh R | | | Correctly handle EC reconstruction work caused by not enough racks | Major | datanode, namenode | Takuya Fukudome | Jing Zhao | | | Remove commons-httpclient dependency from hadoop-azure | Major | . | Akira Ajisaka | Masatake Iwasaki | | | Pagination, sorting and filtering of files/directories in the HDFS Web UI | Major | ui | Ravi Prakash | Ravi Prakash | | | Resource update during NM graceful decommission | Major | graceful, nodemanager, resourcemanager | Junping Du | Brook Zhou | | | Erasure Coding: allow to use multiple EC policies in striping related tests [Part 3] | Major | erasure-coding, test | Rui Li | Rui Li | | | TimerTasks leak in ATS

Writer | Major | timelineserver | Xuan Gong | Xuan Gong | | | Rename the RS coder from HDFS-RAID as legacy | Major | . | Rui Li | Rui Li | | | BlockManager#countNodes should be able to detect duplicated internal blocks | Major | . | Jing Zhao | Jing Zhao | | | BlockManager#chooseExcessReplicasStriped may weaken rack fault tolerance | Major | namenode | Takuya Fukudome | Jing Zhao | | | Missing block exception should carry locatedBlocks information | Minor | erasure-coding, hdfs-client | Mingliang Liu | Mingliang Liu | | | TestMiniYarnClusterNodeUtilization sometimes fails on trunk | Major | test | Takashi Ohnishi | Takashi Ohnishi | | | Migrate TestRPC and related codes to rebase on ProtobufRpcEngine | Major | . | Kai Zheng | Kai Zheng | | | shouldProcessOverReplicated should not count number of pending replicas | Major | erasure-coding, namenode | Takuya Fukudome | Jing Zhao | | | ATS storage has one extra record each time the RM got restarted | Major | timelineserver | Li Lu | Naganarasimha G R | | | Generalize config file handling in container-executor | Major | nodemanager | Sidharta Seethana | Sidharta Seethana | | | Erasure coding: use simple replication for internal blocks on decommissioning datanodes | Major | . | Zhe Zhang | Rakesh R | | | reconcile the two versions of the timeline service performance tests | Minor | . | Sangjin Lee | Sangjin Lee | | | Erasure coding: NameNode manages multiple erasure coding policies | Major | . | Kai Zheng | Rui Li | | | Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency | Major | timelineserver | Steve Loughran | Steve Loughran | | | Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart | Major | namenode | Patrick Kling | Xiaobing Zhou | | | Migrate MR Client test cases part 1 | Trivial | test | Dustin Cote | Dustin Cote | | | Improve and restructure native ISAL support | Major | io | Kai Zheng | Kai Zheng | | | Allow YARN distributed shell to use ATS v1.5 APIs | Major | timelineserver | Li Lu | Li Lu | | | Rename the new Java coder and make it default | Major | . | Rui Li | Rui Li | | | Change Log Level to DEBUG for putDomain call in ATS 1.5 | Trivial | timelineserver | Xuan Gong | Xuan Gong | | | Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1] | Major | erasure-coding | Rakesh R | Rakesh R | | | CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request | Major | capacity scheduler | Wangda Tan | Wangda Tan | | | Add the flush and compaction functionality via coprocessors and scanners for flow run table | Major | timelineserver | Vrushali C | Vrushali C | | | CPU Usage Metric is not captured properly in YARN-2928 | Major | timelineserver | Naganarasimha G R | Naganarasimha G R | | | lz4.c does not detect 64-bit mode properly | Major | native | Alan Burlison | Alan Burlison | | | Migrate TestSaslRPC and related codes to rebase on ProtobufRpcEngine | Major | . | Kai Zheng | Kai Zheng | | | ATS 1.5 timelineclient impl try to create attempt directory for every event call | Major | timelineserver | Xuan Gong | Xuan Gong | | | ATS

timelineclient impl call flush after every event write | Major | timelineserver | Xuan Gong | Xuan Gong | | | Make existing DFSClient#getFileChecksum() work for striped blocks | Major | . | Kai Zheng | Kai Zheng | | | Don't go through all schedulers in ParameterizedTestBase | Major | . | Karthik Kambatla | Karthik Kambatla | | | Remove hsftp from DistCp in trunk | Major | distcp | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Keep NM resource updated through dynamic resource config for RM/NM restart | Major | graceful, nodemanager, scheduler | Junping Du | Junping Du | | | NM is going down with NPE's due to single thread processing of events by Timeline client | Critical | timelineserver | Naganarasimha G R | Naganarasimha G R | | | Migrate MR Client test cases part 2 | Trivial | test | Dustin Cote | Dustin Cote | | | Support reconfiguring fs.protected.directories without NN restart | Major | namenode | Xiaobing Zhou | Xiaobing Zhou | | | Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates | Major | . | Wangda Tan | Wangda Tan | | | Generate histograms in ContainerMetrics for actual container resource usage | Major | . | Varun Vasudev | Varun Vasudev | | | Scheduler UI/Metrics need to consider cases like non-queue label mappings | Major | . | Sunil Govindan | Sunil Govindan | | | ListStatus on empty dir in S3A lists itself instead of returning an empty list | Major | fs/s3 | Pieter Reuse | Pieter Reuse | | | Add additional github web site for ISA-L library | Major | . | Li Bo | Li Bo | | | DirectoryScanner: Improve logs by adding full path of both actual and expected block directories | Minor | datanode | Rakesh R | Rakesh R | | | Refactoring ErasureCodingWorker into smaller reusable constructs | Major | . | Kai Zheng | Kai Zheng | | | Consolidate flow name/version/run defaults | Major | timelineserver | Zhijie Shen | Sangjin Lee | | | S3A JUnit tests failing if using HTTP proxy | Minor | fs/s3 | Zoran Rajic | Zoran Rajic | | | Document configuration of ReservationSystem for CapacityScheduler | Minor | capacity scheduler | Subru Krishnan | Subru Krishnan | | | Support lazy seek in S3AInputStream | Major | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Some yarn.server.timeline.\* tests fail on Windows attempting to use a test root path containing a colon | Minor | test | Gergely Novk | Gergely Novk | | | Test TestLogAggregationService.testLocalFileDeletionOnDiskFull failing | Critical | test | Steve Loughran | Takashi Ohnishi | | | Support complex filters in TimelineReader | Major | . | Varun Saxena | Varun Saxena | | | Erasure Coding: Sort located striped blocks based on decommissioned states | Major | . | Rakesh R | Rakesh R | | | make DU pluggable | Major | . | Elliott Clark | Elliott Clark | | | Add HDFS caller context for EntityGroupFSTimelineStore | Major | timelineserver | Li Lu | Li Lu | | | Support enable caller context in HDFS namenode audit log without restart namenode | Major | . | Xiaoyu Yao | Xiaobing Zhou | | | Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties | Major | . | Xiaoyu Yao | Xiaobing Zhou | | | o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | |

StripedFileTestUtil#readAll flaky | Major | erasure-coding, test | Mingliang Liu | Mingliang Liu | | | Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter | Blocker | . | Brahma Reddy Battula | Bibin A Chundatt | | | Document the general ReservationSystem functionality, and the REST API | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Carlo Curino | | | Configure raw erasure coders for supported codecs | Minor | . | Rui Li | Rui Li | | | o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | Queuing of container requests in the NM | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate | Major | . | Wangda Tan | Sunil Govindan | | | Support enable Hadoop IPC backoff without namenode restart | Major | . | Xiaoyu Yao | Xiaobing Zhou | | | A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. | Major | . | Wangda Tan | Wangda Tan | | | Clearly Document the Password Details for Keystore-based Credential Providers | Major | documentation | Larry McCay | Larry McCay | | | [Aggregation] App-level aggregation and accumulation for YARN system metrics | Major | timelineserver | Junping Du | Li Lu | | | Random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | Respect labels in CapacityScheduler when computing headroom | Major | capacityscheduler | Wangda Tan | Naganarasimha G R | | | Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Arun Suresh | | | API documentation for dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2] | Major | erasure-coding | Rakesh R | Rakesh R | | | Fix ContainerRequest Constructor to set nodelabelExpression correctly | Major | test | Bibin A Chundatt | Bibin A Chundatt | | | Implement asynchronous rename for DistributedFileSystem | Major | fs, hdfs-client | Xiaobing Zhou | Xiaobing Zhou | | | Improve yarn logs to fetch container logs without specifying nodeId | Major | . | Xuan Gong | Xuan Gong | | | MockAM#waitForState sleep duration is too long | Major | . | Karthik Kambatla | Yufei Gu | | | findbug issue on LevelDBCacheTimelineStore | Major | timelineserver | Xuan Gong | Zhiyuan Yang | | | Add support in the NodeManager to re-launch containers | Major | . | Jun Gong | Jun Gong | | | Add a check in the coprocessor for table to operated on | Major | timelineserver | Vrushali C | Vrushali C | | | Ensure non-metric values are returned as is for flow run table from the coprocessor | Major | timelineserver | Vrushali C | Vrushali C | | | [Documentation] Documenting the timeline service v2 | Major | . | Zhijie Shen | Sangjin Lee | | | Store MR counters as timeline metrics instead of event | Major | . | Junping Du | Naganarasimha G R | | | Provide a mechanism to represent complex filters and parse them at the REST layer | Major | timelineserver | Varun Saxena | Varun Saxena | | | Metric improvements for ATS

storage components | Major | timelineserver | Li Lu | Li Lu | | | Store job configurations in Timeline Service v2 | Major | applicationmaster | Junping Du | Varun Saxena | | | bin/hadoop work for dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | bin/mapred work for dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | bin/hdfs work for dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | bin/yarn work for dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | modify hadoop-tools to take advantage of dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | ATS/NM should support a link to dowload/get the logs in text format | Major | yarn | Xuan Gong | Xuan Gong | | | enable daemonization of dynamic commands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | hadoop distcp adds client opts twice when dynamic | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | env var doc update for dynamic commands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | fix shellprofiles in hadoop-tools to allow replacement | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Improve "yarn logs" command-line to optionally show log metadata also | Major | . | Xuan Gong | Xuan Gong | | | hadoop-common unit tests for dynamic commands | Major | scripts, test | Allen Wittenauer | Allen Wittenauer | | | hadoop-hdfs unit tests for dynamic commands | Major | scripts, test | Allen Wittenauer | Allen Wittenauer | | | Add support for configurable read-only mounts when launching Docker containers | Major | yarn | Billie Rinaldi | Billie Rinaldi | | | clean up how rumen is executed | Major | scripts, tools | Allen Wittenauer | Allen Wittenauer | | | dynamic subcommands need a way to manipulate arguments | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | add a streaming subcommand to mapred | Major | . | Allen Wittenauer | Allen Wittenauer | | | hbase unit tests fail due to dependency issues | Blocker | timelineserver | Sangjin Lee | Sangjin Lee | | | convert hadoop gridmix to be dynamic | Major | scripts, tools | Allen Wittenauer | Allen Wittenauer | | | dynamic subcommand docs should talk about exit vs. continue program flow | Major | documentation, scripts | Allen Wittenauer | Allen Wittenauer | | | clarify daemonization and security vars for dynamic commands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | rename sub-project shellprofiles to match the rest of Hadoop | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | add a --debug message when dynamic commands have been used | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Support specifying resources for task containers in SLS | Major | . | Wangda Tan | Wangda Tan | | | "yarn logs" command should not require the appOwner argument | Major | . | Ram Venkatesh | Xuan Gong | | | Refactor startContainerInternal() in ContainerManager to remove unused parameter | Minor | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors | Major | . | Sidharta Seethana | Sidharta Seethana | | | Implement asynchronous setPermission/setOwner for DistributedFileSystem | Major | hdfs, hdfs-client | Xiaobing Zhou | Xiaobing Zhou | | | Customize User-Agent header sent in HTTP requests by

| Minor | fs/s3 | Chris Nauroth | Chris Nauroth | | | Recover when S3A fails on IOException in read() | Major | fs/s3 | Pieter Reuse | Pieter Reuse | | | add low level counter metrics for S3A; use in read performance tests | Major | fs/s3, metrics | Steve Loughran | Steve Loughran | | | ShellBasedIdMapping needs suport for Solaris | Major | security | Alan Burlison | Alan Burlison | | | Add maintenance states to AdminStates | Major | . | Ming Ma | Ming Ma | | | Enable parallel test execution for hadoop-aws. | Minor | test | Chris Nauroth | Chris Nauroth | | | Corrective mechanisms for rebalancing NM container queues | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Arun Suresh | | | fix typo in dynamic subcommand docs | Trivial | scripts | Sean Busbey | Allen Wittenauer | | | Underscores should be escaped in dynamic subcommands document | Minor | documentation | Akira Ajisaka | Allen Wittenauer | | | fix typo in debug statement for dynamic subcommands | Major | scripts | Allen Wittenauer | Allen Wittenauer | | | Notify the RM about the status of OPPORTUNISTIC containers | Major | nodemanager, resourcemanager | Konstantinos Karanasos | Konstantinos Karanasos | | | Fix findbugs warning in hadoop-yarn-common module | Major | . | Akira Ajisaka | Arun Suresh | | | Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers | Major | nodemanager, resourcemanager | Arun Suresh | Arun Suresh | | | S3AFileSystem#toString might throw NullPointerException due to null cannedACL. | Minor | fs/s3 | Chris Nauroth | Chris Nauroth | | | timeline service build fails with java 8 | Blocker | timelineserver | Sangjin Lee | Sangjin Lee | | | NM side resource value should get updated if change applied in RM side | Critical | nodemanager, resourcemanager | Junping Du | Junping Du | | | Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster | Major | . | Arun Suresh | Arun Suresh | | | timelinereader has a lot of logging that's not useful | Minor | timelineserver | Sangjin Lee | Sangjin Lee | | | FileSystem#initialize must not attempt to create StorageStatistics objects with null or empty schemes | Major | fs | Brahma Reddy Battula | Mingliang Liu | | | Yarn logs should take a -out option to write to a directory | Major | . | Xuan Gong | Xuan Gong | | | saveNamespace should not throw IOE when only one storage directory fails to write VERSION file | Major | . | Todd Lipcon | Andras Bokor | | | Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds | Major | . | Arun Suresh | Konstantinos Karanasos | | | Code cleanup for TestDistributedShell | Major | timelineserver | Li Lu | Li Lu | | | Add support for a minimum retry interval for container retries | Major |

| Varun Vasudev | Jun Gong | | | s3a failures can surface as RTEs, not IOEs | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect | Major | ui, webhdfs | Ravi Prakash | Ravi Prakash | | | created time shows 0 in most REST output | Critical | timelineserver | Sangjin Lee | Varun Saxena | | | Online aggregation logic should not run immediately after collectors got started | Major | timelineserver | Li Lu | Li Lu | | | Add node label usage in RM CapacityScheduler web UI | Major | capacityscheduler, resourcemanager, webapp | Wangda Tan | Naganarasimha G R | | | Implement asynchronous setAcl/getAclStatus for DistributedFileSystem | Major | fs | Xiaobing Zhou | Xiaobing Zhou | | | Avoid setting CONTENT-DISPOSITION header in the container-logs web-service | Major | . | Xuan Gong | Xuan Gong | | | Localization failures should be available in container diagnostics | Critical | . | Hitesh Shah | Vinod Kumar Vavilapalli | | | Move BlockReader implementation to the client implementation package | Major | hdfs-client | Tsz Wo Nicholas Sze | Takanobu Asanuma | | | Fix intermittent test failure of TestDataNodeErasureCodingMetrics | Major | . | Rakesh R | Rakesh R | | | Checks for SPARC architecture need to include 64-bit SPARC | Major | conf | Alan Burlison | Alan Burlison | | | NPE in Separator.joinEncoded() | Critical | timelineserver | Sangjin Lee | Vrushali C | | | flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled | Critical | timelineserver | Sangjin Lee | Varun Saxena | | | Add getNewReservation in ApplicationClientProtocol | Major | applications, client, resourcemanager | Subru Krishnan | Sean Po | | | Refactor raw erasure coders | Major | . | Kai Zheng | Kai Zheng | | | Minor cleanups needed for CMake 3.X | Minor | native | Alan Burlison | Alan Burlison | | | fix "no findbugs output file" error for hadoop-yarn-server-timelineservice-hbase-tests | Major | timelineserver | Vrushali C | Vrushali C | | | Refactor and speedup TestAsyncDFSRename | Minor | test | Xiaobing Zhou | Xiaobing Zhou | | | Erasure Coding: Rename replication-based names in BlockManager to more generic [part-3] | Major | erasure-coding | Rakesh R | Rakesh R | | | timestamps are stored unencoded causing parse errors | Blocker | timelineserver | Sangjin Lee | Varun Saxena | | | Improve "yarn log" command-line "logFiles" option to support regex | Major | . | Xuan Gong | Xuan Gong | | | Read cache concurrency issue between read and evict in EntityGroupFS timeline store | Critical | . | Li Lu | Li Lu | | | Expose ExecutionType in Container api record | Major | . | Arun Suresh | Hitesh Sharma | | | Reuse FileSystem#access in TestAsyncDFS | Major | hdfs | Xiaobing Zhou | Xiaobing Zhou | | | QueuingContainerManager does not start GUARANTEED Container even if Resources are available | Major | . | Arun Suresh | Konstantinos Karanasos | | | YARN container system metrics are not aggregated to application | Critical | timelineserver | Sangjin Lee | Naganarasimha G R | | | Add support for different network setups when launching the docker container | Major | nodemanager | Varun Vasudev | Sidharta Seethana | | | Get Container logs for the Running application from Yarn Logs CommandLine | Major | . | Xuan Gong | Xuan Gong | | | entire time series is returned for YARN container system metrics (CPU and memory) | Major | timelineserver | Sangjin Lee | Varun Saxena | | | Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data | Major | . | Kai Zheng | Rakesh R | | | Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs | Minor | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Improve "yarn log" command-line to read the last K bytes for the log files | Major |

| Xuan Gong | Xuan Gong | | | Add tests to verify that S3A supports SSE-S3 encryption | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Fix Exceptions thrown during registerAM call when Distributed Scheduling is Enabled | Major | . | Arun Suresh | Hitesh Sharma | | | Add StorageStatistics to S3A; instrument some more operations | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Work preserving recovery of Unmanged AMs | Critical | resourcemanager | Karthik Kambatla | Subru Krishnan | | | Fix NoOvercommitPolicy to take advantage of RLE representation of plan | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Carlo Curino | | | Make HBaseTimeline[Reader\|Writer]Impl default and move FileSystemTimeline\*Impl | Major | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | HDFS Namenode UI: Fix browsing directories that need to be encoded | Major | ui | Ravi Prakash | Ravi Prakash | | | Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI | Major | ui, webhdfs | Ravi Prakash | Ravi Prakash | | | fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests | Major | timelineserver | Vrushali C | Vrushali C | | | Escaping occurences of encodedValues | Critical | timelineserver | Joep Rottinghuis | Sangjin Lee | | | StageAllocaterGreedyRLE: Fix NPE in corner case | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Carlo Curino | | | Fix bug in RLESparseResourceAllocation.getRangeOverlapping(...) | Major | . | Ishai Menache | Ishai Menache | | | Allow ResourceRequest to specify an enforceExecutionType flag | Major | . | Arun Suresh | Arun Suresh | | | More test cases for queuing of containers at the NM | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | Properly report status of killed/stopped queued containers | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | s3a initialization against public bucket fails if caller lacks any credentials | Minor | fs/s3 | Steve Loughran | Chris Nauroth | | | NPE in Distributed Shell while publishing DS\CONTAINER\START event and other miscellaneous issues | Major | timelineserver | Varun Saxena | Varun Saxena | | | Close LogReader in in AHSWebServices#getStreamingOutput and FileInputStream in NMWebServices#getLogs | Major | . | Xuan Gong | Xuan Gong | | | Rename the download=true option for getLogs in NMWebServices and AHSWebServices | Major | . | Xuan Gong | Xuan Gong | | | Eliminate singleton converters and static method access | Major | timelineserver | Joep Rottinghuis | Joep Rottinghuis | | | Allow files / directories to be moved | Major | ui | Ravi Prakash | Ravi Prakash | | | Add jdiff support to YARN | Blocker | . | Li Lu | Li Lu | | | [Documentation] Update timeline service v2 documentation to capture information about filters | Major | timelineserver | Varun Saxena | Varun Saxena | | | Modify AMRMClient to set the ExecutionType in the ResourceRequest | Major | . | Arun Suresh | Arun Suresh | | | Run existing ContainerManager tests using QueuingContainerManagerImpl | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | | | AM-RM protocol changes for identifying resource-requests explicitly | Major | applications, resourcemanager | Subru Krishnan | Subru Krishnan | | | Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId,

| Critical | api | Thomas Graves | Wangda Tan | | | Container line in yarn logs output for a live application should include the hostname for the container | Major | . | Siddharth Seth | Xuan Gong | | | "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete | Major | . | Vinod Kumar Vavilapalli | Jian He | | | fix several rebase and other miscellaneous issues before merge | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | upgrade HBase version for first merge | Critical | timelineserver | Sangjin Lee | Vrushali C | | | document s3a better | Minor | documentation, fs/s3 | Steve Loughran | Steve Loughran | | | Add jitter to CachingGetSpaceUsed's thread | Major | . | Elliott Clark | Elliott Clark | | | Yarn CLI to obtain App logs for last 'n' bytes fails with 'java.io.IOException' and for 'n' bytes fails with NumberFormatException | Blocker | . | Sumana Sathish | Xuan Gong | | | FileSystemStorageStatistics#getLong(readOps) should return readOps + largeReadOps | Major | fs | Mingliang Liu | Mingliang Liu | | | Guard null stats key in FileSystemStorageStatistics | Major | fs | Mingliang Liu | Mingliang Liu | | | Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented | Major | fs | Mingliang Liu | Mingliang Liu | | | Remove AsyncDistributedFileSystem API | Major | hdfs-client | Xiaobing Zhou | Xiaobing Zhou | | | Add per-cache-pool default replication num configuration | Minor | caching | xupeng | xupeng | | | S3A: Support fadvise "random" mode for high performance readPositioned() reads | Major | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Allow only suitable storage policies to be set on striped files | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node | Major | . | Arun Suresh | igo Goiri | | | Inconsistent values of counters across tasks and job reported to timeline service. | Major | applicationmaster | Varun Saxena | Varun Saxena | | | Erasure Coding: Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block | Major | datanode | Rakesh R | Rakesh R | | | Implement ErasureCodec for HitchHiker XOR coding | Major | io | Kai Sasaki | Kai Sasaki | | | Fix inconsistent subsection titles in filesystem.md | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Add in-page TOC to filesystem specification pages | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Define common statistics names across schemes | Major | fs | Mingliang Liu | Mingliang Liu | | | [documentation] several updates/corrections to timeline service documentation | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | yarn logs command: no need to specify -applicationId when specifying containerId | Major | . | Jian He | Gergely Novk | | | Logs for a completed container are not available in the yarn logs output for a live application | Major | . | Siddharth Seth | Xuan Gong | | | fix hadoop-aws pom not to do the exclusion | Major | timelineserver | Sangjin Lee | Sangjin Lee | | | maven enforcer plugin to ban java 8 incompatible dependencies | Minor |

| Steve Loughran | Tsuyoshi Ozawa | | | Support reset operation for new global storage statistics and per FS storage stats | Major | fs | Mingliang Liu | Mingliang Liu | | | Support for specifying a path for ATS plugin jars | Major | timelineserver | Li Lu | Li Lu | | | Cannot obtain logs using YARN CLI -am for either KILLED or RUNNING AM | Critical | yarn | Sumana Sathish | Xuan Gong | | | Improve yarn logs to get Container List | Major | . | Xuan Gong | Xuan Gong | | | Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc | Minor | documentation | Rakesh R | Rakesh R | | | Log Docker run command when container fails | Major | yarn | Varun Vasudev | Varun Vasudev | | | Fix signal handling for docker containers | Major | yarn | Sidharta Seethana | Shane Kumpf | | | For AM containers, or for containers of running-apps, "yarn logs" incorrectly only (tries to) shows syslog file-type by default | Major | log-aggregation | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | | PlanCommand#getThrsholdPercentage should not use throughput value. | Major | diskbalancer | Lei (Eddy) Xu | Yiqun Lin | | | Mount usercache and NM filecache directories into Docker container | Major | yarn | Varun Vasudev | Sidharta Seethana | | | Obtaining logs for completed container says 'file belongs to a running container ' at the end | Critical | . | Sumana Sathish | Xuan Gong | | | [YARN-3368] Add resource usage by applications and queues to cluster overview page. | Major | . | Wangda Tan | Wangda Tan | | | Available Resource calculation for a queue is not correct when used with labels | Major | capacity scheduler | Sunil Govindan | Sunil Govindan | | | DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation | Major | fs | Mingliang Liu | Mingliang Liu | | | Distributed Scheduling: Ensure sort order of allocatable nodes returned by the RM is not lost | Major | . | Arun Suresh | Arun Suresh | | | Provide an option to set the socket buffers in S3AFileSystem | Minor | fs/s3 | Rajesh Balamohan | Rajesh Balamohan | | | Optimize conversion from path string to components | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Erasure Coding: Document about the current allowed storage policies for EC Striped mode files | Major | documentation | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Specify FileSystem listStatus, listFiles and RemoteIterator | Major | documentation, fs | Steve Loughran | Steve Loughran | | | S3A file-create should throw error rather than overwrite directories | Minor | fs/s3 | Raymie Stata | Steve Loughran | | | Use plan RLE to improve CapacityOverTimePolicy efficiency | Major | capacityscheduler, fairscheduler, resourcemanager | Carlo Curino | Carlo Curino | | | Adding tests for coder utilities | Major | . | Kai Sasaki | Kai Sasaki | | | TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently | Major | datanode, test | Mingliang Liu | Mingliang Liu | | | Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey | Major | . | Arun Suresh | Arun Suresh | | | ResourceRequest should take ExecutionType into account during comparison | Major | . | Konstantinos Karanasos | Konstantinos Karanasos | |

TestDataNodeMXBean#testDataNodeMXBeanBlockCount fails intermittently | Major | test | Mingliang Liu | Mingliang Liu | | | Add -client\|server argument for graceful decom | Blocker | graceful | Robert Kanter | Robert Kanter | | | Refactoring and other clean-up for distributed scheduling | Major | . | Arun Suresh | Konstantinos Karanasos | | | Raw Reed-Solomon coder using Intel ISA-L library | Major | . | Zhe Zhang | Kai Zheng | | | Rename DockerStopCommandTest to TestDockerStopCommand | Trivial | . | Shane Kumpf | Shane Kumpf | | | Add support for docker inspect command | Major | yarn | Shane Kumpf | Shane Kumpf | | | Dispose of unnecessary SASL servers | Major | ipc | Daryn Sharp | Daryn Sharp | | | remove AHS enable check from LogsCLI#fetchAMContainerLogs | Major | . | Xuan Gong | Xuan Gong | | | More efficiently build IPC responses | Major | . | Daryn Sharp | Daryn Sharp | | | Optimize conversion of byte arrays back to path string | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Optimize creating a full path from an inode | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Improve non-exclusive node partition resource allocation in Capacity Scheduler | Major | . | Wangda Tan | Sunil Govindan | | | Optimize IPC server protobuf decoding | Major | . | Daryn Sharp | Daryn Sharp | | | Add support for docker rm | Minor | yarn | Shane Kumpf | Shane Kumpf | | | False alarm in datanode log - ERROR - Disk Balancer is not enabled | Major | datanode, hdfs | Weiwei Yang | Weiwei Yang | | | Fix @return related javadoc warnings in yarn-api | Major | . | Vrushali C | Vrushali C | | | Changes in scheduler to identify resource-requests explicitly by allocation-id | Major | resourcemanager | Subru Krishnan | Subru Krishnan | | | Fix javadoc warnings by JDK8 in hadoop-nfs package | Major | . | Kai Sasaki | Kai Sasaki | | | Differentiate exactly match with regex in yarn log CLI | Major | . | Xuan Gong | Xuan Gong | | | Remove bind-mount /etc/passwd for Docker containers | Major | yarn | Zhankun Tang | Zhankun Tang | | | DiskBalancer: query command should report Plan file path apart from PlanID | Minor | diskbalancer | Manoj Govindassamy | Manoj Govindassamy | | | Make DiskChecker pluggable in NodeManager | Major | nodemanager | Ray Chiang | Yufei Gu | | | Benchmark framework and test for erasure coders | Major | io | Kai Zheng | Rui Li | | | Erasure Coding: revisit buffer used for encoding and decoding. | Major | . | Yi Liu | Sammi Chen | | | Document the caller context config keys | Minor | ipc, namenode | Mingliang Liu | Mingliang Liu | | | Documenting NNThroughputBenchmark tool | Major | benchmarks, test | Mingliang Liu | Mingliang Liu | | | TestBlockManager#testBlockReportQueueing fails intermittently | Major | namenode, test | Mingliang Liu | Daryn Sharp | | | Clarify DecommissionType.FORCEFUL comment | Minor | documentation | Robert Kanter | Vrushali C | | | s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints) | Major | fs/s3 | Steve Loughran | Steve Loughran | | | S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | doc for

indicates incorrect values | Minor | fs/s3 | Shen Yinjie | Shen Yinjie | | | Automatic and Asynchronous Decommissioning Nodes Status Tracking | Major | resourcemanager | Daniel Zhi | Daniel Zhi | | | Refactor jni\_common to conform to C89 restrictions imposed by Visual Studio 2010 | Major | io | Sammi Chen | Sammi Chen | | | Optimize FSPermissionChecker group membership check | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Tune S3A provider plugin mechanism | Minor | fs/s3 | Steve Loughran | Steve Loughran | | | Support running isolated unit tests separate from AWS integration tests. | Major | fs/s3 | Chris Nauroth | Chris Nauroth | | | Refactor DistributedScheduling framework to pull out common functionality | Major | resourcemanager | Arun Suresh | Arun Suresh | | | Pass IIP for file status related methods | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Refactor DFSInputStream#ReaderStrategy | Major | erasure-coding | Kai Zheng | Sammi Chen | | | Reduce byte/string conversions for get listing | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Mount /sys/fs/cgroup into Docker containers as read only mount | Major | yarn | Varun Vasudev | luhuichun | | | Fix typo in RM\SCHEDULER\RESERVATION\THRESHOLD\INCREMENT\_MULTIPLE | Trivial | fairscheduler | Ray Chiang | Ray Chiang | | | Application moveToQueue should check NodeLabel permission | Major | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | | | Add localize API to the ContainerManagementProtocol | Major | . | Jian He | Jian He | | | Optimize mkdir ops | Major | hdfs | Daryn Sharp | Daryn Sharp | | | DiskBalancer does not execute multi-steps plan. | Critical | diskbalancer | Lei (Eddy) Xu | Lei (Eddy) Xu | | | API changes required to support recurring reservations in the YARN ReservationSystem | Major | resourcemanager | Subru Krishnan | Sangeetha Abdu Jyothi | | | Changes in AMRMClient for identifying resource-requests explicitly | Major | resourcemanager | Subru Krishnan | Arun Suresh | | | Fix path related byte array conversion bugs | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Optimize UTF8 string/byte conversions | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Optimize FSPermissionChecker's internal path usage | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Internally optimize path component resolution | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Add REST API to retrieve scheduler activity | Major | capacity scheduler, resourcemanager | Sunil Govindan | Chen Ge | | | Directly resolve paths into INodesInPath | Major | hdfs | Daryn Sharp | Daryn Sharp | | | Fix javadoc warnings by JDK8 in hadoop-common package | Major | . | Kai Sasaki | Kai Sasaki | | | RPC Server signals backoff to clients when all request queues are full | Major | . | Ming Ma | Ming Ma | | | Allowed deferred sending of call responses | Major | ipc | Daryn Sharp | Daryn Sharp | | | Do surgical preemption based on reserved container in CapacityScheduler | Major | capacity scheduler | Eric Payne | Wangda Tan | | | Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource | Blocker | api | Wangda Tan | Wangda Tan | | | AM-RM protocol changes to support container resizing | Blocker | api | MENG DING | MENG DING | | | Add support for async edit logging | Major | namenode | Daryn Sharp | Daryn Sharp | | | Provide configuration support for upgrade domain | Major |

| Ming Ma | Ming Ma | | | Display upgrade domain information in fsck | Major | . | Ming Ma | Ming Ma | | | Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes | Minor | . | Chris Trezzo | Chris Trezzo | | | Allow Balancer to run faster | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | RM container allocation delayed incase of app submitted to Nodelabel partition | Major | scheduler | Bibin A Chundatt | Bibin A Chundatt | | | Avoid unnecessary getBlockCollection calls in BlockManager | Major | namenode | Jing Zhao | Jing Zhao | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Missing interface annotation on JournalSet | Trivial | namenode | Todd Lipcon | Brandon Li | | | Support cross-project Jenkins builds | Major | build | Tom White | Tom White | | | Move TestFifoScheduler to FIFO package | Major | scheduler, test | Rohith Sharma K S | Rohith Sharma K S | | | Reformat hadoop-annotations, o.a.h.classification.tools | Minor | . | Li Lu | Li Lu | | | Remove deprecation warnings from JobHistoryServer classes | Minor | jobhistoryserver | Ray Chiang | Ray Chiang | | | Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp | Major | capacityscheduler | Wangda Tan | Wangda Tan | | | Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager | Major | . | Mingliang Liu | Mingliang Liu | | | Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method | Major | . | Mingliang Liu | Mingliang Liu | | | Replace usage of deprecated NameNode.DEFAULT\_PORT in TestFileSystem | Major | . | Mingliang Liu | Mingliang Liu | | | Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials | Major | test | Mingliang Liu | Mingliang Liu | | | Use GenericTestUtils#setLogLevel to the logging level | Major | . | Mingliang Liu | Mingliang Liu | | | Undeprecate createNonRecursive() | Major | . | Ted Yu | Ted Yu | | | Clean up checkstyle warnings in hadoop-hdfs-client module | Major | . | Mingliang Liu | Mingliang Liu | | | Update src/site/markdown/releases to include old versions of Hadoop | Major | build, documentation | Allen Wittenauer | Allen Wittenauer | | | Make static fields in GenericTestUtils for assertExceptionContains() package-private and final | Minor | test | Mingliang Liu | Mingliang Liu | | | Fix findbugs warnings in FSDirSnapshotOp | Major | namenode | Mingliang Liu | Mingliang Liu | | | NPE in SaslRpcServer | Major | . | Sergey Shelukhin | Sergey Shelukhin | | | Fix checkstyle error in CapacityScheduler.java | Trivial | . | Rohith Sharma K S | Naganarasimha G R | | | Document YARN security model from the perspective of Application Developers | Major | site | Steve Loughran | Steve Loughran | | | Docs for WebHDFS still describe GETDELEGATIONTOKENS operation | Trivial | documentation | Wellington Chevreuil | Wellington Chevreuil | | | Avoid printing a stack trace when recovering an app after the RM restarts | Major | . | Siddharth Seth | Junping Du | | | Clean up some configuration related codes about legacy block reader | Minor | . | Kai Zheng | Youwei Wang | | | Code and comment mismatch in JavaKeyStoreProvider | Trivial | . | nijel |