Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.2.1.1-beta.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add user info in the YARN ClientToken | Blocker | . | Bikas Saha | Jason Lowe | | | yarn proto definitions should specify package as 'hadoop.yarn' | Blocker | . | Arun C Murthy | Binglin Chang | | | RpcRequestHeaderProto defines callId as uint32 while ipc.Client.CONNECTION\CONTEXT\CALL\_ID is signed (-3) | Blocker | . | Arun C Murthy | Arun C Murthy | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Support server advertised kerberos principals | Critical | ipc, security | Daryn Sharp | Daryn Sharp | | | Add MXBean methods to query NN's transaction information and JournalNode's journal status | Minor | . | Jing Zhao | Jing Zhao | | | Provide testing support for DFSClient to drop RPC responses | Major | . | Jing Zhao | Jing Zhao | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Inefficient comparison with the empty string. Use isEmpty() instead | Minor | conf, fs, fs/s3, ha, io, metrics, performance, record, security, util | Brandon Li | Brandon Li | | | Improve getSplits performance by using listLocatedStatus | Major | job submission | Hairong Kuang | Hairong Kuang | | | Add generic type parameter to RetryInvocationHandler | Minor | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Augment MockNM to use multiple cores | Minor | . | Bikas Saha | Karthik Kambatla | | | Local jobs all use same local working directory | Major | . | Sandy Ryza | Sandy Ryza | | | Make FSNameSystem#auditLoggers an unmodifiable list | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Add appendToFile command to "hdfs dfs" | Minor | tools | Arpit Agarwal | Arpit Agarwal | | | ClientId should have getMsb/getLsb methods | Minor | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | namenode webserver's page has a tooltip that is inconsistent with the datanode HTML link | Trivial | namenode | Joseph Lorenzini | Vivek Ganesan | | | move slf4j to version 1.7.5 | Minor | build | Steve Loughran | Kousuke Saruta | | | Make checknative shell command accessible on Windows. | Minor | bin | Chris Nauroth | Chris Nauroth | | | Expose a REST API for monitoring the fair scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | | | Upgrade Avro dependency to 1.7.4 | Major | . | Sandy Ryza | Sandy Ryza | | | Support Snappy codec on Windows. | Major | io | Chris Nauroth | Chris Nauroth | | | Support Kerberos HTTP SPNEGO authentication for non-SUN JDK | Major | security | Yu Gao | Yu Gao | | | Move the version info of zookeeper dependencies to hadoop-project/pom | Minor | build | Karthik Kambatla | Karthik Kambatla | | | Easy access to final parameters in Configuration | Major | conf | Jason Lowe | Jason Lowe | | | Turn warning message in RetryInvocationHandler to debug | Minor |

| Arpit Gupta | Arpit Gupta | | | Add more unit tests for retry cache to cover all AtMostOnce methods | Minor | . | Jing Zhao | Jing Zhao | | | TeraInputFormat unnecessarily defines its own FileSplit subclass | Minor | examples | Sandy Ryza | Sandy Ryza | | | Clean up YARN CLI app list to show only running apps. | Major | client | Tassapol Athiapinya | Xuan Gong | | | Add metrics and web UI for cluster version summary | Major | namenode | Todd Lipcon | Ravi Prakash | | | Allow multiple net interfaces to be used with HA namenode RPC server | Critical | . | Kihwal Lee | Kihwal Lee | | | Minor improvement to output header for $ yarn node -list | Minor | client | Tassapol Athiapinya | Akira Ajisaka | | | Improve help message for $ yarn logs | Major | client | Tassapol Athiapinya | Xuan Gong | | | Improve help message for $ yarn applications and $yarn node | Major | client | Tassapol Athiapinya | Xuan Gong | | | Add addIfService() to CompositeService | Minor | . | Karthik Kambatla | Karthik Kambatla | | | Enable multiple states to to be specified in Resource Manager apps REST call | Major | resourcemanager | Trevor Lorimer | Trevor Lorimer | | | Allow auxiliary services to listen for container starts and completions | Major | nodemanager | Sandy Ryza | Alejandro Abdelnur | | | HAServiceState should have a state for stopped services | Minor | ha | Karthik Kambatla | Karthik Kambatla | | | Add support whitelist for system users to Yarn container-executor.c | Major | nodemanager | Alejandro Abdelnur | Roman Shaposhnik | | | in order to avoid dependency divergence within Hadoop itself lets enable DependencyConvergence | Major | build | Roman Shaposhnik | Roman Shaposhnik | | | Include token tracking ids in jobconf | Major | job submission, security | Sandy Ryza | Karthik Kambatla | | | Reduce the number of byte array creations and copies in XDR data manipulation | Major | nfs | Tsz Wo Nicholas Sze | Haohui Mai | | | Clarify WebHDFS REST API that all JSON respsonses may contain additional properties | Minor | documentation, webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | FileInputFormat doesn't read files recursively in the input path dir | Major | mrv1, mrv2 | Ramgopal N | Devaraj K | | | MRAppMaster throws invalid transitions for JobImpl | Major | mr-am | Devaraj K | Devaraj K | | | Support building the JNI code against the IBM JVM | Major | build | Tian Hong Wang | Tian Hong Wang | | | Stale files left behind for failed jobs | Major | mrv2 | Ravi Prakash | Ravi Prakash | | | Reducer should not implicate map attempt if it has insufficient space to fetch map output | Major | mrv2 | Jason Lowe | Ashwin Shankar | | | chown and chgrp reject users and groups with spaces on platforms where spaces are otherwise acceptable | Major | fs | Chris Nauroth | Chris Nauroth | | | JobContext cache files api are broken | Blocker |

| Omkar Vinit Joshi | Omkar Vinit Joshi | | | RM should validate the release container list before actually releasing them | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | | | The thread of ContainerLaunch#call will fail without any signal if getLocalizedResources() is called when the container is not at LOCALIZED | Major | . | Zhijie Shen | Zhijie Shen | | | For HdfsFileStatus, set default value of childrenNum to -1 instead of 0 to avoid confusing applications | Major | . | Brandon Li | Brandon Li | | | PortmapInterface should check if the procedure is out-of-range | Major | nfs | Brandon Li | Brandon Li | | | HistoryFileManager doesn't stop threads when service is stopped | Major | jobhistoryserver, mrv2 | Jason Lowe | Karthik Kambatla | | | Configuration#writeXml uses platform defaulting encoding, which may mishandle multi-byte characters. | Major | conf | Chris Nauroth | Chris Nauroth | | | DistributedShell throwing Errors in logs after successfull completion | Major | applications/distributed-shell | Abhishek Kapoor | Omkar Vinit Joshi | | | LeaseRenewer throw java.util.ConcurrentModificationException when timeout | Major | . | yunjiong zhao | yunjiong zhao | | | TestCopyCommitter Fails on JDK7 | Major | mrv2 | Robert Parker | Robert Parker | | | TestJobHistoryEvents and TestJobHistoryParsing have race conditions | Major | mrv2, test | Jason Lowe | Jason Lowe | | | Junit in TestJobHistoryServer failing in jdk 7 | Major | jobhistoryserver | Ashwin Shankar | Robert Parker | | | Supress logging of full stack trace of quota and lease exceptions | Major | namenode | Kihwal Lee | Robert Parker | | | Port HADOOP-9249 hadoop-maven-plugins Clover fix to branch-2 to fix build failures | Major | build | Dennis Y | Chris Nauroth | | | fsck can fail if a file is renamed or deleted | Major | . | Kihwal Lee | Robert Parker | | | Har metadata cache can grow without limit | Major | fs | Jason Lowe | Cristina L. Abad | | | Remove unused private RawLocalFileSystem#execCommand method from branch-2. | Trivial | fs | Chris Nauroth | Chris Nauroth | | | Tests block and sometimes timeout on Windows due to invalid entropy source. | Major | build, test | Chris Nauroth | Chris Nauroth | | | TestDFSIO fails intermittently on JDK7 | Major | test | Karthik Kambatla | Karthik Kambatla | | | Support for spnego keytab separate from the JournalNode keytab for secure HA | Minor | . | Jing Zhao | Jing Zhao | | | RM handles killed application tracking URL poorly | Major | resourcemanager | Jason Lowe | Jason Lowe | | | Document dfs cp -f option | Trivial | . | Keegan Witt | Keegan Witt | | | nn fails to download checkpointed image from snn in some setups | Blocker | namenode | Allen Wittenauer | Vinayakumar B | | |

fails to properly fallback to local rack | Minor | namenode | Eric Sirianni | Tsz Wo Nicholas Sze | | | globStatus using backslash for escaping does not work on Windows | Major | test | Chris Nauroth | Chuan Liu | | | BootstrapStandby not working with QJM when the existing NN is active | Major | ha, qjm | Jing Zhao | Jing Zhao | | | Server must not advertise kerberos realm | Blocker | ipc | Daryn Sharp | Daryn Sharp | | | Namenode#copyEditLogSegmentsToSharedDir should close EditLogInputStreams upon finishing | Major | namenode | Chuan Liu | Chuan Liu | | | TestNamenodeRetryCache fails on Windows due to incorrect cleanup | Minor | test | Chuan Liu | Chuan Liu | | | TestDirectoryScanner fails on Windows | Minor | test | Chuan Liu | Chuan Liu | | | Snapshot names should not be allowed to contain slash characters | Major | snapshots | Aaron T. Myers | Jing Zhao | | | SASL changes from HADOOP-9421 breaks Secure HA NN | Blocker | . | Kihwal Lee | Daryn Sharp | | | TestDatanodeBlockScanner fails on Windows due to incorrect path format | Minor | test | Chuan Liu | Chuan Liu | | | ClientRMService.forceKillApplication() should handle the non-RUNNING applications properly | Major | resourcemanager | Devaraj K | Xuan Gong | | | TestFsck fails on Windows | Minor | . | Chuan Liu | Chuan Liu | | | WHY appToken is removed both in BaseFinalTransition and AMUnregisteredTransition AND clientToken is removed in FinalTransition and not BaseFinalTransition | Major | . | Jian He | Xuan Gong | | | Nodes list web page on the RM web UI is broken | Major | . | Jian He | Xuan Gong | | | LocalJobRunner has race condition resulting in job failures | Major | . | Brock Noland | Sandy Ryza | | | Remove duplicated error message for snapshot commands when processing invalid arguments | Minor | snapshots | Jing Zhao | Jing Zhao | | | WebHDFS open sets Content-Length header to what is specified by length parameter rather than how much data is actually returned. | Minor | webhdfs | Arpit Gupta | Chris Nauroth | | | Priority#compareTo method seems to be wrong. | Major | . | Jian He | Jian He | | | Secure RM with recovery enabled and rm state store on hdfs fails with gss exception | Blocker | resourcemanager | Arpit Gupta | Vinod Kumar Vavilapalli | | | AM recovery does not work for map only jobs | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | | | DelegationTokenSecretManager#retrievePassword can cause deadlock in NameNode | Blocker | namenode | Deepesh Khandelwal | Daryn Sharp | | | Remove the debug message added by HADOOP-8855 | Minor | security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | LocalJobRunner does not work on Windows. | Major | . | Chris Nauroth | Sandy Ryza | | | RM restart throws Null pointer Exception in Secure Env | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | | | MiniYARNCluster with multiple nodemanagers, all nodes have same key for allocations | Major | nodemanager | Alejandro Abdelnur | Alejandro Abdelnur | | | Corrections to Fair Scheduler documentation | Major | documentation | Wing Yew Poon | | | | In Fair Scheduler documentation, inconsistency on which properties have prefix | Major | scheduler | Sandy Ryza | Akira Ajisaka | | | Deadlock in NameNode between SafeModeMonitor#run and DatanodeManager#handleHeartbeat | Blocker | namenode | Arpit Gupta | Kihwal Lee | | | ResourceManager should fail when

is set less than heartbeat interval | Major | resourcemanager | Yesha Vora | Zhijie Shen | | | NodeManager should mandatorily set some Environment variables into every containers that it launches | Major | . | Xuan Gong | Kenji Kikushima | | | HeartBeat thread in AMRMClientAsync does not handle runtime exception correctly | Major | . | Xuan Gong | Xuan Gong | | | proxy server start and stop documentation wrong | Minor | . | Andr Kelpe | | | | Move HAZKUtil to o.a.h.util.ZKUtil and make inner-classes public | Minor | ha | Karthik Kambatla | Karthik Kambatla | | | Active nodes can be decremented below 0 | Major | resourcemanager | Robert Parker | Robert Parker | | | revert MAPREDUCE-5357 | Major | distcp | Alejandro Abdelnur | Robert Kanter | | | RawLocalFileSystem.listStatus() return absolute paths when input path is relative on Windows | Major | fs | shanyu zhao | shanyu zhao | | | Too many safemode monitor threads being created in the standby namenode causing it to fail with out of memory error | Blocker | ha | Arpit Gupta | Jing Zhao | | | YARN/MR2/Job-history /logs link does not have correct content | Major | . | Xuan Gong | Jian He | | | Make ApplicationConstants.Environment.USER definition OS neutral | Minor | . | Chuan Liu | Chuan Liu | | | TestContainerLaunch fails on Windows | Minor | . | Chuan Liu | Chuan Liu | | | Allow per NN SPN for internal SPNEGO. | Blocker | . | Kihwal Lee | Kihwal Lee | | | By default yarn application -list should display all the applications in a state other than FINISHED / FAILED | Blocker | . | Omkar Vinit Joshi | Xuan Gong | | | FileUtil.createJarWithClassPath() does not generate relative classpath correctly | Major | fs | shanyu zhao | shanyu zhao | | | Race condition in ipc.Client causes TestIPC timeout | Minor | . | Binglin Chang | Binglin Chang | | | Improper synchronization in RetryCache | Blocker | . | Kihwal Lee | Kihwal Lee | | | MRClientService does not verify ACLs properly | Blocker | mr-am, mrv2 | Jason Lowe | Jason Lowe | | | Secondary NameNode fails to checkpoint if error occurs downloading edits on first checkpoint | Major | namenode | Aaron T. Myers | Aaron T. Myers | | | Capacity Scheduler tries to reserve the memory more than what node manager reports. | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | | | Job submitted with Delegation token in secured environment causes RM to fail during RM restart | Blocker | resourcemanager | Arpit Gupta | Omkar Vinit Joshi | | | TestTaskAttempt fails jdk7 with NullPointerException | Major | test | Nemon Lou | Nemon Lou | | | Unmanaged AMs registering a tracking URI should not be proxy-fied | Critical | resourcemanager | Alejandro Abdelnur | Alejandro Abdelnur | | | Invalid key to HMAC computation error when getting application report for completed app attempt | Blocker | resourcemanager | Jason Lowe | Jason Lowe | | | Compile failure with JDK8 | Major | client | Trevor Robinson | Trevor Robinson | | | ResourceManager and NodeManager do not load native libraries on Windows. | Major | nodemanager, resourcemanager | Chris Nauroth | Chris Nauroth | | | '5s sleep' in MRAppMaster.shutDownJob is only needed before stopping ClientService | Major |

| Jian He | Jian He | | | Audit logging of delegation tokens for MR tracing | Major | namenode, security | Andrew Wang | Andrew Wang | | | RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes | Critical | resourcemanager | Thomas Graves | Jonathan Eagles | | | command "mapred job" and "mapred queue" omit HADOOP\CLIENT\OPTS | Major | . | Nemon Lou | Nemon Lou | | | TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows | Minor | . | Chuan Liu | Chuan Liu | | | Add old constructor back to DelegationTokenInformation to unbreak downstream builds | Major | . | Andrew Wang | Andrew Wang | | | NameNode may fail to start when dfs.client.test.drop.namenode.response.number is set | Minor | . | Jing Zhao | Jing Zhao | | | TestContainerLogsPage fails with native builds | Minor | nodemanager | Roman Shaposhnik | Roman Shaposhnik | | | Upgrade Jersey version to 1.9 | Blocker | . | Brock Noland | Karthik Kambatla | | | NMTokenSecretManagerInNM is not being told when applications have finished | Blocker | . | Jason Lowe | Omkar Vinit Joshi | | | hadoop-client excludes commons-httpclient | Major | build | Lohit Vijayarenu | Lohit Vijayarenu | | | versions of a few transitive dependencies diverged between hadoop subprojects | Minor | build | Roman Shaposhnik | Roman Shaposhnik | | | In-memory map outputs can be leaked after shuffle completes | Blocker | mrv2 | Jason Lowe | Jason Lowe | | | Add configuration keys for retry policy in WebHDFSFileSystem | Major | webhdfs | Haohui Mai | Haohui Mai | | | Fix broken links in the document of HDFS Federation | Minor | . | Haohui Mai | Haohui Mai | | | Hadoop services won't start with different keypass and keystorepass when https is enabled | Major | security | Yesha Vora | Chris Nauroth | | | If lease soft limit is recovered successfully the append can fail | Major | . | Todd Lipcon | Tao Luo | | | NPE in FSNamesystem.commitBlockSynchronization() | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Use enum for nfs constants | Minor | nfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | RM crash with NPE on NODE\_REMOVED event with FairScheduler | Major | . | Lohit Vijayarenu | Mayank Bansal | | | Shared data structures in Public Localizer and Private Localizer are not Thread safe. | Critical | . | Omkar Vinit Joshi | Omkar Vinit Joshi | | | Cancelling ContainerLaunch#call at KILLING causes that the container cannot be completed | Major |

| Zhijie Shen | Zhijie Shen | | | Change hdfs-nfs parent project to hadoop-project | Major | nfs | Kihwal Lee | Brandon Li | | | Add script changes/utility for starting NFS gateway | Major | nfs | Brandon Li | Brandon Li | | | In map-side sort, swap entire meta entries instead of indexes for better cache performance | Major | performance, task | Sandy Ryza | Sandy Ryza | | | Support dotdot name in NFS LOOKUP operation | Major | nfs | Brandon Li | Brandon Li | | | Fix array copy error in Readdir and Readdirplus responses | Major | nfs | Brandon Li | Brandon Li | | | Change FSDataOutputStream to HdfsDataOutputStream for opened streams to fix type cast error | Major | nfs | Brandon Li | Brandon Li | | | Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for NFS deployment | Major | nfs | Brandon Li | Brandon Li | | | Historyserver does not refresh the result of restarted jobs after RM restart | Blocker | . | Yesha Vora | Jian He | | | Job can fail when RM restarts after staging dir is cleaned but before MR successfully unregister with RM | Blocker | . | Jian He | Jian He | | | Add NFS server export table to control export by hostname or IP range | Major | nfs | Brandon Li | Jing Zhao | | | Yarn and MRv2 should do HTTP client authentication in kerberos setup. | Blocker | nodemanager, resourcemanager | Jaimin Jetly | Omkar Vinit Joshi | | | Support file append in NFSv3 gateway to enable data streaming to HDFS | Major | nfs | Brandon Li | Brandon Li | | | JobClient exit whenever RM issue Reboot command to 1st attempt App Master. | Major | applicationmaster, client | Rohith Sharma K S | Jian He | | | AMRMClient support for resource blacklisting | Major | . | Bikas Saha | Junping Du | | | MNT EXPORT should give the full group list which can mount the exports | Major | nfs | Brandon Li | Brandon Li | | | ContainerExistStatus should define a status for preempted containers | Blocker | api | Alejandro Abdelnur | Alejandro Abdelnur | | | ResourceManager throws ArrayIndexOutOfBoundsException while handling CONTAINER\_ALLOCATED for application attempt | Major | resourcemanager | Devaraj K | Zhijie Shen | | | Refactor o.a.h.nfs to support different types of authentications | Major | nfs | Brandon Li | Jing Zhao | | | Support symlink operations | Major | nfs | Brandon Li | Brandon Li | | | Add more debug trace for NFS READ and WRITE | Trivial | nfs | Brandon Li | Brandon Li | | | Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts | Major | resourcemanager | Jian He | Jian He | | | Race condition causing RM to potentially relaunch already unregistered AMs on RM restart | Major | resourcemanager | Jian He | Jian He | | | ClassCastException is thrown during preemption When a huge job is submitted to a queue B whose resources is used by a job in queueA | Major | capacityscheduler, resourcemanager | J.Andreina | Chris Douglas | | | Refactor RpcMessage and NFS3Response to support different types of authentication information | Major | nfs | Jing Zhao | Jing Zhao | | | Move RpcFrameDecoder out of the public API | Minor | nfs | Haohui Mai | Haohui Mai | |