Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.2.6.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Shell command fs -count should include human readable option | Trivial | . | Jonathan Allen | Jonathan Allen | | | TokenIdentifier serialization should consider Unknown fields | Blocker | . | Siddharth Seth | Junping Du | | | YARN Admission Control/Planner: enhancing the resource allocation model with time. | Major | capacityscheduler, resourcemanager, scheduler | Carlo Curino | Carlo Curino | | | ClientToAMTokenIdentifier and DelegationTokenIdentifier should allow extended fields | Blocker | . | Junping Du | Junping Du | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Create an API to Separate Credentials/Password Storage from Applications | Major | security | Larry McCay | Larry McCay | | | Add generateEncryptedKey and decryptEncryptedKey methods to KeyProvider | Major | security | Alejandro Abdelnur | Arun Suresh | | | Support for encrypting Intermediate data and spills in local filesystem | Major | security | Alejandro Abdelnur | Arun Suresh | | | Add a way to format the RMStateStore | Major | resourcemanager | Karthik Kambatla | Robert Kanter | | | Add support to distcp to preserve raw.\* namespace extended attributes | Major | distcp | Charles Lamb | Charles Lamb | | | Hadoop cryptographic file system | Major | security | Yi Liu | Yi Liu | | | Transparent data at rest encryption | Major | security | Alejandro Abdelnur | Charles Lamb | | | isolated classloader on the client side | Major | util | Sangjin Lee | Sangjin Lee | | | FairScheduler: Add the notion of steady fair share | Major | fairscheduler | Ashwin Shankar | Wei Yan | | | FairScheduler: Preemption timeout should be configurable per queue | Major | fairscheduler | Ashwin Shankar | Wei Yan | | | inotify in HDFS | Major | hdfs-client, namenode, qjm | James Thomas | James Thomas | | | The ReplaceDatanodeOnFailure policies could have a forgiving option | Minor | hdfs-client | Harsh J | Tsz Wo Nicholas Sze | | | FairScheduler: Configure fairSharePreemptionThreshold per queue | Major | fairscheduler | Ashwin Shankar | Wei Yan | | | Make the HDFS home directory location customizable. | Minor | . | Kevin Odell | Yongjun Zhang | | | Capture aggregate memory allocation at the app-level for chargeback | Major | resourcemanager | Kendall Thrapp | Eric Payne | | | Incorporate new S3A FileSystem implementation | Major | fs, fs/s3 | Jordan Mendelson | Jordan Mendelson | | | Support Archival Storage | Major | balancer & mover, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Write to single replica in memory | Major | datanode, hdfs-client, namenode | Arpit Agarwal | Arpit Agarwal | | | Create Docker analog of the LinuxContainerExecutor in YARN | Major | . | Arun C Murthy | Abin Shahab | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the

| Minor | conf | Ravi Prakash | Ravi Prakash | | | Make DFSOuputSteram$Packet default constructor reuse the other constructor | Trivial | hdfs-client | Jing Zhao | Jing Zhao | | | RandomDatum overrides equals(Object) but no hashCode() | Minor | test | Brandon Li | Brandon Li | | | Refactor SSLFactory to expose static method to determine HostnameVerifier | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | KeyProvider implementations should be thread safe | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | KeyStores should have methods to generate the materials themselves, KeyShell should use them | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | JavaKeyStoreProvider should accept keystore password via configuration falling back to ENV VAR | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Change visibility of KeyStore.Options getter methods to public | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | KeyProvider Metadata should have an optional description, there should be a method to retrieve the metadata from all keys | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Key Management Server based on KeyProvider API | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Add optional attributes to KeyProvider Options and Metadata | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | KMSClientProvider should respect a configurable timeout. | Major | . | Andrew Wang | Mike Yoder | | | Improve logging in caching classes | Minor | caching | Andrew Wang | Andrew Wang | | | BlockManager#computeInvalidateWork() could do nothing | Minor | . | Juan Yu | Juan Yu | | | KeyProvider KeyVersion should provide the key name | Major | security | Alejandro Abdelnur | Arun Suresh | | | Create KeyProvider extension to handle delegation tokens | Major | security | Alejandro Abdelnur | Arun Suresh | | | Rename DataNode#checkWriteAccess to checkReadAccess. | Major | datanode | Liang Xie | Liang Xie | | | Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Delegate KeyProviderExtension#toString to underlying KeyProvider | Trivial | . | Andrew Wang | Andrew Wang | | | Remove unused native code for munlock. | Minor | native | Chris Nauroth | Chris Nauroth | | | Implement Windows equivalent of mlock. | Major | native | Chris Nauroth | Chris Nauroth | | | Support Centralized Cache Management on Windows. | Major | datanode | Colin P. McCabe | Chris Nauroth | | | Add key attributes to the key shell | Major | security | Mike Yoder | Mike Yoder | | | FairScheduler: Add debug information about cluster capacity, availability and reservations | Trivial | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Fix block protocol so that Datanodes don't require root or jsvc | Major | datanode, security | Owen O'Malley | Chris Nauroth | | | Add common tests for ACLs in combination with viewfs. | Major | fs, test | Chris Nauroth | Stephen Chu | | | Refactor KMSACLs to avoid locking | Major | security | Benoy Antony | Benoy Antony | | | Add unregisterSource() to MetricsSystem API | Major | metrics | shanyu zhao | shanyu zhao | | | Move the default options for distcp -p to DistCpOptionSwitch | Trivial | distcp | Charles Lamb | Charles Lamb | | | Deduplicate xattr names in memory | Major | namenode | Andrew Wang | Andrew Wang | | | Upgrade S3n

to support multi directories | Minor | fs/s3 | Theodore michael Malaska | Theodore michael Malaska | | | Add 'header banner' to 'explorer.html' also in Namenode UI | Major | . | Vinayakumar B | Vinayakumar B | | | EncryptedKeyVersion should have a key name property | Major | security | Alejandro Abdelnur | Arun Suresh | | | Add a new "-skipSharedEditsCheck" option for BootstrapStandby | Minor | ha, namenode | Liang Xie | Rakesh R | | | Add a new option to NN upgrade to terminate the process after upgrade on NN is completed | Major | namenode | Danilo Vunjak | Danilo Vunjak | | | BlockPlacementPolicy shoud choose storage but not datanode for deletion | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | ProxyUsers configuration should support configurable prefixes | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | KMSKeyProviderCache should be in hadoop-common | Major | security | Alejandro Abdelnur | Arun Suresh | | | FairShareComparator creates too many Resource objects | Minor | fairscheduler | Hong Zhiguo | Hong Zhiguo | | | KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API | Major | security | Alejandro Abdelnur | Arun Suresh | | | Support negative caching of user-group mapping | Major | security | Andrew Wang | Lei (Eddy) Xu | | | Iteration on KeyProviderFactory.serviceLoader is thread-unsafe | Major | security | Benoy Antony | Benoy Antony | | | Make seed optional in NetworkTopology#sortByDistance | Major | namenode | Ashwin Shankar | Ashwin Shankar | | | Allow Text to be read with a known length | Minor | io | Todd Lipcon | Todd Lipcon | | | Clarify usage of encryption and encrypted encryption key in KeyProviderCryptoExtension | Major | . | Andrew Wang | Andrew Wang | | | Add EncryptedKeyVersion factory method to KeyProviderCryptoExtension | Major | . | Andrew Wang | Andrew Wang | | | FairScheduler: preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness | Major | scheduler | Ashwin Shankar | Ashwin Shankar | | | There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode | Major |

| Mit Desai | Mit Desai | | | Add getDatanodeStorageReport to ClientProtocol | Major | hdfs-client, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Enable TCP\_NODELAY by default for IPC | Major | ipc | Todd Lipcon | Todd Lipcon | | | KMS audit log should consolidate successful similar requests | Major | security | Alejandro Abdelnur | Arun Suresh | | | add api that enables checking if a user has certain permissions on a file | Major | hdfs-client, namenode, webhdfs | Thejas M Nair | Jitendra Nath Pandey | | | Add ability to exclude/include specific datanodes while balancing | Major | balancer & mover | Benoy Antony | Benoy Antony | | | Expose YARN/MR endpoints on multiple interfaces | Major | nodemanager, resourcemanager, webapp | Arpit Agarwal | Craig Welch | | | Balancer should preserve storage type of replicas | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add test case for incorrect data node condition during balancing | Major | balancer & mover | Benoy Antony | Benoy Antony | | | Improving the argument check during balancer command line parsing | Minor | balancer & mover | Benoy Antony | Benoy Antony | | | Deletion of directories with snapshots will not output reason for trash move failure | Minor | . | Stephen Chu | Stephen Chu | | | Update BlockManager methods to use DatanodeStorageInfo where possible | Minor | namenode | Arpit Agarwal | Arpit Agarwal | | | KeyShell args should use single-dash style | Major | security | Mike Yoder | Andrew Wang | | | Use block ID-based block layout on datanodes | Major | datanode | James Thomas | James Thomas | | | Improve error message on token expire exception | Trivial | . | Li Lu | Li Lu | | | Enhance hadoop classpath command to expand wildcards or write classpath into jar manifest. | Major | scripts, util | Chris Nauroth | Chris Nauroth | | | Change default KeyProvider bitlength to 128 | Major | . | Andrew Wang | Andrew Wang | | | AuthenticationFilter should support externalizing the secret for signing and provide rotation support | Major | security | Alejandro Abdelnur | Robert Kanter | | | Move some Balancer's inner classes to standalone classes | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Remove addBlock and replaceBlock from DatanodeDescriptor | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Separate HDFS commands from CommandsManual.apt.vm | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Refactor HTTP delegation support out of httpfs to common | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Get DN storages out of blockContentsStale state faster after NN restarts | Major | . | Ming Ma | Ming Ma | | | FairScheduler: Collect metrics on duration of critical methods that affect performance | Major | scheduler | Karthik Kambatla | Karthik Kambatla | | | Porting libhdfs to Windows | Major | libhdfs | Ziliang Guo | Chris Nauroth | | | Separate block replica dispatching from Balancer | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add waitFor to AMRMClient(Async) | Major | client | Zhijie Shen | Tsuyoshi Ozawa | | | ResourceManager sets ClientRMService in RMContext multiple times | Trivial | resourcemanager | zhihai xu | zhihai xu | | | RMAppAttempt state machine entries for KILLED state has duplicate event entries | Trivial | resourcemanager | zhihai xu | zhihai xu | | | Code cleanup for Balancer and Dispatcher | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Implement HTTP proxyuser support in HTTP authentication client/server libraries | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | HDFS INFO logging is verbose & uses file appenders | Major | datanode | Gopal V | Nathan Yao | | | FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | Separate mapred commands from

| Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Replace HttpFS custom proxyuser handling with common implementation | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Byte array native checksumming | Major | performance | James Thomas | James Thomas | | | harchive: Document how to unarchive | Minor | documentation, harchive | Koji Noguchi | Akira Ajisaka | | | Javadoc for KeyValueTextInputFormat should include default separator and how to change it | Minor | documentation | Matt Lavin | Akira Ajisaka | | | Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent" | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Add some components in Native Libraries document | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Add a link to YARN CHANGES.txt in the left side of doc | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Typo in description and error message for 'yarn.resourcemanager.cluster-id' | Trivial | . | Devaraj K | Anandha L Ranganathan | | | Move NFS out of order write unit tests into TestWrites class | Minor | nfs | Zhe Zhang | Zhe Zhang | | | KMS add delegation token support | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Improve DefaultCryptoExtension#generateEncryptedKey performance | Major | security | Yi Liu | Yi Liu | | | KMS, add proxyuser support | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | An ip whilelist based implementation to resolve Sasl properties per connection | Major | . | Benoy Antony | Benoy Antony | | | [Capacity Scheduler] support simple user and group mappings to queues | Major | capacityscheduler | Ram Venkatesh | Ram Venkatesh | | | java.net.SocketTimeoutException in Fetcher caused jobs stuck for more than 1 hour | Critical | mr-am, task | yunjiong zhao | yunjiong zhao | | | org.apache.hadoop.util.DataChecksum should support native checksum calculation | Major | performance | James Thomas | James Thomas | | | An ip whitelist based implementation of TrustedChannelResolver | Major | security | Benoy Antony | Benoy Antony | | | Allow dfs.data.transfer.saslproperties.resolver.class default to hadoop.security.saslproperties.resolver.class | Minor | security | Benoy Antony | Benoy Antony | | | block writer should pass the expected block size to DataXceiverServer | Major | datanode, hdfs-client | Arpit Agarwal | Arpit Agarwal | | | Javadoc points to Wrong Reader and Writer classes in SequenceFile | Trivial | documentation, io | Timothy Mann | Ray Chiang | | | Fix bash tab completion code to work | Trivial | scripts | Jim Hester | Jim Hester | | | Allow changing MiniDFSCluster volumes per DN and capacity per volume | Major | datanode, test | Arpit Agarwal | Arpit Agarwal | | | Add missing job config options to mapred-default.xml | Major | documentation | Sandy Ryza | Ray Chiang | | | MiniDFSCluster should skip edit log fsync by default | Major | namenode | Daryn Sharp | Stephen Chu | | | Add log category for long-running DFSClient notices | Minor | . | Todd Lipcon | Zhe Zhang | | | Use native CRC32 in DFS write path | Major | datanode, hdfs-client, performance | Todd Lipcon | James Thomas | | | Support Datanode layout changes with rolling upgrade | Major | datanode | Colin P. McCabe | James Thomas | | | Configurable replication factor in the hadoop archive command | Minor | . | Zhe Zhang | Zhe Zhang | | | Define a variable jackson.version instead of using constant at multiple places | Minor |

| Juan Yu | Juan Yu | | | Remove unused cache in UserProvider | Major | security | Benoy Antony | Benoy Antony | | | Add missed NFSv3 request and response classes | Major | nfs | Brandon Li | Brandon Li | | | KMS should have a blacklist for decrypting EEKs | Major | security | Alejandro Abdelnur | Arun Suresh | | | Add a KeyProvider instantiation based on a URI | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Http server/client utils to propagate and recreate Exceptions from server to client | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | | | Use single editlog record for creating file + overwrite. | Critical | namenode | Yi Liu | Yi Liu | | | JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs | Trivial | client | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Improve NN allocateBlock log to include replicas' datanode IPs | Minor | namenode | Ming Ma | Ming Ma | | | BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid | Major | hdfs-client | Colin P. McCabe | Colin P. McCabe | | | checknative command to probe for winutils.exe on windows | Minor | native | Steve Loughran | Xiaoyu Yao | | | RM should expose the resource types considered during scheduling when AMs register | Major | . | Varun Vasudev | Varun Vasudev | | | KMS: add ACLs on per key basis. | Major | security | Alejandro Abdelnur | Arun Suresh | | | yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented | Trivial | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing. | Minor | fs/s3, test | Hari | Hari | | | create tools/hadoop-amazon for aws/EMR support | Major | fs/s3 | Steve Loughran | Steve Loughran | | | HAadmin transtionToActive with forceActive option can show confusing message. | Minor | . | Rushabh S Shah | Rushabh S Shah | | | Add server-side encryption functionality to s3a | Major | fs/s3 | David S. Wang | David S. Wang | | | CGroups - Admins should be allowed to enforce strict cpu limits | Major | . | Varun Vasudev | Varun Vasudev | | | User documentation for CredentialShell | Major | . | Andrew Wang | Larry McCay | | | KMS should support signing cookies with zookeeper secret manager | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Document considerations of HAR and Encryption | Minor | documentation | Andrew Wang | Charles Lamb | | | mapreduce.job.credentials.binary should allow all URIs | Major | security | zhihai xu | zhihai xu | | | KMS: Support for multiple Kerberos principals | Major | . | Andrew Wang | Alejandro Abdelnur | | | Cleanup KMS configuration keys | Major | . | Andrew Wang | Andrew Wang | | | FairScheduler: Set the default value for maxAMShare to 0.5 | Minor | . | Wei Yan | Wei Yan | | | MiniKDC to use locale EN\_US for case conversions | Minor | tools | Steve Loughran | Steve Loughran | | | KMS delegation token secret manager should be able to use zookeeper as store | Major | security | Alejandro Abdelnur | Arun Suresh | | | Add Timestamp Preservation to DistCp | Major | tools/distcp | Gary Steelman | Gary Steelman | | | How about inputstream close statement from catch block to finally block in FileContext#copy() ? | Minor |

| skrho | skrho | | | Clarify ACL delimiter and how to configure ACL groups only | Trivial | documentation, fairscheduler | Miklos Christine | Miklos Christine | | | Adding site documents of hadoop-tools | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | There are Chinese Characters in the FairScheduler's document | Minor | documentation | Fengdong Yu | Fengdong Yu | | | Remove @date JavaDoc comment in ProgramDriver class | Trivial | documentation | Henry Saputra | Henry Saputra | | | Add config key to restrict setStoragePolicy | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | CapacityScheduler: Improve reservations | Major | capacityscheduler | Thomas Graves | Thomas Graves | | | Remove INSTALL document | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Document oiv\_legacy command | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | Make number of KMS threads configurable | Major | kms | Andrew Wang | Andrew Wang | | | Add missing version subcommand for hdfs | Minor | scripts | Allen Wittenauer | Sasaki Toru | | | Add logs when attemptFailuresValidityInterval is enabled | Major | . | Xuan Gong | Xuan Gong | | | Add storagePolicy to NN edit log during file creation | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Reduce the memory usage of WebImageViewer | Major | . | Haohui Mai | Haohui Mai | | | Reinstate building of ant tasks support | Major | build, fs | Jason Lowe | Jason Lowe | | | Decommission slows way down when it gets towards the end | Major | namenode | Ming Ma | Ming Ma | | | Better batching of IBRs | Major | . | Kihwal Lee | Kihwal Lee | | | Update user doc of secure mode about Datanodes don't require root or jsvc | Major | documentation, security | Yi Liu | Chris Nauroth | | | Update Hadoop's lz4 to r123 | Major | native | Colin P. McCabe | Colin P. McCabe | | | Localization exception stack traces are not passed as diagnostic info | Major | nodemanager | Gera Shegalov | Gera Shegalov | | | Add an SSD policy into the default BlockStoragePolicySuite | Major | . | Jing Zhao | Jing Zhao | | | Provide a boolean switch to enable MR-AM profiling | Minor | applicationmaster, client | Gera Shegalov | Gera Shegalov | | | Make DFSOutputStream.MAX\_PACKETS configurable | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add JvmPauseMonitor to NFS gateway | Minor | nfs | Brandon Li | Brandon Li | | | Remove the need to run NodeManager as privileged account for Windows Secure Container Executor | Major | . | Remus Rusanu | Remus Rusanu | | | Replace AM resync/shutdown command with corresponding exceptions | Major | . | Jian He | Jian He | | | Optimize HDFS Encrypted Transport performance | Major | datanode, hdfs-client, security | Yi Liu | Yi Liu | | | Move Id-Name mapping in NFS to the hadoop-common area for better maintenance | Major | nfs, security | Yongjun Zhang | Yongjun Zhang | | | Match hadoop.auth cookie format to jetty output | Major | security | Gregory Chanan | Gregory Chanan | | | Improve Openssl library finding | Major | security | Yi Liu | Colin P. McCabe | | | Add rolling downgrade documentation | Major | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Support optional configuration of AES cipher suite on

| Major | datanode, hdfs-client, security | Chris Nauroth | Chris Nauroth | | | Limit the number of byte arrays used by DFSOutputStream | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException | Major | namenode | Rushabh S Shah | Rushabh S Shah | | | add an SCM-ignored XML filename to keep secrets in (auth-keys.xml?) | Minor | build | Steve Loughran | | | | Update sample program and documentations for writing YARN Application | Major | . | Li Lu | | | | [Umbrella] Support rolling upgrades in YARN | Major | graceful, rolling upgrade | Siddharth Seth | | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Remove unnecessary method (tokenRefetchNeeded) in DFSClient | Trivial | hdfs-client | Uma Maheswara Rao G | Uma Maheswara Rao G | | | hdfs balancer throws ArrayIndexOutOfBoundsException if option is specified without arguments | Minor | balancer & mover | Stephen Chu | madhukara phatak | | | Interrupting hadoop fs -put from the command line causes a LeaseExpiredException | Major | . | Todd Lipcon | Daryn Sharp | | | Faulty sanity check in FsDirectory.unprotectedSetQuota | Trivial | namenode | Binglin Chang | Binglin Chang | | | FsShell's Text command does not read avro data files stored on HDFS | Major | fs | Allan Yan | Allan Yan | | | Create an API to separate encryption key storage from applications | Major | security | Owen O'Malley | Owen O'Malley | | | Create CLI tools for managing keys via the KeyProvider API | Major | security | Owen O'Malley | Larry McCay | | | TestKeyShell improperly tests the results of a Delete | Major | security | Larry McCay | Larry McCay | | | JavaKeyStoreProvider needs to set keystore permissions properly | Major | security | Larry McCay | Larry McCay | | | TestKeyProviderFactory fails randomly | Major | test | Alejandro Abdelnur | Alejandro Abdelnur | | | KeyProvider API should using windowing for retrieving metadata | Major | . | Owen O'Malley | Owen O'Malley | | | bin/hadoop key throws NPE with no args and assorted other fixups | Minor | bin | Charles Lamb | Charles Lamb | | | KeyShell doesn't allow setting Options via CLI | Minor | bin | Charles Lamb | Charles Lamb | | | TestKMS fails because race condition writing acl files | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | KMS, keyVersion name should not be assumed to be keyName@versionNumber | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Avoid negative elapsed time in JHS/MRAM web UI and services | Major | . | Zhijie Shen | Zhijie Shen | | | Unportable getgrouplist() usage breaks FreeBSD | Major | . | Dmitry Sivachenko | Dmitry Sivachenko | | | [ HDFS Rolling Upgrade - Shell ] shutdownDatanode and getDatanodeInfo usage is missed | Major | tools | Brahma Reddy Battula | Brahma Reddy Battula | | | FsShell setfacl can throw ArrayIndexOutOfBoundsException when no perm is specified | Minor | fs | Stephen Chu | sathish | | | Add preemption info to RM Web UI and add logs when preemption occurs | Major | resourcemanager, webapp | Wangda Tan | Wangda Tan | | | Fix code bug in GetApplicationsRequestPBImpl#mergeLocalToBuilder | Major | . | Binglin Chang | Binglin Chang | | | External links need to be removed from YARN UI | Major |

| Yesha Vora | Craig Welch | | | [ Web HDFS ] Syntax for MKDIRS, CREATESYMLINK, and SETXATTR are given wrongly(missed webhdfs/v1).). | Major | documentation, webhdfs | Brahma Reddy Battula | Stephen Chu | | | Unable to fetch the block information by Browsing the file system on Namenode UI through IE9 | Major | namenode | J.Andreina | Haohui Mai | | | hadoop\user\info\_alloc fails on FreeBSD due to incorrect sysconf use | Major | . | Dmitry Sivachenko | Dmitry Sivachenko | | | Clean up native code compilation warnings. | Minor | native | Chris Nauroth | Chris Nauroth | | | MiniDFSCluster may still be partially running after initialization fails. | Minor | test | Chris Nauroth | Chris Nauroth | | | BlockPoolSliceScanner and high speed hdfs appending make datanode to drop into infinite loop | Critical | datanode | ikweesung | Colin P. McCabe | | | NFS should throw error for invalid entry in dfs.nfs.exports.allowed.hosts | Major | nfs | Yesha Vora | Abhiraj Butala | | | Update rpc metrics when the call throws an exception | Major | . | Ming Ma | Ming Ma | | | daemon scripts should remove pid file on stop call after stop or process is found not running | Major | . | Vinayakumar B | Vinayakumar B | | | Race in DrainDispatcher can cause random test failures | Major | . | Siddharth Seth | Li Lu | | | NFS doesn't return correct lookup access for directories | Major | nfs | Yesha Vora | Brandon Li | | | KeyShell returns -1 on error to the shell, should be 1 | Major | security | Mike Yoder | Mike Yoder | | | AMs and NMs can get exceptions after recovery but before scheduler knowns apps and app-attempts | Major | resourcemanager | Ashwin Shankar | Jian He | | | RemoteException can't be retried properly for non-HA scenario | Major | . | Ming Ma | Ming Ma | | | Update without holding write lock in JavaKeyStoreProvider#innerSetCredential() | Minor | . | Ted Yu | Ted Yu | | | Potential null dereference in CredentialShell#promptForCredential() | Minor | . | Ted Yu | Ted Yu | | | Compression codecs must used pooled direct buffers or deallocate direct buffers when stream is closed | Major | . | Hari Shreedharan | Colin P. McCabe | | | TestDFSAdminWithHA fails on windows | Major | test, tools | Vinayakumar B | Vinayakumar B | | | In HDFS HA mode, Distcp/SLive with webhdfs on secure cluster fails with Client cannot authenticate via:[TOKEN, KERBEROS] error | Major | security | Jian He | Jing Zhao | | | CryptoExtension generateEncryptedKey method should receive the key name | Major | security | Alejandro Abdelnur | Arun Suresh | | | bestNode shouldn't always return the first DataNode | Minor | webhdfs | yunjiong zhao | yunjiong zhao | | | FairScheduler missing handling of containers for unknown application attempts | Critical | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used | Major | . | Sangjin Lee | Sangjin Lee | | | Native Libraries Guide doen't mention a dependency on openssl-development package | Major | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | CombineFileInputFormat.getSplits() including directories in its results | Major |

| Jason Dere | Jason Dere | | | NodeManager web UI can incorrectly report Pmem enforcement | Major | nodemanager | Leitao Guo | Leitao Guo | | | RawLocalFileSystem fails to read symlink targets via the stat command when the format of stat command uses non-curly quotes | Major | . | Yongjun Zhang | Yongjun Zhang | | | DFSClient should create blocks using StorageType | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Fix the command to launch JournalNode in HDFS-HA document | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | NPE in ContinuousScheduling thread when we lose a node | Major | fairscheduler, resourcemanager | Andy Skelton | Wei Yan | | | Run "hdfs zkfc-formatZK" on a server in a non-namenode will cause a null pointer exception. | Major | auto-failover, ha | WenJin Ma | Masatake Iwasaki | | | Livelock can occur in FairScheduler when there are lots of running apps | Major | fairscheduler | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Missing lock in JavaKeyStoreProvider.createCredentialEntry | Major | security | Benoy Antony | Benoy Antony | | | Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp\block\verification.log.curr | Critical | datanode | Vinayakumar B | Vinayakumar B | | | NFS: Exception should be added in NFS log for invalid separator in nfs.exports.allowed.hosts | Major | nfs | Yesha Vora | Abhiraj Butala | | | Add XAttrs to ViewFs and make XAttrs + ViewFileSystem internal dir behavior consistent | Major | fs, test | Stephen Chu | Stephen Chu | | | client lacks delegation token exception details when application submit fails | Minor | resourcemanager | Jason Lowe | Chen He | | | webhdfs wont fail over when it gets java.io.IOException: Namenode is in startup mode | Major | ha, webhdfs | Arpit Gupta | Jing Zhao | | | FileJournalManager doesn't purge empty and corrupt inprogress edits files | Major | namenode | Vinayakumar B | Vinayakumar B | | | container-executor shouldn't require o-r permissions | Minor | nodemanager | Aaron T. Myers | Aaron T. Myers | | | FSNamesystem methods should call resolvePath | Major | namenode | Charles Lamb | Charles Lamb | | | The extended attributes javadoc should simply refer to the user docs | Major | . | Charles Lamb | Charles Lamb | | | Using com.sun.org.apache.xml.internal.serialize.\* in XmlEditsVisitor.java is JVM vendor specific. Breaks IBM JAVA | Major | tools | Amir Sanjar | | | | Fix a few unit tests that use hard-coded port numbers | Major | test | Arpit Agarwal | Arpit Agarwal | | | MapReduce changes for exposing YARN/MR endpoints on multiple interfaces. | Major | . | Xuan Gong | Craig Welch | | | DataNode logs wrong layoutversion during upgrade | Major | datanode | Benoy Antony | Benoy Antony | | | CredentialShell args should use single-dash style | Minor | . | Andrew Wang | Andrew Wang | | | site plugin couldn't parse hadoop-kms index.apt.vm | Minor | documentation | Ted Yu | Akira Ajisaka | | | Some tests in TestDFSClientFailover are missing @Test annotation | Major | test | Akira Ajisaka | Akira Ajisaka | | | Improve synchronization in BPOfferService with read write lock | Major | datanode | Yongjun Zhang | Yongjun Zhang | | | Compilation fails in native link0 function on

| Blocker | native | Chris Nauroth | Chris Nauroth | | | StorageReport array is initialized with wrong size in DatanodeDescriptor#getStorageReports | Minor | namenode | Ted Yu | Tsz Wo Nicholas Sze | | | Fix comment in o.a.h.y.server.resourcemanager.schedulerAppSchedulingInfo | Trivial | resourcemanager | Wenwu Peng | Wenwu Peng | | | Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction | Major | namenode | Vinayakumar B | Vinayakumar B | | | DN fails to startup if one of the data dir is full | Critical | datanode | Vinayakumar B | Vinayakumar B | | | Remove duplicate code in FSDirectory#unprotectedConcat | Major | namenode | Yi Liu | Yi Liu | | | NFS should not return NFS3ERR\_IO for AccessControlException | Major | nfs | Brandon Li | Abhiraj Butala | | | Incorrect usage on 'hadoop credential list' | Trivial | security | Josh Elser | Josh Elser | | | Fix CredentialShell help behavior and error codes | Minor | security | Josh Elser | Josh Elser | | | Need to set version name correctly before decrypting EEK | Major | security | Arun Suresh | Arun Suresh | | | JMXJsonServlet fails when used within Tomcat | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | | | New task status field in task attempts table can lead to an empty web page | Major | . | Mit Desai | Mit Desai | | | Fix TestKeyProviderFactory testcases to use default 128 bit length keys | Major | . | Arun Suresh | Arun Suresh | | | DFSUtil Should Use configuration.getPassword for SSL passwords | Major | . | Larry McCay | Larry McCay | | | Remove hadoop-metrics2.properties from hdfs project | Major | . | Akira Ajisaka | Akira Ajisaka | | | YARN trunk build failing TestDistributedShell.testDSShell | Major | . | Varun Vasudev | Varun Vasudev | | | A block could remain under replicated if all of its replicas are on decommissioned nodes | Major | . | Ming Ma | Ming Ma | | | Application hangs when it fails to launch AM container | Critical | resourcemanager | zhihai xu | zhihai xu | | | MR AM should have working directory in LD\LIBRARY\PATH | Major | mr-am | Jason Lowe | Jason Lowe | | | Typo in Configuration.getPasswordFromCredentialProviders | Trivial | security | Larry McCay | Larry McCay | | | dfs.web.authentication.kerberos.principal shows up in logs for insecure HDFS | Minor | namenode | Allen Wittenauer | Allen Wittenauer | | | Miscellaneous trivial corrections to KMS classes | Major | security | Alejandro Abdelnur | Arun Suresh | | | JavaKeyStoreProvider has to protect against corrupting underlying store | Major | security | Larry McCay | Arun Suresh | | | Fair scheduler: Consider only active queues for computing fairshare | Major | scheduler | Ashwin Shankar | Ashwin Shankar | | | Configuration.getValByRegex does not substitute for variables | Major | . | Robert Kanter | Robert Kanter | | | TestAMRestart fails intermittently | Major | . | Jian He | Jian He | | | Cleanup notifyDone\* methods in RMStateStore | Major | . | Jian He | Varun Saxena | | | Code cleanup for unnecessary INode replacement | Minor | namenode | Jing Zhao | Jing Zhao | | | Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler) | Minor | nfs | Ted Yu | Abhiraj Butala | | | Throw an exception in GenericOptionsParser when passed an empty Path | Minor |

| Alex Holmes | zhihai xu | | | WebAppUtils Should Use configuration.getPassword for Accessing SSL Passwords | Major | . | Larry McCay | Larry McCay | | | BlockInfo.addStorage fails when DN changes the storage for a block replica | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | NetgroupCache does not remove group memberships | Major | security | Benoy Antony | Benoy Antony | | | Flags for posix\_fadvise are not valid in some architectures | Major | native | David Villegas | David Villegas | | | Normalize the order of public final in HdfsFileStatus | Major | . | Haohui Mai | Tassapol Athiapinya | | | Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer | Major | balancer & mover, datanode | Vinayakumar B | Vinayakumar B | | | Hadoop Common native compilation broken in windows | Blocker | native | Vinayakumar B | David Villegas | | | TestGridmixRecord unit tests failure on PowerPC | Major | test, tools | Jinghui Wang | Jinghui Wang | | | Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus | Minor | mrv1, mrv2 | Sandy Ryza | Akira Ajisaka | | | Typo in MergeManagerImpl.java | Trivial | . | Efe Gencer | Akira Ajisaka | | | Missing alternatives in javadocs for deprecated constructors in mapreduce.Job | Minor | client, documentation, job submission | Christopher Tubbs | Akira Ajisaka | | | HistoryServerFileSystemStateStore fails to update tokens | Major | jobhistoryserver | Jason Lowe | Jason Lowe | | | incorrect description in distcp2 document | Major | documentation | Yongjun Zhang | Akira Ajisaka | | | CompositeInputFormat javadoc is broken | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Fix dead link in InputFormat javadoc | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Fix javadoc spelling for HadoopArchives#writeTopLevelDirs | Trivial | documentation | Akira Ajisaka | Akira Ajisaka | | | RM and TS web interfaces sometimes return request is a replay error in secure mode | Critical | . | Varun Vasudev | Varun Vasudev | | | some standard JDK APIs are not part of system classes defaults | Major | mrv2 | Sangjin Lee | Sangjin Lee | | | Small fix for NetworkTopologyWithNodeGroup#sortByDistance | Minor | . | Yi Liu | Yi Liu | | | Unable to check mapreduce job status if submitted using a non-default namenode | Major | jobhistoryserver | Benjamin Zhitomirsky | Benjamin Zhitomirsky | | | Fix HDFS CacheReplicationMonitor rescan logic | Major | caching | Yi Liu | Yi Liu | | | RPC authentication and authorization metrics overflow to negative values on busy clusters | Minor | metrics | Jason Lowe | Tsuyoshi Ozawa | | | Native Libraries Guide contains format error | Minor | documentation | Peter Klavins | Peter Klavins | | | Edit log corruption due to delayed block removal | Major | namenode | Yongjun Zhang | Yongjun Zhang | | | TestJobEndNotifier fails intermittently in branch-2 | Major | . | Mit Desai | Chang Li | | | DBInputSplit creates invalid ranges on Oracle | Major |

| Julien Serdaru | Wei Yan | | | Native Libraries Guide contains mis-spelt build line | Major | documentation | Peter Klavins | Peter Klavins | | | OOB message can't be sent to the client when DataNode shuts down for upgrade | Major | datanode | Brandon Li | Brandon Li | | | Active to StandBy transition does not stop rmDispatcher that causes 1 AsyncDispatcher thread leak. | Critical | resourcemanager | Nishan Shetty | Rohith Sharma K S | | | hadoop native build fails to detect java\_libarch on ppc64le | Major | build | Dinar Valeev | | | | Fix TestOptionsParser | Major | security | Charles Lamb | Charles Lamb | | | portmap and nfs3 are documented as hadoop commands instead of hdfs | Major | documentation, nfs | Allen Wittenauer | Brandon Li | | | Description for yarn.nodemanager.localizer.cache.target-size-mb is incorrect | Minor | nodemanager | Jason Lowe | Chen He | | | Blocks and INodes could leak for Rename with overwrite flag | Major | namenode | Yi Liu | Yi Liu | | | Potential NPE in EmbeddedElectorService#stop | Minor | resourcemanager | Devaraj K | Tsuyoshi Ozawa | | | LCE should support non-cgroups, non-secure mode | Blocker | nodemanager | Allen Wittenauer | Allen Wittenauer | | | NFS readdirplus doesn't return dotdot attributes | Major | nfs | Brandon Li | Brandon Li | | | Work around buggy getgrouplist() implementations on Linux that return 0 on failure | Major | native | Chris Nauroth | Chris Nauroth | | | fs-encryption merge triggered release audit failures | Blocker | . | Allen Wittenauer | Charles Lamb | | | Fully qualified intermediate done directory will break per-user dir creation on Windows | Major | jobhistoryserver | Zhijie Shen | Zhijie Shen | | | DFSAdmin refreshSuperUserGroupsConfiguration failed in security cluster | Minor | tools | yunjiong zhao | yunjiong zhao | | | libhdfs documentation is out of date | Minor | . | Andrew Wang | Chris Nauroth | | | build/test/test.mapred.spill causes release audit warnings | Major | test | Jason Lowe | Chen He | | | incorrect snapshot directory diff generated by snapshot deletion | Critical | snapshots | Juan Yu | Juan Yu | | | FileWriter should be closed in finally block in BlockReceiver#receiveBlock() | Minor | . | Ted Yu | Tsuyoshi Ozawa | | | Move HTTP delegation tokens out of URL querystring to a header | Blocker | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Fix typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | | | Fix HTTP content type for ReconfigurationServlet | Minor | conf | Lei (Eddy) Xu | Lei (Eddy) Xu | | | NPE in FairSchedulerAppsBlock | Major | . | Maysam Yabandeh | Tsuyoshi Ozawa | | | Timelineserver returns invalid Delegation token in secure kerberos enabled cluster when hadoop.http.filter.initializers are not set | Critical | timelineserver | Karam Singh | Varun Vasudev | | | Fix typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | | | hadoop.auth cookie after HADOOP-10710 still not proper according to RFC2109 | Major | security | Gregory Chanan | | | | RM web services app submission doesn't pass secrets correctly | Major | . | Varun Vasudev | Varun Vasudev | | | Update Tomcat version used by HttpFS and KMS to latest 6.x version | Major | . | Alejandro Abdelnur | Robert Kanter | | | Validate SleepJob command line parameters | Minor | test | Gera Shegalov | Gera Shegalov | | | TestNodeManagerResync#testBlockNewContainerRequestsOnStartAndResync should have a test timeout | Major | . | Jason Lowe | Eric Payne | | | TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly | Major | . | Yongjun Zhang | Yongjun Zhang | | | Add build directory to

| Minor | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | hadoop fs -text of zero-length file causes EOFException | Major | fs | Eric Payne | Eric Payne | | | With crypto, no native lib systems are too verbose | Major | encryption | Allen Wittenauer | Charles Lamb | | | Fix typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | | | Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart). | Major | namenode | Uma Maheswara Rao G | Yi Liu | | | compile error on project "Apache Hadoop OpenStack support" | Minor | build, fs/swift | xukun | | | | In sortAndSpill of MapTask.java, size is calculated wrongly when bufend \< bufstart. | Major | mrv1, mrv2 | zhihai xu | zhihai xu | | | Lack of synchronization on access to datanodeUuid in DataStorage#format() | Minor | . | Ted Yu | Xiaoyu Yao | | | OsSecureRandom.setConf() might leak file descriptors. | Major | security | Yongjun Zhang | Yongjun Zhang | | | KMS cannot deploy on Windows, because class names are too long. | Blocker | . | Chris Nauroth | Chris Nauroth | | | SnapshotDiff report can hit IndexOutOfBoundsException when there are nested renamed directory/file | Major | snapshots | Jing Zhao | Jing Zhao | | | TestBlocksScheduledCounter#testBlocksScheduledCounter should shutdown cluster | Minor | test | Vinayakumar B | Vinayakumar B | | | Inconsistency between 'hdfs dfsadmin' and 'hdfs dfsadmin -help' | Minor | . | Akira Ajisaka | Xiaoyu Yao | | | Distcp data between two HA clusters requires another configuration | Major | datanode, federation, hdfs-client | Dave Marion | Dave Marion | | | hdfs.dll does not produce .pdb files | Minor | hdfs-client | Remus Rusanu | Chris Nauroth | | | Add missing timeout annotations to tests | Major | test | Arpit Agarwal | Xiaoyu Yao | | | warning message 'ssl.client.truststore.location has not been set' gets printed for hftp command | Major | . | Yesha Vora | Xiaoyu Yao | | | KMSClientProvider should use getAuthenticationMethod() to determine if in proxyuser mode or not | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | DN must reserve space for a full block when an RBW block is created | Major | datanode | Gopal V | Arpit Agarwal | | | DFS input streams do not timeout | Critical | hdfs-client | Daryn Sharp | Daryn Sharp | | | Fix DN upgrade with layout version change | Major | datanode | James Thomas | Arpit Agarwal | | | Newly moved block replica been invalidated and deleted in TestBalancer | Major | balancer & mover, test | Binglin Chang | Binglin Chang | | | SLS can deadlock when all the threads are taken by AMSimulators | Critical | scheduler-load-simulator | Wei Yan | Wei Yan | | | Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work | Major | . | Yongjun Zhang | Yongjun Zhang | | | NPE if hosts not specified in ProxyUsers | Major | security | Gregory Chanan | Gregory Chanan | | | Bug introduced in HADOOP-9374, which parses the -tokenCacheFile as binary file but set it to the configuration as JSON

| Major | security, util | Jinghui Wang | zhihai xu | | | RM crashes if App gets rejected for any reason and HA is enabled | Major | resourcemanager | Mayank Bansal | Mayank Bansal | | | HistoryServerFileSystemStateStore can create zero-length files | Major | jobhistoryserver | Jason Lowe | Jason Lowe | | | Cgroups should allow YARN containers to be limited to allocated cores | Major | . | Varun Vasudev | Varun Vasudev | | | FairScheduler: Zero weight can lead to livelock | Major | scheduler | qingwu.fu | zhihai xu | | | Hadoop Balancer prematurely exits iterations | Major | balancer & mover | Benjamin Bowman | Rafal Wojdyla | | | Fix NameNode deadlock when opening file under /.reserved path | Critical | namenode | Yi Liu | Yi Liu | | | FairScheduler: Potential integer overflow calculating totalMaxShare | Major | scheduler | zhihai xu | zhihai xu | | | Upgrade fails for Windows HA cluster due to file locks held during rename in JournalNode. | Blocker | journal-node | Chris Nauroth | Chris Nauroth | | | Excessive logging by org.apache.hadoop.util.Progress when value is NaN | Major | . | Mit Desai | Mit Desai | | | After refactoring of HTTP proxyuser to common, doAs param is case sensitive | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Fix ResourceManagerRest.apt.vm syntax error | Major | . | Jian He | Jian He | | | FileSystemRMStateStore#readFile/writeFile should close FSData(In\|Out)putStream in final block | Trivial | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | SharedFileDescriptorFactory should not allocate sparse files | Minor | caching | Gopal V | Colin P. McCabe | | | Eliminate old configuration parameter names from s3a | Minor | fs/s3 | David S. Wang | David S. Wang | | | S3AOutputStream not reading new config knobs for multipart configs | Major | fs/s3 | David S. Wang | David S. Wang | | | ExportSnapshot fails on kerberized cluster using s3a | Major | fs/s3 | David S. Wang | David S. Wang | | | NN continues to issue block locations for DNs with full disks | Major | namenode | Daryn Sharp | Rushabh S Shah | | | Add a parameter "attempt\Failures\Validity\_Interval" in DistributedShell | Major | applications/distributed-shell | Xuan Gong | Xuan Gong | | | The invalidate method in SimulatedFSDataset.java failed to remove (invalidate) blocks from the file system. | Minor | datanode, test | Megasthenis Asteris | Megasthenis Asteris | | | TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7 | Major | test | Rushabh S Shah | Akira Ajisaka | | | KMS: KeyAuthorizationKeyProvider should verify the keyversion belongs to the keyname on decrypt | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | kms docs say proxyusers, not proxyuser for config params | Trivial | documentation | Charles Lamb | Charles Lamb | | | CryptoCodec testcases requiring OpenSSL should be run only if -Pnative is used | Major | security, test | Alejandro Abdelnur | Arun Suresh | | | KMS return HTTP UNAUTHORIZED 401 on ACL failure | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | hadoop-fuse-dfs fails because it cannot find JavaKeyStoreProvider$Factory | Major | . | Colin P. McCabe | Colin

McCabe | | | Return value of read(ByteBuffer buf) in CryptoInputStream is incorrect in some cases | Major | security | Yi Liu | Yi Liu | | | ResourceManager sometime become un-responsive due to NPE in SystemMetricsPublisher | Major | resourcemanager, timelineserver | Karam Singh | Zhijie Shen | | | Submitted applications occasionally lack a tracking URL | Major | resourcemanager | Jason Lowe | Jason Lowe | | | mapred hsadmin getGroups fails to connect in some cases | Major | client | Robert Kanter | Robert Kanter | | | On secure clusters call to timeline server fails with authentication errors when running a job via oozie | Blocker | timelineserver | Arpit Gupta | Zhijie Shen | | | MetricsSystemImpl could leak memory in registered callbacks | Major | metrics | Chuan Liu | Chuan Liu | | | Site build is broken | Major | . | Jian He | Jian He | | | Clients are always sent to the same datanode when read is off rack | Critical | . | Jason Lowe | Andrew Wang | | | TestAMRMClientOnRMRestart test fails | Major | . | Jian He | Jian He | | | DN rejects blocks if it has older UC block | Major | . | Daryn Sharp | Eric Payne | | | Pipeline close recovery race can cause block corruption | Critical | datanode | Kihwal Lee | Kihwal Lee | | | Fix TestRpcProgramNfs3 to use DFS\ENCRYPTION\KEY\PROVIDER\URI | Minor | test | Charles Lamb | Charles Lamb | | | RM shouldn't use the old RMApplicationHistoryWriter unless explicitly setting FileSystemApplicationHistoryStore | Major | resourcemanager, timelineserver | Karam Singh | Zhijie Shen | | | YARNRunner.getJobStatus() fails with ApplicationNotFoundException if the job rolled off the RM view | Major | client | Sangjin Lee | Sangjin Lee | | | Fix a bunch of typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | | | Remove obsolete entries from yarn-default.xml | Minor | . | Ray Chiang | Ray Chiang | | | HA NN can NPE upon transition to active | Critical | namenode | Daryn Sharp | Kihwal Lee | | | TestKMSWithZK does not use KEY\PROVIDER\URI | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Fix TestJournalNode#testFailToStartWithBadConfig to match log output change | Minor | test | Ray Chiang | Ray Chiang | | | Allow falling back to a non-SASL connection on DataTransferProtocol in several edge cases. | Major | datanode, hdfs-client, security | Chris Nauroth | Chris Nauroth | | | Avoid Findbugs warning for synchronization on AbstractNNFailoverProxyProvider#fallbackToSimpleAuth. | Trivial | ha | Chris Nauroth | Chris Nauroth | | | TestDataStorage does not release file locks between tests. | Minor | test | Chris Nauroth | Chris Nauroth | | | Skip tests related to short-circuit read on platforms that do not currently implement short-circuit read. | Minor | test | Chris Nauroth | Chris Nauroth | | | TestRMApplicationHistoryWriter fails with FairScheduler | Major | . | zhihai xu | zhihai xu | | | NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if excluded nodes passed are not part of the cluster tree | Major | . | Vinayakumar B | Vinayakumar B | | | TestProportionalCapacityPreemptionPolicy fails with FairScheduler | Major | . | zhihai xu | zhihai xu | | | Enable DistributedCache for uber-mode Jobs | Major | applicationmaster, distributed-cache | Gera Shegalov | Gera Shegalov | | | Jobs can deadlock if headroom is limited by cpu instead of memory | Critical | mrv2, scheduler | Peng Zhang | Peng Zhang | | | Reconfiguring DataNode volumes does not release the lock files in removed

| Major | datanode | Chris Nauroth | Chris Nauroth | | | FairScheduler: Queue filters not working on scheduler page in RM UI | Major | scheduler | Ashwin Shankar | Ashwin Shankar | | | Tests in TestTracing should not depend on the order of execution | Minor | . | Masatake Iwasaki | Masatake Iwasaki | | | Fix headroom calculation in FairScheduler | Major | . | Sandy Ryza | Anubhav Dhoot | | | Intermittent failure of TestFairScheduler.testContinuousScheduling | Major | scheduler | Ratandeep Ratti | | | | hdfs namenode -metadataVersion command does not honor configured name dirs | Minor | namenode | Charles Lamb | Charles Lamb | | | TestDataTransferKeepalive fails intermittently on Windows. | Major | test | Chris Nauroth | Chris Nauroth | | | Fix build on macosx: HDFS parts | Minor | . | Binglin Chang | Binglin Chang | | | Fix build on macosx: YARN parts | Major | . | Binglin Chang | Binglin Chang | | | TestJobHistoryParsing.testPartialJob fails in branch-2 | Major | . | Mit Desai | Mit Desai | | | TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2 | Minor | test | Juan Yu | Juan Yu | | | UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due to NativeCRC32 method changes | Blocker | native | Steve Loughran | Chris Nauroth | | | REST API for application creation/submission is using strings for numeric & boolean values | Major | api | Doug Haigh | Varun Vasudev | | | TestSafeMode assumes Unix line endings in safe mode tip. | Trivial | test | Chris Nauroth | Chris Nauroth | | | Fix minor typo in distcp -p usage text | Trivial | distcp | Charles Lamb | Charles Lamb | | | TestLeaseRecovery leaks MiniDFSCluster instances. | Major | test | Chris Nauroth | Chris Nauroth | | | minor distcp doc edits | Trivial | distcp, documentation | Charles Lamb | Charles Lamb | | | ResourceManager UI showing negative value for "Decommissioned Nodes" field | Major | resourcemanager, webapp | Nishan Shetty | Rohith Sharma K S | | | During HA upgrade, JournalNode should create a new committedTxnId file in the current directory | Major | qjm | Jing Zhao | Jing Zhao | | | FairScheduler: Potential deadlocks in loading alloc files and clock access | Major | . | Wei Yan | Wei Yan | | | TestEncryptionZones#testIsEncryptedMethod fails on branch-2 after archival storage merge | Major | encryption | Andrew Wang | Andrew Wang | | | hadoop-aws only need test-scoped dependency on hadoop-common's tests jar | Major | . | Juan Yu | Juan Yu | | | Old MR client is not compatible with new MR application | Blocker | client, mr-am | Zhijie Shen | Junping Du | | | Update FsShell documentation to mention deprecation of some of the commands, and mention alternatives | Major | documentation, fs | Hemanth Yamijala | Akira Ajisaka | | | Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags | Trivial | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | HDFS permissions guide documentation states incorrect default group mapping class. | Trivial | documentation | Chris Nauroth | Ray Chiang | | | user/custom LogManager fails to load if the client classloader is enabled | Minor | util | Sangjin Lee | Sangjin Lee | | | Use current version of the archive name in DistributedCacheDeploy document | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Fix usage and example at FileSystemShell.apt.vm | Trivial | documentation | Kenji Kikushima | Kenji Kikushima | |

MRJobConfig#MR\CLIENT\TO\AM\IPC\MAX\RETRIES\ON\TIMEOUTS config name is wrong | Major | . | Jian He | Akira Ajisaka | | | NetUtils.wrapException loses inner stack trace on BindException | Minor | net | Steve Loughran | Steve Loughran | | | Fsck documentation is outdated. | Major | documentation | Konstantin Shvachko | Masahiro Yamaguchi | | | javax package system class default is too broad | Minor | util | Sangjin Lee | Sangjin Lee | | | Document dfs.namenode.resource.\* | Major | documentation | Eli Collins | Daisuke Kobayashi | | | TestMRCJCFileInputFormat.testAddInputPath() fails on trunk | Minor | test | Sangjin Lee | Akira Ajisaka | | | Fix and clarify INodeInPath getter functions | Minor | . | Zhe Zhang | Zhe Zhang | | | JavaKeystoreProvider should not report a key as created if it was not flushed to the backing file | Major | . | Andrew Wang | Arun Suresh | | | Application History Server tries to access hdfs before doing secure login | Major | timelineserver | Mit Desai | Mit Desai | | | Use of ThreadLocal\<Random\> results in poor block placement | Blocker | namenode | Jeff Buell | Andrew Wang | | | NFS updateMaps OS check is reversed | Major | nfs | Allen Wittenauer | Brandon Li | | | Update BUILDING.txt to state that CMake 3.0 or newer is required on Mac. | Trivial | documentation, native | Chris Nauroth | Chris Nauroth | | | TestFairCallQueue fails | Major | test | Akira Ajisaka | Akira Ajisaka | | | NPE while running Mover if the given path is for a file | Major | balancer & mover | Prabushankar Chinnasamy | Jing Zhao | | | Hamlet should close table tags | Major | . | Ray Chiang | Ray Chiang | | | Resource Manager crashes with NPE due to lack of synchronization | Blocker | . | Mit Desai | Mit Desai | | | Potential deadlock in RM when querying ApplicationResourceUsageReport | Blocker | resourcemanager | Karam Singh | Wangda Tan | | | Generic History Service of TimelineServer sometimes not able to handle NPE | Major | timelineserver | Karam Singh | Zhijie Shen | | | Namenode not able to reconnect to KMS after KMS restart | Major | security | Arun Suresh | Arun Suresh | | | TestNamenodeCapacityReport.testXceiverCount may sometimes fail due to lack of retry | Major | . | Mit Desai | Mit Desai | | | Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. | Trivial | test | Chris Nauroth | Chris Nauroth | | | The namenode usage message doesn't include "-rollingupgrade started" | Minor | namenode | Colin P. McCabe | Colin P. McCabe | | | TestDistributedShell#testDSRestartWithPreviousRunningContainers fails | Major |

| Jian He | Jian He | | | Additional unit test for replica write with full disk | Major | test | Arpit Agarwal | Arpit Agarwal | | | NM does not need to send finished container whose APP is not running to RM | Major | nodemanager | Jun Gong | Jun Gong | | | Resource Localization fails on a cluster due to existing cache directories | Blocker | nodemanager | Anubhav Dhoot | Anubhav Dhoot | | | Wrong path when deleting through fuse-dfs a file which already exists in trash | Major | fuse-dfs | Chengbing Liu | Chengbing Liu | | | Fix typo in nfs3 server duplicate entry reporting | Trivial | nfs | Charles Lamb | Charles Lamb | | | NPE in ApplicationACLsManager | Major | resourcemanager | Benoy Antony | Benoy Antony | | | Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free | Major | capacityscheduler | Varun Vasudev | Varun Vasudev | | | Automatically refresh auth token and retry on auth failure | Major | security | zhubin | Arun Suresh | | | ContainerId@toString() is unreadable for epoch \>0 after YARN-2182 | Critical | . | Vinod Kumar Vavilapalli | Tsuyoshi Ozawa | | | TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS | Major | . | Wei Yan | Wei Yan | | | MetricsSystemImpl may miss a registered source | Minor | metrics | Chuan Liu | Chuan Liu | | | Remove synchronized blocks from SnappyCodec and ZlibCodec buffering inner loop | Major | performance | Gopal V | Gopal V | | | Block should be placed in the client's 'rack-local' node if 'client-local' node is not available | Major | namenode | Vinayakumar B | Vinayakumar B | | | Fix a findbugs warning in ReplaceDatanodeOnFailure | Minor | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Some accesses to DomainSocketWatcher#closed are not protected by lock | Minor | . | Ted Yu | Colin P. McCabe | | | Remove duplicated entry "dfs.webhdfs.enabled" in the user doc | Trivial | documentation | Yi Liu | Yi Liu | | | TestCommitterEventHandler fails in trunk | Major | . | Ted Yu | Mit Desai | | | Fix DelegationTokenAuthenticatedURL to pass the connection Configurator to the authenticator | Major | . | Arun Suresh | Arun Suresh | | | Flaky test TestAMRMRPCNodeUpdates | Major | . | Ming Ma | Ming Ma | | | Tarball as local resource type archive fails to localize on Windows | Major | . | Hitesh Shah | Craig Welch | | | Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources | Major | . | Arun Suresh | Arun Suresh | | | Should trim the content of keystore password file for JavaKeyStoreProvider | Minor | security | zhubin | Yi Liu | | | Fix several issues of hadoop security configuration in user doc. | Trivial | documentation, security | Yi Liu | Yi Liu | | | Fix findbugs exclude file | Minor | build | Arun Suresh | Arun Suresh | | | Delegation token for KMS should only be got once if it already exists | Major | kms, security | Yi Liu | Yi Liu | | | ApplicationSubmissionContext change breaks the existing app submission | Blocker | resourcemanager | Zhijie Shen | Wangda Tan | | | TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. | Trivial | test | Chris Nauroth | Chris Nauroth | | | TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. | Trivial | test | Chris Nauroth | Chris Nauroth | | | TestCgroupsLCEResourcesHandler leaks file descriptors. | Minor | test | Chris Nauroth | Chris Nauroth | | | Fix uninitialized variables in NativeIO.c | Major | native | Xiaoyu Yao | Xiaoyu Yao | | | TestContainerLauncherImpl sometimes fails | Major | test | Mit Desai | Mit Desai | | | Fix the release audit warning caused by hadoop-yarn-registry | Minor | . | Yi Liu | Yi Liu | | |

TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots | Major | . | Yongjun Zhang | Yongjun Zhang | | | Broken Link for GFS in package.html | Minor | . | Suraj Nayak | Suraj Nayak | | | NPE happened when RM restart after CapacityScheduler queue configuration changed | Critical | resourcemanager, scheduler | Wangda Tan | Chang Li | | | KMSClientProvider authentication fails when both currentUgi and loginUgi are a proxied user | Major | . | Arun Suresh | Arun Suresh | | | namenode -rollingUpgrade throws ArrayIndexOutOfBoundsException | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Fix typo in javadoc for FileSystem#listStatus() | Minor | . | Ted Yu | Li Lu | | | o.a.h.security.token.delegation.DelegationTokenManager should be more generalized to handle other DelegationTokenIdentifier | Major | security | Zhijie Shen | Zhijie Shen | | | RM web services authentication filter should add support for proxy user | Major | resourcemanager | Varun Vasudev | Zhijie Shen | | | Shuffle bandwidth computation includes time spent waiting for maps | Major | . | Siqi Li | Siqi Li | | | The active NameNode will not accept an fsimage sent from the standby during rolling upgrade | Major | namenode | Colin P. McCabe | Jing Zhao | | | GraphiteSink emits wrong timestamps | Major | . | Sascha Coenen | Ravi Prakash | | | NN doesn't schedule replication when a DN storage fails | Major | namenode | Ming Ma | Ming Ma | | | When a LayoutVersion support SNAPSHOT, it must support FSIMAGE\NAME\OPTIMIZATION. | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | WindowsSecureContainerExecutor should not depend on DefaultContainerExecutor#getFirstApplicationDir. | Minor | nodemanager | zhihai xu | zhihai xu | | | Standby RM does not transitionToActive if previous transitionToActive is failed with ZK exception. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | Killing a job just as it finishes can generate an NPE in client | Major | client, mrv2 | Jason Lowe | Rohith Sharma K S | | | DelegationTokenAuthenticationHandler needs to support DT operations for proxy user | Major | security | Zhijie Shen | Zhijie Shen | | | Ignore .keep files | Major | . | Karthik Kambatla | Karthik Kambatla | | | Unresponseive NFS mount point due to deferred COMMIT response | Major | nfs | Brandon Li | Brandon Li | | | Windows: Wildcard classpath variables not expanded against resources contained in archives | Major | nodemanager | Craig Welch | Craig Welch | | | TestDNFencingWithReplication fails consistently | Minor | test | Charles Lamb | Charles Lamb | | | Proxy user is problem for RPC interface if yarn.resourcemanager.webapp.proxyuser is not set. | Blocker | resourcemanager | Zhijie Shen | Zhijie Shen | | | Race condition: ZKRMStateStore retry logic may throw NodeExist exception | Blocker | . | Jian He | Jian He | | | TestDNFencing.testQueueingWithAppend failed often in latest test | Major | ha | Yongjun Zhang | Yongjun Zhang | | | (Rumen) Rumen tool returns error "ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type" | Major | . | Junping Du | Junping Du | | | Fix findbugs in ZK DelegationTokenSecretManagers | Blocker | . | Karthik Kambatla | Arun Suresh | | | Fix syntax error in SecureContainer.apt.vm | Major | . | Jian He | Jian He | | | ZKDelegationTokenSecretManager fails to renewToken created by a peer | Major |

| Arun Suresh | Arun Suresh | | | HDFS concat operation should not be allowed in Encryption Zone | Major | encryption, namenode | Yi Liu | Charles Lamb | | | If an unreadable file is encountered during log aggregation then aggregated file in HDFS badly formed | Major | log-aggregation | Sumit Mohanty | Xuan Gong | | | winutils task: unsecure path should not call AddNodeManagerAndUserACEsToObject | Major | . | Remus Rusanu | Remus Rusanu | | | YARN unable to renew delegation token fetched via webhdfs due to incorrect service port | Critical | webhdfs | Varun Vasudev | Jitendra Nath Pandey | | | ContainerManagementProtocolProxy can create thousands of threads for a large cluster | Critical | client | Jason Lowe | Jason Lowe | | | Yarn jobs via oozie fail with failed to renew token (secure) or digest mismatch (unsecure) errors when RM is being killed | Blocker | resourcemanager | Arpit Gupta | Jian He | | | If a sub-folder is encountered by log aggregator it results in invalid aggregated file | Major | log-aggregation | Sumit Mohanty | Xuan Gong | | | HdfsConstants#MEMORY\STORAGE\POLICY\ID and HdfsConstants#MEMORY\STORAGE\POLICY\ID are missing in branch-2 | Minor | . | Jing Zhao | Jing Zhao | | | hadoop.security.kms.client.encrypted.key.cache.expiry property spelled wrong in core-default | Minor | conf | Steve Loughran | Stephen Chu | | | Completely remove word 'experimental' from FairScheduler docs | Trivial | documentation | Harsh J | Harsh J | | | Windows: Node manager cannot serve up log files via the web user interface when yarn.nodemanager.log-dirs to any drive letter other than C: (or, the drive that nodemanager is running on) | Major | nodemanager | Craig Welch | Craig Welch | | | NFSv3 gateway frequently gets stuck due to GC | Critical | nfs | Eric Zhiqiang Ma | Brandon Li | | | Disable SSLv3 in HttpFS | Blocker | webhdfs | Robert Kanter | Robert Kanter | | | SSLFactory shouldn't allow SSLv3 | Blocker | . | Wei Yan | Wei Yan | | | map\input\file is missing from streaming job environment | Major | . | Jason Lowe | Jason Lowe | | | Timeline server domain not set correctly when using shell\_command on Windows | Major | applications/distributed-shell | Varun Vasudev | Varun Vasudev | | | The OfflineImageViewer (OIV) can output invalid XML depending on the filename | Major | . | Ravi Prakash | Ravi Prakash | | | The getMaxNodesPerRack() method in BlockPlacementPolicyDefault is flawed | Critical | . | Kihwal Lee | Kihwal Lee | | | NPE seen in wbhdfs FS while running SLive | Minor | webhdfs | Arpit Gupta | Jing Zhao | | | Fix a couple javac warnings in NFS | Major | nfs | Brandon Li | Brandon Li | | | NM fails to clean up usercache\DEL\\<timestamp\> dirs after YARN-661 | Critical | . | Siqi Li | Siqi Li | | | fix endmacro of set\find\shared\library\without\_version in CMakeLists | Minor | build | Yi Liu | Yi Liu | | | JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative | Major | util | Jinghui Wang | Jinghui Wang | | | SystemMetricsPublisher can use Kerberos directly instead of timeline DT | Critical | resourcemanager, timelineserver | Zhijie Shen | Zhijie Shen | | | XMLUtils.mangleXmlString doesn't seem to handle less than sign | Minor | . | Ravi Prakash | Colin

McCabe | | | Potential race condition in startLocalizer when using LinuxContainerExecutor | Blocker | . | Xuan Gong | Xuan Gong | | | Potential null dereference in FSDownload | Minor | . | Ted Yu | Gera Shegalov | | | Promoting AccessControlList to be public | Major | . | Zhijie Shen | Zhijie Shen | | | NM can't aggregate logs past HDFS delegation token expiry. | Critical | nodemanager | Tassapol Athiapinya | Jian He | | | Support overriding log4j.properties per job | Major | . | Junping Du | Junping Du | | | YarnClient doesn't need to translate Kerberos name of timeline DT renewer | Blocker | timelineserver | Arpit Gupta | Zhijie Shen | | | yarn logs -applicationId on 2.6.0 should support logs written by 2.4.0 | Blocker | log-aggregation | Gopal V | Xuan Gong | | | DefaultContainerExecutor runs only one localizer at a time | Critical | . | Siqi Li | Siqi Li | | | Update archival storage user documentation | Blocker | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Process group remains alive after container process is killed externally | Major | nodemanager | Billie Rinaldi | Billie Rinaldi | | | TestTraceAdmin assumes Unix line endings. | Trivial | test | Chris Nauroth | Chris Nauroth | | | make rollingUpgrade start/finalize idempotent | Major | ha | Arpit Gupta | Jing Zhao | | | Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures | Minor | test | Charles Lamb | Charles Lamb | | | ContainerExecutor always append "nice -n" in command on branch-2 | Critical | . | Xuan Gong | Xuan Gong | | | Patch up Jetty to disable SSLv3 | Blocker | security | Karthik Kambatla | Mike Yoder | | | Handle app-recovery failures gracefully | Blocker | resourcemanager | bc Wong | Karthik Kambatla | | | Timeline server .out log have JAXB binding exceptions and warnings. | Critical | . | Zhijie Shen | Zhijie Shen | | | FSNamesystem ACL operations should write to audit log on failure | Minor | namenode | Charles Lamb | Charles Lamb | | | Credential and Key Shell Commands not available on Windows | Major | scripts | Larry McCay | Larry McCay | | | DFSOutputStream should not silently drop data if DataStreamer crashes with an unchecked exception | Critical | hdfs-client | Jason Lowe | Rushabh S Shah | | | NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. | Major | journal-node, namenode | Chris Nauroth | Chris Nauroth | | | RM2 in HA setup tries to login using the RM1's kerberos principal | Blocker | resourcemanager | Arpit Gupta | Wangda Tan | | | Deadlock when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | | | NPE from MemoryTimelineStore.getDomains | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol. | Major | hdfs-client | Chris Nauroth | Chris Nauroth | | | JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar

| Major | client | Gera Shegalov | Gera Shegalov | | | Remove the logic to inject entity owner as the primary filter | Critical | timelineserver | Zhijie Shen | Zhijie Shen | | | Wrong reduce task progress if map output is compressed | Minor | . | Emilio Coppa | Emilio Coppa | | | Balancer always shows zero Bytes Already Moved | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | TestWinUtils#testChmod fails after removal of NO\PROPAGATE\INHERIT\_ACE. | Trivial | native, test | Chris Nauroth | Chris Nauroth | | | NullPointerException in RM HA enabled 3-node cluster | Critical | resourcemanager | Gour Saha | Jian He | | | Fix bugs of yarn queue CLI | Critical | client | Wangda Tan | Wangda Tan | | | MR distributed cache not working correctly on Windows after NodeManager privileged account changes. | Critical | nodemanager | Chris Nauroth | Craig Welch | | | TestBalancer#testBalancerWithRamDisk creates test files incorrectly | Minor | test | Xiaoyu Yao | Xiaoyu Yao | | | Skip NFS TestShellBasedIdMapping tests that are irrelevant on Windows. | Trivial | test | Chris Nauroth | Chris Nauroth | | | Container leak on NM | Critical | . | Jian He | Jian He | | | NPE in ATS Timeline Domains when upgrading from 2.4 to 2.6 | Critical | timelineserver | Gopal V | Zhijie Shen | | | User-Group mappings not updated by RM when a user is removed from a group. | Critical | . | Sidharta Seethana | Wangda Tan | | | DataNode in secure mode may throw NullPointerException if client connects before DataNode registers itself with NameNode. | Minor | datanode, security | Chris Nauroth | Chris Nauroth | | | Map/Reduce dangerously adds Guava @Beta class to CryptoUtils | Blocker | . | Christopher Tubbs | | | | Add backwords compatible ContainerId.newInstance constructor for use within Tez Local Mode | Blocker | . | Jonathan Eagles | Jonathan Eagles | | | DataNode.requestShortCircuitFdsForRead may throw NullPointerException | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Resource manager crashed with Null Pointer Exception | Blocker | . | Yesha Vora | Jian He | | | Make NetUtils.wrapException throw EOFException instead of wrapping it as IOException | Major | . | Jian He | Steve Loughran | | | NFS may only do partial commit due to a race between COMMIT and write | Critical | nfs | Brandon Li | Brandon Li | | | Fix log msgs about distributing system-credentials | Major | . | Jian He | Jian He | | | Disable SSLv3 in KMS | Blocker | kms | Robert Kanter | Robert Kanter | | | Renable SSLv2Hello in HttpFS | Blocker | webhdfs | Robert Kanter | Robert Kanter | | | Fetcher - connect() doesn't handle connection refused correctly | Blocker | . | Sidharta Seethana | Junping Du | | | Killing app may hang while AM is unregistering | Major | . | Jian He | Jian He | | | Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager

| Blocker | nodemanager | Junping Du | Junping Du | | | ThreadLocal used in FSEditLog class causes FSImage permission mess up | Blocker | namenode | jiangyu | jiangyu | | | ResourceManger is not able renew WebHDFS token when application submitted by Yarn WebService | Major | nodemanager, resourcemanager, webapp | Karam Singh | Varun Vasudev | | | LZ4 Compression fails to recognize PowerPC Little Endian Architecture | Major | io, native | Ayappan | Bert Sanders | | | s3n read truncated, but doesn't throw exception | Major | fs/s3 | David Rosenstrauch | | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Flake TestDFSZKFailoverController.testManualFailoverWithDFSHAAdmin due to a long edit log sync op | Minor | auto-failover, test | Liang Xie | Liang Xie | | | TestFixedLengthInputFormat fails in windows | Major | client, test | Varun Vasudev | Varun Vasudev | | | shorten test run time with a smaller retry timeout setting | Major | test | Liang Xie | Liang Xie | | | Add test for successive Snapshots between XAttr modifications | Minor | snapshots, test | Stephen Chu | Stephen Chu | | | Add HDFS tests for ACLs in combination with viewfs. | Major | hdfs-client, test | Chris Nauroth | Stephen Chu | | | Add tests for XAttrs in combination with viewfs | Major | hdfs-client | Stephen Chu | Stephen Chu | | | TestTimelineWebServices fails on trunk after HADOOP-10791 | Major | . | Zhijie Shen | Zhijie Shen | | | Change MiniDFSCluster to support StorageType configuration for individual directories | Minor | test | Tsz Wo Nicholas Sze | Arpit Agarwal | | | Create a CryptoCodec test that verifies interoperability between the JCE and OpenSSL implementations | Major | security | Alejandro Abdelnur | Yi Liu | | | Create MiniKMS for testing | Major | security, test | Alejandro Abdelnur | Alejandro Abdelnur | | | Credential Provider related unit tests failed on Windows | Major | webapp | Xiaoyu Yao | Xiaoyu Yao | | | HDFS Credential Provider related Unit Test Failure | Major | encryption | Xiaoyu Yao | Xiaoyu Yao | | | Credential Provider related Unit Tests Failure on Windows | Major | security | Xiaoyu Yao | Xiaoyu Yao | | | KMSClientProvider should drain the local generated EEK cache on key rollover | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | | | TestRMWebServicesAppsModification sometimes fails in trunk | Minor | . | Ted Yu | Varun Vasudev | | | TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator. | Minor | datanode, test | Chris Nauroth | Chris Nauroth | | | Unittest TestKeyShell, TestCredShell and TestKMS assume UNIX path separator for JECKS key store path | Major | security | Xiaoyu Yao | Xiaoyu Yao | | | TestContainerLaunch fails due to classpath problem with hamcrest classes. | Minor | nodemanager, test | Chris Nauroth | Chris Nauroth | | | Test encryption zones with KMS | Major | security, test | Alejandro Abdelnur | Alejandro Abdelnur | | | TestEncryptionZones assumes Unix path separator for KMS key store path | Major | encryption | Xiaoyu Yao | Xiaoyu Yao | | | TestContainerManagerSecurity fails on trunk | Major | . | Zhijie Shen | Jian He | | | TestEncryptionZonesWithHA assumes Unix path separator for KMS key store path | Minor | security, test | Xiaoyu Yao | Xiaoyu Yao | | | TestWorkPreservingRMRestart fails with FairScheduler | Major | . | Junping Du | Karthik Kambatla | | | TestPipeApplication#testSubmitter fails in trunk | Minor | . | Ted Yu | Binglin Chang | | | Update TestApplicationHistoryClientService to use the new generic history store | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | TestAggregatedLogFormat fails in trunk | Major |

| Xuan Gong | Xuan Gong | | | Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows | Major | tools | Varun Vasudev | Varun Vasudev | | | TestDefaultContainerExecutor#testContainerLaunchError fails on Windows | Major | . | Varun Vasudev | Varun Vasudev | | | TestNMSimulator fails sometimes due to timing issue | Major | . | Varun Vasudev | Varun Vasudev | | | TestContainerResourceUsage fails intermittently | Major | . | Varun Vasudev | Varun Vasudev | | | TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. | Trivial | test | Chris Nauroth | Chris Nauroth | | | TestJavaSerialization fails in trunk build | Minor | . | Ted Yu | Varun Vasudev | | | RM web services - add test case to ensure the http static user cannot kill or submit apps in secure mode | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | TestApplicationHistoryServer is likely to fail on less powerful machine | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | TestRMProxyUsersConf fails on Windows VMs | Major | resourcemanager | Varun Vasudev | Varun Vasudev | | | TestDistributedShell fails in trunk | Major | . | Ted Yu | Wangda Tan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add Listing Support to Key Management APIs | Major | security | Larry McCay | Larry McCay | | | Minor improvements to Crypto input and output streams | Major | security | Alejandro Abdelnur | Yi Liu | | | Add a method to CryptoCodec to generate SRNs for IV | Major | security | Alejandro Abdelnur | Yi Liu | | | After restart NM should resync with the RM without killing containers | Major | resourcemanager | Bikas Saha | Anubhav Dhoot | | | Use DirectorySnapshottableFeature to represent a snapshottable directory | Major | namenode | Jing Zhao | Jing Zhao | | | Add containers to launchedContainers list in RMNode on container recovery | Major | resourcemanager | Jian He | Jian He | | | TimelineServer should load pseudo authentication filter when authentication = simple | Major | . | Zhijie Shen | Zhijie Shen | | | Recover NMTokens upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | | | AMRMTokenManager need to have a way to roll over AMRMToken | Major | resourcemanager | Xuan Gong | Xuan Gong | | | Data persisted in NM should be versioned | Major | nodemanager | Junping Du | Junping Du | | | The diagnostics is always the ExitCodeException stack when the container crashes | Major | nodemanager | Zhijie Shen | Tsuyoshi Ozawa | | | Improve exception information on AM launch crashes | Major | . | Li Lu | Li Lu | | | Refactor YARN distributed shell with existing public stable API | Major | . | Li Lu | Li Lu | | | ShuffleHandler DB schema should be versioned with compatible/incompatible changes | Major |

| Junping Du | Junping Du | | | Recover container tokens upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | | | RMStateStore needs to save AMRMToken master key for recovery when RM restart/failover happens | Major | resourcemanager | Xuan Gong | Xuan Gong | | | The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale | Major | datanode, hdfs-client | Colin P. McCabe | Colin P. McCabe | | | DistributedShell may allocate more containers than client specified after it restarts | Major | . | Jian He | Li Lu | | | Consolidate RMStateVersion and NMDBSchemaVersion into StateVersion in yarn-server-common | Major | . | Junping Du | Junping Du | | | CryptoCodec#getCodecclasses throws NPE when configurations not loaded. | Major | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Recover applications upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | | | FileBasedKeyStoresFactory Should use Configuration.getPassword for SSL Passwords | Major | security | Larry McCay | Larry McCay | | | Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config | Minor | nfs | Jeff Hansen | Brandon Li | | | LdapGroupsMapping Should use configuration.getPassword for SSL and LDAP Passwords | Major | security | Larry McCay | Larry McCay | | | Data persistent in timelinestore should be versioned | Major | timelineserver | Junping Du | Junping Du | | | CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure | Major | . | Chen He | Craig Welch | | | Dynamically add new volumes to DataStorage, formatted if necessary. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Make FSDataset support adding data volumes dynamically | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Display readable last contact time for dead nodes on NN webUI | Major | . | Ming Ma | Ming Ma | | | ApplicationMaster needs to find a way to update the AMRMToken periodically | Major | resourcemanager | Xuan Gong | Xuan Gong | | | MRAppMaster changes for AMRMToken roll-up | Major | resourcemanager | Xuan Gong | Xuan Gong | | | Refactor TimelineWebServices | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Recover containers upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | | | Update documentation about how to write YARN applications | Major | documentation | Li Lu | Li Lu | | | Create a scheduler, which assigns schedulables a priority level | Major | . | Chris Li | Chris Li | | | Fair scheduler to re-populate container allocation state | Major | fairscheduler | Bikas Saha | Anubhav Dhoot | | | Remove MRv1 commands from CommandsManual.apt.vm | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Add Cross-Origin support to the ATS REST API | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | | | DistributedShell publishes unfriendly user information to the timeline server | Minor |

| Zhijie Shen | Robert Kanter | | | Adding support for moving apps between queues in Capacity Scheduler | Major | capacityscheduler | Subru Krishnan | Subru Krishnan | | | Capacity Scheduler changes for moving apps between queues | Major | capacityscheduler | Janos Matyas | Janos Matyas | | | Adding support for drainig a queue, ie killing all apps in the queue | Major | capacityscheduler, fairscheduler | Subru Krishnan | Subru Krishnan | | | Add ability to specify a reverse ACL (black list) of users and groups | Major | security | Benoy Antony | Benoy Antony | | | Fix dead link in Configuration javadoc | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | AM release request may be lost on RM restart | Major | resourcemanager | Jian He | Jian He | | | Enabling HTTPs for the writer REST API of TimelineServer | Major | . | Zhijie Shen | Zhijie Shen | | | Allow specifying multiple MapOutputCollectors with fallback | Major | task | Todd Lipcon | Todd Lipcon | | | RM should not recover containers from previously failed attempt when AM restart is not enabled | Major | . | Jian He | Jian He | | | Create a FairCallQueue: a multi-level call queue which schedules incoming calls and multiplexes outgoing calls | Major | . | Chris Li | Chris Li | | | RM should log using RMStore at startup time | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | FileSystemApplicationHistoryStore blocks RM and AHS while NN is in safemode | Major | . | Jonathan Eagles | Jonathan Eagles | | | Add LazyPersist flag to FileStatus | Major | . | Arpit Agarwal | Arpit Agarwal | | | Update ContainerId#toString() to avoid conflicts before and after RM restart | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Add XDR packaging method for each NFS request | Major | nfs | Brandon Li | Brandon Li | | | Adding tracing to Hadoop RPC | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | Ensure distributed shell work with RM work-preserving recovery | Major | resourcemanager | Jian He | Jian He | | | Propagate LazyPersist flag to DNs via DataTransferProtocol | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Add unit tests | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | DN support for saving replicas to persistent storage and evicting in-memory replicas | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Add new RAM\_DISK storage type | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | NN periodically unlinks lazy persist files with missing replicas from namespace | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | 'hdfs put' command should accept lazyPersist flag for testing | Major | datanode | Tassapol Athiapinya | Arpit Agarwal | | | Byte array native checksumming on client side (HDFS changes) | Major | hdfs-client, performance | James Thomas | James Thomas | | | Move RM recovery related proto to yarn\server\resourcemanager\_recovery.proto | Major | . | Jian He | Tsuyoshi Ozawa | | | Move lazily persisted replicas to finalized directory on DN startup | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Bugfix in LazyWriter, fix test case and some refactoring | Major | datanode, test | Arpit Agarwal | Arpit Agarwal | | | Make FsDataset and DataStore support removing volumes. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Move TimelineClient to yarn-common project | Major | client | Zhijie Shen | Zhijie Shen | | | Add Additional unit tests for HDFS-6581 | Major |

| Xiaoyu Yao | Xiaoyu Yao | | | MapReduce AM should not use maxAttempts to determine if this is the last retry | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Wangda Tan | | | Improve replica eviction from RAM disk | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | NM restart: cgroup is not removed for reacquired containers | Major | nodemanager | Jason Lowe | Jason Lowe | | | Allow All Origins by default when Cross Origin Filter is enabled | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | | | Enable Cross Origin Filter for timeline server only and not all Yarn servers | Major | timelineserver | Jonathan Eagles | Mit Desai | | | Cross Origin configuration parameters prefix are not honored | Major | timelineserver | Jonathan Eagles | Mit Desai | | | DistributedFileSystem must get delegation tokens from configured KeyProvider | Major | security | Alejandro Abdelnur | Zhe Zhang | | | Supporting consistent edit log reads when in-progress edit log segments are included | Major | qjm | James Thomas | James Thomas | | | Allow for origin pattern matching in cross origin filter | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | | | Document Cross Origin Filter Configuration for ATS | Major | documentation, timelineserver | Jonathan Eagles | Jonathan Eagles | | | Update ConverterUtils#toContainerId to parse epoch | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Delete all copies when a block is deleted from the block space | Major | datanode | Nathan Yao | Arpit Agarwal | | | Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long | Major | caching, datanode | Colin P. McCabe | Colin P. McCabe | | | Notify NN of evicted block before deleting it from RAM disk | Major | datanode, namenode | Arpit Agarwal | Arpit Agarwal | | | Correctly persist raw namespace xattrs to edit log and fsimage | Major | encryption | Stephen Chu | Charles Lamb | | | Add additional unit tests for encryption zones | Major | encryption | Stephen Chu | Stephen Chu | | | Move s3-related FS connector code to hadoop-aws | Major | fs/s3 | David S. Wang | David S. Wang | | | Merging generic-history into the Timeline Store | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | | | Add logs when RM send new AMRMToken to ApplicationMaster | Major | resourcemanager | Xuan Gong | Xuan Gong | | | ContainerId can overflow with RM restart | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Cross Origin Filter throws UnsupportedOperationException upon destroy | Major | timelineserver | Jonathan Eagles | Mit Desai | | | Possible livelock in CapacityScheduler when RM is recovering apps | Major | resourcemanager | Jian He | Jian He | | | "yarn application -status \<appId\>" throws NPE when retrieving the app from the timelineserver | Major |

| Zhijie Shen | Zhijie Shen | | | Making the CapacityScheduler more dynamic | Major | capacityscheduler | Carlo Curino | Carlo Curino | | | ReservationSystem: replan upon capacity reduction | Major | resourcemanager | Carlo Curino | Carlo Curino | | | Admission Control: Reservation subsystem | Major | resourcemanager | Carlo Curino | Subru Krishnan | | | Add a public API to reserve resources (part of YARN-1051) | Major | resourcemanager | Carlo Curino | Subru Krishnan | | | Cross Origin Filter Http response split vulnerability protection rejects valid origins | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | | | Directory scanner should correctly reconcile blocks on RAM disk | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Add an AM retry count reset window to YARN RM | Major | resourcemanager | Chris Riccomini | Xuan Gong | | | Add test to verify encryption zone creation after NameNode restart without saving namespace | Minor | encryption, test | Stephen Chu | Stephen Chu | | | Add WebHDFS support for reading and writing to encryption zones | Major | encryption, webhdfs | Stephen Chu | Charles Lamb | | | Generic history service RPC interface doesn't work when service authorization is enabled | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | LazyWriter#evictBlocks misses a null check for replicaState | Minor | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Admission Control: agents to allocate reservation | Major | resourcemanager | Carlo Curino | Carlo Curino | | | Create a ZooKeeper-backed secret provider | Major | security | Robert Kanter | Robert Kanter | | | CapacityOverTimePolicy: a policy to enforce quotas over time for YARN-1709 | Major | . | Carlo Curino | Carlo Curino | | | Adding tracing to DataNode data transfer protocol | Major | . | Masatake Iwasaki | Masatake Iwasaki | | | Fix unit test failures in HDFS-6581 branch | Major | test | Arpit Agarwal | Xiaoyu Yao | | | Admission Control: plan follower | Major | capacityscheduler, resourcemanager | Carlo Curino | Carlo Curino | | | Refactor EncryptionZoneWithId and EncryptionZone | Major | namenode, security | Charles Lamb | Charles Lamb | | | Few more unit test fixes for HDFS-6581 | Major | test | Arpit Agarwal | Arpit Agarwal | | | Generic history service should support application-acls | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | | | Create an XAttr that disallows the HDFS admin from accessing a file | Major | namenode, security | Charles Lamb | Charles Lamb | | | Create FileStatus isEncrypted() method | Major | namenode, security | Charles Lamb | Charles Lamb | | | Updating ContainerTokenIdentifier#read/write to use ContainerId#getContainerId | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Fix finalize and upgrade unit test failures | Major | test | Arpit Agarwal | Arpit Agarwal | | | Update KeyProvider instantiation to create by URI | Major | encryption | Andrew Wang | Andrew Wang | | | Fix listEZs to work correctly with snapshots | Major | encryption | Andrew Wang | Andrew Wang | | | Handle AMRMTokens across RM failover | Blocker | resourcemanager | Karthik Kambatla | Jian He | | | FsDatasetImpl#copyBlockFiles debug log can be improved | Minor | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Threshold for RM to accept requests from AM after failover | Major | resourcemanager | Jian He | Jian He | | | Expose FileStatus#isEncrypted in libhdfs | Major | encryption | Andrew Wang | Colin P. McCabe | | | MR job client cannot reconnect to AM after NM restart. | Blocker | . | Tassapol Athiapinya | Junping Du | | | Add NFS Gateway support for reading and writing to encryption zones | Major | encryption, nfs | Stephen Chu | Charles Lamb | | | Improved shuffle error handling across NM restarts | Major |

| Jason Lowe | Junping Du | | | Refresh data volumes on DataNode based on configuration changes | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Admission Control: Integrate Reservation subsystem with ResourceManager | Major | resourcemanager | Subru Krishnan | Subru Krishnan | | | Move startFile EDEK retries to the DFSClient | Major | encryption | Andrew Wang | Andrew Wang | | | Add forwarding constructor for INodeFile for existing callers | Minor | namenode, test | Arpit Agarwal | Arpit Agarwal | | | TestStorageMover often fails in Jenkins | Minor | test | Tsz Wo Nicholas Sze | Jing Zhao | | | Make eviction scheme pluggable | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Fix unit test failures in SimulatedFsDataset | Major | test | Arpit Agarwal | Arpit Agarwal | | | Ensure all completed containers are reported to the AMs across RM restart | Major | resourcemanager | Bikas Saha | Anubhav Dhoot | | | Add unit test for evict/delete RAM\_DISK block with open handle | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | API changes for handling logs of long-running services | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | Add new DistributedFileSystem API for getting all the existing storage policies | Major | balancer & mover, namenode | Jing Zhao | Jing Zhao | | | More generalized timeline ACLs | Major | . | Zhijie Shen | Zhijie Shen | | | Move CipherSuite xattr information up to the encryption zone root | Major | encryption | Andrew Wang | Zhe Zhang | | | Unit test for creating encryption zone on root path | Minor | . | Zhe Zhang | Zhe Zhang | | | Fix hftp to work with encryption | Major | namenode | Charles Lamb | Charles Lamb | | | NMs need to find a way to get LogAggregationContext | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | Prepare yarn-1051 branch for merging with trunk | Major | capacityscheduler, resourcemanager, scheduler | Subru Krishnan | Subru Krishnan | | | Add a tool to list all the existing block storage policies | Minor | hdfs-client, namenode | Jing Zhao | Jing Zhao | | | Fix findbugs warnings in HDFS-6581 branch | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Balancer and Mover tools should ignore replicas on RAM\_DISK | Major | datanode | Arpit Agarwal | Xiaoyu Yao | | | Improve diagnostics on storage directory rename operations by using NativeIO#renameTo in Storage#rename. | Major | journal-node, namenode | Chris Nauroth | Chris Nauroth | | | Fix findbugs warnings in RamDiskReplicaTracker | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Add command line option to ask DataNode reload configuration. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Split error checks in AtomicFileOutputStream#close into separate conditions to improve diagnostics. | Minor | journal-node | Chris Nauroth | Chris Nauroth | | | Separate CipherSuite from crypto protocol version | Major | encryption | Andrew Wang | Andrew Wang | | | Allow dynamically changing the tracing level in Hadoop servers | Major | datanode, namenode | Colin P. McCabe | Colin

McCabe | | | Fix jenkins findbugs warning and test case failures for trunk merge patch | Major | capacityscheduler, resourcemanager, scheduler | Subru Krishnan | Subru Krishnan | | | Update the description of GenericOptionsParser -jt option | Minor | documentation | Akira Ajisaka | Akira Ajisaka | | | Bugfix in createLocatedFileStatus caused by bad merge | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Using Time.now() for recording start/end time of reconfiguration tasks | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | Remove EncryptionZoneManager.NULL\_EZ | Minor | namenode | Charles Lamb | Charles Lamb | | | Use block storage policy to set lazy persist preference | Major | namenode | Arpit Agarwal | Arpit Agarwal | | | Metrics to track usage of memory for writes | Major | datanode | Arpit Agarwal | Xiaoyu Yao | | | Fix Jenkins failures in HDFS-6581 branch | Major | datanode | Arpit Agarwal | Arpit Agarwal | | | Winutils needs ability to create task as domain user | Major | nodemanager | Kyle Leckie | Remus Rusanu | | | Add XDR parser method for each NFS response | Major | nfs | Brandon Li | Brandon Li | | | Implement secure Windows Container Executor | Major | nodemanager | Remus Rusanu | Remus Rusanu | | | Using TimelineNamespace to shield the entities of a user | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | DFSClient should instantiate a KeyProvider, not a KeyProviderCryptoExtension | Critical | encryption | Andrew Wang | Andrew Wang | | | Remove incorrect precondition check on key length in FileEncryptionInfo | Critical | encryption | Andrew Wang | Andrew Wang | | | Log handling for LRS | Major | log-aggregation, nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | Recalculate headroom more frequently to keep it accurate | Major | . | Craig Welch | Craig Welch | | | CapacityScheduler headroom doesn't account for other AM's running | Critical | capacityscheduler | Thomas Graves | Chen He | | | LazyWriter should use either async IO or one thread per physical disk | Major | datanode | Arpit Agarwal | Xiaoyu Yao | | | add hadoop-yarn-registry package under hadoop-yarn | Major | api | Steve Loughran | Steve Loughran | | | API changes for users | Major | api | Wangda Tan | Wangda Tan | | | Make distributed shell use the domain-based timeline ACLs | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Common server side PB changes (not include user API PB changes) | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | Modify the LogDeletionService to support Log aggregation for LRS | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | Node label manager API and storage implementations | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Changes in AMRMClient to support labels | Major | resourcemanager | Wangda Tan | Wangda Tan | | | yarn-registry JAR won't link against ZK 3.4.5 | Major | client | Steve Loughran | Steve Loughran | | | Spin off the LogRollingInterval from LogAggregationContext | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | Use unbuffered writes when persisting in-memory replicas | Major | datanode | Arpit Agarwal | Xiaoyu Yao | | | DefaultContainerExecutor should pick a working directory randomly | Critical | nodemanager | zhihai xu | zhihai xu | | |

ContainerId#getId as deprecated | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | | | Changes for capacity scheduler to support allocate resource respect labels | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Miscellaneous changes in ResourceManager to support labels | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Resource on each label not correct when multiple NMs in a same host and some has label some not | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | TestSecureRMRegistryOperations failing on windows: secure ZK won't start | Major | api, resourcemanager | Steve Loughran | Steve Loughran | | | Simplify the output when the user doesn't have the access for getDomain(s) | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Fix test timeout in TestResourceTrackerOnHA#testResourceTrackerOnHA | Blocker | client | Wangda Tan | Wangda Tan | | | Mark Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over | Critical | resourcemanager | Jian He | Tsuyoshi Ozawa | | | Changes of RM node label manager default configuration | Critical | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | Timeline authentication filter should add support for proxy user | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Support get/add/remove/change labels in RM admin CLI | Critical | resourcemanager | Wangda Tan | Wangda Tan | | | Add retry for timeline client put APIs | Major | . | Li Lu | Li Lu | | | Log related CLI and Web UI changes for Aggregated Logs in LRS | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | Fix returning value of starting reconfiguration task | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | | | containerLogNotFound log shows multiple time for the same container | Major | log-aggregation | Xuan Gong | Xuan Gong | | | NodeManager should identify failed disks becoming good again | Major | nodemanager | Ravi Gummadi | Varun Vasudev | | | Add retry for timeline client getDelegationToken method | Major | . | Li Lu | Li Lu | | | ktutil test hanging on some machines/ktutil versions | Major | . | Steve Loughran | Steve Loughran | | | TestSecureRMRegistryOperations failing on windows: auth problems | Major | api, resourcemanager | Steve Loughran | Steve Loughran | | | Improve HDFS-6581 eviction configuration | Major | datanode | Arpit Agarwal | Xiaoyu Yao | | | Add logUploadedTime into LogValue for better display | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | | | rmadmin -replaceLabelsOnNode does not correctly parse port | Major | client | Phil D'Amore | Naganarasimha G R | | | ClientToAMTokenMasterKey should be provided to AM at launch time | Blocker | . | Hitesh Shah | Jason Lowe | | | CapacityScheduler should explicitly log when an accessible label has no capacity | Minor | capacityscheduler | Phil D'Amore | Wangda Tan | | | AHSWebServices should return FORBIDDEN(403) if the request user doesn't have access to the history data | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Move checksum computation off the hot path when writing to RAM disk | Major | datanode, hdfs-client | Arpit Agarwal | Chris Nauroth | | | Localization and log-aggregation will fail if hdfs delegation token expired after token-max-life-time | Critical |

| Jian He | Jian He | | | Enable MR AM to post history events to the timeline server | Major | mr-am | Zhijie Shen | Robert Kanter | | | Create a framework specific config to enable timeline server | Major | . | Jonathan Eagles | Robert Kanter | | | Add UTs to cover timeline server authentication | Major | timelineserver | Zhijie Shen | Zhijie Shen | | | Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | | | Changes in distributed shell to support specify labels | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Changes in RM Web UI to better show labels to end users | Major | resourcemanager | Wangda Tan | Wangda Tan | | | Test failure in TestJobHistoryEventHandler and TestMRTimelineEventHandling | Critical | . | Zhijie Shen | Zhijie Shen | | | registry punycoding of usernames doesn't fix all usernames to be DNS-valid | Major | api, resourcemanager | Steve Loughran | Steve Loughran | | | Show when block deletion will start after NameNode startup in WebUI | Major | . | Jing Zhao | Chris Nauroth | | | Move getClusterNodeLabels and getNodeToLabels to YarnClient instead of AdminService | Critical | api, client, resourcemanager | Wangda Tan | Wangda Tan | | | YARN node CLI should display labels on returned node reports | Major | client | Wangda Tan | Wangda Tan | | | Timeline delegation tokens need to be automatically renewed by the RM | Critical | timelineserver | Zhijie Shen | Zhijie Shen | | | Resource Manager fails startup with HDFS label storage and secure cluster | Major | resourcemanager | Phil D'Amore | Wangda Tan | | | Improved Yarn Registry service record structure | Major | api, resourcemanager | Gour Saha | Steve Loughran | | | Under some scenario, it is possible to end up with capacity scheduler configuration that uses labels that no longer exist | Critical | capacityscheduler | Sumit Mohanty | Wangda Tan | | | Add yarn queue CLI to get queue infos | Major | client | Wangda Tan | Sunil Govindan | | | Capacity of labels should be zero by default | Critical | resourcemanager | Wangda Tan | Wangda Tan | | | Fix potential issues and code clean up for \*NodeLabelsManager | Major | . | zhihai xu | zhihai xu | | | Document NM Restart feature | Blocker | nodemanager | Junping Du | Junping Du | | | Support get/add/remove/change labels in RM REST API | Major | resourcemanager | Wangda Tan | Craig Welch | | | RMProxy should retry EOFException | Critical | resourcemanager | Jian He | Jian He | | | NodeLabels manager should trim all inputs for hosts and labels | Major | resourcemanager | Sushmitha Sreenivasan | Wangda Tan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | MRAppMaster should handle Resync from RM instead of shutting down. | Major | applicationmaster | Rohith Sharma K S | Rohith Sharma K S | | | Move DirectBufferPool into common util | Minor | util | Todd Lipcon | Todd Lipcon | | | Add ability to roll over AMRMToken | Major | resourcemanager | Xuan Gong | Xuan Gong | | | Merge KMS to branch-2 | Major | security | Alejandro Abdelnur |