Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.2.7.1.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Non-HA rollback compatibility broken | Blocker | . | J.Andreina | J.Andreina | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern | Minor | security | Harsh J | Harsh J | | | ZKRMStateStore: Avoid setting watches that are not required | Minor | . | Jun Gong | Jun Gong | | | Mover/Balancer should not schedule two replicas to the same DN | Minor | balancer & mover | Walter Su | Walter Su | | | Need document for storage type label of data node storage locations under dfs.datanode.data.dir | Major | documentation | Xiaoyu Yao | Xiaoyu Yao | | | Compatibility doc to state that ATS v1 is a stable REST API | Major | documentation | Steve Loughran | Steve Loughran | | | RMServerUtils.validateResourceRequests should only obtain queue info once | Major | resourcemanager | Jason Lowe | Varun Saxena | | | Add @VisibleForTesting annotation to {{BlockPoolSlice#selectReplicaToDelete}} | Trivial | . | Colin P. McCabe | Colin P. McCabe | | | Choose SSD over DISK in block placement | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | | | AppMaster tracking URL is broken in HA | Major | webapp | Anubhav Dhoot | Anubhav Dhoot | | | dfs.datanode.du.reserved is not considered while computing available space | Major | datanode | Brahma Reddy Battula | Brahma Reddy Battula | | | NPE when NodeManager.serviceInit fails and stopRecoveryStore called | Minor | nodemanager | Bibin A Chundatt | Bibin A Chundatt | | | Use LinkedHashMap to preserve order of resource requests | Major | nodemanager | zhihai xu | zhihai xu | | | ACLStatus and XAttributes not properly presented to INodeAttributesProvider before returning to client | Major | namenode | Arun Suresh | Arun Suresh | | | Split getAdditionalBlock() into two methods. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column | Major | resourcemanager, webapp | Jason Lowe | Jason Lowe | | | DistributedFIleSystem should not look for keyProvider in cache if Encryption is disabled | Minor | hdfs-client | Arun Suresh | Arun Suresh | | | Some of UserMetricsInfo metrics are incorrectly set to root queue metrics | Major | webapp | Rohit Agarwal | Rohit Agarwal | | | Possible leak in DelegationTokenRenewer#allTokens | Major |

| Jian He | Rohith Sharma K S | | | NameNode Failover during HA upgrade can cause DataNode to finalize upgrade | Blocker | ha | Jing Zhao | Jing Zhao | | | Patches applied for YARN-2424 are inconsistent between trunk and branch-2 | Major | . | Sidharta Seethana | Naganarasimha G R | | | Always use snapshot path as source when invalid snapshot names are used for diff based distcp | Minor | distcp | Sushmitha Sreenivasan | Jing Zhao | | | Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN | Critical | documentation | J.Andreina | J.Andreina | | | The footer of the Web UI "Hadoop, 2014" is old | Major | . | Akira Ajisaka | Brahma Reddy Battula | | | Task list sort by task id broken | Minor | . | Siqi Li | Siqi Li | | | Error Message points to wrong parent directory in case of path component name length error | Major | namenode | Anu Engineer | Anu Engineer | | | RM fails to come up with error "Failed to load/recover state" when mem settings are changed | Critical | yarn | Sumana Sathish | Jian He | | | ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy | Major | client | Jason Lowe | Jason Lowe | | | MR2 can't run local jobs with -libjars command options which is a regression from MR1 | Critical | mrv2 | zhihai xu | zhihai xu | | | DFSClient#getServerDefaults returns null within 1 hour of system start | Blocker | . | Xiaoyu Yao | Xiaoyu Yao | | | Using monotonicNow for block report scheduling causes test failures on recently restarted systems | Blocker | datanode | Arpit Agarwal | Arpit Agarwal | | | Invalid user logins trigger large backtraces in server log | Major | . | Chang Li | Chang Li | | | DistributedShell uses the wrong user to put timeline data | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | | | Mover should not schedule two replicas to the same DN storage | Major | balancer & mover | Surendra Singh Lilhore | Surendra Singh Lilhore | | | "hadoop dfs" command prints message about using "yarn jar" on Windows(branch-2 only) | Minor | scripts | Varun Vasudev | Varun Vasudev | | | Regression: s3n read failure recovery broken | Major | fs/s3 | Takenori Sato | Takenori Sato | | | killing ContainerLocalizer action doesn't take effect when private localizer receives FETCH\_FAILURE status. | Minor | nodemanager | zhihai xu | zhihai xu | | | DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm | Major | . | Eric Payne | Colin P. McCabe | | | Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode | Blocker | caching | Gopal V | Colin P. McCabe | | | NPE when

fails and stopRecoveryStore invoked | Major | nodemanager | Brahma Reddy Battula | Brahma Reddy Battula | | | Race condition in LocalizerRunner kills localizer before localizing all resources | Critical | nodemanager | zhihai xu | zhihai xu | | | JobHistoryServer should not fail when encountering a missing directory | Major | jobhistoryserver | Craig Welch | Craig Welch | | | Uber jobs fail to update AMRM token when it rolls over | Blocker | mr-am | Jason Lowe | Jason Lowe | | | Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler | Blocker | . | Eric Payne | Eric Payne | | | FSNamesystem#Delete() should not call logSync() when holding the lock | Blocker | namenode | Jing Zhao | Haohui Mai | | | FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies | Critical | fairscheduler | Karthik Kambatla | Karthik Kambatla | | | getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime | Blocker | . | Yesha Vora | Haohui Mai | | | Job history file is not flushed correctly because isTimerActive flag is not set true when flushTimerTask is scheduled. | Critical | mrv2 | zhihai xu | zhihai xu | | | OsSecureRandom should lazily fill its reservoir | Major | security | Arun Suresh | Arun Suresh | | | DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace | Critical | . | Billie Rinaldi | Colin P. McCabe | | | SnappyCodec#checkNativeCodeLoaded may unexpectedly fail when native code is not loaded | Major | . | Zhijie Shen | Zhijie Shen | | | IllegalArgumentException due to missing job submit time | Major | jobhistoryserver | zhihai xu | zhihai xu | | | HDFS INotify: the destination field of RenameOp should always end with the file name | Major | . | Colin P. McCabe | Colin P. McCabe | | | Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion. | Critical | resourcemanager | zhihai xu | zhihai xu | | | Incremental BlockReport will dramatically slow down the startup of a namenode | Major | . | Hui Zheng | Walter Su | | | Audit log not present while refreshing Service ACLs | Minor | resourcemanager | Varun Saxena | Varun Saxena | | | Default value for maximum nodemanager connect wait time is too high | Major | . | Jason Lowe | Naganarasimha G R | | | Rolling upgrade readiness is not updated in jmx until query command is issued. | Critical | . | Kihwal Lee | Brahma Reddy Battula | | | Standby namenode doesn't process DELETED\_BLOCK if the add block request is in edit log. | Major |

| Rushabh S Shah | Rushabh S Shah | | | Nodemanager can fail to delete local logs if log aggregation fails | Major | log-aggregation, nodemanager | Jason Lowe | Rohith Sharma K S | | | Remove description about Java 6 from docs | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop | Critical | datanode | Vinayakumar B | Rushabh S Shah | | | Interaction between reservations and userlimit can result in significant ULF violation | Major | capacityscheduler | Thomas Graves | Thomas Graves | | | NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host | Critical | . | Junping Du | Junping Du | | | JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases | Major | jobhistoryserver, mrv2 | Craig Welch | Craig Welch | | | Prevent multiple balancers from running simultaneously | Critical | balancer & mover | Rakesh R | Rakesh R | | | Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. | Critical | scripts | Chris Nauroth | Chris Nauroth | | | ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster | Major | resourcemanager, webapp | Weiwei Yang | Weiwei Yang | | | Fix a typo in NamenodeFsck | Minor | namenode | Tsz Wo Nicholas Sze | Takanobu Asanuma | | | Fix UT TestRMFailover.testRMWebAppRedirect | Critical | resourcemanager, webapp | Weiwei Yang | Weiwei Yang | | | Pending block replication can get stuck using older genstamp | Major | namenode | Nathan Roberts | Nathan Roberts | | | Ensure ZkDelegationTokenSecretManager namespace znodes get created with ACLs | Major | security | Gregory Chanan | Gregory Chanan | | | Fix findbugs warnings in yarn-server-resourcemanager | Minor | resourcemanager | Akira Ajisaka | Vinod Kumar Vavilapalli | | | yarn cmd says "could not find main class 'queue'" in windows | Blocker | yarn | Sumana Sathish | Varun Saxena | | | Fix dead link for TimelineServer REST API | Minor | documentation | Akira Ajisaka | Jagadesh Kiran N | | | Applications are getting stuck some times in case of retry policy forever | Major | client | Raju Bairishetti | Raju Bairishetti | | | DFSClient probe for encryption testing interprets empty URI property for "enabled" | Blocker | encryption | Steve Loughran | Steve Loughran | | | FairScheduler: RM quits when node removal races with continousscheduling on the same node | Critical | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | | | Isolating the error of generating a single app report when getting all apps from generic history service | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | | | filtering on UI sticks even if I move away from the page | Major | webapp | Sangjin Lee | Jian He | | | On Windows localized resources are not moved to the front of the classpath when they should be | Major | yarn | Craig Welch | Craig Welch | | | Need to clearly document primaryFilter and otherInfo value type | Critical | timelineserver | Zhijie Shen | Zhijie Shen | | | Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop | Blocker | security | Mike Yoder | Larry McCay | | | App submission via REST API is broken in secure mode due to Timeline DT service address is empty | Blocker | resourcemanager, timelineserver | Zhijie Shen | Zhijie Shen | | | RM failed to come up with

Wait for ZKClient creation timed out" | Critical | yarn | Sumana Sathish | Jian He | | | DN startup may cause severe data loss | Blocker | datanode | Daryn Sharp | Daryn Sharp | | | create() always retried with hardcoded timeout when file already exists with open lease | Major | hdfs-client | Andrey Stepachev | J.Andreina | | | Remove usage information on unsupported operation "fsck -showprogress" from branch-2 | Major | documentation | J.Andreina | J.Andreina | | | NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled | Critical | . | Peng Zhang | Rohith Sharma K S | | | Fix dead links to DistCp and Hadoop Archives pages. | Minor | documentation, site | Kazuho Fujii | Kazuho Fujii | | | CapacityScheduler should forbid moving LeafQueue from one parent to another | Blocker | . | Wangda Tan | Wangda Tan | | | Fix DominantRC#compare() does not work as expected if cluster resource is empty | Blocker | resourcemanager | Bibin A Chundatt | Rohith Sharma K S | | | Serialize the recently added Task#encryptedSpillKey field at the end | Minor | . | Arun Suresh | Arun Suresh | | | Change heavily recorded NN logs from INFO to DEBUG level | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | | | HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands | Major | documentation | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Document that NFS gateway does not work with rpcbind on SLES 11 | Major | documentation | Arpit Agarwal | Arpit Agarwal | | | DN always uses HTTP/localhost@REALM principals in SPNEGO | Blocker | . | Haohui Mai | Haohui Mai | | | TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute | Blocker | datanode | Yongjun Zhang | Yongjun Zhang | | | TestCommitBlockSynchronization fails in branch-2.7 | Major | test | Arpit Agarwal | Arpit Agarwal | | | TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7 | Major | webhdfs | Arpit Agarwal | Arpit Agarwal | | | The default retry policy does not handle RetriableException correctly | Critical | ipc | Arpit Agarwal | Arpit Agarwal | | | Lease recovery should return true if the lease can be released and the file can be closed | Major | namenode | J.Andreina | J.Andreina | | | Incorrect port specified in HFTP Guide document in branch-2 | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | | | Block corruption can happen during pipeline recovery | Blocker | datanode | Peng Zhang | Kihwal Lee | | | Both RM are on standBy state when kerberos user not in yarn.admin.acl | Critical | resourcemanager | Bibin A Chundatt | Varun Saxena | | | ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission | Major | . | Robert Kanter | Bibin A Chundatt | | | Fix setting of dfs.datanode.readahead.bytes in

to match DFSConfigKeys | Minor | datanode | Ray Chiang | Ray Chiang | | | Reserved RBW space is not released if creation of RBW File fails | Blocker | . | Kanaka Kumar Avvaru | Kanaka Kumar Avvaru | | | Small refactoring of DelegationTokenAuthenticationFilter to allow code sharing | Minor | security | Yongjun Zhang | Yongjun Zhang | | | Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them | Critical | . | Zhe Zhang | Zhe Zhang | | | NMProxy should retry on NMNotYetReadyException | Critical | . | Karthik Kambatla | Robert Kanter | | | Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings in secure cluster | Critical | . | Zhang Wei | Varun Saxena | | | Failed to launch new attempts because ApplicationMasterLauncher's threads all hang | Major | resourcemanager | Jun Gong | Jun Gong | | | Resource Localization fails on a cluster due to existing cache directories | Critical | nodemanager | Ranga Swamy | Brahma Reddy Battula | | | NM fails to read files from full disks which can lead to container logs being lost and other issues | Blocker | log-aggregation, nodemanager | Varun Saxena | Varun Saxena | | | BlockScanner is incorrectly disabled by default | Blocker | datanode | Andrew Wang | Arpit Agarwal | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue | Critical | capacityscheduler | Jason Lowe | Jason Lowe | | | [RM HA] Rest api endpoints doing redirect incorrectly | Major | resourcemanager | bc Wong | Xuan Gong | | | AM logs link missing in the RM UI for a completed app | Blocker | . | Hitesh Shah | Xuan Gong | | | Improve the error message when attempting manual failover with auto-failover enabled | Minor | . | Akira Ajisaka | Akira Ajisaka | | | Fix the format issue of the new RM web UI and AHS web UI after YARN-3272 / YARN-3262 | Major | resourcemanager | Xuan Gong | Xuan Gong | | | Don't fail RM if queue's configured labels are not existed in cluster-node-labels | Major | resourcemanager | Rohith Sharma K S | Wangda Tan | | | Move load labels from storage from serviceInit to serviceStart to make it works with RM HA case. | Major | resourcemanager | Wangda Tan | Wangda Tan | | | CapacityScheduler should trim default\node\label\_expression | Critical | api, client, resourcemanager | Wangda Tan | Sunil Govindan | | | Application (Attempt and Container) Not Found in AHS results in Internal Server Error (500) | Major | timelineserver | Jonathan Eagles | Mit Desai | | | Documentation of ResourceManager HA should explain configurations about listen addresses | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | | | Fix TestFSImage#testZeroBlockSize on Windows | Major | datanode, test | Xiaoyu Yao | Xiaoyu Yao | | | Feature documentation for HDFS-6581 | Major | documentation | Arpit Agarwal | Arpit Agarwal |