Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.1.2.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | HDFS Block Placement - Ignore Locality for First Block Replica | Minor | block placement, hdfs-client | BELUGA BEHR | BELUGA BEHR | | | WASB: Add support for StreamCapabilities | Major | fs/azure | Thomas Marquardt | Thomas Marquardt | | | Plugin interface to support more complex usernames in Hadoop | Major | security | Eric Yang | Bolke de Bruin | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve anti-affinity section description in YARN Service API doc | Major | docs, documentation | Charan Hebri | Gour Saha | | | Retry KMS calls when SSLHandshakeException occurs | Major | common, kms | Kitti Nanasi | Kitti Nanasi | | | Improve exception when tfile fails to load LzoCodec | Major | . | Gera Shegalov | Gera Shegalov | | | make DEFAULT\MAX\CORRUPT\FILEBLOCKS\RETURNED configurable | Minor | hdfs | Lantao Jin | Lantao Jin | | | Log full stack trace if DiskBalancer exits with an unhandled exception | Minor | diskbalancer | Stephen O'Donnell | Gabor Bota | | | Several typos in Log Aggregation related classes | Minor | . | Szilard Nemeth | Szilard Nemeth | | | Disk Balancer should not fail if volume usage is greater than capacity | Minor | diskbalancer | Stephen O'Donnell | Stephen O'Donnell | | | Fix Typos - Node Not Chosen | Trivial | namenode | BELUGA BEHR | BELUGA BEHR | | | Print ExecutionType in Container report CLI | Minor | . | Bilwa S T | Bilwa S T | | | Expose HighestPriorityLowRedundancy blocks statistics | Major | hdfs | Kitti Nanasi | Kitti Nanasi | | | Replace the deprecated zk-address property in the HA config example in ResourceManagerHA.md | Minor | yarn | Antal Blint Steinbach | Antal Blint Steinbach | | | Make QJM HTTP URL connection timeout configurable | Minor | qjm | Chao Sun | Chao Sun | | | Remove super user privilege requirement for NameNode.getServiceStatus | Minor | namenode | Chao Sun | Chao Sun | | | Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint | Major | resourcemanager | Anna Savarin | Weiwei Yang | | | Exit NameNode if dangling child inode is detected when saving FsImage | Major | hdfs, namenode | Siyao Meng | Siyao Meng | | | Expose SecurityEnabled boolean field in JMX for other services besides NameNode | Minor | . | Ray Burgemeestre | Adam Antal | | | Audit log all EC policy names during addErasureCodingPolicies | Major | erasure-coding | liaoyuxiangqin | liaoyuxiangqin | | | Create a new touch command to allow modifying atime and mtime | Minor | tools | Brian Burton | Hrishikesh Gadre | | | YARN NM: OOM error while reading back the state store on recovery | Critical | yarn | Kanwaljeet Sachdev | Pradeep Ambati | | | RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache | Major | hdfs | Fei Hui | Fei Hui | | | RBF: Illegal Router Admin command leads to printing usage for all commands | Major | . | Ayush Saxena | Ayush Saxena | | | Make block increment deletion number configurable | Major | . | Yiqun Lin | Ryan Wu | |

TestRMEmbeddedElector#testCallbackSynchronization is flakey | Major | test | Robert Kanter | Robert Kanter | | | RBF: Choose to enable the default nameservice to read/write files | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | Fix the inconsistent default refresh interval on Caching documentation | Trivial | documentation | BELUGA BEHR | Hrishikesh Gadre | | | Allow linux container runtimes to be pluggable | Minor | nodemanager | Craig Condit | Craig Condit | | | Add JMX, conf and stacks menus to the datanode page | Minor | datanode | fengchuang | fengchuang | | | YARN NM: Implement Iterable Abstraction for LocalResourceTracker state | Critical | yarn | Pradeep Ambati | Pradeep Ambati | | | Create utility to limit frequency of log statements | Major | common, util | Erik Krogen | Erik Krogen | | | Allow updating application tracking url after registration | Major | . | Jonathan Hung | Jonathan Hung | | | Support getting PreemptionMessage when using AMRMClientAsync | Major | yarn | Krishna Kishore | Zian Chen | | | Limit the maximum number of container assignments per heartbeat | Major | . | Weiwei Yang | Zhankun Tang | | | upgrade to commons-compress 1.18 | Major | . | PJ Fanning | Akira Ajisaka | | | Fix errors in yarn-default.xml related to GPU/FPGA | Major | . | Zhankun Tang | Zhankun Tang | | | make storageId in BlockPoolTokenSecretManager.checkAccess optional | Major | . | Ajay Kumar | Ajay Kumar | | | Sleep in TestLazyPersistFiles should be put into a loop | Trivial | hdfs | Adam Antal | Adam Antal | | | Update the doc about the default value of "maximum-container-assignments" for capacity scheduler | Minor | . | Zhankun Tang | Zhankun Tang | | | Review hadoop credential doc, including object store details | Minor | documentation, security | Steve Loughran | Steve Loughran | | | Add queue and partition info to RM audit log | Major | resourcemanager | Jonathan Hung | Eric Payne | | | AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting | Major | . | Wanqiang Ji | Wanqiang Ji | | | Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode | Trivial | . | Wanqiang Ji | Wanqiang Ji | | | Refactor NameNodeWebHdfsMethods to allow better extensibility | Major | . | CR Hota | CR Hota | | | distcp documentation is woefully out of date | Critical | documentation, tools/distcp | Allen Wittenauer | Dinesh Chitlangia | | | AliyunOSS: Enable Yarn to use OSS | Major | fs/oss | wujinhu | wujinhu | | | AliyunOSS: add missing owner & group attributes for oss FileStatus | Major | fs/oss | wujinhu | wujinhu | | | FileOutputCommitter does not report progress while merging paths. | Major | . | Kuhu Shukla | Kuhu Shukla | | | Fix SchedulerInfo#getSchedulerType for custom schedulers | Minor | . | Bilwa S T | Bilwa S T | | | EC: Track Erasure Coding commands in DFS statistics | Major | erasure-coding | Ayush Saxena | Ayush Saxena | | | Escape newlines in health report in YARN UI | Major | . | Jonathan Hung | Keqiu Hu | | | Add Guaranteed and MaxCapacity to CSQueueMetrics | Major |

| Jonathan Hung | Jonathan Hung | | | Harden Token service loader use | Major | security | Steve Loughran | Steve Loughran | | | Add table of contents to YARN Service API document | Minor | documentation | Akira Ajisaka | Zhankun Tang | | | Performance improvement in Tailing EditLog | Major | namenode | Kenneth Yang | Kenneth Yang | | | revert HADOOP-12751 | Minor | security | Steve Loughran | Steve Loughran | | | ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration | Minor | common | luhuachao | luhuachao | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list' | Blocker | federation | Yiran Wu | Yiran Wu | | | Upgrade jquery-ui to 1.12.1 in YARN | Major | webapp | Sunil Govindan | Sunil Govindan | | | Avoid static initialization of ObjectMappers | Minor | performance | Todd Lipcon | Todd Lipcon | | | NPE in DataNode due to uninitialized DiskBalancer | Major | datanode, diskbalancer | Xiao Chen | Xiao Chen | | | Yarn service conflict between restart policy and NM configuration | Critical | yarn | Chandni Singh | Suma Shivaprasad | | | Support bind propagation options for mounts in docker runtime | Major | . | Billie Rinaldi | Billie Rinaldi | | | [DS] AM registration fails when hadoop authorization is enabled | Blocker | . | Bibin A Chundatt | Bibin A Chundatt | | | AllocationRespose proto setNMToken initBuilder not done | Major | . | Bibin A Chundatt | Bilwa S T | | | TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events | Major | capacityscheduler | Haibo Chen | Weiwei Yang | | | Fix the broken anchor in SLS site-doc | Minor | documentation | Weiwei Yang | Weiwei Yang | | | CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps | Critical | capacity scheduler, capacityscheduler | Karam Singh | Manikandan R | | | Avoid publishing reserved container to ATS from RM | Critical | yarn-native-services | Yesha Vora | Suma Shivaprasad | | | Improve diagnostic message when artifact is not set properly | Major | . | Yesha Vora | Gour Saha | | | Validate service principal format prior to launching yarn service | Major | security, yarn | Eric Yang | Eric Yang | | | LocalFs#listLocatedStatus does not filter out hidden .crc files | Minor | fs | Erik Krogen | Erik Krogen | | | New AM attempt could not retrieve previous attempt component data | Critical | . | Yesha Vora | Gour Saha | | | Potential thread leak in ActivitiesManager | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | [UI2] Container diagnostic information is missing from container page | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | Nodemanager logs failed to download file with INFO level | Major | yarn | Eric Yang | Eric Yang | | | Yarn Service Upgrade: Typo in Error message | Major | . | Chandni Singh | Chandni Singh | | | Add RM web service endpoint to get user information | Major | resourcemanager | Akhil PB | Akhil PB | | | [UI2] Display current logged in user | Major | . | Akhil PB | Akhil PB | | | [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome browser | Major |

| Akhil S Naik | Akhil PB | | | [UI2] IP address in component page shows N/A | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | NodeManager metrics may return wrong negative values when NM restart | Major | . | Yang Wang | Szilard Nemeth | | | [UI2] YARN Services UI new submission failures are not debuggable | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu. | Major | yarn-ui-v2 | Sumana Sathish | Akhil PB | | | Fix FSN read/write lock metrics name | Trivial | documentation, metrics | Chao Sun | Chao Sun | | | Container cleanup fails while trying to delete Cgroups | Critical | . | Yesha Vora | Suma Shivaprasad | | | Container launch exception in AM log should be printed in ERROR level | Major | . | Yesha Vora | Yesha Vora | | | TestEditLogTailer#testTriggersLogRollsForAllStandbyNN fails due to missing synchronization between rollEditsRpcExecutor and tailerThread shutdown | Minor | ha | Hrishikesh Gadre | Hrishikesh Gadre | | | EC: Display erasure coding policy for sub-directories is not working | Major | erasure-coding | Souryakanta Dwivedy | Ayush Saxena | | | EC: "removePolicy" is not working for built-in/system Erasure Code policies | Minor | documentation, erasure-coding | Souryakanta Dwivedy | Ayush Saxena | | | Update DataTables version in yarn-common in line with JQuery 3 upgrade | Major | yarn | Akhil PB | Akhil PB | | | Race condition in NM container launched after done | Major | . | Yang Wang | Pradeep Ambati | | | NPE in AllocationTagsManager when a container is removed more than once | Major | resourcemanager | Weiwei Yang | Weiwei Yang | | | Avoid committing allocation proposal to unavailable nodes in async scheduling | Major | capacityscheduler | Tao Yang | Tao Yang | | | FSPermissionChecker may throws AIOOE when check inode permission | Major | namenode | He Xiaoqiao | He Xiaoqiao | | | KMS Accept Queue Size default changed from 500 to 128 in Hadoop 3.x | Major | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | NameNode UI : "Utilities -\> Browse the file system -\> open a file -\> Head the file" is not working | Major | ui | Nanda kumar | Nanda kumar | | | fsck -list-corruptfileblocks has infinite loop if user is not privileged. | Major | tools | Wei-Chiu Chuang | Yuen-Kuei Hsueh | | | DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction | Major | namenode | Wei-Chiu Chuang | chencan | | | Fix few annotation typos in YarnConfiguration | Trivial | . | Sen Zhao | Sen Zhao | | | TestDirectoryScanner#testDirectoryScannerInFederatedCluster is flaky | Minor | hdfs | Daniel Templeton | Daniel Templeton | | | container-executor should not write cgroup tasks files for docker containers | Major | . | Jim Brennan | Jim Brennan | | | sleeper service fails to launch with "Authentication Required" | Critical | yarn | Sumana Sathish | Billie Rinaldi | | | Cleanup symlinks when container restarted by NM to solve issue "find: File system loop detected;" for tar ball artifacts. | Critical | . | Rohith Sharma K S | Chandni Singh | | | Race between close/recoverLease leads to missing block | Major |

| zhouyingchao | Jinglun | | | Enhance KMS client retry behavior | Critical | kms | Kitti Nanasi | Kitti Nanasi | | | Fix NM Collector Service Port issue in YarnConfiguration | Major | ATSv2 | Prabha Manepalli | Prabha Manepalli | | | Statistic for list\located\status is incremented incorrectly by listStatusIterator | Minor | hdfs-client | Todd Lipcon | Antal Mihalyi | | | Test failure TestSSLHttpServer.testExcludedCiphers with TLS\ECDHE\RSA\WITH\AES\128\CBC\_SHA256 cipher suite | Major | common | Gabor Bota | Szilard Nemeth | | | Restore previous state in container-executor after failure | Major | . | Jim Brennan | Jim Brennan | | | [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked | Major | . | Rohith Sharma K S | Wangda Tan | | | Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled | Trivial | erasure-coding | Souryakanta Dwivedy | Ayush Saxena | | | NPE in localizer hearbeat processing if a container is killed while localizing | Major | . | lujie | lujie | | | Typo correction for yarn configuration in OpportunisticContainers(federation) docs | Major | documentation, federation | Y. SREENIVASULU REDDY | Y. SREENIVASULU REDDY | | | Setting hostname of docker container breaks with "host" networking mode for Apps which do not run as a YARN service | Major | . | Yesha Vora | Suma Shivaprasad | | | RBF: Add check to have single valid argument to safemode command | Major | federation | Soumyapn | Ayush Saxena | | | ReencryptionUpdater fails with ConcurrentModificationException during processCheckpoints | Major | encryption | Xiao Chen | Zsolt Venczel | | | Fix a typo in CS init error message when resource calculator is not correctly set | Minor | . | Weiwei Yang | Abhishek Modi | | | Typo in the definition of "stable" in the interface classification | Minor | . | Daniel Templeton | Daniel Templeton | | | FsDatasetImpl should log DiskOutOfSpaceException | Major | hdfs | Fei Hui | Fei Hui | | | KMS log4j is not initialized properly at startup | Major | kms | Kitti Nanasi | Kitti Nanasi | | | ITestNativeAzureFileSystemConcurrencyLive times out | Major | . | Andras Bokor | Andras Bokor | | | Typo in compatibility doc: SHOUD -\> SHOULD | Trivial | . | Daniel Templeton | Laszlo Kollar | | | Handle possible NPEs due to deleted blocks in race condition | Major | namenode | Vinayakumar B | Vinayakumar B | | | Fix DistributedShell unit tests | Major | distributed-shell, timelineservice | Eric Yang | Abhishek Modi | | | RBF: Add validation for max arguments for Router admin ls, clrQuota, setQuota, rm and nameservice commands | Major | . | Ayush Saxena | Ayush Saxena | | | EC: "hdfs ec -getPolicy" is not retrieving policy details when the special REPLICATION policy set on the directory | Minor | erasure-coding | Souryakanta Dwivedy | Ayush Saxena | | | ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns | Major | ipc | Steve Loughran | Kihwal Lee | | | RBF: Add check to order command | Major | federation | Soumyapn | Ranith Sardar | | | KMS performance regression due to too many open file descriptors after Jetty migration | Blocker | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate | Major |

| Anthony Hsu | Anthony Hsu | | | RBF: Handle mount table znode with null value | Major | federation, hdfs | yanghuafeng | yanghuafeng | | | Container-executor permission check errors cause the NM to be marked unhealthy | Critical | . | Shane Kumpf | Craig Condit | | | RBF: Router logs are not capturing few of the dfsrouteradmin commands | Major | . | Soumyapn | Ayush Saxena | | | 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available | Major | datanode | usharani | Ranith Sardar | | | EC: Fix Intermittent Failure in TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | Major | erasure-coding | Ayush Saxena | Ayush Saxena | | | CS preemption monitor always fails since one under-served queue was deleted | Major | capacityscheduler, scheduler preemption | Tao Yang | Tao Yang | | | Fix dead lock during async editlog rolling if edit queue is full | Major | namenode | zhangwei | Daryn Sharp | | | ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster when ACls are enabled | Major | . | Rohith Sharma K S | Rohith Sharma K S | | | Node status updater thread could be lost after it is restarted | Critical | nodemanager | Tao Yang | Tao Yang | | | Fix DN UI logs link broken when https is enabled after HDFS-13902 | Minor | datanode | Jianfei Jiang | Jianfei Jiang | | | History Server task attempts REST API returns invalid data | Major | jobhistoryserver | Oleksandr Shevchenko | Oleksandr Shevchenko | | | WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status | Major | hdfs, webhdfs | Siyao Meng | Siyao Meng | | | Correct the log when Invalid emptier Interval configured | Major | trash | Harshakiran Reddy | Ayush Saxena | | | CapacityScheduler does not enforce max resource allocation check at queue level | Major | capacity scheduler, capacityscheduler, resourcemanager | Tarun Parimi | Tarun Parimi | | | Fix exception message in Resource.throwExceptionWhenArrayOutOfBound | Minor | . | Szilard Nemeth | Gergely Pollak | | | Fix the fmt\_bytes function in the dfs-dust.js | Minor | hdfs, ui | yanghuafeng | yanghuafeng | | | [UI2] YARN UI2 is not accessible when config.env file failed to load | Critical | yarn-ui-v2 | Akhil PB | Akhil PB | | | [UI2] YARN UI2 breaks if getUserInfo REST API is not available in older versions. | Critical | . | Akhil PB | Akhil PB | | | Fix broken list items in PlacementConstraints documentation | Minor | documentation | Masahiro Tanaka | Masahiro Tanaka | | | Arrays.fill() wrong index in BlockSender.readChecksum() exception handling | Major | datanode | Tao Zhang | Tao Zhang | | | Container cgroups are leaked when using docker | Major | . | Jim Brennan | Jim Brennan | | | StringUtils#createStartupShutdownMessage throws NPE when args is null | Major | . | Lokesh Jain | Dinesh Chitlangia | | | org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently | Major | mrv2 | Robert Joseph Evans | Jason Lowe | | | MRAppBenchmark.benchmark1() fails with NullPointerException | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko | | | ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko | | | HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit | Major | httpfs | Siyao Meng | Siyao Meng | | | WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is

| Major | hdfs, webhdfs | Siyao Meng | Pranay Singh | | | CapacityScheduler fails to unreserve when cluster resource contains empty resource type | Critical | capacityscheduler | Tao Yang | Tao Yang | | | TestDataNodeMultipleRegistrations is flaky | Major | . | igo Goiri | Ayush Saxena | | | triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. | Critical | ha | Rong Tang | Rong Tang | | | Remove the 'Path ... should be specified as a URI' warnings on startup | Major | conf | Arpit Agarwal | Ayush Saxena | | | DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk | Major | nodemanager | Jason Lowe | Eric Badger | | | Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. | Major | . | Ayush Saxena | Ayush Saxena | | | [UI2] Few duplicated or inconsistent information displayed in UI2 | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | [UI2] Container logs on Application / Service pages on UI2 are not available many case, improve error messages in such cases. | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | JobResourceUploader creates LocalFileSystem when it's not necessary | Major | job submission | Peter Cseh | Peter Cseh | | | RM fails to recover finished unmanaged AM | Critical | . | Rakesh Shah | Bibin A Chundatt | | | yarn-registry.md has wrong word ong-lived, it should be long-lived | Trivial | documentation | leiqiang | leiqiang | | | App Nodelabel missed after RM restart for finished apps | Major | . | Bibin A Chundatt | Bibin A Chundatt | | | RBW Blocks which are having less GS should be added to Corrupt | Minor | . | Brahma Reddy Battula | Brahma Reddy Battula | | | resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Memory leak when CapacityScheduler allocates from reserved container with non-default label | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Reuse Object Mapper in KMSJSONReader | Major | kms | Jonathan Eagles | Jonathan Eagles | | | TestNMProxy unit test is failing | Major | yarn | Eric Yang | Eric Yang | | | Fix incorrect option used in description of InMemoryAliasMap | Minor | documentation | Yiqun Lin | Yiqun Lin | | | [UI2] YARN service delete option disappears after stopping application | Critical | yarn-ui-v2 | Yesha Vora | Akhil PB | | | [UI2] Improve error pages in new YARN UI | Major | yarn-ui-v2 | Akhil PB | Akhil PB | | | ZStandardDecompressor native code sets an integer field as a long | Blocker | . | Jason Lowe | Jason Lowe | | | RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice | Major | . | Ayush Saxena | Ayush Saxena | | | [UI2] Maintain constant horizontal application info bar for Application Attempt page | Major | yarn-ui-v2 | Yesha Vora | Akhil PB | | | Adding replicas to volume map makes DataNode start slowly | Major | . | Yiqun Lin | Surendra Singh Lilhore | | | Add null check for add-replica pool to avoid lock acquiring | Major | . | Yiqun Lin | Surendra Singh Lilhore | | | [UI2] Application Attempts tab is not shown correctly when there are no attempts | Major | . | Charan Hebri | Akhil PB | | | hadoop.registry.rm.enabled is not used | Major |

| igo Goiri | igo Goiri | | | ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads | Major | erasure-coding | Xiao Chen | Hrishikesh Gadre | | | [UI2] Remove application tab from YARN Queue Page | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | [UI2] Lost nodes representation missing from Nodemanagers Chart | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora | | | ShutdownHookManager shutdown time needs to be configurable & extended | Major | util | Steve Loughran | Steve Loughran | | | TestDataNodeVolumeFailure is Flaky | Major | . | Ayush Saxena | Ayush Saxena | | | RBF: Remove FSCK from Router Web UI | Major | . | Fei Hui | Fei Hui | | | YARN Service Client might not work correctly with RM REST API for Kerberos authentication | Blocker | . | Eric Yang | Eric Yang | | | JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used | Major | client, mrv2 | Peter Bacsko | Peter Bacsko | | | YARN UI2 doc changes to update security status (verified under security environment) | Blocker | . | Sunil Govindan | Sunil Govindan | | | Yarn Service: discrepancy between hashcode and equals of ConfigFile | Minor | . | Chandni Singh | Chandni Singh | | | start-build-env.sh creates an invalid /etc/sudoers.d/hadoop-build-${USER\_ID} file entry | Minor | build | Jon Boone | Jon Boone | | | Move DelegationTokenIssuer to the right path | Blocker | security | Wei-Chiu Chuang | Wei-Chiu Chuang | | | ZStandardDecompressor.c mistakes a class for an instance | Blocker | . | Ben Lau | Jason Lowe | | | RBF: Documentation should reflect right scripts for v3.0 and above | Major | . | CR Hota | CR Hota | | | Set HTTPOnly attribute to Cookie | Major | . | Chandni Singh | Chandni Singh | | | Fix findbugs warning in trunk for FSImageFormatPBINode | Major | . | Yiqun Lin | Yiqun Lin | | | RBF: Web UI update to bootstrap-3.3.7 | Major | . | igo Goiri | igo Goiri | | | Misleading log statement in NM when max retries is -1 | Minor | . | Chandni Singh | Chandni Singh | | | CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0 | Critical | tools/distcp | Ted Yu | Ted Yu | | | Nodemanager restart docs should describe nodemanager supervised property | Major | documentation | Jason Lowe | Suma Shivaprasad | | | Renamed HADOOP\SECURITY\GROUP\SHELL\COMMAND\_TIMEOUT keys break compatibility | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Fix lingering timeline collector after serviceStop in TimelineCollectorManager | Trivial | ATSv2 | Prabha Manepalli | Prabha Manepalli | | | zstd compressor can fail with a small output buffer | Major | . | Jason Lowe | Jason Lowe | | | TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently | Major | erasure-coding, test | Xiao Chen | Xiao Chen | | | RMContainerAllocator#handleJobPriorityChange expects application\_priority always | Major |

| Bibin A Chundatt | Bilwa S T | | | Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances | Major | kms | Wei-Chiu Chuang | Xiao Chen | | | HDFS OIV temporary dir deletes folder | Major | hdfs | Adam Antal | Adam Antal | | | DFSStripedOutputStream should implement both hsync methods | Critical | erasure-coding | Xiao Chen | Xiao Chen | | | Compilation fails with dependency convergence error for hbase.profile=2.0 | Blocker | . | Rohith Sharma K S | Rohith Sharma K S | | | Update AWS Java SDK versions in NOTICE.txt | Major | . | Akira Ajisaka | Akira Ajisaka | | | Update JSch versions in LICENSE.txt | Major | . | Akira Ajisaka | Akira Ajisaka | | | Tolerate corrupted seen\_txid file | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak | | | CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. | Major | . | Sumana Sathish | Wangda Tan | | | Improve the debug message in CS#allocateContainerOnSingleNode | Trivial | . | Weiwei Yang | Zhankun Tang | | | RMStateStore contains large number of expired RMDelegationToken | Major | resourcemanager | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | DFSOutputStream close() throws exception on subsequent call after DataNode restart | Major | hdfs-client | Erik Krogen | Erik Krogen | | | NullPointerException when reaching max shuffle connections | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | Fix a parsing error for crossdomain.xml | Major | build, yarn-ui-v2 | Takanobu Asanuma | Takanobu Asanuma | | | ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3) | Major | erasure-coding | Xiao Chen | Kitti Nanasi | | | Upgrade Maven Surefire plugin to 3.0.0-M1 | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | YARN Service keytab does not support s3, wasb, gs and is restricted to HDFS and local filesystem only | Major | yarn-native-services | Gour Saha | Gour Saha | | | NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null | Critical | capacityscheduler | Tao Yang | Tao Yang | | | Failed Storage Locations shows nothing in the Datanode Volume Failures | Major | . | Ayush Saxena | Ayush Saxena | | | create-release script should set max-cache-ttl as well as default-cache-ttl for gpg-agent | Blocker | build | Akira Ajisaka | Akira Ajisaka | | | start-build-env.sh still creates an invalid /etc/sudoers.d/hadoop-build-${USER\_ID} file entry after HADOOP-15802 | Major | build | Akira Ajisaka | Akira Ajisaka | | | BlockDecompressorStream#decompress should not return -1 in case of IOException. | Major | . | Surendra Singh Lilhore | Surendra Singh Lilhore | | | Inefficient Flush Logic in JobHistory EventWriter | Major | . | Zichen Sun | Zichen Sun | | | Exclude MD5 checksum files from release artifact | Critical | build | Akira Ajisaka | Akira Ajisaka | | | The config and log of gpg-agent are removed in create-release script | Major | build | Akira Ajisaka | Dinesh Chitlangia | | | NN UI is broken with IE11 | Minor | namenode, ui | Daisuke Kobayashi | Ayush Saxena | | | Fix error messages in HDFS-12716 | Minor | hdfs | Adam Antal | Ayush Saxena | | | TestEvents#testEvents fails | Critical | jobhistoryserver, test | Zhaohui Xin | Zhaohui Xin | | | Yarn Service Upgrade: Instance state changes from UPGRADING to READY without performing a readiness check | Critical | . | Chandni Singh | Chandni Singh | | | Containers fail during creating a symlink which started with hyphen for a resource file | Minor | . | Oleksandr Shevchenko | Oleksandr Shevchenko | | | YARN Resource Manager is running OOM because of leak of Configuration Object | Major | yarn-native-services | Eric Yang | Eric Yang | | |

is misformatted in single node setup document | Major | documentation | Akira Ajisaka | Zhaohui Xin | | | Upgrade plexus-utils from 2.0.5 to 3.1.0 | Major | security | Akira Ajisaka | Akira Ajisaka | | | Upgrade Curator version to 2.13.0 to fix ZK tests | Major | . | Jason Lowe | Akira Ajisaka | | | NM and service AM don't have updated status for reinitialized containers | Critical | . | Billie Rinaldi | Chandni Singh | | | FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used | Major | mrv2 | Peter Bacsko | Peter Bacsko | | | Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs | Minor | . | OrDTesters | OrDTesters | | | Doc typo in PluggableShuffleAndPluggableSort.md | Minor | documentation | Zhaohui Xin | Zhaohui Xin | | | LevelDBCacheTimelineStore in ATS 1.5 leaks native memory | Major | timelineserver | Tarun Parimi | Tarun Parimi | | | Service Upgrade: With default readiness check, the status of upgrade is reported to be successful prematurely | Major | . | Chandni Singh | Chandni Singh | | | Ls command with e option fails when the filesystem is not HDFS | Major | erasure-coding, tools | Takanobu Asanuma | Takanobu Asanuma | | | Configuration: Included properties are not cached if resource is a stream | Critical | . | Eric Payne | Eric Payne | | | Fix itemization in YARN service quickstart document | Minor | documentation | Akira Ajisaka | Ayush Saxena | | | Ls with -e option not giving the result in proper format | Major | . | Soumyapn | Shubham Dewan | | | In-Maintenance ICON is missing in datanode info page | Major | datanode | Harshakiran Reddy | Ranith Sardar | | | MapReduce example wordmedian should handle generic options | Major | . | Fei Hui | Fei Hui | | | Shutdown NM may cause NPE when opportunistic container scheduling is enabled | Critical | . | lujie | lujie | | | KMS client uses wrong UGI after HADOOP-14445 | Blocker | kms | Wei-Chiu Chuang | Wei-Chiu Chuang | | | JSON License is included in the transitive dependency of aliyun-sdk-oss 3.0.0 | Blocker | . | Akira Ajisaka | Akira Ajisaka | | | AliyunOSS: bring fixes back from HADOOP-15671 | Blocker | fs/oss | wujinhu | wujinhu | | | FairShare calculation broken for large values after YARN-8833 | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | Avoid potential integer overflow when computing fair shares | Major | fairscheduler | liyakun | liyakun | | | [UI2] YARN UI2 page loading failed due to js error under some time zone configuration | Critical | webapp | collinma | collinma | | | Invalid event: REGISTERED and LAUNCH\_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM | Critical | . | lujie | lujie | | | RM fails to start if absolute resource is specified for partition capacity in CS queues | Blocker | yarn | Jiandan Yang | Jiandan Yang | | | RM nodes web page can not display node info | Blocker | yarn | Jiandan Yang | Jiandan Yang | | | When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER\THEN\MAX\_ALLOCATION) | Critical |

| Zhankun Tang | Zhankun Tang | | | S3A assumed role tests failing due to changed error text in AWS exceptions | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | Incorrect getSnapshottableDirListing() javadoc | Major | snapshots | Wei-Chiu Chuang | Dinesh Chitlangia | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Modify a logging message in TestCapacityScheduler | Trivial | . | Zhankun Tang | Zhankun Tang | | | TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey | Minor | test | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | | TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896 | Minor | capacity scheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | RBF: Router admin clrQuota is not synchronized with nameservice | Major | . | Dibyendu Karmakar | Dibyendu Karmakar | | | DockerClient still touches hadoop.tmp.dir | Minor | . | Jason Lowe | Craig Condit | | | Update documentation and yarn-default related to the Docker runtime | Minor | . | Shane Kumpf | Craig Condit | | | Cleanup ENTRYPOINT documentation | Minor | . | Craig Condit | Craig Condit | | | Add version attribute to site doc examples and quickstart | Major | site | Gour Saha | Eric Yang | | | Logging improvements for better debuggability | Major | . | Suma Shivaprasad | Suma Shivaprasad | | | Document best practice for user management | Major | documentation, yarn | Eric Yang | Eric Yang | | | RBF: Router ID in RouterRpcClient is always null | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Improve error message for invalid value in fields attribute | Minor | ATSv2 | Charan Hebri | Abhishek Modi | | | Refactor NameNode failover proxy providers | Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko | | | RBF: Configurable value in xml for async connection request queue size. | Major | federation | CR Hota | CR Hota | | | Add support for tmpfs mounts with the Docker runtime | Major | . | Shane Kumpf | Craig Condit | | | Stabilize/tune S3A committers; review correctness & docs | Blocker | fs/s3 | Steve Loughran | Steve Loughran | | | TestDistributedShell fails on Windows | Major | . | Botong Huang | Botong Huang | | | [Documentation] RBF: Mount points across multiple subclusters | Minor | . | igo Goiri | igo Goiri | | | AliyunOSS: update oss-sdk version to 3.0.0 | Major | fs/oss | wujinhu | wujinhu | | | S3 listing inconsistency can raise NPE in globber | Major | fs | Steve Loughran | Steve Loughran | | | java doc comments in docker-util.h is confusing | Minor | . | Zian Chen | Zian Chen | | | AliyunOSS: Support Assume Roles in AliyunOSS | Major | fs/oss | wujinhu | wujinhu | | | Update Docker examples to use image which exists | Minor | . | Craig Condit | Craig Condit | | | RBF: Move ClientProtocol APIs to its own module | Major | . | igo Goiri | Chao Sun | | | Improve the error message when a bind mount is not whitelisted | Major |

| Simon Prewo | Simon Prewo | | | Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way | Major | timelineserver | Vrushali C | Abhishek Modi | | | Serve aggregated logs of historical apps from timeline service | Critical | timelineserver | Varun Saxena | Rohith Sharma K S | | | Provide Java client for fetching Yarn specific entities from TimelineReader | Critical | timelinereader | Rohith Sharma K S | Abhishek Modi | | | DynamoDB table Update can fail S3A FS init | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys | Minor | conf | Steve Loughran | Steve Loughran | | | YARN service example is out-dated | Major | yarn-native-services | Eric Yang | Suma Shivaprasad | | | Add documentation for Delete Queue | Major | capacity scheduler, documentation | Naganarasimha G R | Suma Shivaprasad | | | AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream | Critical | . | wujinhu | wujinhu | | | AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy | Major | fs/oss | wujinhu | wujinhu | | | Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds | Minor | metrics, security | Harshakiran Reddy | LiXin Ge | | | AliyunOSS: fix incorrect ReadOps and WriteOps in statistics | Major | fs/oss | wujinhu | wujinhu | | | YarnClient should contact TimelineReader for application/attempt/container report | Critical | . | Rohith Sharma K S | Abhishek Modi | | | Yarn Service Upgrade: Add GET APIs that returns instances matching query params | Critical | . | Chandni Singh | Chandni Singh | | | Yarn Service Upgrade: Support upgrade of service that use docker containers | Major | . | Chandni Singh | Chandni Singh | | | Yarn Service Upgrade: Support express upgrade of a service | Major | . | Chandni Singh | Chandni Singh | | | publish all exposed ports to random ports when using bridge network | Minor | yarn | Charo Zhang | Charo Zhang | | | Oozie unable to create sharelib in s3a filesystem | Critical | fs, fs/s3 | Soumitra Sulav | Steve Loughran | | | Yarn Service Upgrade: Support cancelling upgrade | Major | . | Chandni Singh | Chandni Singh | | | Replace the url of the repository in Apache Hadoop source code | Major | documentation | Akira Ajisaka | Akira Ajisaka | | | AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore | Major | fs/oss | wujinhu | wujinhu | | | Nvidia-docker v2 support for YARN GPU feature | Critical | . | Zhankun Tang | Charo Zhang | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | YARN native service should return container if launch failed | Critical | . | Wangda Tan | Chandni Singh | | | Update EC documentation about rack fault tolerance | Major | documentation, erasure-coding | Xiao Chen | Kitti Nanasi | | | Upgrade Apache Zookeeper version due to security concerns | Major | . | Boris Vulikh | Akira Ajisaka | | | Upgrade maven-shade-plugin from 2.4.3 to 3.2.0 | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Upgrade Eclipse Jetty version to 9.3.24 | Major | . | Boris Vulikh | Boris Vulikh | | | YARN service/components/instances should have SUCCEEDED/FAILED states