Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.3.3.6.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add recoverLease(), setSafeMode(), isFileClosed() APIs to FileSystem | Major | fs | Wei-Chiu Chuang | Tak-Lon (Stephen) Wu | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally | Major | . | Xiaoqiao He | Xiaoqiao He | | | RBF: Disable watch in ZKDelegationSecretManager for performance | Major | . | Fengnan Li | Fengnan Li | | | EC: Remove unnecessary method (getWeight) in StripedReconstructionInfo | Trivial | . | Haiyang Hu | Haiyang Hu | | | RBF: Rename data to the Trash should be based on src locations | Major | rbf | Xiangyi Zhu | Xiangyi Zhu | | | BPServiceActor add a new thread to handle IBR | Minor | . | JiangHua Zhu | Viraj Jasani | | | Remove lock contention in SelectorPool of SocketIOWithTimeout | Major | common | Xuesen Liang | Xuesen Liang | | | Fix typo: indicies -\> indices | Minor | . | Jiale Qi | Jiale Qi | | | Limit the findbugs suppression IS2\INCONSISTENT\SYNC to S3AFileSystem field | Minor | fs/s3 | Viraj Jasani | Viraj Jasani | | | Parallelize MultipleOutputs#close call | Major | . | Prabhu Joseph | Ashutosh Gupta | | | Add number of decommissioning/shutdown nodes to YARN cluster metrics. | Major | client, resourcemanager | Chris Nauroth | Chris Nauroth | | | Upgrade to snakeyaml 1.33 | Major | . | PJ Fanning | PJ Fanning | | | Support DecommissionBackoffMonitor parameters reconfigurable | Major | . | Haiyang Hu | Haiyang Hu | | | Fix main thread name. | Major | common, ipc | zhengchenyu | zhengchenyu | | | RBF: Add a utility to dump the StateStore | Major | rbf | Owen O'Malley | Owen O'Malley | | | It should consider EC reconstruction work when we determine if a node is busy | Major | ec, erasure-coding | Kidd5368 | Kidd5368 | | | Log start and end of phase/step in startup progress | Minor | namenode | Viraj Jasani | Viraj Jasani | | | Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel | Major | namenode | Viraj Jasani | Viraj Jasani | | | Add compile platform in the hadoop version output | Major | . | Ayush Saxena | Ayush Saxena | | | BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout should be volatile | Major | . | Haiyang Hu | Haiyang Hu | | | Sasl connection failure should log remote address | Major | . | Viraj Jasani | Viraj Jasani | | | Fix method name of RPC.Builder#setnumReaders | Minor | ipc | Haiyang Hu | Haiyang Hu | | | RBF: Add cache hit rate metric

MountTableResolver#getDestinationForPath | Minor | rbf | farmmamba | farmmamba | | | Add LastHeartbeatResponseTime for BP service actor | Major | . | Viraj Jasani | Viraj Jasani | | | Server connection should log host name before returning VersionMismatch error | Minor | ipc | Viraj Jasani | Viraj Jasani | | | Distcp -update between different cloud stores to use modification time while checking for file skip. | Major | tools/distcp | Mehakmeet Singh | Mehakmeet Singh | | | Enhance WritableName to be able to return aliases for classes that use serializers | Minor | . | Bryan Beaudreault | Bryan Beaudreault | | | Upgrade ant to 1.10.13 | Major | . | Aleksandr Nikolaev | Aleksandr Nikolaev | | | Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. | Major | resourcemanager | Shilun Fan | Shilun Fan | | | Implement token storage solution based on MySQL | Major | . | Hector Sandoval Chaverri | Hector Sandoval Chaverri | | | Upgrade Netty to 4.1.89.Final | Major | build | Aleksandr Nikolaev | Aleksandr Nikolaev | | | S3A filesystem to support binding to other URI schemes | Major | . | Harshit Gupta | Harshit Gupta | | | Publish SBOM artifacts | Major | build | Dongjoon Hyun | Dongjoon Hyun | | | Simplify single node instructions for creating directories for Map Reduce | Trivial | documentation | Nikita Eshkeev | Nikita Eshkeev | | | Add a CallerContext getter on the Schedulable interface | Major | . | Christos Bisias | Christos Bisias | | | Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project | Major | common | Ayush Saxena | | | | Improve NameServices info at JournalNode web UI | Minor | . | Zhaohui Wang | Zhaohui Wang | | | S3A to support upload of files greater than 2 GB using DiskBlocks | Major | fs/s3 | Harshit Gupta | Harshit Gupta | | | S3A: reject multipart copy requests when disabled | Minor | fs/s3 | Steve Loughran | Steve Loughran | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RBF: Correct target directory when move to trash for kerberos login user. | Major | rbf | Xiping Zhang | Xiping Zhang | | | Reserved Space For Replicas is not released on some cases | Major | hdfs | Prabhu Joseph | Ashutosh Gupta | | | Erasure Coding: Recovery may cause excess replicas when busy DN exsits | Major | ec, erasure-coding | Kidd5368 | Kidd5368 | | | An unhandled ArrayIndexOutOfBoundsException in DefaultStringifier.storeArray() if provided with an empty input | Minor | common, io | FuzzingTeam | FuzzingTeam | | | EC striped block is not sufficient when doing in maintenance | Major | ec, erasure-coding | dingshun | dingshun | | |

Shutdown nodes should be 0 now expected: \<1\> but was: \<0\> | Major | yarn | Bence Kosztolnik | Bence Kosztolnik | | | Register the shutdown hook only when not in shutdown for KeyProviderCache constructor | Minor | hdfs | Xing Lin | Xing Lin | | | LogThrottlingHelper: the dependent recorder is not triggered correctly | Major | . | Chengbing Liu | | | | Resource Manager UI, cluster/appattempt/\*, can not present FINAL\_SAVING state | Critical | yarn | Bence Kosztolnik | Bence Kosztolnik | | | ClientRMService implemented getCallerUgi and verifyUserAccessForRMApp methods but forget to use sometimes, caused audit log missing. | Major | yarn | Beibei Zhao | Beibei Zhao | | | Fix log throttling by declaring LogThrottlingHelper as static members | Major | . | Chengbing Liu | | | | Fix a typo in Trash | Minor | documentation | xiaoping.huang | xiaoping.huang | | | ObserverNamenode handles addBlock rpc and throws a FileNotFoundException | Critical | . | ZanderXu | ZanderXu | | | [NFS GW] Fix regression after netty4 migration | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang | | | Cancel fileMonitoringTimer even if trustManager isn't defined | Major | common, test | Steve Vaughan | Steve Vaughan | | | Java 11 JavaDoc fails due to missing package comments | Major | build, common | Steve Loughran | Steve Vaughan | | | Avoid mixing canonical and non-canonical when performing comparisons | Minor | common, test | Steve Vaughan | Steve Vaughan | | | Namenode audit log to only include IP address of client | Major | . | Viraj Jasani | Viraj Jasani | | | fix test AbstractContractDistCpTest#testDistCpUpdateCheckFileSkip | Major | tools/distcp | Mehakmeet Singh | Mehakmeet Singh | | | No need to clean tmp files in distcp direct mode | Major | tools/distcp | 10000kang | 10000kang | | | LocalDirAllocator cannot recover from directory tree deletion during the life of a filesystem client | Minor | fs, fs/azure, fs/s3 | Steve Loughran | Steve Loughran | | | TestFsDatasetImpl.testReportBadBlocks brittle | Minor | test | Steve Loughran | Viraj Jasani | | | Send error to datanode if FBR is rejected due to bad lease | Major | datanode, namenode | Stephen O'Donnell | Stephen O'Donnell | | | ListFiles with recursive fails with FNF | Major | . | Ayush Saxena | Ayush Saxena | | | Insufficient heap during full test runs in Docker container. | Minor | build | Chris Nauroth | Chris Nauroth | | | Wrong StringUtils.join() called in AbstractContractRootDirectoryTest | Trivial | test | Attila Doroszlai | Attila Doroszlai | | | ABFS should exclude incompatible credential providers | Major | fs/azure | Tamas Domok | Tamas Domok | | | Filesystem Spelling Mistake | Trivial | fs | Sebastian Baunsgaard | Sebastian Baunsgaard | | | MR Fetcher class to use an AtomicInteger to generate

| Major | build, client | Steve Loughran | Steve Loughran | | | Fix lease interval comparison in BlockReportLeaseManager | Trivial | namenode | dzcxzl | dzcxzl | | | Fix bug of DRF comparison DominantResourceFairnessComparator2 in fair scheduler | Major | fairscheduler | Xiaoqiao He | Xiaoqiao He | | | Fix abundant Broken pipe exception in BlockSender | Minor | hdfs | fanluo | fanluo | | | Add debug log for getting details of tokenKindMap | Minor | . | Pralabh Kumar | Pralabh Kumar | | | [UI2] Refresh buttons don't work after EmberJS upgrade | Minor | yarn-ui-v2 | Brian Goerlitz | Susheel Gupta | | | Open file fails with NumberFormatException for S3AFileSystem | Critical | fs, fs/azure, fs/s3 | Ayush Saxena | Steve Loughran | | | Path.suffix raises NullPointerException | Minor | hdfs-client | Patrick Grandjean | Patrick Grandjean | | | Fix the exception message to print the Identifier pattern | Minor | . | Nishtha Shah | Nishtha Shah | | | Fix the issue of arguments number limit in report command in DFSAdmin. | Major | . | Haiyang Hu | Haiyang Hu | | | openFile builder new optLong() methods break hbase-filesystem | Major | fs | Steve Loughran | Steve Loughran | | | Fix the metric of "HttpPort" at DataNodeInfo | Minor | . | Zhaohui Wang | Zhaohui Wang | | | Erasure Coding: invalidate wrong block after reporting bad blocks from datanode | Critical | namenode | farmmamba | farmmamba | | | Fix several maven build warnings | Minor | build | Dongjoon Hyun | Dongjoon Hyun | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Prevent resource leaks in TestClientRMService. | Minor | test | Chris Nauroth | Chris Nauroth | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | TestBPOfferService#testMissBlocksWhenReregister fails intermittently | Major | datanode | Ahmed Hussein | Ahmed Hussein | | | TestBPOfferService#testMissBlocksWhenReregister fails on trunk | Major | . | Ahmed Hussein | Masatake Iwasaki | | | fs.s3a.prefetch.block.size to be read through longBytesOption | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | s3a prefetching to use SemaphoredDelegatingExecutor for submitting work | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | hadoop-aws maven build to add a prefetch profile to run all tests with prefetching | Major | fs/s3, test | Steve Loughran | Viraj Jasani | | | s3a prefetching Executor should be closed | Major | fs/s3 | Viraj Jasani | Viraj Jasani | | | Implement readFully(long position, byte[] buffer, int offset, int length) | Minor | fs/s3 | Ahmar Suhail | Alessandro Passaro | | | S3PrefetchingInputStream to support status probes when closed | Minor | fs/s3 | Steve Loughran | Viraj Jasani | | | Address JavaDoc warnings in classes like MarkerTool, S3ObjectAttributes,

| Minor | fs/s3 | Mukund Thakur | Ankit Saurabh | | | ITestS3APrefetchingInputStream does not skip if no CSV test file available | Minor | fs/s3 | Daniel Carl Jones | Daniel Carl Jones | | | assertion failure in ITestS3APrefetchingInputStream | Major | fs/s3, test | Steve Loughran | Ashutosh Gupta | | | Avoid using grizzly-http-\* APIs | Major | . | Viraj Jasani | Viraj Jasani | | | S3A prefetching: Error logging during reads | Minor | . | Ahmar Suhail | Ankit Saurabh | | | compatibility table in directory\_markers.md doesn't render right | Minor | fs/s3, site | Steve Loughran | Masatake Iwasaki | | | Add reason in in x-ms-client-request-id on a retry API call. | Minor | fs/azure | Pranav Saxena | Pranav Saxena | | | ABFS: Add changes for expect hundred continue header with append requests | Major | fs/azure | Anmol Asrani | Anmol Asrani | | | x-ms-client-request-id to have some way that identifies retry of an API. | Minor | fs/azure | Pranav Saxena | Pranav Saxena | | | ABFS: Enable config controlled ETag check for Rename idempotency | Major | fs/azure | Sneha Vijayarajan | Sree Bhattacharyya | | | S3A ITestS3ABucketExistence access point test failure | Major | fs/s3, test | Steve Loughran | Steve Loughran | | | S3A Prefetch - SingleFilePerBlockCache to use LocalDirAllocator | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | Backport S3A prefetching stream to branch-3.3 | Major | fs/s3 | Steve Loughran | Steve Loughran | | | Fix transient failure of ITestS3APrefetchingInputStream#testRandomReadLargeFile | Major | fs/s3, test | Steve Loughran | Viraj Jasani | | | S3A audit header to include count of items in delete ops | Major | fs/s3 | Steve Loughran | Viraj Jasani | | | s3a prefetch cache blocks should be accessed by RW locks | Major | . | Viraj Jasani | Viraj Jasani | | | Upgrade aws-java-sdk to 1.12.367+ | Major | fs/s3 | Steve Loughran | Viraj Jasani | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | HostRestrictingAuthorizationFilter should pass through requests if they don't access WebHDFS API | Major | . | Takanobu Asanuma | Takanobu Asanuma | | | Fix documentation for StateStoreRecordOperations#get(Class ..., Query ...) | Trivial | . | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | Add Namenode status to BPServiceActor metrics and improve logging in offerservice | Major | . | Viraj Jasani | Viraj Jasani | | | RBF: Routers should propagate the real user in the UGI via the caller context | Major | . | Simbarashe Dzinamarira | Simbarashe Dzinamarira | | | snakeyaml dependency: upgrade to v2.0 | Major | . | PJ Fanning | PJ Fanning | | | Upgrade Apache Derby from 10.10.2.0 to 10.14.2.0 due to CVEs | Major | build, test | PJ Fanning | | | | Upgrade to jetty 9.4.51 due to cve | Major | common | PJ Fanning | PJ Fanning | | | Fix WriteOperations.listMultipartUploads function description | Trivial | fs/s3 | Dongjoon Hyun | Dongjoon Hyun | | | Remove mysql-connector-java | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang |