av9ash/GitBugs huggingface dataset:
Bug: ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 18/Aug/20 13:47
Affects Version/s: 3.4.0

The test ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics fails when the property fs.azure.test.appendblob.enabled is set to true
Bug: ITestS3ADirectoryPerformance.testListOperations failing
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/Jul/20 09:21
Affects Version/s: 3.4.0

Because of HADOOP-17022
[INFO] Running org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
[ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 670.029 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
[ERROR] testListOperations(org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance)  Time elapsed: 44.089 s  <<< FAILURE!
java.lang.AssertionError: object_list_requests starting=166 current=167 diff=1 expected:<2> but was:<1>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:645)
	at org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance.testListOperations(ITestS3ADirectoryPerformance.java:117)
Bug: ABFS: Run the integration tests with various combinations of configurations and publish consolidated results
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 06/Aug/20 19:08
Affects Version/s: 3.3.0

ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms.
Currently the developer has to manually run the tests with different combinations of configurations ex: HNS account with SharedKey and OAuth, NonHNS account with SharedKey etc..
The expectation is to automate these runs with different combinations. This will help the developer to run the integration tests with different variants of configurations automatically.
Bug: ITestS3GuardOutOfBandOperations testListingDelete[auth=false] fails on unversioned bucket
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 16/Jul/20 16:23
Affects Version/s: 3.4.0

transient failure of 
{code}
[ERROR] Tests run: 24, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 369.68 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations
[ERROR] testListingDelete[auth=false](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 19.103 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: No such file or directory: s3a://stevel-ireland/fork-0001/test/dir-e34a122f-a04d-48c3-90c1-9d35427fa939/file-1-e34a122f-a04d-48c3-90c1-9d35427fa939
{code}

config is unguarded codepath, s3a status was not passed down. Test run was really, really slow (8 threads)

hypothesis: test run took so long that a TTL expired and the open operation did a HEAD to s3 even when a s3guard record was found.
Bug: S3AFileSystem.listLocatedStatus(file) does a LIST even with S3Guard
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 16/Jul/20 14:10
Affects Version/s: 3.4.0

This is minor and we may want to WONTFIX; noticed during work on directory markers.

If you call listLocatedStatus(file) then a LIST call is always made to S3, even when S3Guard is present and has the record to say "this is a file"

Does this matter enough to fix? 

# The HADOOP-16465 work moved the list before falling back to getFileStatus
# that listing calls s3guard.listChildren(path) to list the children.
# which only returns the chlldren of a path, not a record of the path itself.
# so we get an empty list back, triggering the LIST
# its only after that LIST fails that we fall back to getFileStatus and hence look for the actual file record.
Bug: ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 23/Jul/20 07:29
Affects Version/s: 3.3.1

When authentication is set to SharedKey, below test fails.

 

[ERROR]   ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 Expecting org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException with text "Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.", 403 but got : "void"

 

This test fails when the newly introduced config "fs.azure.account.hns.enabled" is set. This config will avoid network call to check if namespace is enabled, whereas the test expects thsi call to be made. 

 

The assert in test to 403 needs check too. Should ideally be 401.
Bug: Add debug-level logs in Filesystem#close
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 17/Sep/20 07:57
Affects Version/s: 3.3.0

HDFS reuses the same cached FileSystem object across the file system. If the client calls FileSystem.close(), closeAllForUgi(), or closeAll() (if it applies to the instance) anywhere in the system it purges the cache of that FS instance, and trying to use the instance results in an IOException: FileSystem closed.

It would be a great help to clients to see where and when a given FS instance was closed. I.e. in close(), closeAllForUgi(), or closeAll(), it would be great to see a DEBUG-level log of
 * calling method name, class, file name/line number
 * FileSystem object's identity hash (FileSystem.close() only)

For the full calling stack, turn on TRACE logging.
Bug: Add  ViewFileSystem/InodeTree Mount points Resolution Cache
Status: Open | Priority: Minor | Resolution: 
Created: 02/Sep/20 02:06

Now the ViewFileSystem walkthrough InodeTree to resolve the passed-in path. This could be optimized by adding a cache of resolution result. It will benefit all kinds of mount points. And it is also very useful to reduce the Regex parse cost of Regex Mount points introduced in HADOOP-15891
Bug: ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 30/Jul/20 14:12
Affects Version/s: 3.3.0

The test ITestS3AEncryptionWithDefaultS3Settings fails if

* the test run is set up with a KMS key
* the test bucket has a default encryption of AES (maybe also unencrypted)

Proposed: downgrade the test to skip if the default encryption is determined to be something other than S3-KMS
Bug: Support for AWS STSAssumeRoleWithWebIdentitySessionCredentialsProvider based credential provider to support use of IRSA on deployments on AWS EKS Cluster
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 06/Aug/20 05:31
Affects Version/s: 3.3.0

The latest version of AWS SDK has support to use IRSA for providing credentials to Kubernetes pods which can potentially replace the use of Kube2IAM. For our Apache Spark on Kubernetes use cases, this feature will be useful. The current Hadoop AWS component does support adding custom credential provider but I think if we could add STSAssumeRoleWithWebIdentitySessionCredentialsProvider support to (using roleArn, role session name, web Identity Token File) to the hadoop-aws library, it will be useful for the community as such who use AWS EKS.

[https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/STSAssumeRoleWithWebIdentitySessionCredentialsProvider.html]

[https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/STSAssumeRoleWithWebIdentitySessionCredentialsProvider.Builder.html

] [https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/]
Bug: Intermittent ITestTerasortOnS3A.test_120_terasort failure
Status: Open | Priority: Minor | Resolution: 
Created: 06/Aug/20 16:10
Affects Version/s: 3.3.0

[*INFO*] Running org.apache.hadoop.fs.s3a.commit.terasort.*ITestTerasortOnS3A*

[*ERROR*] *Tests* *run: 14*, *Failures: 2*, Errors: 0, *Skipped: 2*, Time elapsed: 110.43 s *<<< FAILURE!* - in org.apache.hadoop.fs.s3a.commit.terasort.*ITestTerasortOnS3A*

[*ERROR*] test_120_terasort[directory](org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A)  Time elapsed: 6.261 s  <<< FAILURE!

java.lang.AssertionError: terasort(s3a://mthakur-data/terasort-directory/sortin, s3a://mthakur-data/terasort-directory/sortout) failed expected:<0> but was:<1>

 at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.executeStage(ITestTerasortOnS3A.java:241)

 at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.test_120_terasort(ITestTerasortOnS3A.java:291)

 

[*ERROR*] test_120_terasort[magic](org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A)  Time elapsed: 5.962 s  <<< FAILURE!

java.lang.AssertionError: terasort(s3a://mthakur-data/terasort-magic/sortin, s3a://mthakur-data/terasort-magic/sortout) failed expected:<0> but was:<1>

 at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.executeStage(ITestTerasortOnS3A.java:241)

 at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.test_120_terasort(ITestTerasortOnS3A.java:291)
Bug: Fix findbugs warnings in hadoop-tools on branch-2.10
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 11/Aug/20 03:06

{noformat}
M D UC_USELESS_OBJECT UC: Useless object stored in variable keysToUpdateAsFolder of method org.apache.hadoop.fs.azure.NativeAzureFileSystem.mkdirs(Path, FsPermission, boolean)  At NativeAzureFileSystem.java:[line 3013]
M D DLS_DEAD_LOCAL_STORE DLS: Dead store to op in org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.access(Path, FsAction)  At AzureBlobFileSystemStore.java:[line 901]
M B CO_COMPARETO_INCORRECT_FLOATING Co: org.apache.hadoop.mapred.gridmix.InputStriper$1.compare(Map$Entry, Map$Entry) incorrectly handles double value  At InputStriper.java:[line 136]
M V MS_MUTABLE_COLLECTION_PKGPROTECT MS: org.apache.hadoop.mapred.gridmix.emulators.resourceusage.TotalHeapUsageEmulatorPlugin$DefaultHeapUsageEmulator.heapSpace is a mutable collection which should be package protected  At TotalHeapUsageEmulatorPlugin.java:[line 132]
M D RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT RV: Return value of org.codehaus.jackson.map.ObjectMapper.getJsonFactory() ignored, but method has no side effect  At JsonObjectMapperWriter.java:[line 59]
H D RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT RV: Return value of new org.apache.hadoop.tools.rumen.datatypes.DefaultDataType(String) ignored, but method has no side effect  At MapReduceJobPropertiesParser.java:[line 212]
{noformat}
Bug: Add python2 to required package on CentOS 8 for building documentation
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 14/Aug/20 06:41

{{mvn site}} fails without python2.

{noformat}
[INFO] --- exec-maven-plugin:1.3.1:exec (shelldocs) @ hadoop-common ---
/usr/bin/env: ‘python2’: No such file or directory
{noformat}
Bug: S3A rename operation not the same as HDFS when dest is empty dir
Status: Open | Priority: Minor | Resolution: 
Created: 27/Jul/20 12:13
Affects Version/s: 3.2.1

When I run the test ITestS3ADeleteManyFiles, I change the [https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADeleteManyFiles.java#L97]

to 

{code}

fs.mkdirs(finalDir);

{code}

So before rename operator, "finalParent/final" has been created.

But after the rename operation,  all the files will be moved from "srcParent/src" to "finalParent/final"

So this is not the same with the HDFS rename operation:

HDFS rename includes the calculation of the destination path. If the destination exists and is a directory, the final destination of the rename becomes the destination + the filename of the source path.
let dest = if (isDir(FS, src) and d != src) :
        d + [filename(src)]
    else :
        d
Bug: DF implementation of CachingGetSpaceUsed makes DFS Used size not correct
Status: Resolved | Priority: Minor | Resolution: Not A Problem
Created: 27/Jul/20 06:24

When   we calculate DN's storage used, we add each Volume's used size together and each volume's size comes from it's BP's size. 

When we use DF instead of DU, we know that DF check disk space usage (not disk size of a directory). so when check BP dir path,  What you're actually checking is the corresponding disk directory space. 

 

When we use this with federation, under each volume  may have more than one BP, each BP return it's corresponding disk directory space. 

 

If we have two BP under one volume, we will make DN's storage info's Used size double than real size.
Bug: Fix outdated properties of JournalNode when performing rollback
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 21/Jul/20 07:11

When rollback HDFS cluster, properties in JNStorage won't be refreshed after the storage dir changed. It leads to exceptions when starting namenode.

The exception like:
{code:java}
2020-07-09 19:04:12,810 FATAL [IPC Server handler 105 on 8022] org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [10.0.118.217:8485, 10.0.117.208:8485, 10.0.118.179:8485], stream=null))
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
10.0.118.217:8485: Incompatible namespaceID for journal Storage Directory /mnt/vdc-11176G-0/dfs/jn/nameservicetest1: NameNode has nsId 647617129 but storage has nsId 0
	at org.apache.hadoop.hdfs.qjournal.server.JNStorage.checkConsistentNamespace(JNStorage.java:236)
	at org.apache.hadoop.hdfs.qjournal.server.Journal.newEpoch(Journal.java:300)
	at org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.newEpoch(JournalNodeRpcServer.java:136)
	at org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.newEpoch(QJournalProtocolServerSideTranslatorPB.java:133)
	at org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25417)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2278)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2274)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2274)
{code}
Bug: Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 13/Jul/20 21:58

While making an internal change to use {{TimeUnit.MICROSECONDS}} instead of {{TimeUnit.MILLISECONDS}} for rpc details, we found that we also had to modify this code in DecayRpcScheduler.addResponseTime() to initialize {{queueTime}} and {{processingTime}} with the correct units.
{noformat}
    long queueTime = details.get(Timing.QUEUE, TimeUnit.MILLISECONDS);
    long processingTime = details.get(Timing.PROCESSING, TimeUnit.MILLISECONDS);
{noformat}
If we change these to use {{RpcMetrics.TIMEUNIT}} it is simpler.

We also found one test case in TestRPC that was assuming the units were milliseconds.
Bug: Fix typo in Hadoop KMS document
Status: Resolved | Priority: Trivial | Resolution: Fixed
Created: 12/Aug/20 05:27

[https://hadoop.apache.org/docs/r3.3.0/hadoop-kms/index.html#HTTP_Kerberos_Principals_Configuration]

bq. In order to be able to access directly a specific KMS instance, the KMS instance must also have Keberos service name with its own hostname. This is required for monitoring and admin purposes.

Keberos -> Kerberos
Bug: S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 18/Oct/23 10:31
Affects Version/s: 3.4.0

Failures in impala test VMs using iAM for auth

{code}
Failed to open file as a parquet file: java.net.SocketTimeoutException: re-open s3a://impala-test-uswest2-1/test-warehouse/test_pre_gregorian_date_parquet_2e80ae30.db/hive2_pre_gregorian.parquet at 84 on s3a://impala-test-uswest2-1/test-warehouse/test_pre_gregorian_date_parquet_2e80ae30.db/hive2_pre_gregorian.parquet: org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: +: Failed to load credentials from IMDS

{code}
Bug: Build failure while trying to create apache 3.3.7 release locally.
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 10/Oct/23 15:43
Affects Version/s: 3.3.6

{noformat}
[ESC[1;34mINFOESC[m] ESC[1m-------< ESC[0;36morg.apache.hadoop:hadoop-client-check-test-invariantsESC[0;1m >--------ESC[m
[ESC[1;34mINFOESC[m] ESC[1mBuilding Apache Hadoop Client Packaging Invariants for Test 3.3.9-SNAPSHOT [105/111]ESC[m
[ESC[1;34mINFOESC[m] ESC[1m--------------------------------[ pom ]---------------------------------ESC[m
[ESC[1;34mINFOESC[m] 
[ESC[1;34mINFOESC[m] ESC[1m--- ESC[0;32mmaven-enforcer-plugin:3.0.0-M1:enforceESC[m ESC[1m(enforce-banned-dependencies)ESC[m @ ESC[36mhadoop-client-check-test-invariantsESC[0;1m ---ESC[m
[ESC[1;34mINFOESC[m] Adding ignorable dependency: org.apache.hadoop:hadoop-annotations:null
[ESC[1;34mINFOESC[m]   Adding ignore: *
[ESC[1;33mWARNINGESC[m] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
Duplicate classes found:


  Found in:
    org.apache.hadoop:hadoop-client-minicluster:jar:3.3.9-SNAPSHOT:compile
    org.apache.hadoop:hadoop-client-runtime:jar:3.3.9-SNAPSHOT:compile
  Duplicate classes:
    META-INF/versions/9/module-info.class

{noformat}
CC [~stevel@apache.org]  [~weichu]
Bug: Setup pre-commit CI for Windows 10
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 24/Dec/23 17:41
Affects Version/s: 3.4.0

We need to setup a pre-commit CI for validating the Hadoop PRs against Windows 10.

On a sidenote, we've got the nightly Jenkins CI running for Hadoop on Windows 10 - https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-win10-x86_64/.
Bug: ABFS: Remove commons IOUtils.close() from AbfsOutputStream
Status: Open | Priority: Critical | Resolution: 
Created: 17/Oct/23 09:51
Affects Version/s: 3.4.0


Hive is raising  java.lang.NoSuchMethodError when attempting to run Hive queries against abfs because it's version of commons IO doesn't match that hadoop is built with.

Moving to use Hadoop's IOUtil.cleanup methods rather than commons IOUtils avoids this
Bug: NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 16/Oct/23 11:08
Affects Version/s: 3.4.0

NPE in error handling code of RetryOnErrorCodeCondition.shouldRetry(); in bundle-2.20.128.jar

This is AWS SDK code; fix needs to go there. 

{code}
Caused by: java.lang.NullPointerException
	at software.amazon.awssdk.awscore.retry.conditions.RetryOnErrorCodeCondition.shouldRetry(RetryOnErrorCodeCondition.java:45) ~[bundle-2.20.128.jar:?]
	at software.amazon.awssdk.core.retry.conditions.OrRetryCondition.lambda$shouldRetry$0(OrRetryCondition.java:46) ~[bundle-2.20.128.jar:?]
	at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90) ~[?:1.8.0_382]
{code}
Bug: fs.getXattrs(path) for S3FS doesn't have x-amz-server-side-encryption-aws-kms-key-id header.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Dec/23 20:52
Affects Version/s: 3.3.6

Once a path while uploading has been encrypted with SSE-KMS with a key id and then later when we try to read the attributes of the same file, it doesn't contain the key id information as an attribute. should we add it?

 

while cherry-picking please include https://issues.apache.org/jira/browse/HADOOP-19190
Bug: coalesce AWS S3 client proxy settings
Status: Open | Priority: Major | Resolution: 
Created: 12/Oct/23 12:50
Affects Version/s: 3.4.0

all the code to build up proxy settings for the two clients is nearly identical.

* coalesce into on method to load all the options, another to apply them to the different bullders
* extend TestS3AProxy to test async client too
* test for invalid options, rather than just the good paths.
Bug: S3A: retry on credential expiry
Status: Open | Priority: Major | Resolution: 
Created: 25/Nov/23 13:11
Affects Version/s: 3.4.0

Reported in AWS SDK https://github.com/aws/aws-sdk-java-v2/issues/3408

bq. In RetryableStage execute method, the "AwsCredentails" does not attempt to renew if it has expired. Therefore, if a method called with the existing credential is expiring soon, the number of retry is less than intended due to the expiration of the credential.

The stack from this report doesn't show any error detail we can use to identify the 400 exception as something we should be retrying on. This could be due to the logging, or it could actually hold. we've have to generate some socket credentials, let them expire and then see how hadoop fs commands failed. Something to do by hand as an STS test to do this is probably slow. *unless we expire all session credentials of a given role?*. Could be good, would be traumatic for other test runs though.

{code}
software.amazon.awssdk.services.s3.model.S3Exception: The provided token has expired. (Service: S3, Status Code: 400, Request ID: 3YWKVBNJPNTXPJX2, Extended Request ID: GkR56xA0r/Ek7zqQdB2ZdP3wqMMhf49HH7hc5N2TAIu47J3HEk6yvSgVNbX7ADuHDy/Irhr2rPQ=)

{code}
Bug: S3A: Support dynamic region resolution
Status: Open | Priority: Major | Resolution: 
Created: 20/Nov/23 17:06
Affects Version/s: 3.4.0

AWS v1 SDK used to do dynamic region inference including EC2 metadata. See HADOOP-17771 for that at work with the chain of: sysprops, env vars, EC2 metadata.

I think we *try* to do this, but I'm not sure it is working, maybe related to the region=null vs region="" options: its very hard to differentiate the two
Bug: S3A: transfer manager not wired up to s3a executor pool
Status: Open | Priority: Major | Resolution: 
Created: 08/Dec/23 11:57
Affects Version/s: 3.4.0

S3ClientFactory.createS3TransferManager() doesn't use the executor declared in S3ClientCreationParameters.transferManagerExecutor

* method needs to take S3ClientCreationParameters
* and set the transfer manager executor