av9ash/GitBugs huggingface dataset:
Bug: ABFS: eTag Check to mitigate Rename Failures
Status: Resolved | Priority: Minor | Resolution: Duplicate
Created: 15/Mar/23 12:08

RenameFilePath on its first try receives a Request timed out error with code 500. On retrying the same operation, a Source file not found (404) error is received. 

Possible temporary mitigation: Check whether etags remain the same before and after the retry and accordingly send an Operation Successful result, instead of source file not found. For this, adding a check and an additional call to get the source eTag, in case it is not present already, to check after a rename retry whether the source and destination eTags match.
Bug: you can't launch create-release --docker from a build file
Status: Open | Priority: Minor | Resolution: 
Created: 14/Mar/23 14:31
Affects Version/s: 3.3.5

the {{create-release}} script launches docker with -i for interactivity, which
breaks when launching it from an ant build file or other scripts


{code}
        [x] 
        [x] Failed!
        [x] 
        [x] the input device is not a TTY
        [x]         3.12 real         0.29 user         0.39 sys

{code}

Fix: don't do that
Bug: S3A delegation token implementations to be able to update tokens from the user credentials
Status: Open | Priority: Minor | Resolution: 
Created: 07/Feb/23 17:06
Affects Version/s: 3.3.5


Spark never renews tokens, instead it can create new ones and attach them to the current users credentials.

This means long-running S3A instances which can pick up new tokens/credentials need a way to look for new tokens in the credential chain.

Proposed
* class AbstractDelegationTokenBinding adds a CallableRaisingIOE field which can be updated with a callback
* S3ADelegationTokens to add method boolean maybeUpdateTokenFromOwner() to look for any new token and switch to it if new
* S3ADelegationTokens serviceInit() to pass the method down to the instantiated DT binding as the callback

It is up to the token binding implementation to decide what to do about it; the standard implementations will do: nothing.
Bug: AbstractSTestS3AHugeFiles put request assert is using wrong iostatistic
Status: Open | Priority: Minor | Resolution: 
Created: 02/Feb/23 12:52
Affects Version/s: 3.3.5

The huge file test case {{AbstractSTestS3AHugeFiles.test_010_CreateHugeFile()}} asserts that the put file count > 0. But this stat measures the count of HTTP PUT requests sent to S3; if a multipart PUT was used then the count is zero. As a result, this test case can fail on bigger files than the default.

Fix. assert on OBJECT_PUT_REQUESTS_COMPLETED; this will be 1 for a simple PUT, and #of multipart blocks uploaded on a larger file.
Bug: Simplify single node instructions for creating directories for Map Reduce
Status: Resolved | Priority: Trivial | Resolution: Fixed
Created: 18/Jan/23 08:59
Affects Version/s: 3.3.4

The {{mkdir}} command supports the {{-p}} option which instructs {{hdfs}} to create all the parent directories if needed. The single nose setup instructions now ask a user to create both /user and /user/<username> directories explicitly, which can be simplified to creating just the /user/<username> with the help from the -p option of mkdir