av9ash/GitBugs huggingface dataset:
Bug: intermittent ITestS3AInconsistency.testGetFileStatus failure.
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 06/Jan/21 06:45
Affects Version/s: 3.3.1

{code}
[*ERROR*] *Tests* *run: 3*, *Failures: 1*, Errors: 0, Skipped: 0, Time elapsed: 30.944 s *<<< FAILURE!* - in org.apache.hadoop.fs.s3a.*ITestS3AInconsistency*

[*ERROR*] testGetFileStatus(org.apache.hadoop.fs.s3a.ITestS3AInconsistency)  Time elapsed: 6.471 s  <<< FAILURE!

java.lang.AssertionError: getFileStatus should fail due to delayed visibility.

 at org.junit.Assert.fail(Assert.java:88)

 at org.apache.hadoop.fs.s3a.ITestS3AInconsistency.testGetFileStatus(ITestS3AInconsistency.java:114)

 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

 at java.lang.reflect.Method.invoke(Method.java:498)

 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)

 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)

 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)

 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)

 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)

 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)

 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)

 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)

 at java.util.concurrent.FutureTask.run(FutureTask.java:266)

 at java.lang.Thread.run(Thread.java:748)
{code}
Bug: abfs listLocatedStatus to support incremental/async page fetching
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Jan/21 10:39
Affects Version/s: 3.3.0

listLocatedStatus uses listStatus[] then serves up as an iterator.

Once HADOOP-17475 is in, it can do the list incrementally with async prefetch.

Although not used much in the Hadoop codebase, the LocationStatusFetcher class is used during query planning across applications
Bug: Upgrade Jackson databind to 2.10.5.1
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Feb/21 10:51
Affects Version/s: 3.2.2

Hey everyone, we've done a container scan of Hadoop 3.2.2 we are using to build a shaded version of a Flink uber jar with, and noticed several apparent problems that are primarily related to com.faster.xml.jackson.core_jackson-databind.

 

Specifically the report claims version 2.4.0 of the library is used (am not sure about this part personally so I may be mistaken) and the fix suggestion I see is to move up to either 2.10.5.1, 2.9.10.8, 2.6.7.4 as appropriate.

 

I believe 2.10.3 is actually what's currently in use based on [https://github.com/apache/hadoop/blob/4cf35315838a6e65f87ed64aaa8f1d31594c7fcd/hadoop-project/pom.xml#L75|https://github.com/apache/hadoop/blob/4cf35315838a6e65f87ed64aaa8f1d31594c7fcd/hadoop-project/pom.xml#L75.]

 

Hopefully not a far-reaching change as I know changing dependencies can sometimes have a big knock-on effect, anyway - figured I'd report it incase someone plans to work on it.

 

Again do note that this is using a scan of an image built for Flink 1.11.3, but using Hadoop so it has a bunch of the same classes in, and I do believe that in Flink itself, the version of Jackson pulled in does not have the same problems, thus my thinking it is related to the Hadoop dependencies.

Thanks!
Bug: port UGI#getGroupsSet optimizations into 2.10
Status: In Progress | Priority: Major | Resolution: 
Created: 21/Jan/21 17:33
Affects Version/s: 2.10.1

HADOOP-17079 introduced an optimization adding a UGI#getGroupsSet and use Set#contains() instead of List#contains() to speed up large group look up while minimize List->Set conversions in Groups#getGroups() call.

This ticket is to port the changes into branch-2.10.

 

CC: [~Jim_Brennan], [~xyao]
Bug: S3Guard import can OOM on large imports
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 02/Mar/21 10:39
Affects Version/s: 3.3.1

I know I'm closing ~all S3Guard issues as wontfix, but this is pressing so I'm going to do it anyway

S3guard import of directory tree containing many, many files will OOM. Looking at the code this is going to be because

* import tool builds a map of all dirs imported, which as the comments note "superfluous for DDB". - *cut*
* DDB AncestorState tracks files as well as dirs, purely as a safety check to make sure current op doesn't somehow write a file entry above a dir entry in the same operation

We've been running S3Guard for a long time, and condition #2 has never arisen.

Propose: don't store filenames there, so memory consumption goes from O(files + dirs) to O(dirs)

Code straightforward, can't think of any tests
Bug: Upgrade Jackson databind in branch-2.10 to 2.9.10.7
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Mar/21 16:19

Two known vulnerabilities found in Jackson-databind

[CVE-2021-20190|https://nvd.nist.gov/vuln/detail/CVE-2021-20190] high severity
[CVE-2020-25649|https://nvd.nist.gov/vuln/detail/CVE-2020-25649] high severity
Bug: ABFS: Partially obfuscate SAS object IDs in Logs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 31/Mar/21 10:59
Affects Version/s: 3.3.1

Delegation SAS tokens are created using various parameters for specifying details such as permissions and validity. The requests are logged, along with values of all the query parameters. This change will partially mask values logged for the following object IDs representing the security principal: skoid, saoid, suoid
Bug: Add a MkdirOperation for chained S3 operations during mkdir
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 26/Jan/21 16:18
Affects Version/s: 3.3.1


on S3A mkdirs is implemented as a walk up the tree looking for any parent dir, exiting fast when found; failing if a file is found.

Proposed: pull this out into its own MkDir operation with the goals of efficiency, track duration and isolate/test

* specific callbacks to getFileStatus and  createFakeDirectory(key);
* Move from simple getFileStatus call to probes assuming dir exists before looking for a file: HEAD + path /, LIST path,
* the probe for a file  HEAD path will only succeed in a failure mode where higher cost is a detail

I think I'd also like to start looking at what we can do with some context tracking/telemetry where something provides an operation ID and ability to attach this to the requests
Bug: ABFS : add high performance listStatusIterator
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Jan/21 04:34
Affects Version/s: 3.4.0

The ABFS connector now implements listStatusIterator() with
asynchronous prefetching of the next page(s) of results.
For listing large directories this can provide tangible speedups.

If for any reason this needs to be disabled, set
fs.azure.enable.abfslistiterator to false.
Bug: ABFS: Change default Readahead Queue Depth from num(processors) to const
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 22/Mar/21 11:05
Affects Version/s: 3.3.1

The default value of readahead queue depth is currently set to the number of available processors. However, this can result in one inputstream instance consuming more processor time. To ensure equal thread allocation during read for all inputstreams created in a session, we change the default readahead queue depth to a constant (2).
Bug: Upgrade JUnit to 4.13.1
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Mar/21 16:36

A reported vulnerability reported in JUnit4.7-4.13.
The JUnit4 test rule [TemporaryFolder on unix-like systems does not limit access to created files|https://github.com/junit-team/junit4/security/advisories/GHSA-269g-pwp5-87pp]
Bug: Magic committer to downgrade abort in cleanup if list uploads fails with access denied
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Feb/21 11:21
Affects Version/s: 3.3.0

If the caller doesn't have "s3:ListBucketMultipartUploads" permissions on a bucket, then magic committer cleanup fails. 
{code}
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:247)
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:112)
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:315)
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:407)
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:311)
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:286)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.listMultipartUploads(S3AFileSystem.java:4549)
	at org.apache.hadoop.fs.s3a.commit.CommitOperations.listPendingUploadsUnderPath(CommitOperations.java:361)
	at org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.abortPendingUploadsInCleanup(AbstractS3ACommitter.java:671)
	at org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.cleanup(AbstractS3ACommitter.java:770)
{code}
it should just swallow this, given it's best effort
Bug: property 'ssl.server.keystore.location' has not been set in the ssl configuration file
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 26/Feb/21 11:01
Affects Version/s: 2.8.5

I trying to install hadoop cluster HA , but datanode does not start properly; I get this errror:

2021-02-23 17:13:26,934 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: java.security.GeneralSecurityException: The property 'ssl.server.keystore.location' has not been set in the ssl configuration file.
at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:199)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:905)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1303)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:481)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2609)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2497)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2544)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2729)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2753)
Caused by: java.security.GeneralSecurityException: The property 'ssl.server.keystore.location' has not been set in the ssl configuration file.
at org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory.init(FileBasedKeyStoresFactory.java:152)
at org.apache.hadoop.security.ssl.SSLFactory.init(SSLFactory.java:148)
at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:197)
... 8 more

But in my ssl-server.xml i correctly set this property:

<property>
<name>ssl.server.keystore.location</name>
<value>/data/hadoop/server.jks</value>
<description>Keystore to be used by clients like distcp. Must be
specified.
</description>
</property>

<property>
<name>ssl.server.keystore.password</name>
<value>xxxx</value>
<description>Optional. Default value is "".
</description>
</property>

<property>
<name>ssl.server.keystore.keypassword</name>
<value>xxxxx</value>
<description>Optional. Default value is "".
</description>
</property>

<property>
<name>ssl.server.keystore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>

Do you have any suggestion to solve this problem?
my hadoop version is: 2.8.5
java version: 8
SO: centos 7
Bug: Allow ProtobufRpcEngine to be extensible
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 22/Mar/21 19:33

The ProtobufRpcEngine class doesn't allow for new RpcEngine implementations to extend some of its inner classes (e.g. Invoker and Server.ProtoBufRpcInvoker). Also, some of its methods are long enough such that overriding them would result in a lot of code duplication (e.g. Invoker#invoke and Server.ProtoBufRpcInvoker#call).

When implementing a new RpcEngine, it would be helpful to reuse most of the code already in ProtobufRpcEngine. This would allow new fields to be added to the RPC header or message with minimal code changes.
Bug: skip-dir option is not processed by Yetus
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Mar/21 18:55

Running test patch locally does not work anymore after the Yetus upgrade


{code:bash}
dev-support/bin/test-patch --plugins="maven,checkstyle" --test-parallel=true patch-file.patch
{code}

Error is 

{code:bash}
Testing  patch on trunk.
ERROR: Unprocessed flag(s): --skip-dir

    environment {
        SOURCEDIR = 'src'
        // will also need to change notification section below
        PATCHDIR = 'out'
        DOCKERFILE = "${SOURCEDIR}/dev-support/docker/Dockerfile"
        YETUS='yetus'
        // Branch or tag name.  Yetus release tags are 'rel/X.Y.Z'
        YETUS_VERSION='rel/0.13.0'
/skip-
                        # URL for user-side presentation in reports and such to our artifacts

 _____     _ _                _
|  ___|_ _(_) |_   _ _ __ ___| |
| |_ / _` | | | | | | '__/ _ \ |
|  _| (_| | | | |_| | | |  __/_|
|_|  \__,_|_|_|\__,_|_|  \___(_)



| Vote |    Subsystem |  Runtime   | Comment
============================================================================
|  -1  |       yetus  |   0m 05s   | Unprocessed flag(s): --skip-dir
{code}

It seems that the "{{--skip-dir}}" option supported Yetus release prior to 0.11.
Bug: Replace currentTimeMillis with monotonicNow in elapsed time
Status: Resolved | Priority: Major | Resolution: Won't Fix
Created: 08/Jan/21 20:39

I noticed that there is a widespread incorrect usage of {{System.currentTimeMillis()}}  throughout the hadoop code.

For example:

{code:java}
// Some comments here
long start = System.currentTimeMillis();
while (System.currentTimeMillis() - start < timeout) {
  // Do something
}
{code}

Elapsed time should be measured using `monotonicNow()`.
Bug: Decommission S3Guard
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 08/Feb/21 18:10
Affects Version/s: 3.3.0

S3Guard is obsolete in functionality and performance. Remove, carefully.

Goals: 
* make it easy for the 3.3.1 release to turn off s3guard
* remove it from the code for ease of maintenance
* strip down the tests
* be happy
Bug: ITestAssumeRole.testAssumeRoleBadInnerAuth failure
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Jan/21 16:13
Affects Version/s: 3.3.0

failure of test {{ITestAssumeRole.testAssumeRoleBadInnerAuth}} where a failure was expected, but the error text was wrong.

Either STS has changed its error text or something is changing where the failure happens.

Given the nature of the test, it may be simplest to keep the expectation of an FS init faiure, but remove the text match
Bug: Collect more S3A IOStatistics
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Jan/21 14:23

collect more stats of 
* how long FS API calls are taking
* what is going wrong

API
* Duration of rename(), delete() other common API Calls.
* openFile: how long the open operation is taking, split of open and openfile
* time to initialize FS instance (maybe better in FileSystem.cache?)
* create: Time to complete the create checks
* finishedWrite: time to execute finishedWrite operations (which may include bulk deletes)

Failure tracking: what failures are we seeing as counters
Interrupts, connections, auth failures:

This would be done in S3ARetryPolicy with an IOStatisticsStore passed in.
Bug: Change String.getBytes() to DFSUtil.string2Bytes(String) to avoid Unsupported Encoding Exception
Status: Open | Priority: Major | Resolution: 
Created: 27/Mar/21 09:24

Hello,
I found that DFSUtil.string2Bytes(String) can be used here instead of String.getBytes(). Otherwise, the API String.getBytes() may cause potential risk of UnsupportedEncodingException since the behavior of this method when the string cannot be encoded in the default charset is unspecified. One recommended API is DFSUtil.string2Bytes(String) which provides more control over the encoding process and can avoid this exception.
Bug: ABFS: Toggle Store Mkdirs request overwrite parameter
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Feb/21 03:31
Affects Version/s: 3.3.1

The call to mkdirs with overwrite set to true results in an additional call to set properties (LMT update, etc) at the backend, which is not required for the HDFS scenario. Moreover, mkdirs on an existing file path returns success. This PR provides an option to set the overwrite parameter to false, and ensures that mkdirs on a file throws an exception.
Bug: Upgrade hadoop default version in hadoop.tools.dynamometer.TestDynamometerInfra
Status: Resolved | Priority: Major | Resolution: Duplicate
Created: 15/Jan/21 04:13

Currently hadoop.tools.dynamometer.TestDynamometerInfra in trunk is failed.

{code}
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.tools.dynamometer.TestDynamometerInfra
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.531 s <<< FAILURE! - in org.apache.hadoop.tools.dynamometer.TestDynamometerInfra
[ERROR] org.apache.hadoop.tools.dynamometer.TestDynamometerInfra  Time elapsed: 0.53 s  <<< ERROR!
java.io.FileNotFoundException: http://mirrors.ocf.berkeley.edu/apache/hadoop/common/hadoop-3.1.3/hadoop-3.1.3.tar.gz
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1923)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1523)
        at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1506)
        at org.apache.hadoop.tools.dynamometer.DynoInfraUtils.fetchHadoopTarball(DynoInfraUtils.java:151)
        at org.apache.hadoop.tools.dynamometer.TestDynamometerInfra.setupClass(TestDynamometerInfra.java:176)
{code}
Bug: ABFS: ITestAzureBlobFileSystemCheckAccess test failure if test doesn't have oauth keys
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 18/Feb/21 13:52
Affects Version/s: 3.3.1

If the test isn't set up with oauth tokens, then ITestAzureBlobFileSystemCheckAccess raises illegal argument exception

# Should this downgrade to skip?
# if not:  someone needs to document how to get get oauth tokens for azure storage as the instructions documenting my last previous successful event are out of date with the latest Azure UI.
Bug: ABFS: Disable throttling update for auth failures
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 11/Mar/21 09:21
Affects Version/s: 3.3.1

Throttling metrics are updated post the execution of each request. Failures related to fetching access tokens and signing requests do not occur at the Store. Hence, such operations should not contribute to the measured Store failures, and are therefore excluded from the metric update for throttling.
Bug: Distcp parallel file copy breaks the modification time
Status: Patch Available | Priority: Major | Resolution: 
Created: 29/Mar/21 15:11

The commit HADOOP-11794. Enable distcp to copy blocks in parallel. (bf3fb585aaf2b179836e139c041fc87920a3c886) broke the modification time of large files.

 

In CopyCommitter.java inside concatFileChunks Filesystem.concat is called which changes the modification time therefore the modification times of files copeid by distcp will not match the source files. However this only occurs for large enough files, which are copied by splitting them up by distcp.

In concatFileChunks before calling concat extract the modification time and apply that to the concatenated result-file after the concat. (probably best -after- before the rename()).
Bug: Fix TestLogLevel
Status: Open | Priority: Major | Resolution: 
Created: 11/Mar/21 13:05

Found when fixing HADOOP-17572: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2755/2/testReport/org.apache.hadoop.log/TestLogLevel/testLogLevelByHttp/
{noformat}
Expected to find 'Unrecognized SSL message' but got unexpected exception:javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
 at sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(SSLSocketInputRecord.java:448)
 at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:184)
 at sun.security.ssl.SSLTransport.decode(SSLTransport.java:108)
 at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1143)
 at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1054)
 at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:394)
 at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
 at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
 at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)
 at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:186)
 at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
 at org.apache.hadoop.log.LogLevel$CLI.connect(LogLevel.java:271)
 at org.apache.hadoop.log.LogLevel$CLI.process(LogLevel.java:293)
 at org.apache.hadoop.log.LogLevel$CLI.doGetLevel(LogLevel.java:234)
 at org.apache.hadoop.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:127)
 at org.apache.hadoop.log.LogLevel$CLI.run(LogLevel.java:110)
 at org.apache.hadoop.log.TestLogLevel.getLevel(TestLogLevel.java:301)
 at org.apache.hadoop.log.TestLogLevel.access$000(TestLogLevel.java:63)
 at org.apache.hadoop.log.TestLogLevel$1.call(TestLogLevel.java:279)
 at org.apache.hadoop.log.TestLogLevel$1.call(TestLogLevel.java:275)
 at org.apache.hadoop.security.authentication.KerberosTestUtils$1.run(KerberosTestUtils.java:102)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.authentication.KerberosTestUtils.doAs(KerberosTestUtils.java:99)
 at org.apache.hadoop.security.authentication.KerberosTestUtils.doAsClient(KerberosTestUtils.java:115)
 at org.apache.hadoop.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:275)
 at org.apache.hadoop.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:234)
 at org.apache.hadoop.log.TestLogLevel.testLogLevelByHttp(TestLogLevel.java:354)
{noformat}
Bug: Upgrade tomcat-embed-core to 7.0.108
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Mar/21 18:27

[CVE-2021-25329|https://nvd.nist.gov/vuln/detail/CVE-2021-25329] critical severity.
Impact: [CVE-2020-9494|https://nvd.nist.gov/vuln/detail/CVE-2020-9494]
7.0.0-7.0.107 are all affected by the vulnerability.
Bug: TFileDumper creates malformed hex sample output
Status: Open | Priority: Major | Resolution: 
Created: 27/Mar/21 17:38
Affects Version/s: 3.2.2

{{org.apache.hadoop.io.file.tfile.TFileDumper}} creates malformed hex output:
{code}
out.print("0X");
for (int j = 0; j < sampleLen; ++j) {
  byte b = key[i];
  out.printf("%X", b);
}
{code}
It should use {{"%02X"}} (zero-pad and use width 2) as formatting pattern; otherwise bytes < 16 will only have on hex char and for example both {{1, 0}} and {{16}} would have the result {{"10"}}.