av9ash/GitBugs huggingface dataset:
Bug: Operating on / in ChecksumFileSystem throws NPE
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 27/Sep/24 08:25

Operating on / on ChecksumFileSystem throws NPE

{noformat}
java.lang.NullPointerException
	at org.apache.hadoop.fs.Path.<init>(Path.java:151)
	at org.apache.hadoop.fs.Path.<init>(Path.java:130)
	at org.apache.hadoop.fs.ChecksumFileSystem.getChecksumFile(ChecksumFileSystem.java:121)
	at org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:774)
	at org.apache.hadoop.fs.ChecksumFileSystem.setReplication(ChecksumFileSystem.java:884)
{noformat}

Internally I observed it for SetPermission but on my Mac LocalFs doesn't let me setPermission on "/", so I reproduced it via SetReplication which goes through the same code path
Bug: hadoop-client-runtime exclude dnsjava InetAddressResolverProvider
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 24/Sep/24 14:32
Affects Version/s: 3.4.1

[https://github.com/dnsjava/dnsjava/issues/338]

 
{code:java}
Exception in thread "main" java.util.ServiceConfigurationError: java.net.spi.InetAddressResolverProvider: Provider org.apache.hadoop.shaded.org.xbill.DNS.spi.DnsjavaInetAddressResolverProvider not found
    at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
    at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1219)
    at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
    at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
    at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
    at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
    at java.base/java.util.ServiceLoader.findFirst(ServiceLoader.java:1812)
    at java.base/java.net.InetAddress.loadResolver(InetAddress.java:508)
    at java.base/java.net.InetAddress.resolver(InetAddress.java:488)
    at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1826)
    at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:1139)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1818)
    at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1931)
    at org.apache.logging.log4j.core.util.NetUtils.getLocalHostname(NetUtils.java:56)
    at org.apache.logging.log4j.core.LoggerContext.lambda$setConfiguration$0(LoggerContext.java:625) {code}
Bug: NPE on maven enforcer with -Pnative on arm mac
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 30/Sep/24 16:28
Affects Version/s: 3.4.1

If you try to build on an arm mac with -Pnative you get an npe in enforcer.

This is independent of whether or not maven can actually compare the native code.

Upgrading maven to 3.5.0 works.
Bug: Support force close a DomainSocket for server service
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 03/Sep/24 08:58

Currently the DomainSocket#close will check the reference count to be 0 before it goes on to close the socket. In server service case, server calls DomainSocket#listen will add 1 to the reference count. When trying to close the server socket which is blocked by accept, the close call will doing endless count > 0 check, which prevent the server socket to be closed.
Bug: MetricsSystemImpl should not print INFO message in CLI
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Sep/24 18:11
Affects Version/s: 3.4.0

Below is an example:
{code}
# hadoop fs  -Dfs.s3a.bucket.probe=0 -Dfs.s3a.change.detection.version.required=false -Dfs.s3a.change.detection.mode=none -Dfs.s3a.endpoint=http://some.site:9878 -Dfs.s3a.access.keysome=systest -Dfs.s3a.secret.key=8...1 -Dfs.s3a.endpoint=http://some.site:9878  -Dfs.s3a.path.style.access=true -Dfs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem   -ls  -R s3a://bucket1/
24/09/17 10:47:48 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
24/09/17 10:47:48 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
24/09/17 10:47:48 INFO impl.MetricsSystemImpl: s3a-file-system metrics system started
24/09/17 10:47:48 WARN impl.ConfigurationHelper: Option fs.s3a.connection.establish.timeout is too low (5,000 ms). Setting to 15,000 ms instead
24/09/17 10:47:50 WARN s3.S3TransferManager: The provided S3AsyncClient is an instance of MultipartS3AsyncClient, and thus multipart download feature is not enabled. To benefit from all features, consider using S3AsyncClient.crtBuilder().build() instead.
drwxrwxrwx   - root root          0 2024-09-17 10:47 s3a://bucket1/dir1
24/09/17 10:47:53 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system...
24/09/17 10:47:53 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped.
24/09/17 10:47:53 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown complete. 
{code}
Bug: upgrade dnsjava to 3.6.1 due to CVEs
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 25/Jul/24 11:48

See https://github.com/apache/hadoop/pull/6955 - but this is missing the necessary change to LICENSE-binary (which already has an out of date version for dnsjava).

* CVE-2024-25638 https://github.com/advisories/GHSA-cfxw-4h78-h7fw
* https://github.com/advisories/GHSA-mmwx-rj87-vfgr
* https://github.com/advisories/GHSA-crjg-w57m-rqqf
Bug: S3A: AWS SDK 2.25.53 warnings logged about transfer manager not using CRT client
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 13/Sep/24 15:44
Affects Version/s: 3.4.0


When an S3 transfer manager is created for renaming/download a new message is logged telling off the caller for not using the CRT client.

{code}
5645:2024-09-13 16:29:17,375 [setup] WARN  s3.S3TransferManager (LoggerAdapter.java:warn(225)) - The provided S3AsyncClient is an instance of MultipartS3AsyncClient, and thus multipart download feature is not enabled. To benefit from all features, consider using S3AsyncClient.crtBuilder().build() instead.
{code}

This is a change in the SDK to tell us developers off -yet it is visible to end users who don't benefit from it and for which it only creates confusion.

It appears to have been downgraded to debug in the AWS trunk code in PR "S3 Async Client - Multipart download (#5164) -but:

* it is too late to upgrade and qualify a new version for 3.4.1; downgrading is all we can do
* there is no guarantee this log message or similar will reoccur.

Plan
1. Revert from 3.4.1
2. lift code from cloudstore library which uses reflection to access and manipulate log4j logs where present
3. downgrade all transfer manager log levels to NONE. 
4. File an AWS report about how this is an incompatible regression, identify how their process can evolve, particularly in the area of code guidelines about safe logging use.

I also intend to tighten up our review process to support more rigorous detection of new .warn() messages in the AWS SDK. I'm going to propose that as well as requiring review of our test/CLI output, we require ripgrep scans of .warn(/.error( in SDK source, audit of any new changes. by saving the output of the previous iteration, it'll be straightforward to identify new changes -but not changes in codepaths which change their frequency of appearance.

I think we should revisit whether or not to move off the xfer manager in the past. We've discussed it in the past, and avoided it just due to maintenance costs. However, it is pushing maintenance costs anyway.

meanwhile: no new AWS SDK updates until we are confident we have our processes under control.
Bug: dtutil cancel command fails with NullPointerException
Status: In Progress | Priority: Major | Resolution: 
Created: 14/Sep/24 15:01

The hadoop dtutil cancel command gives NullPointerException.
{code}
java.lang.NullPointerException
	at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:422)
	at org.apache.hadoop.security.User.<init>(User.java:48)
	at org.apache.hadoop.security.User.<init>(User.java:43)
	at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1418)
	at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1402)
	at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.getUser(AbstractDelegationTokenIdentifier.java:80)
	at org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier.getUser(DelegationTokenIdentifier.java:81)
	at org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier.toString(DelegationTokenIdentifier.java:91)
	at java.base/java.lang.String.valueOf(String.java:2951)
	at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
	at org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier.stringifyToken(DelegationTokenIdentifier.java:123)
	at org.apache.hadoop.hdfs.DFSClient$Renewer.cancel(DFSClient.java:804)
	at org.apache.hadoop.security.token.Token.cancel(Token.java:527)
	at org.apache.hadoop.security.token.DtFileOperations.removeTokenFromFile(DtFileOperations.java:283)
	at org.apache.hadoop.security.token.DtUtilShell$Remove.execute(DtUtilShell.java:320)
	at org.apache.hadoop.tools.CommandShell.run(CommandShell.java:72)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
	at org.apache.hadoop.security.token.DtUtilShell.main(DtUtilShell.java:361)
{code}

This is because it is trying to create a User and in User.java we are calling [this|https://github.com/apache/hadoop/blob/ea6e0f7cd58d0129897dfc7870aee188be80a904/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/User.java#L48] line.

But due to this direct call it doesn't initialize the auth_to_local rules and causes NPE in [here|https://github.com/apache/hadoop/blob/ea6e0f7cd58d0129897dfc7870aee188be80a904/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java#L422] in KerberosName.java
Bug: Fix test TestServiceInterruptHandling.testRegisterAndRaise
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 09/Aug/24 11:04

If test on some slow server, testRegisterAndRaise may fail, error stack are

{code}

[ERROR] testRegisterAndRaise(org.apache.hadoop.service.launcher.TestServiceInterruptHandling) Time elapsed: 0.513 s <<< FAILURE! java.lang.AssertionError: interrupt data at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertNotNull(Assert.java:713) at org.apache.hadoop.service.launcher.TestServiceInterruptHandling.testRegisterAndRaise(TestServiceInterruptHandling.java:48) at
{code}


h3. backporting note:

there are two commits for this issue, please cherrypick both when backporting
Bug: S3A: Commiter ITests failing unless job.id set
Status: Open | Priority: Major | Resolution: 
Created: 13/Sep/24 16:37
Affects Version/s: 3.4.1

the job.id maven setup is broken on parallel runs: unless explicitly set on the command line tests which try to parse its traling bits as a number will fail.

{code}
[ERROR] testBinding[File committer in task-fs=[]-task=[file]-[class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter]](org.apache.hadoop.fs.s3a.commit.ITestS3ACommitterFactory)  Time elapsed: 0.181 s  <<< ERROR!
java.lang.Exception: Failed to parse b-00
{code}

Emergency workaround: set it
{code}
-Djob.id=0001
{code}
Bug: Add Options.Rename.THROW_NON_ATOMIC
Status: Open | Priority: Major | Resolution: 
Created: 14/Aug/24 14:16
Affects Version/s: 3.3.6

I propose we add an option `Options.Rename.THROW_NON_ATOMIC` to change `rename()` behavior to throw when the underlying filesystem's rename operation is not atomic.

This would be useful for callers that expect to perform an atomic op but want to fail if when an atomic rename fails.

 

At first this might seem something that can be done by querying capabilities of the filesystem but that would only work on real filesystems. A motivating example would be a virtual filesystem for which paths can resolve to any concrete filesystem (s3, etc). If `rename()` is called with two virtual paths that resolve to different filesystems (s3 and gcs for example) then obviously the operation can't be atomic since bytes must be copied from one fs to another.

 

What do you think [~steve_l] ?
Bug: Upgrade maven-shade-plugin to 3.6.0
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 05/Sep/24 16:35

I found an issue when testing with Jackson 2.18.0-rc1.
Hadoop bundles the Jackson and other 3rd party classes into Hadoop fat jars using maven-shade-plugin. 
Jackson is a Multi-Release jar. https://openjdk.org/jeps/238
The most recent jackson-core jars have classes that support META-INF/versions/21 classes. The existing maven-shade-plugin version in Hadoop build fails with ASM issues with Java 21 classes. maven-shade-plugin handles them fine.
Bug: Improve Logging for Empty or Missing Socket Factory Configuration
Status: Open | Priority: Major | Resolution: 
Created: 05/Sep/24 08:48
Affects Version/s: 3.3.6

In the current implementation of the {{NetUtils.getDefaultSocketFactory}} method, when the configuration parameter {{hadoop.rpc.socket.factory.class.default}} is not set or is empty, the method silently defaults to using the standard {{{}SocketFactory{}}}. However, this lack of logging can make it difficult to diagnose configuration issues. We propose enhancing the code with logging to warn users when this situation occurs.
 
*Expected Behavior:* 
When the {{hadoop.rpc.socket.factory.class.default}} configuration parameter is missing or empty, a warning log should be generated, informing the user that the system is defaulting to the standard {{{}SocketFactory{}}}.
 
*How-to-Fix:*
We propose to expose this relationship by logging a warning when the {{hadoop.rpc.socket.factory.class.default}} parameter is missing or empty.
Bug: Logging Enhancement for Shutdown Timeout Configuration in ShutdownHookManager
Status: Open | Priority: Major | Resolution: 
Created: 05/Sep/24 08:46
Affects Version/s: 3.3.6

The issue was encountered in scenarios where Hadoop services are expected to shut down gracefully within a specified time frame. Misconfigured shutdown timeouts can lead to abrupt terminations or extended shutdown periods. The original code does not provide sufficient visibility into misconfigurations related to the shutdown timeout duration, which can result in unexpected behavior during service shutdowns.
 
*Expected Behavior:* 
The system should log a warning message when the configured shutdown timeout is less than the minimum allowed value, and it should reset the duration to this minimum value. Additionally, the user should be informed to set the {{hadoop.service.shutdown.timeout}} configuration parameter to a valid value.
 
*How-to-Fix:* 
Enhance the {{getShutdownTimeout}} method by adding logging statements that warn when the shutdown timeout is below the allowed minimum, and reset it to the minimum value. This logging will help users identify and correct misconfigurations.
Bug: Improve Logging for User Account Update Time Configuration in ShellBasedIdMapping Constructor
Status: Open | Priority: Major | Resolution: 
Created: 05/Sep/24 08:45
Affects Version/s: 3.3.6

The original implementation of the {{ShellBasedIdMapping}} constructor lacks detailed logging about the configuration of the user account update time ({{{}usergroupid.update.millis{}}}). When this time is set to a value less than 1 minute, the system automatically adjusts it to the minimum of 1 minute without providing clear feedback to the user. This can lead to confusion and difficulty in troubleshooting if the user is unaware that their configuration was overridden. By adding detailed logs, we can ensure that users are informed about the exact configuration being applied, making it easier to diagnose potential issues related to configuration.
 
*Expected Behavior:* 
When the {{ShellBasedIdMapping}} constructor is invoked, the system should log the user-configured update time for user accounts. If the update time is below the minimum threshold (1 minute), the system should log a warning indicating that the minimum value of 1 minute will be used instead.
 
*How-to-Fix:*
We propose to expose such a relationship by adding detailed logging statements to inform the user about the configuration status.
Bug: Enhance Logging for Localized Trash Configuration in moveToAppropriateTrash Method
Status: Open | Priority: Major | Resolution: 
Created: 05/Sep/24 08:43
Affects Version/s: 3.3.6

The {{moveToAppropriateTrash}} method in {{Trash.java}} handles file deletion by moving files to the appropriate trash directory based on the configuration. However, the current implementation lacks logging to indicate whether the localized trash feature is enabled or not. This enhancement proposal introduces logging to provide better visibility into the decision-making process when moving files to the trash, particularly in environments where the ViewFileSystem is used.
 
*Expected Behavior:* 
When {{CONFIG_VIEWFS_TRASH_FORCE_INSIDE_MOUNT_POINT}} is enabled, the system should log that the localized trash feature is enabled and that the {{Trash}} is being initialized with a {{ViewFileSystem}} object. Conversely, if the feature is disabled, the system should log that it is using a fully resolved {{FileSystem}} object.
 
*How-to-Fix:*
This logging enhancement adds informative messages to indicate whether the localized trash feature is enabled or not, thus providing more transparency during the file deletion process. The test cases verify the functionality under different configurations.
Bug:  Logging Enhancements for setConf Method in WhitelistBasedResolver
Status: Open | Priority: Major | Resolution: 
Created: 05/Sep/24 08:42
Affects Version/s: 3.3.6

The original implementation does not provide any logging information about the status of the variable whitelist configuration. As a result, it is difficult to determine whether the variable whitelist is enabled or disabled, which file is being used, and the expiry time set for the whitelist. This lack of visibility can lead to confusion during troubleshooting and configuration, as administrators may not be aware if the variable whitelist is correctly configured or if it is using the intended file and settings.
 
*Expected Behavior:* 
The system should log detailed information about whether the variable whitelist is enabled, the file being used, and the expiry time. This will assist in debugging and ensure that the configuration is applied as expected.
 
*How-to-Fix:*
We propose exposing the configuration behavior through logging. This includes indicating whether the variable whitelist is enabled, specifying the file in use, and logging the expiry time to provide greater visibility into the configuration process.
Bug: Enhance Logging for Timeout Configuration in Client.getTimeout
Status: Open | Priority: Major | Resolution: 
Created: 05/Sep/24 08:40
Affects Version/s: 3.3.6

The original implementation of the {{Client.getTimeout}} method does not provide detailed logging when it falls back to default behaviors based on the configuration. Specifically, when the {{ipc.client.rpc.timeout}} parameter is set to an invalid value (less than or equal to zero), the method either uses the ping interval as the timeout or returns a default value of {{-1}} depending on whether {{ipc.client.ping}} is enabled. This lack of logging can make it difficult for users to understand why certain timeout values are being used, leading to potential confusion and difficulty in troubleshooting. By adding logging statements, the system can inform users about the decisions being made based on their configurations.
 
*Expected Behavior:* 
When the {{Client.getTimeout}} method is invoked, the system should log detailed information about the timeout configuration, including whether the {{ipc.client.ping}} is enabled or disabled, and how this affects the timeout value being used.
 
*How-to-Fix:*
We propose to enhance the method by adding logging statements that inform the user about the configuration decisions being made. This will improve transparency and make the system easier to debug.
Bug: removal of gcm TLS cyphers blocking abfs access "No negotiable cipher suite"
Status: Open | Priority: Major | Resolution: 
Created: 02/Sep/24 12:24
Affects Version/s: 3.4.0

we've seen instances of client-abfs TLS negotiation failing "No negotiable cipher suite". this can be fixed by switching to using "Default_JSSE_with_GCM" as the SSL options.

However, DelegatingSSLSocketFactory "Default" attempts OpenSSL, falling back to 
{code}
Default indicates Ordered, preferred OpenSSL, if failed to load then fall
 back to Default_JSSE
{code}

And " Default_JSSE is not truly the the default JSSE implementation because
the GCM cipher is disabled when running on Java "

What does that mean? it means that if you use the "Default" TLS option of "try openssl and fall back to java" doesn't ever turn on gcm encryption.

Proposed:
* "Default" falls back to GCM
* add an option {{Default_JSSE_No_GCM}}

Once we move off java8 turning off GCM is no longer needed for performance, hopefully (benchmarks would be good here)
Bug: upgrade to jackson 2.14.3
Status: Open | Priority: Major | Resolution: 
Created: 19/Jul/24 21:37

Follow up to HADOOP-18332

I have what I believe fixes the Jackson JAX-RS incompatibility.

https://github.com/pjfanning/jsr311-compat/

The reason that I want to start by just going to Jackson 2.14 is that Jackson has new StreamReadConstraints in Jackson 2.15 to protect against malicious JSON inputs. The constraints are generous but can cause issues with very large or deeply nested inputs.

Jackson has had a lot of security hardening fixes recently and it seems problematic to be stuck on an unsupported version of Jackson (2.12).
Bug: add JacksonUtil to centralise some code
Status: Open | Priority: Major | Resolution: 
Created: 21/Jul/24 14:54
Affects Version/s: 3.4.0

To future proof Hadoop against Jackson changes, it makes sense to not just create ObjectMappers and JsonFactories in many different places in the Hadoop code.

One of the main drivers of this is https://www.javadoc.io/doc/com.fasterxml.jackson.core/jackson-core/latest/com/fasterxml/jackson/core/StreamReadConstraints.html 

Jackson 3 (not yet scheduled for release) has some fairly big API and behaviour changes too.