av9ash/GitBugs huggingface dataset:
Bug: Upgrade aws-java-sdk to 1.12.720
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 13/May/24 14:10
Affects Version/s: 3.3.6

Update to the latest AWS SDK, to stop anyone worrying about the ion library CVE https://nvd.nist.gov/vuln/detail/CVE-2024-21634

This isn't exposed in the s3a client, but may be used downstream. 

on v2 sdk releases, the v1 sdk is only used during builds; 3.3.x it is shipped
Bug: [DOC] Drop Migrating from Apache Hadoop 1.x to Apache Hadoop 2.x
Status: Open | Priority: Minor | Resolution: 
Created: 07/May/24 18:35

Reading the docs, found this page, which is pretty irrelevant in current context or upcoming 3.x releases, can explore dropping it

https://apache.github.io/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html
Bug: Fix TestZKSignerSecretProvider failing unit test
Status: Open | Priority: Minor | Resolution: 
Created: 23/Apr/24 07:14
Affects Version/s: 3.4.0

* {{TestZKSignerSecretProvider and }}{{{}TestRandomSignerSecretProvider{}}}{{{{}} unit test o{}}}ccasional failure
 * The reason was that the MockZKSignerSecretProvider class rollSecret method is {{synchronized}}
 * {{{}s{}}}ometimes verify (secretProvider, timeout (timeout). AtLeastOnce ()). RollSecret () method first in RolloverSignerSecretProvider scheduler thread lock, this results in a timeout
Bug: Invalid GPG commands in Download page
Status: Resolved | Priority: Trivial | Resolution: Fixed
Created: 29/Jun/24 18:47

Instructions in [Download page|https://hadoop.apache.org/releases.html] shows GPG commands with {{--}} converted to &ndash;, which makes the commands invalid.

{code}
gpg –import KEYS
gpg –verify hadoop-X.Y.Z-src.tar.gz.asc
{code}
Bug: Switch the baseurl for Centos 8
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 03/Mar/22 17:23
Affects Version/s: 3.4.0

Centos 8 has reached its End-of-Life and thus its packages are no longer accessible from  mirror.centos.org. We need to switch the *baseurl* to vault.centos.org where the packages are archived.

Please see https://www.centos.org/centos-linux-eol/ for more details.
Bug: Disable S3A auditing by default.
Status: Resolved | Priority: Blocker | Resolution: Fixed
Created: 24/Jan/22 13:32
Affects Version/s: 3.3.2

See HADOOP-18091. S3A auditing leaks memory through ThreadLocal references

* Adds a new option fs.s3a.audit.enabled to controls whether or not auditing
is enabled. This is false by default.

* When false, the S3A auditing manager is NoopAuditManagerS3A,
which was formerly only used for unit tests and
during filsystem initialization.

* When true, ActiveAuditManagerS3A is used for managing auditing,
allowing auditing events to be reported.

* updates documentation and tests.
Bug: Distcp: Sync moves filtered file to home directory rather than deleting
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 27/Jan/22 18:21
Affects Version/s: 3.3.2

Distcp sync with snapshot, if the file being copied is renamed to a path which is in the exclusion filter, tries to delete the file.

But instead of deleting, it moves the file to home directory
Bug: Implement paging during S3 multi object delete.
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 03/Feb/22 10:08
Affects Version/s: 3.3.1

 

{*}Error{*}:

Rename operation fails during multi object delete of size more than 1000. We see an exception during multi object delete of more than 1000 keys in one go during rename operation.

 
{noformat}
org.apache.hadoop.fs.s3a.AWSBadRequestException: rename  com.amazonaws.services.s3.model.AmazonS3Exception
: The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: MalformedXML; Request ID: XZ8PGAQHP0FGHPYS; S3 Extended Request ID: vTG8c+koukzQ8yMRGd9BvWfmRwkCZ3fAs/EOiAV5S9E
JjLqFTNCgDOKokuus5W600Z5iOa/iQBI=; Proxy: null), S3 Extended Request ID: vTG8c+koukzQ8yMRGd9BvWfmRwkCZ3fAs/EOiAV5S9EJjLqFTNCgDOKokuus5W600Z5iOa/iQBI=:MalformedXML: The XML you provided was not well-formed or did not validate against our published schema 
(Service: Amazon S3; Status Code: 400; Error Code: MalformedXML; Request ID: XZ8PGAQHP0FGHPYS; S3 Extended Request ID: vTG8c+koukzQ8yMRGd9BvWfmRwkCZ3fAs/EOiAV5S9EJjLqFTNCgDOKokuus5W600Z5iOa/iQBI=; Proxy: null)
        at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:247)
        at org.apache.hadoop.fs.s3a.s3guard.RenameTracker.convertToIOException(RenameTracker.java:267)
        at org.apache.hadoop.fs.s3a.s3guard.RenameTracker.deleteFailed(RenameTracker.java:198)
        at org.apache.hadoop.fs.s3a.impl.RenameOperation.removeSourceObjects(RenameOperation.java:706)
        at org.apache.hadoop.fs.s3a.impl.RenameOperation.completeActiveCopiesAndDeleteSources(RenameOperation.java:274)
        at org.apache.hadoop.fs.s3a.impl.RenameOperation.recursiveDirectoryRename(RenameOperation.java:484)
        at org.apache.hadoop.fs.s3a.impl.RenameOperation.execute(RenameOperation.java:312)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.innerRename(S3AFileSystem.java:1912)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$rename$7(S3AFileSystem.java:1759)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:499)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:444)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2250)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.rename(S3AFileSystem.java:1757)
        at org.apache.hadoop.fs.FileSystem.rename(FileSystem.java:1605)
        at org.apache.hadoop.fs.TrashPolicyDefault.moveToTrash(TrashPolicyDefault.java:186)
        at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:110){noformat}
 

{*}Solution{*}:

So implementing paging of requests to reduce the number of keys in a single request. Page size can be configured

using "fs.s3a.bulk.delete.page.size"
Bug: Setup Jenkins nightly CI for Windows 10
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 20/Feb/22 15:22
Affects Version/s: 3.4.0

Need to run the Jenkins Nightly CI for Windows 10 environment so that we catch any breaking changes for Hadoop on the Windows 10 platform. Need to get Yetus to run on Windows 10 with against the Hadoop codebase.
Bug: toString method of RpcCall throws IllegalArgumentException
Status: Resolved | Priority: Critical | Resolution: Won't Fix
Created: 24/Feb/22 16:00

We have observed breaking tests such as TestApplicationACLs. We have located the root cause, which is HADOOP-18082. It seems that there is a concurrency issue within ProtobufRpcEngine2. When using a debugger, the missing fields are there, hence the suspicion of concurrency problem. The stack trace:
{noformat}
java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:244)
    at org.apache.hadoop.ipc.RpcWritable$ProtobufWrapper.readFrom(RpcWritable.java:131)
    at org.apache.hadoop.ipc.RpcWritable$Buffer.getValue(RpcWritable.java:232)
    at org.apache.hadoop.ipc.ProtobufRpcEngine2$RpcProtobufRequest.getRequestHeader(ProtobufRpcEngine2.java:645)
    at org.apache.hadoop.ipc.ProtobufRpcEngine2$RpcProtobufRequest.toString(ProtobufRpcEngine2.java:663)
    at java.lang.String.valueOf(String.java:3425)
    at java.lang.StringBuilder.append(StringBuilder.java:516)
    at org.apache.hadoop.ipc.Server$RpcCall.toString(Server.java:1328)
    at java.lang.String.valueOf(String.java:3425)
    at java.lang.StringBuilder.append(StringBuilder.java:516)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3097){noformat}
Bug: Hadoop auth does not handle HTTP Headers in a case-insensitive way
Status: Resolved | Priority: Critical | Resolution: Fixed
Created: 10/Feb/22 20:33

According to [RFC-2616|https://www.ietf.org/rfc/rfc2616.txt] HTTP Headers are case-insensitive. There are proxies / load balancers (e.g.: newer versions of HA-proxy) which deliberately make some of the HTTP headers lower-case results in an authentication / authorization failure inside the Hadoop codebase.

I've created a small patch (I'm from Cloudera):  [^hadoop-auth-headers.patch]. This resolves our authentication issue. Can someone please have a look at this?
Bug: Jenkins jobs intermittently failing in the end
Status: Open | Priority: Critical | Resolution: 
Created: 21/Mar/22 07:05

The PR results are not completed sometime:

{noformat}
13:52:57  Recording test results
13:53:18  Remote call on hadoop10 failed
[Pipeline] echo
13:53:18  junit processing: java.io.IOException: Remote call on hadoop10 failed
{noformat}
Ref:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4081/8/consoleFull

Daily Builds also face similar issues:

{noformat}
java.io.IOException: Pipe closed after 0 cycles
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
	at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
	at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
	at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
	at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
	at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Caused: java.io.IOException: Backing channel 'hadoop19' is disconnected.
{noformat}
Ref:
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/810/console

https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/807/console
Bug: Exclude log4j2 dependency from hadoop-huaweicloud module
Status: Resolved | Priority: Critical | Resolution: Duplicate
Created: 24/Jan/22 05:46

[https://github.com/apache/hadoop/pull/3906#issuecomment-1018401121]

The following log4j2 dependencies must be excluded.
{code:java}
[INFO] \- org.apache.hadoop:hadoop-huaweicloud:jar:3.4.0-SNAPSHOT:compile
[INFO]    \- com.huaweicloud:esdk-obs-java:jar:3.20.4.2:compile
[INFO]       +- com.jamesmurty.utils:java-xmlbuilder:jar:1.2:compile
[INFO]       +- com.squareup.okhttp3:okhttp:jar:3.14.2:compile
[INFO]       +- org.apache.logging.log4j:log4j-core:jar:2.12.0:compile
[INFO]       \- org.apache.logging.log4j:log4j-api:jar:2.12.0:compile {code}
Bug: S3A Authentication to support WebIdentity
Status: Open | Priority: Major | Resolution: 
Created: 05/Mar/22 13:21
Affects Version/s: 2.10.2

We are using the latest version of [delta-sharing|https://github.com/delta-io/delta-sharing] which takes advantage of [hadoop-aws|https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html] (S3A) connector in [Hadoop release version 2.10.1|https://github.com/apache/hadoop/tree/rel/release-2.10.1] to mount an AWS S3 File System. In our particular setup, all services are operated in Amazon Elastic Kubernetes Service (EKS) and need to comply to the AWS security concept [IAM roles for service accounts|https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html] (IRSA).

As [Delta sharing S3 connection|https://github.com/delta-io/delta-sharing#s3] doesn't offer any corresponding support, we patched hadoop-aws-2.10.1 to address this need via a new credentials provider class org.apache.hadoop.fs.s3a.OIDCTokenCredentialsProvider. We also upgraded dependency aws-java-sdk-bundle to its latest version 1.12.167 as [AWS WebIdentityTokenCredentialsProvider class|https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/WebIdentityTokenCredentialsProvider.html%E2%80%A6] was not yet available in original version 1.11.271.

We believe that other delta-sharing users could benefit from this short-term contribution. Then sooner or later, delta-sharing owners will have to upgrade their project to a more recent version of hadoop-aws that is probably more widely used. The effort to promote this change is probably low.

Additional note: AWS WebIdentityTokenCredentialsProvider class is directly supported by Spark applications submitted with configuration properties `spark.hadoop.fs.s3a.aws.credentials.provider`and `spark.kubernetes.authenticate.submission.oauthToken` ([doc|https://spark.apache.org/docs/latest/running-on-kubernetes.html#spark-properties]). So bringing this support to Hadoop will primarily be interesting for non-Spark users.
Bug: Implement a variant of ElasticByteBufferPool which uses weak references for garbage collection.
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Feb/22 13:52

Currently in hadoop codebase, we have two classes which implements byte buffers pooling.

One is ElasticByteBufferPool which doesn't use weak references and thus could cause memory leaks in production environment. 

Other is DirectBufferPool which uses weak references but doesn't support caller's preference for either on-heap or off-heap buffers. 

 

The idea is to create an improved version of ElasticByteBufferPool by subclassing it ( as it is marked as public and stable and used widely in hdfs ) with essential functionalities required for effective buffer pooling. This is important for the parent Vectored IO work.
Bug: S3A: Upgrade AWS SDK to V2
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 08/Jan/22 00:22
Affects Version/s: 3.3.1

This task tracks upgrading Hadoop's AWS connector S3A from AWS SDK for Java V1 to AWS SDK for Java V2.

Original use case:
{quote}We would like to access s3 with AWS SSO, which is supported in software.amazon.awssdk:sdk-core:2.*.

In particular, from [https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html], when to set 'fs.s3a.aws.credentials.provider', it must be "com.amazonaws.auth.AWSCredentialsProvider". We would like to support "software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider" which supports AWS SSO, so users only need to authenticate once.
{quote}
Bug: High performance vectored read API in Hadoop
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Feb/22 13:13
Affects Version/s: 3.3.4

Add support for multiple ranged vectored read api in PositionedReadable. The default iterates through the ranges to read each synchronously, but the intent is that FSDataInputStream subclasses can make more efficient readers especially object stores implementation.
Bug: Replace log4j 1.x with reload4j
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 21/Jan/22 02:10

As proposed in the dev mailing list (https://lists.apache.org/thread/fdzkv80mzkf3w74z9120l0k0rc3v7kqk) let's replace log4j 1 with reload4j in the maintenance releases (i.e. 3.3.x, 3.2.x and 2.10.x)
Bug: Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2. CVE-2020-36518
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Mar/22 12:03
Affects Version/s: 3.3.2

CVE-2020-36518
https://github.com/FasterXML/jackson-databind/issues/2816
Bug: UserGroupInformation shouldn't log exception 
Status: Open | Priority: Major | Resolution: 
Created: 21/Feb/22 21:14

In {{UserGroupInformation.doAs}}, we currently create a new {{Exception}} and log it in {{LOG.debug}}. This doesn't look necessary:
{code}
      if (LOG.isDebugEnabled()) {
        LOG.debug("PrivilegedAction [as: {}][action: {}]", this, action,
            new Exception());
      }
{code}
Bug: Produce Windows binaries of Hadoop
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 20/Feb/22 18:31
Affects Version/s: 3.4.0

We currently only provide Linux libraries and binaries. We need to provide the same for Windows. We need to port the [create-release script|https://github.com/apache/hadoop/blob/5f9932acc4fa2b36a3005e587637c53f2da1618d/dev-support/bin/create-release] to run on Windows and produce the Windows binaries.
Bug: ABFS: Skip testEtagConsistencyAcrossRename for Non-HNS accounts
Status: Open | Priority: Major | Resolution: 
Created: 13/Jan/22 09:32
Affects Version/s: 3.3.2

The rename operation is not supported for non-HNS accounts. Hence, tests verifying matching etag for file before and after rename should not be run against non-HNS accounts.
Bug: FileNotFoundException in abfs mkdirs() call
Status: Open | Priority: Major | Resolution: 
Created: 13/Jan/22 10:23
Affects Version/s: 3.3.1

seen in production: calling mkdirs in FileOutputCommitter setupJob is triggering an FNFE

{code}
 java.io.FileNotFoundException: Operation failed: "The specified path does not exist.", 404, PUT, https://bcket.dfs.core.windows.net/table1/_temporary/0?resource=directory&timeout=90, PathNotFound, "The specified path does not exist."
 	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.checkException(AzureBlobFileSystem.java:1131)
 	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.mkdirs(AzureBlobFileSystem.java:445)
 	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2347)
{code}

I suspect what is happening is that while this job is setting up, a previous job is doing cleanup/abort on the same path

assuming that abfs mkdirs is like the posix one -nonatomic, as it goes up/down the chain of parent dirs, something else gets in the way.

if so, this is something which can be handled in the client -when we get an FNFE we could warn and retry.

in the manifest committer each job will have a unique id under _temporary and there will be the option to skip deleting the temp dir entirely, for better coexistence of active jobs.
Bug: ABFS: Toggle Store Mkdirs request overwrite parameter with default value
Status: Open | Priority: Major | Resolution: 
Created: 17/Jan/22 12:23
Affects Version/s: 3.3.2

 mkdirs config is set to overwrite=false as default as the related backend deployment is completed.

PR :- [ABFS: Toggle Store Mkdirs request overwrite parameter by anmolanmol1234 · Pull Request #3830 · apache/hadoop (github.com)|https://github.com/apache/hadoop/pull/3830]
Bug: Utility to identify git commit / Jira fixVersion discrepancies for RC preparation
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 14/Feb/22 09:42
Affects Version/s: 2.10.2

As part of RC preparation,  we need to identify all git commits that landed on release branch, however their corresponding Jira is either not resolved yet or does not contain expected fixVersions. Only when we have git commits and corresponding Jiras with expected fixVersion resolved, we get all such Jiras included in auto-generated CHANGES.md as per Yetus changelog generator.

Proposal of this Jira is to provide such script that can be useful for all upcoming RC preparations and list down all Jiras where we need manual intervention. This utility script should use Jira API to retrieve individual fields and use git log to loop through commit history.

The script should identify these issues:
 # commit is reverted as per commit message
 # commit does not contain Jira number format (e.g. HADOOP-XXXX / HDFS-XXXX etc) in message
 # Jira does not have expected fixVersion
 # Jira has expected fixVersion, but it is not yet resolved
 # Jira has release corresponding fixVersion and is resolved, but no corresponding commit yet found

It can take inputs as:
 # First commit hash to start excluding commits from history
 # Fix Version
 # JIRA Project Name
 # Path of project's working dir
 # Jira server url
Bug: Change URI[] in INodeLink to String[] to reduce memory footprint of ViewFileSystem
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 17/Feb/22 18:37
Affects Version/s: 2.10.2

Around 40k instances of INodeLink each of which is taking memory ranging from 1680bytes to 1160bytes. Multiplying 40k with 1160bytes will give us approximate 45mb.

With changing from URI to String in INodeLink the memory consumed by each of the INodeLink objects has reduced from ~1160 bytes to ~320 bytes. Overall size becomes (40k X 320) 12mb
Bug: Bump aliyun-sdk-oss to 3.13.2 and jdom2 to 2.0.6.1
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 01/Feb/22 00:08
Affects Version/s: 3.4.0

The current aliyun-sdk-oss 3.13.0 is affected by [CVE-2021-33813|https://github.com/advisories/GHSA-2363-cqg2-863c] due to jdom 2.0.6. maven-shade-plugin is also affected by the CVE. 

Bumping aliyun-sdk-oss to 3.13.2 and jdom2 to 2.0.6.1 will resolve this issue
{code:java}
[INFO] +- org.apache.maven.plugins:maven-shade-plugin:jar:3.2.1:provided
[INFO] |  +- org.apache.maven.shared:maven-artifact-transfer:jar:0.10.0:provided
[INFO] |  +- org.jdom:jdom2:jar:2.0.6:provided
......
[INFO] +- com.aliyun.oss:aliyun-sdk-oss:jar:3.13.1:compile
[INFO] |  +- org.jdom:jdom2:jar:2.0.6:compile
{code}
Bug: s3a prefetching to use SemaphoredDelegatingExecutor for submitting work
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Mar/22 12:49
Affects Version/s: 3.4.0

Use SemaphoredDelegatingExecutor for each to stream to submit work, if possible, for better fairness in processes with many streams.

this also takes a DurationTrackerFactory to count how long was spent in the queue, something we would want to know

if cherrypicking, follow with HADOOP-18466 and HADOOP-18455
Bug: Collect IOStatistics during S3A prefetching 
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 29/Mar/22 16:13
Affects Version/s: 3.4.0

There is a lot more happening in reads, so there's a lot more data to collect and publish in IO stats for us to view in a summary at the end of processes as well as get from the stream while it is active.

Some useful ones would seem to be:

counters
 * is in memory. using 0 or 1 here lets aggregation reports count total #of memory cached files.
 * prefetching operations executed
 * errors during prefetching

gauges
 * number of blocks in cache
 * total size of blocks
 * active prefetches
+ active memory used

duration tracking count/min/max/ave
 * time to fetch a block
 * time queued before the actual fetch begins
 * time a reader is blocked waiting for a block fetch to complete

and some info on cache use itself
 * number of blocks discarded unread
 * number of prefetched blocks later used
 * number of backward seeks to a prefetched block
 * number of forward seeks to a prefetched block

the key ones I care about are
 # memory consumption
 # can we determine if cache is working (reads with cache hit) and when it is not (misses, wasted prefetches)
 # time blocked on executors

The stats need to be accessible on a stream even when closed, and aggregated into the FS. once we get per-thread stats contexts we can publish there too and collect in worker threads for reporting in task commits
Bug: getTrashRoot/s in ViewFileSystem should return viewFS path, not targetFS path
Status: Resolved | Priority: Major | Resolution: Fixed
Created: 26/Feb/22 01:24
Affects Version/s: 2.10.2

It is probably incorrect that we return a targetFS path from getTrashRoot() in ViewFileSystem, as that path will be used later on by ViewFileSystem in other operations, such as rename. ViewFileSystem is assuming the path that it receives is a viewFS path, but not a target FS path. For example, rename() in ViewFileSystem will call getUriPath() for src/dst path, which will remove the scheme/authority and then try to resolve the path-only component. It thus sometimes leads to incorrect path resolution, as we are doing the path resolution again on a targetFS path. 

 

On the other hand, it is not always trivial/feasible to determine the correct viewFS path for a given trash root in targetFS path. 

Example:

Assume we have a mount point for /user/foo -> abfs:/containerA

User foo calls getTrashRoot("/a/b/c") and "/a/b/c" does not match any mount point. We fall back to the fallback hdfs, which by default returns hdfs://localhost/user/foo/.Trash. In this case, it is incorrect to return the trash root as viewfs:/user/foo, as it will be resolved to the abfs mount point, instead of the fallback hdfs.