Repository: apache/opendal
Pull request #5871: RFC-5871: Read Returns Metadata
Author: login: meteorgan
Created: 2025-03-24T14:49:09Z | Merged: 2025-03-26T15:57:38Z

# Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->

Closes #5425.

# Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->

# What changes are included in this PR?

a new RFC
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->

# Are there any user-facing changes?


<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->

<!---
If there are any breaking changes to public APIs, please add the `breaking-changes` label.
-->


Linked issue #123: Roadmap v0.3
This issue is reserved for roadmap v0.3 progress tracking.

For comments and discussion, please go to https://github.com/datafuselabs/opendal/discussions/117.

---

- [x] https://github.com/datafuselabs/opendal/issues/119
- [x] https://github.com/datafuselabs/opendal/issues/30
- [x] https://github.com/datafuselabs/opendal/issues/71
- [x] https://github.com/datafuselabs/opendal/issues/124

Diff:
diff --git a/core/src/docs/rfcs/5871_read_returns_metadata.md b/core/src/docs/rfcs/5871_read_returns_metadata.md
new file mode 100644
index 000000000000..3c16d0ed44aa
--- /dev/null
+++ b/core/src/docs/rfcs/5871_read_returns_metadata.md
@@ -0,0 +1,112 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be provided alongside existing functionality, allowing users to continue using current `reader` methods without modification.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to struct `raw::RpRead`
+
+The `raw::RpRead` struct will be modified to include a new field `metadata` that stores the metadata returned by the read operation.
+Existing fields will be evaluated and potentially removed if they become redundant.
+
+```rust
+pub struct RpRead {
+    // New field to store metadata 
+    metadata: Metadata,
+}
+```
+
+
+## Implementation Details
+
+For services that return metadata in their read responses:
+- The metadata will be captured from the service response.
+- All available fields (content_type, etag, version_id, last_modified, etc.) will be populated
+
+For services that don't return metadata in read responses:
+- We'll make an additional `stat` call to fetch the metadata and populate the `metadata` field in `raw::RpRead`.
+
+Special considerations:
+- We should always return total object size in the metadata, even if it's not part of the read response
+- For range reads, the metadata should reflect the full object's properties (like total size) rather than the range
+- For versioned objects, the metadata should include version information if available
+
+# Drawbacks
+
+- Additional memory overhead for storing metadata during reads
+- Potential complexity in handling metadata for range reads
+
+# Rationale and alternatives
+
+- Maintains full backward compatibility with existing read operations
+- Improves performance by avoiding additional stat calls
+- Aligns with common storage service APIs (S3, GCS, Azure)
+
+# Prior art
+
+Similar patterns exist in other storage SDKs:
+
+- `object_store` crate returns metadata in `GetResult` after calling `get_opts`
+- AWS S3 SDK returns comprehensive metadata in `GetObjectOutput`
+- Azure Blob SDK returns properties and metadata in `DownloadResponse`
+
+# Unresolved questions
+
+None
+
+# Future possibilities
+
+- Once we return metadata during reader initialization, we can optimize `ReadContext::parse_into_range` by using the 
+`content_length` from `metadata` directly, eliminating the need for an additional `stat` call
diff --git a/core/src/docs/rfcs/mod.rs b/core/src/docs/rfcs/mod.rs
index 4d4980e81782..778da9a40b89 100644
--- a/core/src/docs/rfcs/mod.rs
+++ b/core/src/docs/rfcs/mod.rs
@@ -260,3 +260,7 @@ pub mod rfc_5495_list_with_deleted {}
 /// Write Returns Metadata
 #[doc = include_str!("5556_write_returns_metadata.md")]
 pub mod rfc_5556_write_returns_metadata {}
+
+/// Read Returns Metadata
+#[doc = include_str!("5871_read_returns_metadata.md")]
+pub mod rfc_5871_read_returns_metadata {}


Reviews:
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T15:16:27Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T15:27:11Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T04:10:18Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T11:19:42Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T11:22:12Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T14:08:03Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T14:14:21Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T14:17:18Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T14:22:55Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T14:27:42Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T14:30:04Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T16:10:39Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T04:24:46Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-26T04:55:12Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-26T05:03:22Z
user: login: meteorgan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T14:15:51Z
user: login: Xuanwo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-26T15:54:16Z
user: login: Xuanwo
body: Thank you @meteorgan for this, looks nice!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-26T15:57:28Z

Review comments:
diff_hunk: @@ -0,0 +1,132 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let (data, meta) = reader.read(..).await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+```
+
+The behavior remains backward compatible if users don't need the metadata - they can simply ignore the metadata part of the return tuple.
+
+# Reference-level explanation
+
+## Changes to `Operator` API
+
+The following functions will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`:
+
+- `read()`
+- `read_with()`
+
+## Changes to `Reader` API
+
+- `read()` will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`.
+- `fetch()` will be modified to return `Result<(Vec<Buffer>, Metadata)>` instead of `Result<Buffer>`.
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
+}
+```
+
+## Changes to struct `http_util::HttpBody`
+
+The `HttpBody` struct will be modified to include a new field for metadata.
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: Do we need a new struct,  or could we modify `HttpBody` to `HttpReader` ?
created_at: 2025-03-24T15:16:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,132 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let (data, meta) = reader.read(..).await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+```
+
+The behavior remains backward compatible if users don't need the metadata - they can simply ignore the metadata part of the return tuple.
+
+# Reference-level explanation
+
+## Changes to `Operator` API
+
+The following functions will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`:
+
+- `read()`
+- `read_with()`
+
+## Changes to `Reader` API
+
+- `read()` will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`.
+- `fetch()` will be modified to return `Result<(Vec<Buffer>, Metadata)>` instead of `Result<Buffer>`.
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
+}
+```
+
+## Changes to struct `http_util::HttpBody`
+
+The `HttpBody` struct will be modified to include a new field for metadata.
+
+
+
+## Implementation Details
+
+For services that return metadata in their read responses:
+- The metadata will be captured from the service response.
+- All available fields (content_type, etag, version_id, last_modified, etc.) will be populated
+
+For services that don't return metadata in read responses:
+- for `fs`: we can use `stat` to retrieve the metadata before returning. Since the metadata is cached by the kernel, this should be efficient
+- for other services: A default metadata object will be returned
+
+Special considerations:
+- We should always return total object size in the metadata, even if it's not part of the read response
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: what if `content_range` returns `<unit> <range>/*` ? In that case, we can't directly determine the object size.  Other services, such as `ftp`, `hdfs`, also support range reads, but we can't obtain the object size directly from read responses directly, should we call `stat` ?
created_at: 2025-03-24T15:27:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,132 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let (data, meta) = reader.read(..).await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: This design can breaks every call to `read`, which I don't think is a good idea for us. More importantly, not all users are interested in the metadata returned by `read`. Requiring all users to accommodate these breaking changes doesn't make sense to me.

As an alternative, how about add API in this way:

```rust
impl Reader {
    fn metadata(&self) -> &Metadata;
}
```

Users who want to know about `etag` / `content_length` can use in this way:

```rust
let reader = op.reader(path).await?;
let etag = reader.metadata().etag();
```

This change will also be compatible. No API breaking.
created_at: 2025-03-25T04:10:15Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: How about not to return `Metadata` for `read` call? Users who want to play with returned metadata can just call `op.reader()` and use `metadata()`.

Most users use `op.read()` really don't care about it's related metadata.
created_at: 2025-03-25T11:17:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: typo.
created_at: 2025-03-25T11:17:29Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: Another good thing is that know we know the total size of this file. We can remove the extra `stat` call in `Reader`.
created_at: 2025-03-25T11:18:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+
+The behavior remains backward compatible if users don't need the metadata. The new API will be optional, 
+and users can still use the existing `reader` methods without any changes.
+
+# Reference-level explanation
+
+## Changes to `Operator` API
+
+The following functions will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`:
+
+- `read()`
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: I prefer to not change `read()`.

Think about this: 
![image](https://github.com/user-attachments/assets/5be69769-50f4-4471-ba52-c412de357d44)

created_at: 2025-03-25T11:19:29Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: Unlike the modification of `write(path) -> Result<()>` to `write(path) -> Result<Metadata>`, which mostly allows existing code to compile without issues, changing `Result<Buffer>` to `Result<(Buffer, Metadata)>` could break every call to this API.

We need to make a tradeoff here.
created_at: 2025-03-25T11:22:12Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: I'm afraid we can't always obtain the total size directly, as I mentioned here: https://github.com/apache/opendal/pull/5871#discussion_r2010410558
created_at: 2025-03-25T14:08:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: Alright, this approach looks better. We should aim to keep migration cost as low as possible.
created_at: 2025-03-25T14:14:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: It's fine to just call another `stat` for them. One of our principles is "object storage first."
created_at: 2025-03-25T14:17:18Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: In particular, we need to fetch the object size when initializing  the `reader` before we could track the total bytes size while reading.
created_at: 2025-03-25T14:22:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: Oh, i recall why designed it this way, i wanted to accumulate the total file size during reading, especially if we can't fetch it directly from the read responses.
created_at: 2025-03-25T14:27:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metada();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: > It's fine to just call another `stat` for them. One of our principles is "object storage first."

Ok, then many issues are resolved. 😄 
created_at: 2025-03-25T14:30:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: > Oh, i recall why designed it this way, i wanted to accumulate the total file size during reading, especially if we can't fetch it directly from the read responses.

I don't understand how this could be useful:

- Reader::metadata() should be able to return before read any data.
- If we are reading full data, users can just call bs.len() to get the whole length
- If we are reading partial data, the content_length in metadata is wrong.
created_at: 2025-03-25T16:10:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,141 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: This could address part of the issue when we can't get the data length without running `stat` operations. 

> * If we are reading full data, users can just call bs.len() to get the whole length

Yeah. But right now we're focused on how to return `metadata` - more specifically, how to populate the `content_length` field.

> * If we are reading partial data, the content_length in metadata is wrong.

We'll need other approaches to tackle this.

Anyway, If we agree to use `stat` operations. Those problems are pretty much go away.


created_at: 2025-03-26T04:24:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,132 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+The read operations will be enhanced to return both data and metadata:
+
+```rust
+// Before
+let data = op.read("path/to/file").await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+
+// After
+let (data, meta) = op.read("path/to/file").await?;
+if let Some(content_type) = meta.content_type() {
+    println!("Content-Type: {}", content_type);
+}
+```
+
+For reader operations:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let (data, meta) = reader.read(..).await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+```
+
+The behavior remains backward compatible if users don't need the metadata - they can simply ignore the metadata part of the return tuple.
+
+# Reference-level explanation
+
+## Changes to `Operator` API
+
+The following functions will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`:
+
+- `read()`
+- `read_with()`
+
+## Changes to `Reader` API
+
+- `read()` will be modified to return `Result<(Buffer, Metadata)>` instead of `Result<Buffer>`.
+- `fetch()` will be modified to return `Result<(Vec<Buffer>, Metadata)>` instead of `Result<Buffer>`.
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
+}
+```
+
+## Changes to struct `http_util::HttpBody`
+
+The `HttpBody` struct will be modified to include a new field for metadata.
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: We don't need this change. `Metadata` can be carried in `RpRead` instead.
created_at: 2025-03-26T04:55:12Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: I feel that "optional" is not the right word here. This API is not optional; it's additive.
created_at: 2025-03-26T04:56:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: We don't need this. The returning metadata can be carried in `RpRead` which is created at the same time along with `oio::Read`.
created_at: 2025-03-26T04:58:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: Such API also changes the fact that all APIs for `oio::Read` takes `&mut self`. This can lead to `oio::Read` is not `Sync` anymore.
created_at: 2025-03-26T04:58:58Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
+}
+```
+
+## Changes to struct `http_util::HttpBody`
+
+The `HttpBody` struct will be modified to include a new field for metadata.
+
+
+
+## Implementation Details
+
+For services that return metadata in their read responses:
+- The metadata will be captured from the service response.
+- All available fields (content_type, etag, version_id, last_modified, etc.) will be populated
+
+For services that don't return metadata in read responses:
+- for `fs`: we can use `stat` to retrieve the metadata before returning. Since the metadata is cached by the kernel, this should be efficient
+- for other services: A default metadata object will be returned
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: We can call a `stat` instead.
created_at: 2025-03-26T04:59:37Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
+}
+```
+
+## Changes to struct `http_util::HttpBody`
+
+The `HttpBody` struct will be modified to include a new field for metadata.
+
+
+
+## Implementation Details
+
+For services that return metadata in their read responses:
+- The metadata will be captured from the service response.
+- All available fields (content_type, etag, version_id, last_modified, etc.) will be populated
+
+For services that don't return metadata in read responses:
+- for `fs`: we can use `stat` to retrieve the metadata before returning. Since the metadata is cached by the kernel, this should be efficient
+- for other services: A default metadata object will be returned
+
+Special considerations:
+- We should always return total object size in the metadata, even if it's not part of the read response
+- For range reads, the metadata should reflect the full object's properties (like total size) rather than the range
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: Agreed.
created_at: 2025-03-26T04:59:55Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
+}
+```
+
+## Changes to struct `http_util::HttpBody`
+
+The `HttpBody` struct will be modified to include a new field for metadata.
+
+
+
+## Implementation Details
+
+For services that return metadata in their read responses:
+- The metadata will be captured from the service response.
+- All available fields (content_type, etag, version_id, last_modified, etc.) will be populated
+
+For services that don't return metadata in read responses:
+- for `fs`: we can use `stat` to retrieve the metadata before returning. Since the metadata is cached by the kernel, this should be efficient
+- for other services: A default metadata object will be returned
+
+Special considerations:
+- We should always return total object size in the metadata, even if it's not part of the read response
+- For range reads, the metadata should reflect the full object's properties (like total size) rather than the range
+- For versioned objects, the metadata should include version information if available
+
+# Drawbacks
+
+- Additional memory overhead for storing metadata during reads
+- Potential complexity in handling metadata for range reads
+
+# Rationale and alternatives
+
+- Maintains full backward compatibility with existing read operations
+- Improves performance by avoiding additional stat calls
+- Aligns with common storage service APIs (S3, GCS, Azure)
+
+# Prior art
+
+Similar patterns exist in other storage SDKs:
+
+- `object_store` crate returns metadata in `GetResult` after calling `get_opts`
+- AWS S3 SDK returns comprehensive metadata in `GetObjectOutput`
+- Azure Blob SDK returns properties and metadata in `DownloadResponse`
+
+# Unresolved questions
+
+None
+
+# Future possibilities
+
+None
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: There some size related tricks in `ReadContext::parse_into_range`: 

https://github.com/apache/opendal/blob/8c72db973a97cf94b3acbb9744933b8e0d635815/core/src/types/context/read.rs#L72-L103

After this change, we can remove then entirely.
created_at: 2025-03-26T05:03:00Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: meteorgan
body: > This can lead to `oio::Read` is not `Sync` anymore.

Hmmm, I don't understand this point of view.
created_at: 2025-03-26T14:15:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,117 @@
+- Proposal Name: `read_returns_metadata`
+- Start Date: 2025-03-24
+- RFC PR: [apache/opendal#5871](https://github.com/apache/opendal/pull/5871)
+- Tracking Issue: [apache/opendal#5872](https://github.com/apache/opendal/issues/5872)
+
+# Summary
+
+Enhance read operations by returning metadata along with data in read operations.
+
+# Motivation
+
+Currently, read operations (`read`, `read_with`, `reader`, `reader_with`) only return the data content. Users who need metadata 
+during reads (like `Content-Type`, `ETag`, `version_id`, etc.) must make an additional `stat()` call. This is inefficient and 
+can lead to race conditions if the file is modified between the read and stat operations.
+
+Many storage services (like S3, GCS, Azure Blob) return metadata in their read responses. For example, S3's GetObject API returns 
+important metadata like `ContentType`, `ETag`, `VersionId`, `LastModified`, etc. We should expose this information to users 
+directly during read operations.
+
+# Guide-level explanation
+
+For `reader` API, we will introduce a new method `metadata()` that returns metadata:
+
+```rust
+// Before
+let data = op.reader("path/to/file").await?.read(..).await?;
+let meta = op.stat("path/to/file").await?;
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+
+// After
+let reader = op.reader("path/to/file").await?;
+let meta = reader.metadata();
+if let Some(etag) = meta.etag() {
+    println!("ETag: {}", etag);
+}
+let data = reader.read(..).await?;
+```
+The new API will be optional, and users can still use the existing `reader` methods without any changes.
+
+For backward compatibility and to minimize migration costs, We won't change the existing `read` API. Anyone who wants 
+to obtain metadata during reading can use the new reader operations instead.
+
+# Reference-level explanation
+
+## Changes to `Reader` API
+
+The `impl Reader` will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+impl Reader {
+    // Existing fields...
+    
+    fn metadata(&self) -> &Metadata {}
+}
+```
+
+## Changes to trait `oio::Read`
+
+The `Read` trait will be modified to include a new function `metadata()` that returns metadata.
+
+```rust
+pub trait Read {
+    // Existing functions...
+    
+    fn metadata(&self) -> Metadata;
path: core/src/docs/rfcs/5871_read_returns_metadata.md
user: login: Xuanwo
body: It's an interesting trick involving Rust's `Send` and `Sync`. 

If a trait only provides `&mut self` APIs and there's no way to access the struct itself, we can safely declare it as `Sync` when it's `Send` by self because it cannot be accessed from multiple threads simultaneously. This can eliminate some unnecessary `Mutex<T>` instances when implementing `oio::Read`.
created_at: 2025-03-26T15:54:16Z
author_association: MEMBER

PR comments:
user: login: meteorgan
created_at: 2025-03-26T04:35:11Z
body: I think the crux of the issue lies here, `metadata()` returns a `u64`, not an `Option<u64>`.  I'm not sure if we're required to always return a valid `content_length`. If we are, why does the comment mention it'll be 0 when unset ? That could clash with cases where the content is genuinely empty.
https://github.com/apache/opendal/blob/8c72db973a97cf94b3acbb9744933b8e0d635815/core/src/types/metadata.rs#L223-L226
author_association: CONTRIBUTOR
user: login: Xuanwo
created_at: 2025-03-26T04:53:14Z
body: > I'm not sure if we're required to always return a valid `content_length`. If we are, why does the comment mention it'll be 0 when unset ? That could clash with cases where the content is genuinely empty.

It's definitely a dirty workaround.

We have this because some storage services, like fs, don't return `content_length` in their listings. However, since `Metadata::content_length` is one of the most commonly used metadata fields, we don't want users to have to check for its existence just because a few services don't provide it.
author_association: MEMBER
user: login: meteorgan
created_at: 2025-03-26T16:09:25Z
body: Thanks for the review—it’s practically a whole new design. I’ve learned a ton from it! @Xuanwo 
author_association: CONTRIBUTOR
user: login: Xuanwo
created_at: 2025-03-26T18:05:58Z
body: > Thanks for the review—it’s practically a whole new design. I’ve learned a ton from it! @Xuanwo

Nice work!
author_association: MEMBER
user: login: Xuanwo
created_at: 2025-03-27T04:06:40Z
body: Oh, I recall an issue with the current design while sleeping last night.

We are not calling `Access::read` during `Operator::reader` so we might don't have `Metadata` to use at this stage.
author_association: MEMBER
user: login: meteorgan
created_at: 2025-03-27T04:45:34Z
body: > We are not calling `Access::read` during `Operator::reader` so we might don't have `Metadata` to use at this stage.

I'm aware of this issue, and planning to call `Access::read` before constructing `ReadContext`.

author_association: CONTRIBUTOR
user: login: Xuanwo
created_at: 2025-03-27T04:55:43Z
body: > I'm aware of this issue, and planning to call `Access::read` before constructing `ReadContext`.

But we don't know the range to read :thinking: 
author_association: MEMBER
user: login: meteorgan
created_at: 2025-03-27T13:59:21Z
body: > But we don't know the range to read 🤔

Oh, Got it. After digging a bit deeper, I've come up with a few more questions:

1. Does it make sense to run `stat` before `Access::read` ? Without a `version_id`, these two operations might end up hitting the same object but different `version_id`s
2. Based on how `ReadGenerator` is designed, it looks like every call to `next_reader` triggers another `Access::read`. Couldn't that mean we sometimes end up reading different versions of the same object ?
author_association: CONTRIBUTOR
user: login: meteorgan
created_at: 2025-03-27T14:10:20Z
body: > But we don't know the range to read 🤔

It looks like we need a bit of time to think through this issue. It might actually call for an API change, or add a new API returns both data and metadata ?
author_association: CONTRIBUTOR
user: login: meteorgan
created_at: 2025-03-27T14:29:22Z
body: The returned value like `object_store` might be a better choice.

```rust
/// Result for a get request
#[derive(Debug)]
pub struct GetResult {
    /// The [`GetResultPayload`]
    pub payload: GetResultPayload,
    /// The [`ObjectMeta`] for this object
    pub meta: ObjectMeta,
    /// The range of bytes returned by this request
    ///
    /// Note this is not `usize` as `object_store` supports 32-bit architectures such as WASM
    pub range: Range<u64>,
    /// Additional object attributes
    pub attributes: Attributes,
}
```
author_association: CONTRIBUTOR
user: login: meteorgan
created_at: 2025-03-31T10:28:14Z
body: Hi, @Xuanwo do any new ideas on these issues ?
author_association: CONTRIBUTOR
user: login: Xuanwo
created_at: 2025-03-31T10:44:58Z
body: > Hi, @Xuanwo do any new ideas on these issues ?

I'm guessing we need to start a new one. Returning `GetResult` is not good for us since our `Reader` performs more complex work than `object_store` do like chunking and concurrent reading.
author_association: MEMBER
user: login: meteorgan
created_at: 2025-03-31T11:27:36Z
body: > Oh, Got it. After digging a bit deeper, I've come up with a few more questions:
> 
> 1. Does it make sense to run `stat` before `Access::read` ? Without a `version_id`, these two operations might end up hitting the same object but different `version_id`s
> 2. Based on how `ReadGenerator` is designed, it looks like every call to `next_reader` triggers another `Access::read`. Couldn't that mean we sometimes end up reading different versions of the same object ?

How about these two issues ? Did i overlook anything ? or are there potential bugs in current implementation ? @Xuanwo 
author_association: CONTRIBUTOR
user: login: meteorgan
created_at: 2025-03-31T11:29:14Z
body: > I'm guessing we need to start a new one.

Are you suggesting we open a new issue to dive into these issues ? 
author_association: CONTRIBUTOR
user: login: Xuanwo
created_at: 2025-03-31T11:30:24Z
body: > How about these two issues ? Did overlook anything ? or are there potential bugs in current implementation ? @Xuanwo

Users who want to make sure to read the exact version should specify `version` while calling `reader_with`. We can make this clear in the API docs.

> Are you suggesting we open a new issue to dive into these issues ?

Let's reopen the old issue.
author_association: MEMBER
user: login: meteorgan
created_at: 2025-03-31T13:18:59Z
body: > Users who want to make sure to read the exact version should specify `version` while calling `reader_with`. We can make this clear in the API docs.

These issues don't tie into `Read Returns Metadata`.  I'll go ahead and open a separate issue to cover them.


author_association: CONTRIBUTOR