Title: ERC_5219_Resolve_Mode
Category: Uncategorized

Author: Pandapip1
Date: 2023-05-01T22:40:18.483Z

Adds an ERC-4804 resolve mode for ERC-5219

Author: qizhou
Date: 2023-05-26T05:57:36.979Z

PR is here Add EIP: ERC-5219 Resolve Mode by Pandapip1 · Pull Request #6944 · ethereum/EIPs · GitHub
And the gateways w3eth.io and w3link.io are now supporting the draft.  A test contract can be browsed at testnet https://0x6587e67f1fbeaabdee8b70efb396e750e216283b.w3q-g.w3link.io/request/asdf/1234?abc=567&aaa=bar&xxx=yyy
The source code of the contract can be found at https://explorer.galileo.web3q.io/address/0x6587e67F1FBEAabDEe8b70EFb396E750e216283B/contracts
@Pandapip1 could you please take a look at the code and see if the gateway is working as intended (including customized response headers)

Author: qizhou
Date: 2023-05-29T04:46:07.596Z

One more question is regarding URL encoding/decoding, where we should expect the inputs of the following function
function request(string[] memory resource, KeyValue[] memory params) external view returns (uint16 statusCode, string memory body, KeyValue[] headers);

are the UTF-8 encoded string after URL decoding the Web3 URL?
For example, for the following Web3 URL
web3://0x6587e67f1fbeaabdee8b70efb396e750e216283b:3334/asdf%CE%B1/1234?abc=567&aaa=%20bar&xxx=yyy

the sources will be asdfα / 1234 and query of aaa is  bar (space before bar).
Could you please confirm this? @Pandapip1

Author: Pandapip1
Date: 2023-06-11T15:03:26.404Z

That is correct.
Extra text so that this post is over the character minimum.

Author: nand
Date: 2023-11-02T08:05:06.560Z

Hi!
In the ERC-5219 contract interface, the returned body is of type string, so question:
Is it intended that only text should be returned, or any binary data could be returned, and I guess the interface needs to be updated with bytes instead of string?
Use case: I could easily see non text data be returned, e.g. some stored gzipped content, some crafted binary data, …
ERC-5219 being final, what would be the best way forward?

Add an alteration on ERC-6944 indicating a slight change? It would be a bit messy.
Like ERC-6860 was created to evolve a final ERC-4804, create a new ERC to evolve 5219, with binary return support, and also input HTTP header support?

Author: nand
Date: 2023-11-30T07:14:21.272Z

Hi!
@qizhou had the idea to add support for “big request”, i.e. requests whose returned content needs to be fetched over several RPC calls because of max gas limitations.
One initial idea would be: In a call to request, if the smart contract wants to return more bytes on that already returned :

the smart contract returns in one of the headers a special header with a web3:// pointer to fetch the next chunk of bytes, e.g. x-web3-next-chunk: web3://0xabcd/getFile/xxx?chunk=2
the web3 client send right away the HTTP code and the first bytes it received, and then fetchs this next URL, continue to stream the returned bytes, and loop until there is no longer a x-web3-next-chunk header.

(@qizhou: I removed the use of a special statusCode as we need right away the HTTP code to start streaming the answer, and the presence of x-web3-next-chunk in itself indicate the “big request” mode)
@Pandapip1 : What do you think of this small extension of ERC-5219, and do you think it could be in the non-final ERC-6944, or shall it be in a separate ERC? Thanks!

Author: qizhou
Date: 2023-11-30T22:24:50.636Z

Hi nand, do you want to provide more context about how HTTP chunked encoding solves the issue?

Author: nand
Date: 2023-12-01T07:49:58.675Z

Hi @qizhou,
Sure, I thought about the use of the Transfer-Encoding: chunked header method, which allows the server to send to the client the chunk of binary data right away after each call to the smart contract, effectively streaming the result.
Then I realized this was a HTTP/1 concept, and HTTP/2 has already native streaming support with their concept of DATA frames for a given request, so there isn’t even something special to do for HTTP/2.
On the server libs tested (golang http server lib, electron), they natively give the opportunity to send response chunk by chunk, and handle themselves the switch to chunked transfer encoding if HTTP/1.

Author: qizhou
Date: 2024-01-22T08:23:19.985Z

An interesting idea of using ERC6944 web3:// with a customized header to Content-Encoding: gzip so that the browser will decompress on-chain data returned by the contract
web3://0xb464cac335daec57d4f50657ec846c3109c774b2:3334/gzip.js or
https://0xb464cac335daec57d4f50657ec846c3109c774b2.3334.w3link.io/gzip.js

Author: nand
Date: 2024-01-23T09:27:02.553Z

I was wondering if it is a good idea, because this is part of the Accept-Encoding / Content-Encoding HTTP mechanism : the client sends a list of compression protocol it supports (e.g. Accept-Encoding: gzip, deflate, br) and the HTTP server choose one it supports and returns in which format it is encoded (e.g. Content-Encoding: gzip). If these is no match between client supported encoding and server supported encoding, the data is sent without compression – so this mechanism guarantee 100% that the client will be able to read the data.
In our case, we (the web3 client) are not able to specify the supported compression encodings, so we cannot garantee 100% that the client will be able to read the data.
For example we sent a response with Content-Encoding: br but the HTTP client does not support br.
Possible options I see :

We introduce a special header (e.g. web3-content-encoding) in which the smartcontract return specify the compression algorithm. A list of supported compression algorithms are required by the ERC spec. The web3 client itself will see this header, decompress the data, and send it to the HTTP client decompressed, and without forwarding the web3-content-encoding header – it will be invisible to the HTTP client.
The same than above, but we don’t use a new special header, we consume Content-Encoding (which is not forwarded to the HTTP client, so it stays invisible to the HTTP client). I like it less because it is reusing the name of a header of another mechanism we are not using.
We don’t care : we just forward the Content-Encoding: xxx header to the HTTP client, and if it is not supported by the HTTP client, it breaks. (It will work 99.9% of the time for common algorithms such as gzip, but we cannot guarantee 100%)

For option 1 and 2, there should work with the “large file”/“chunk” feature.
Let me know if you see others options and what are your thoughts.

Author: nand
Date: 2024-02-09T10:56:29.929Z

Hi!
I have put 2 PRs for 2 new features of the resource request mode :

ERC-7617: Chunk support for ERC-5219 mode in Web3 URL : Introduce a special returned HTTP header which contains a web3:// pointing to the next chunk of data of the returned data.
ERC-7618: Content encoding in ERC-5219 mode Web3 URL : Indicate that the returned HTTP header content-encoding is processed at the protocol level, returning decompressed data to the client (as we don’t want to do the Accept-Encoding/Content-Encoding handshake)

These features have been implemented on web3protocol-js v0.5.8 and are usable with web3curl-js v0.1.5.
Chunk test
This call implements chunking and return 3 chunks:
web3curl "web3://0x64850db133e088fc1657c7bf9c00303d36d92736:11155111/getFile/abcd"

returns
startmiddleend

in a visible streaming fashion.
Content-encoding test
This call returned gziped data, and the HTTP header Content-encoding: gzip
web3curl "web3://0xb464cac335daec57d4f50657ec846c3109c774b2:3334/gzip.js"

returns
* HTTP Status code: 200
* HTTP Headers: 
*   Content-Type: application/javascript

  console.log("hello world");

Author: SamWilsn
Date: 2024-05-22T16:39:39.005Z

If the client advertises, for example, Accept-Encoding: br, and the contract provides Content-Encoding: br, why does the gateway have to decode the data?

Author: nand
Date: 2024-05-23T06:57:40.513Z

Good point, I’ll add that decoding is optional if the returned content-encoding match one of the encoding advertised in Accept-Encoding.