Why HTTP streaming upload matters on Web browser
Tags: javascript, webdev, chromium, chrome
Published: 2021-11-10 11:32:42

---
title: Why HTTP streaming upload matters on Web browser
published: true
description: fetch() upload streaming is available since Chrome 85. But, it is going to give up to shipping and the origin trial ends today. So I posted why the Web needs the feature.
tags: javascript,webdev,chromium,chrome
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n70e26r88n94rydfja92.jpg
---
 

We want to make the web better. I believe fetch() upload streaming is a key to making the web simpler, safer and faster.


## Available, but giving up to shipping ...

Looking back in history, the document of fetch() upload streaming seems to have started 7 years ago. The latest document is under whatwg/fetch.

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/avkmuwuivl8tcsjwsea1.png" width="500">

<https://github.com/yutakahirano/fetch-with-streams/commits/master?after=3653e535378143663ade61e13064ce3561fbcf68+34>

Main browser vendors have positive reactions as follows.

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pmdzg3tv2i99rsgj725z.png" width="500">

<https://github.com/whatwg/fetch/pull/425#issuecomment-531680634>

At last, we have the fetch() upload streaming feature since Chrome 85. 

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0wm8mbdco9ald6o304cq.png" width="500">

<chrome://flags/#enable-experimental-web-platform-features>

**However, the Chromium team is going to give up to shipping the feature**. That is the reason why the post is here.
> We worked with a parter but we failed to show benefits of the feature, so we're giving up shipping the feature. 

https://bugs.chromium.org/p/chromium/issues/detail?id=688906#c57

[The origin trial](https://developer.chrome.com/origintrials/#/view_trial/3524066708417413121) ends today, Nov 10, 2021.

## Why the Web browser needs fetch() upload streaming

HTTP is the core transfer technology on the web. It is an open, widely used and trusted protocol. And we are building more rich and complex applications on the web. Some of them may deal with high-resolution videos and others may provide high-level security with E2E encryption. The fetch() upload streaming allows us to do both upload and data transformations at the same. So, we can upload while encoding rich videos in client-side. We can upload while encrypting files in client-side, which archives E2E encryption. In addition, it is easy to combine multiple transformations and build a complex one. **The fetch() upload streaming is more efficient in terms of both time and space** than transforming whole data.

The most important thing is that it has high compatibility. Our world is made up of all kinds of software that are integrated together. Streaming upload in HTTP is already available in common software such as Java (JVM languages), Node.js, Go, Swift, curl, etc. **These common software has built-in HTTP streaming upload**. **I believe that built-in availability is the biggest difference** from WebSocket, WebRTC and upcoming WebTransport. The streaming upload coming to web browser makes web server simpler, instead of creating another WebSocket API, etc. only for web browser. Eliminating extra entrances or branches can reduce code size, reduce unexpected behaviors and improve maintainability and safety. HTTP is a widely accepted protocol and provide practical API than raw TCP. As far as I know, HTTP is the only one that has this power.


## Simple, but powerful

API is very simple as follows. You can pass ReadableStream to `fetch()`. The streaming upload is the most general transfer in HTTP, which allows us to transfer conventional fixed-length data and infinite data stream.

```js
await fetch(url, {
  method: "POST",
  body: myReadableStream
});
```

Here is an example of how to create ReadableStream emitting infinite random bytes.

```js
const myReadableStream = new ReadableStream({
  pull(ctrl) {
    // random bytes
    ctrl.enqueue(window.crypto.getRandomValues(new Uint32Array(128)));
  }
});
```

Here is how to convert MediaStream to ReadableStream. This can apply screen share, web camera, voice or other sources which are from MediaStream.

```js
// Convert MediaStream to ReadableStream
function mediaStreamToReadableStream(mediaStream, timeslice) {
  return new ReadableStream({
    start(ctrl) {
      const recorder = new MediaRecorder(mediaStream);
      recorder.ondataavailable = async (e) => {
        ctrl.enqueue(new Uint8Array(await e.data.arrayBuffer()));
      };
      recorder.start(timeslice);
    }
  });
}

// Usage:
const mediaStream = await navigator.mediaDevices.getDisplayMedia({video: true});
const myReadableStream = mediaStreamToReadableStream(mediaStream, 100);
```

## Everything over HTTP

As another real example, here is a stream encrypting another stream using OpenPGP. The demo shows end-to-end encrypting (E2EE), encrypting, uploading, downloading and decrypting at the same time. In the demo below, the client is a web browser. Yet, `curl` and `gpg` are also compatible with sending and receiving since behind-the-system is a pure HTTP streaming upload.

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kitdhqqck1aoadlg6bzg.gif" width="500">

<https://github.com/nwtgck/piping-ui-web/issues/433#issuecomment-830734031>

Here is another example. This is [SSH over HTTP](https://github.com/nwtgck/piping-ssh-web/blob/31af6a4f130445a184dd3bb0376ed092f5fe2370/src/index.js#L201-L223). Everything including SSH client is complete in web browser. SSH client is also implemented in web browser powered by [SSHy](https://github.com/stuicey/SSHy) project. So, this is also E2E encryption under SSH.

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zhmzvl761y9m9jzb143j.gif" width="400">

<https://github.com/nwtgck/piping-ssh-web>

Here is [VNC over HTTP](https://github.com/nwtgck/piping-vnc-web/blob/8f2df75c7452b515fcc2ba36fe9cbf6218c673fc/core/websock.js#L208-L238). VNC is for remote control. VNC client is complete in web browser powered by [noVNC](https://github.com/novnc/noVNC) project. The application below also provides [E2E encryption using `openssl aes-256-ctr`-command-compatible way](https://github.com/nwtgck/piping-vnc-web/blob/8f2df75c7452b515fcc2ba36fe9cbf6218c673fc/core/websock.js#L206) which encrypts streamingly.

<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ywba7cad4cwk8v57ixwy.gif" width="400">

<https://github.com/nwtgck/piping-vnc-web>

Those two applications tunnel a TCP port over pure HTTP. So, simple commands using `curl` and `nc` or `socat` can also be a client, not only web browser.

## With built-in software

It is worth noting that in those examples, the web browser is just one of clients. This means `java.net.HttpURLConnection` in the **standard library** in JVM languages can be a client, `net/http` in the **standard library** in Go can be a client, `URLSession#uploadTask(withStreamedRequest: request)` in **Foundation** in Swift can be a client. The fetch() upload streaming allows our web browsers to be one. Without it, this highly compatibile feature is not available since there is no alternative and it is impossible to create a polyfill.


## Conclusion

The feature allows us to transfer infinitely in real-time and give efficiency in both time and space, which was never allowed before and it is impossible to create polyfill. It also gives us to communicate with other built-in software seemly and keep web servers simpler, safe and more maintainable.

I love the feature. I love the fetch() upload streaming. Infinite stream has infinite possibilities. I hope this post would be a good one for you to know how great this feature is.


## More about the feature

The links below would be helpful to know more.

* [whatwg spec](https://fetch.spec.whatwg.org/#concept-body-stream)
* [Chromium issue](https://bugs.chromium.org/p/chromium/issues/detail?id=688906)
* [Firefox](https://bugzilla.mozilla.org/show_bug.cgi?id=1387483)
* [Safari](https://bugs.webkit.org/show_bug.cgi?id=203617)
* [web.dev](https://web.dev/fetch-upload-streaming)
* [Chrome Platform Status](https://www.chromestatus.com/feature/5274139738767360)
* [Chromium Design Doc](https://docs.google.com/document/d/e/2PACX-1vQ26giyHPPhxrgVorNtkVSsJPOWecc6c6wHFLFbbl27r0BIBEgRwahh2b37Uk7HfXvZoW1Cu_ed-bRm/pub)
* [my post](https://dev.to/nwtgck/the-power-of-pure-http-screen-share-real-time-messaging-ssh-and-vnc-5ghc)
* [my post in Japanese (SSH)](https://scrapbox.io/nwtgck/SSH%E6%8E%A5%E7%B6%9A%E3%82%92Web%E3%83%96%E3%83%A9%E3%82%A6%E3%82%B6%E3%81%AE%E7%B4%94%E7%B2%8B%E3%81%AAHTTP%E4%B8%8A%E3%81%A7%E5%AE%9F%E7%8F%BE%E3%81%99%E3%82%8B)
* [my post in Japanese (VNC)](https://scrapbox.io/nwtgck/%E3%83%AA%E3%83%A2%E3%83%BC%E3%83%88%E3%83%87%E3%82%B9%E3%82%AF%E3%83%88%E3%83%83%E3%83%97%E6%93%8D%E4%BD%9C%E3%82%92Web%E3%83%96%E3%83%A9%E3%82%A6%E3%82%B6%E3%81%AE%E7%B4%94%E7%B2%8B%E3%81%AAHTTP%E4%B8%8A%E3%81%A7%E5%AE%9F%E7%8F%BE%E3%81%99%E3%82%8B%EF%BC%88VNC%EF%BC%89)

Photo by <a href="https://unsplash.com/@rzunikoff?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Robert  Zunikoff</a> on <a href="https://unsplash.com/s/photos/stream?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>