npm package: @sammacbeth/libdweb
Description: Experimental WebExtensions API for enabling DWeb protocols
Author: Mozilla
Latest version: 0.0.13

# libdweb

[![travis][travis.icon]][travis.url]
[![package][version.icon] ![downloads][downloads.icon]][package.url]
[![styled with prettier][prettier.icon]][prettier.url]

This repository hosts a community effort to implement [experimental APIs][webextension experiments] for Firefox WebExtensions with a goal of enabling dweb protocols in Firefox through browser add-ons. The long term goal of this project is to integrate these APIs into the [WebExtensions][new apis] ecosystem.

## Participation

You can help this effort in following ways:

1. Use these APIs to make something illustrating its value, to build the case for adoption in the core WebExtension API set.
2. Get involved in driving this effort: Help with an API implementation, maintenance, testing, code samples, etc.
3. Help build [API adapters][] to enable seamless integration with existing libraries.
4. Join our IRC channel: #dweb on irc.mozilla.org

## Status: In active development

| API                   | Status |
| --------------------- | ------ |
| [Protocol Handler][]  | 🐥     |
| [Service Discovery][] | 🐣     |
| [File System][]       | 🐣     |
| [UDP Socket][]        | 🐣     |
| [TCP Socket][]        | 🐣     |

- 🥚 : In design phase
- 🐣 : Work in progress
- 🐥 : Try it out
- 🐓 : Usable

## API overview

**Note:** You can try all the examples after you've cloned the repo and got the toolchain setup by running `npm install`. You will also need [Firefox Nightly][] to run the demos.

### Protocol API

The Protocol API allows you to handle custom protocols from your Firefox extension. This is different from the existing [WebExtensions protocol handler API][webextensions protocol_handlers] in that it does not register a website for handling corresponding URLs but rather allows your WebExtension to implement the handler.

The following example implements a simple `dweb://` protocol. When firefox is navigated to `dweb://hello/world`, for example, it will invoke your registered handler and pass it a `request` object containing request URL as `request.url` string property. Your handler is expected to return a repsonse with a `content` that is [async iterator][] of [`ArrayBuffer`][]s. In our example we use a `respond` [async generator][] function to respond with some HTML markup.

```js
browser.protocol.registerProtocol("dweb", request => {
  return {
    contentType: "text/html",
    content: respond(request.url)
  }
})

async function* respond(text) {
  const encoder = new TextEncoder("utf-8")
  yield encoder.encode("<h1>Hi there!</h1>\n").buffer
  yield encoder.encode(
    `<p>You've succesfully loaded <strong>${request.url}</strong><p>`
  ).buffer
}
```

Given that `response.content` is an [async iterator][] it is also possible to stream response content as this next example illustrates.

```js
browser.protocol.registerProtocol("dweb", request => {
  switch (request.url) {
    case "dweb://stream/": {
      return {
        contentType: "text/html",
        content: streamRespond(request)
      }
    }
    default: {
      return {
        contentType: "text/html",
        content: respond(request.url)
      }
    }
  }
})

async function* streamRespond(request) {
  const encoder = new TextEncoder("utf-8")
  yield encoder.encode("<h1>Say Hi to endless stream!</h1>\n").buffer
  let n = 0
  while (true) {
    await new Promise(resolve => setTimeout(resolve, 1000))
    yield encoder.encode(`<p>Chunk #${++n}<p>`).buffer
  }
}
```

You can see the demo of the example above in [Firefox Nightly][] by running following command, and then navigating to [dweb://hello/world](dweb://hello/world) or [dweb://stream/](dweb://stream/)

```
npm run demo:protocol
```

![protocol demo](./demo/protocol/protocol.gif)

### Service Discovery API

API provides DNS-Based Service Discovery API as per [rfc6763][]. Following example illustrates how this API can be used to discover available `http` services in the network.

```js
void (async () => {
  const services = browser.ServiceDiscovery.discover({
    type: "dweb",
    protocol: "tcp" // Must be "tcp" or "udp"
  })

  console.log("Start discovery", services.query)
  for await (const service of services) {
    if (service.lost) {
      console.log("Lost service", service)
    } else {
      console.log("Found service", {
        name: service.name,
        type: service.type,
        protocol: service.protocol
      })

      for (const {
        address,
        port,
        host,
        attributes
      } of await service.addresses()) {
        console.log(
          `Service ${service.name} available at ${host} ${address}:${port}`,
          attributes
        )
      }
    }
  }
  console.log("End discovery", services.query)
})()
```

API also allows you to announce service that others on the network can discover. Following example illustrates that:

```js
void (async () => {
  const service = await browser.ServiceDiscovery.announce({
    name: "My dweb service",
    type: "dweb",
    protocol: "tcp", // must be "tcp" or "udp"
    port: 3000, // ommting port will just assign you available one.
    attributes: {
      // optional txt records
      version: "1.0."
    }
  })

  console.log("Service annouced", {
    name: service.name, // Note: Colud be different like "My dweb service (2)"
    type: service.type,
    protocol: service.protocol,
    port: service.port,
    attributes: service.attributes // Will be null if was omitted
  })

  // Wait for a 1 minute and expire service announcement
  await new Promise(timeout => setTimeout(timeout, 60 * 1000))
  await service.expire()
  console.log(`Service expired`)
})()
```

#### Demo

You can try demo WebExtension that discovers and displays `http`
services in your local network when button in the toolbar is clicked. You can
run it in [Firefox Nightly][] via the following command

```
npm run demo:discovery
```

![discovery button](./demo/discovery/preview.gif)

### FileSystem API

FileSystem API provides access to an OS file system, but restricted to a user chosen directory. Below example illustrates writing a content to a file in user chosen directory.

```js
void (async () => {
  const volume = await browser.FileSystem.mount({
    read: true,
    write: true
  })
  console.log("Mounted", volume)
  localStorage.setItem("volumeURL", volume.url)

  const fileURL = new URL("hello.md", volume.url).href
  const encoder = new TextEncoder()
  const content = encoder.encode("# Hello World\n").buffer
  const size = await browser.FileSystem.writeFile(fileURL, content)
  console.log(`Wrote ${size} bytes to ${fileURL}`)
})()
```

Call to `FileSystem.mount` will notify user that corresponding WebExtension is requesting `read / write` access to the file system, which user can deny or grant by choosing a directory. If user denies to access then promise returned by `mount` will be rejected, if user chooses to grant access to a specific directory the promise will resolve to an object like:

```js
{
  url:"file:///Users/user/dweb/",
  readable:true,
  writable:true
}
```

The rest of the example that writes content into a file should be pretty straight forward.

> **Note:** Granted access will be preserved across sessions, and WebExtension could mount same directory without prompting a user again.

Following is a more complete example that will either mount directory that user has already granted access to or request access to new directory otherwise.

```js
void (async () => {
  const url = localStorage.getItem("volumeURL")
  const volume = await browser.FileSystem.mount({ url, read: true })

  const fileURL = new URL("hello.md", volume.url).href
  const file = await browser.FileSystem.open(fileURL, { read: true })
  const chunk = await browser.File.read(file, { position: 2, size: 5 })
  console.log(`Read file fragment from ${fileURL}`, chunk)
  const decoder = new TextDecoder()
  const content = decoder.decode(chunk)
  console.log(`Decode read fragment`, content)
  await browser.File.close(file)
})()
```

> **Note:** Attempting to mount a URL that user has not previously granted access to will fail without even prompting a user.

FileSystem API has many other functions available. You can follow the links for detailed API interface definitions of [`browser.FileSystem`][] and [`browser.File`][]

You can try demo WebExtension that provides a [REPL][] in the sidebar exposing all of the FileSystem API, which you can run in [Firefox Nightly][] via following command

> **Note:** Commands recognized by [REPL][] correspond to the API functions names and all the parameters are names prefixed by `--` and followed by value.

```
npm run demo:fs
```

![FileSystem](./demo/fs/fs.gif)

### UDPSocket API

API provides an implementation of UDP Datagram sockets. Follow the link for detailed API interface for [`browser.UDPSocket`][] which corresponds to `UDPSocketManager`.

There is also a [@libdweb/dgram-adapter][] project that provides [nodejs dgram][] API adapter.

#### Example

Following example opens UDP socket on port `41234` that will act as a server and will continuously print incoming messages.

```js
void (async () => {
  const server = await browser.UDPSocket.create({
    port: 41234
  })
  console.log(`listening ${server.address.host}:${server.address.port}`)

  const decoder = new TextDecoder()
  for await (const { from, data } of browser.UDPSocket.messages(server)) {
    console.log(`receive message`)
    const message = decoder.decode(data)
    console.log(`server got: ${message} from ${from.host}:${from.port}`)
  }
})()
```

> **Note:** Incoming messages are represented via [async iterator][] which can be consumed via `for await`, but be aware that **messages are not buffered** so if you use `await` inside the `for await` block chances are you will miss message which will be dropped.

Following example opens socket on arbitrary port and then sends a message to the server socket from the above example.

```js
void (async () => {
  const client = await browser.UDPSocket.create({ host: "127.0.0.1" })
  console.log(`opened socket ${client.address.host}:${client.address.port}`)
  const encoder = new TextEncoder()
  const message = encoder.encode("Hello UDP").buffer
  await browser.UDPSocket.send(client, "127.0.0.1", 41234, message)
})()
```

> **Note**: UDPSocket API unlike one in nodejs is not going to resolve hostnames like `"localhost"`. You need to use WebExtensions [dns][webextensions dns] API to resolve hostnames.


#### Demo


##### P2P Chat Demo

You can try demo WebExtension that uses UDP multicasting to do peer-to-peer chat in a firefox sidebar. You can run in [Firefox Nightly][] via following command

> **Note**: This is a demo illustrates UDP and Multicasting API.

```
npm run demo:p2p-chat
```

![p2p-chat](./demo/p2p-chat/chat_demo.png)

##### REPL Demo


You can try demo WebExtension that provides a [REPL][] in the sidebar exposing all of the UDPSocket API, which you can run in [Firefox Nightly][] via following command

> **Note:** Commands recognized by [REPL][] correspond to the API functions names and all the parameters are names prefixed by `--` and followed by corresponding values.

```
npm run demo:dgram
```

### TCPSocket API

TCPSocket API provides a client and server socket APIs for [TCP][] networking.

#### Example

Following example starts echo TCP server on port `8090`. It will accept incoming
connections read first chunk of data, respond by echoing messages back to.

```js
void (async () => {
  const encoder = new TextEncoder()
  const decoder = new TextDecoder()

  const server = await browser.TCPSocket.listen({ port: 8090 })
  console.log("Started TCP Server", server)

  const onconnect = async client => {
    console.log("Client connected:", client)
    const message = await client.read()
    console.log("Received message from client:", decoder.decode(message))
    const response = encoder.encode(`<echo>${decoder.decode(message)}</echo>`)
    await client.write(response.buffer)
  }

  for await (const client of server.connections) {
    onconnect(client)
  }
})()
```

> **Note:** `server.connections` are represented via [async iterator][] which can be consumed via `for await`, but be aware that connections are **not buffered** which is why handle each connection in `onconnect` function so our server can accept more connections. If you use `await` inside the `for await` block chances are you will miss connection, in which case it will be automatically closed.

Following example connects to server from the example above writes a message to it and then reads message received back.

```js
void (async () => {
  const encoder = new TextEncoder()
  const decoder = new TextDecoder()

  const client = await browser.TCPSocket.connect({
    host: "localhost",
    port: 8090
  })
  await client.opened
  console.log("Client connected:", client)

  await client.write(encoder.encode("Hello TCP").buffer)
  const response = await client.read()
  console.log("Received response:", decoder.decode(response))
})()
```


[travis.icon]: https://travis-ci.com/mozilla/libdweb.svg?branch=master
[travis.url]: https://travis-ci.com/mozilla/libdweb
[version.icon]: https://img.shields.io/npm/v/libdweb.svg
[downloads.icon]: https://img.shields.io/npm/dm/libdweb.svg
[package.url]: https://npmjs.org/package/libdweb
[downloads.image]: https://img.shields.io/npm/dm/libdweb.svg
[downloads.url]: https://npmjs.org/package/libdweb
[prettier.icon]: https://img.shields.io/badge/styled_with-prettier-ff69b4.svg
[prettier.url]: https://github.com/prettier/prettier
[webextension experiments]: https://webextensions-experiments.readthedocs.io/en/latest/index.html
[new apis]: https://wiki.mozilla.org/WebExtensions/NewAPIs
[protocol handler]: https://github.com/mozilla/libdweb/blob/master/Readme.md#protocol-api
[udp socket]: https://github.com/mozilla/libdweb/blob/master/Readme.md#udpsocket-api
[tcp socket]: https://github.com/mozilla/libdweb/blob/master/Readme.md#tcpsocket-api
[µtp socket]: https://github.com/mozilla/libdweb/issues/6
[service discovery]: https://github.com/mozilla/libdweb/blob/master/Readme.md#service-discovery-api
[file system]: https://github.com/mozilla/libdweb/blob/master/Readme.md#filesystem-api
[web-ext]: https://www.npmjs.com/package/web-ext
[firefox nightly]: https://blog.nightly.mozilla.org/
[api adapters]: https://github.com/libdweb
[webextensions protocol_handlers]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/manifest.json/protocol_handlers
[async iterator]: https://github.com/tc39/proposal-async-iteration#async-iterators-and-async-iterables
[`arraybuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
[async generator]: https://github.com/tc39/proposal-async-iteration#async-generator-functions
[`browser.filesystem`]: https://github.com/mozilla/libdweb/blob/master/src/FileSystem/FileSystem.js#L8-L39
[`browser.file`]: https://github.com/mozilla/libdweb/blob/master/src/FileSystem/FileSystem.js#L41-L51
[repl]: https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop
[`browser.udpsocket`]: https://github.com/mozilla/libdweb/blob/master/src/UDPSocket/UDPSocket.js
[webextensions dns]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/dns
[@libdweb/dgram-adapter]: https://github.com/libdweb/dgram-adapter
[nodejs dgram]: https://nodejs.org/api/dgram.html
[tcp]: https://en.wikipedia.org/wiki/Transmission_Control_Protocol
[rfc6763]: https://tools.ietf.org/html/rfc6763