Repository: electron
Issue: Support Custom Media Stream Tracks
Labels: enhancement :sparkles:

<!--  As an open source project with a dedicated but small maintainer team, it can sometimes take a long time for issues to be addressed so please be patient and we will get back to you as soon as we can.
-->

### Preflight Checklist
<!-- Please ensure you've completed the following steps by replacing [ ] with [x]-->

* [x] I have read the [Contributing Guidelines](https://github.com/electron/electron/blob/master/CONTRIBUTING.md) for this project.
* [x] I agree to follow the [Code of Conduct](https://github.com/electron/electron/blob/master/CODE_OF_CONDUCT.md) that this project adheres to.
* [x] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

### Problem Description
<!-- Is your feature request related to a problem? Please add a clear and concise description of what the problem is. -->

We need a desktop capturer with some additional features, i.e. excluding certain windows from capture. We implemented such a capturer as a plugin. It periodically produces a bitmap, which then is drawn into a hidden canvas element by JavaScript and then a media stream for WebRTC is produced by Chrome’s canvas capturer.

![Custom Media Stream Tracks](https://user-images.githubusercontent.com/1677767/55440048-57817e80-555b-11e9-801d-30a44a207c65.png)

This solution proved to be workable, but it also quite costly in CPU usage terms. As an experiment we produced plugin that uses internal Chromium API to copy video frames from capturer into MediaStream.

![Custom Media Stream Tracks-2](https://user-images.githubusercontent.com/1677767/55440047-57817e80-555b-11e9-9503-31b6aeaaa755.png)

This optimization significantly reduced CPU usage (from 60% to 25% CPU usage on Windows capturing 2880x1800 desktop). But due to use of Chrome’s internal API optimized plugin requires use of component Electron build.

### Proposed Solution
<!-- Describe the solution you'd like in a clear and concise manner -->
We propose extension of Electron API intended to allow app authors to implement custom media stream tracks.

Example of proposed API use:
      
    const { customMedia } = require("electron");
    let track = customMedia.createTrack({
      info: {
        resolution: { width: 1920, hight: 1080 }
        frameRate: 30
      },
    startCapture: (queue) => {
    this.interval = setInterval({
      let frame = queue.allocateFrame({
        pixelFormat: 'YUV'
            });

      // frame.y, frame.u and frame.v are ArrayBuffers
      // containing pixels from each color plane

      frame.y.fill(76); // filling the buffer with color red
      frame.u.fill(84);
      frame.v.fill(255);
      
      queue.queueFrame(Date.now(), frame);
    });
    },
    stopCapture: () => {
      clearInterval(this.interval);
    }
    })

    let mediaStream = new MediaStream();
    mediaStream.addTrack(track);

    //this is a html5 video element
    video.srcObject = mediaStream

**API description:**

`createTrack(source)`  - creates a standard MediaStreamTrack wrapping video source provided by the client. Being a standard object it can be added to any media stream, and in the stream passed as source to an html5 video element or as source for WebRTC. Its argument is a video source object provided by API client. It should have `info` field containing metadata, `startCapture` and `stopCapture` callbacks.

`source.info` - describes video track. Sets track resolution and framerate. Those will be reported as metadata by video element for example.

`source.startCapture(queue)` - starts video source. queue object is passed to client from API to allocate and queue video frames for playback.

`source.stopCapture()` - stops video source.

`queue.allocateFrame(spec)` - allocates or reuse existing video frame for the client. spec specifies desired frame properties, for example resolution and pixel format.

`queue.queueFrame(timestamp, frame)` - queues frame for playback.

`frame` - video frame object. Should contain means to access pixel data, depending on pixel format. YUV frames should contain y,u,v properties that should be ArrayBuffers pointing to appropriate color planes.

**C++ interoperability**

To ease interoperability of this API with third party C++ plugins queue should wrap a C++ helper object. 


```
enum class Plane {Y,U,V};
struct Format {...}; // pixel format info, resolution etc.

struct VideoFrame {
  virtual Format format() =0;
  virtual void* data(Plane plane) =0;
};

struct Timestamp {};

struct VideoFrameQueue {
  virtual VideoFrame* dequeueFrame() = 0;
  virtual void queueFrame(Timestamp, VideoFrame*) = 0;
};

VideoFrameQueue* unwrapQueue(v8::Handle<Value> value);
```

<!-- ### Alternatives Considered
-->
<!-- A clear and concise description of any alternative solutions or features you've considered. -->

<!-- ### Additional Information
-->
<!-- Add any other context about the problem here. -->