【You don't know about canvas】Replace the green screen video background
Tags: html, canvas, javascript, css
Published: 2021-05-13 08:21:13

#canvas Api Brief introduction
- [Video_and_audio_content](https://developer.mozilla.org/zh-CN/docs/Learn/HTML/Multimedia_and_embedding/Video_and_audio_content)
- [ImageData](https://developer.mozilla.org/zh-CN/docs/Web/API/Canvas_API/Tutorial/Pixel_manipulation_with_canvas)

The `ImageData` object represents the underlying pixel data of an area of a canvas object. It contains the following read-only attributes:：

`width`：The width of the image in pixels.
`height`：The height of the image in pixels.
`data`：A [Uint8ClampedArray](https://developer.mozilla.org/zh-CN/docs/Web/JavaScript/Reference/Global_Objects/Uint8ClampedArray)representing a one-dimensional array containing the data in the RGBA order, with integer values between 0 and 255 (included).

The data property returns a` Uint8ClampedArray` which can be accessed to look at the raw pixel data; each pixel is represented by four one-byte values (red, green, blue, and alpha, in that order; that is, "RGBA" format). Each color component is represented by an integer between 0 and 255. Each component is assigned a consecutive index within the array, with the top left pixel's red component being at index 0 within the array. Pixels then proceed from left to right, then downward, throughout the array.

**In short, we need to group every four pixels to get the `rgba` value of each pixel**

Then we combine the features of 'Canvas' to operate the video to make green screen matting and change the background

First, the renderings：

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2m81hcclpwbq7x8q3gzt.gif)


**Code address**：[canvas-viideo](https://github.com/wzc570738205/canvas-viideo)
**preview address**：[githubPage](https://wzc570738205.github.io/canvas-viideo/)、[阿里云](https://static-8e76dff9-ce38-4577-9e5c-398943705060.bspapp.com/)

Realization idea
**`Video '= = >'video screenshot' = = >'process green pixels as transparent '= = >'map to the top of the background image`**
Take a screenshot of the video, and then make the green pixel block transparent
Then put the processed image above the prepared background image
#Implementation
###1. Prepare videos and canvases
```
<body onload="processor.doLoad()">
    <div>
      <video id="video" src="./q.mp4" width="350" controls="true"></video>
    </div>
    <div>
      <!-- Video capture -->
      <canvas id="c1" width="260" height="190"></canvas>
      <!-- Processing green pixels as transparent -->
      <canvas id="c2" width="260" height="190"></canvas>
      <!-- Map to top of background map -->
      <canvas id="c3" width="260" height="190"></canvas>
    </div>
</body>
```
### 2.Add video playback monitor
```
doLoad: function doLoad() {
  this.video = document.getElementById("video");
  this.c1 = document.getElementById("c1");
  this.ctx1 = this.c1.getContext("2d");
  this.c2 = document.getElementById("c2");
  this.ctx2 = this.c2.getContext("2d");
  this.c3 = document.getElementById("c3");
  this.ctx3 = this.c3.getContext("2d");
  let self = this;
  this.video.addEventListener(
    "play",
    function() {
      self.width = self.video.videoWidth / 5;
      self.height = self.video.videoHeight / 3;
      self.timerCallback();
    },
    false
  );
}
```
### 3.Add timer
Call after video playback, and capture screenshot of each frame
```
timerCallback: function timerCallback() {
  if (this.video.paused || this.video.ended) {
    return;
  }
  this.computeFrame();
  let self = this;
  setTimeout(function () {
    self.timerCallback();
  }, 0);
}
```
### 4.Video frame operation
Set the green background to transparent and map it to the custom background image
```
computeFrame: function computeFrame() {
  this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
  let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
  let l = frame.data.length / 4;

  for (let i = 0; i < l; i++) {
    let r = frame.data[i * 4 + 0];
    let g = frame.data[i * 4 + 1];
    let b = frame.data[i * 4 + 2];
    //rgb(8 204 4)
    if (r > 4 && g > 100 && b < 100) {
      frame.data[i * 4 + 3] = 0;
    }
  }
  this.ctx2.putImageData(frame, 0, 0);
  this.ctx3.putImageData(frame, 0, 0);
  return;
}
```
### 5.Fine tuning
```
    //rgb(8 204 4)
    The green video color is not pure, not always RGB (82004), so a simple fine-tuning has been made..
    if (r > 4 && g > 100 && b < 100) {
      frame.data[i * 4 + 3] = 0;
    }
```

#END
Thank you for reading
My English is not good. I want to share this good technology with you，This is my Chinese blog address:[segmentfault](https://segmentfault.com/a/1190000039975476)

Code address：[canvas-viideo](https://github.com/wzc570738205/canvas-viideo)
preview addrss：[阿里云](https://static-8e76dff9-ce38-4577-9e5c-398943705060.bspapp.com/)、[githubPage](https://wzc570738205.github.io/canvas-viideo/)
green video download：[pixabay](https://pixabay.com/zh/videos/search/green%20screen/)