Real Time Edge Detection in Browser | by Andrew Udell | Towards Data Science

Edge detection is a fundamental tool in computer vision. By using sharp contrasts in brightness, general outlines and shapes can be found in an image. Often used as a filter to find distinctions between items in a photo, edge detection offers a first stepping stone into object recognition.
OpenCV offers a wide range of computer vision tools in C++, Java, Python, and JavaScript. Among them include an edge detection algorithm using the Canny method. Built with performance in mind, OpenCV’s implementation of edge detection can be interfaced with real time video with little to no noticeable latency.
While many examples and tutorials exist in Python, there are fewer in JavaScript. With the increase portability of web technologies, now taking over space in desktop applications and even phone apps, it makes sense to cover the basics of using OpenCV in a web application.
Named after its innovator John Canny, Canny edge detection tries to find edges such that:
it finds as many edges in the image as possible,the edge point should localize on the center of the edge, andedges are only marked once and don’t include background noise
it finds as many edges in the image as possible,
the edge point should localize on the center of the edge, and
edges are only marked once and don’t include background noise
Developed in the early days of computer vision (1986), Canny edge detection is still the gold standard for cutting edge applications. For those few edge detection algorithms that perform better than Canny, they usually require more time and computational power, which make them poor candidates for real time detection.
Luckily there’s no need to write an algorithm from scratch. OpenCV offers an efficient implementation of the Canny edge detection with a few simple lines of code.
The mathematics of how the Canny method finds edges goes beyond the scope of this article, but further details may be found here.
Before jumping into the JavaScript to apply edge detection, the proper elements must be set up on an HTML page. For real time detection to work OpenCV requires a video tag and two canvases.
<!DOCTYPE html><html>    <head>        <Title>            Real Time Edge Detection with OpenCV.js        </Title>    </head>    <body>        <video id="video" playsinline autoplay></video>        <canvas id = "streamCanvas"></canvas>        <canvas id = "edgeDetectionCanvas"></canvas>                <script type="text/javascript" src="lib/CV2/opencv.js"></script>        <script type="text/javascript" src="js/edgeDetection.js"></script>    </body></html>
The reason to include two canvases is because OpenCV can’t read frames from the video tag. Instead, a work around will be used to render frames from the video tags to the first canvas. This canvas will be known as the streamCanvas. Then the frames from the first canvas can be read and modified by OpenCV and rendered on the second canvas, named the edgeDetectionCanvas.
Note that the opencv.js library is included in the first script tag. A copy may be found at “https://docs.opencv.org/{VERSION_NUMBER}/opencv.js". In this article, version 4.5.1 was used, which may be found here.
// Set a variable for the video tagconst video = document.getElementById("video");// Function to stream video from the web camasync function streamVideo() {    // Set video parameters    const constraints = {        // no audio is required        audio: false,        // Set the video size        video: {            width: 800, height: 400        }    };        try {        // Get the video stream                const stream = await navigator.mediaDevices.getUserMedia(constraints);        window.stream = stream;        video.srcObject = stream;              } catch (e) {        alert("An error occurred while accessing the web cam.");    }   }
The first line simply establishes a variable for the video tag, which will be used periodically through the program.
Next, an asynchronous function is defined to stream video from a web cam to the video tag. Within this function, a set of video parameters are defined. The audio was set to False, because this is purely a visual demonstration and the width and height are defined accordingly.
A try/catch actually encapsulates the code for streaming. If there’s some issue accessing the web cam or the user simply doesn’t have one, an alert will inform them. Otherwise the video from the camera will stream directly into the video tag without any further issues with the getUserMedia method.
// Function to draw a video frame onto the canvasfunction drawCanvas() {    // Get context and draw frame from video element    var canvas = document.getElementById("streamCanvas")    var ctx = canvas.getContext('2d');    ctx.drawImage(video, 0, 0); }
Another function is defined to draw a frame from the video tag onto one of the canvases. Within it, the “streamCanvas” is defined and the canvas is specified as a 2D canvas (as opposed to 3D renderings which are also possible).
Finally, the image is drawn from the video tag. The pair of 0’s in the drawImage method simply specify to draw the image at coordinate (0,0) on the canvas.
function edgeDetection() {        // Start video stream    streamVideo();        // Set interval to repeat function every 42 milliseconds    setInterval(() => {        // Draw frame to the intermediate canvas        drawCanvas();                // Get the current frame from the intermediate canvas        var src = cv.imread("streamCanvas");        cv.cvtColor(src, src, cv.COLOR_RGB2GRAY, 0);        cv.Canny(src, src, 50, 100, 3, false);        cv.imshow("edgeDetectionCanvas", src);        src.delete();    }, 42);}
The actual function for edge detection is defined here. Before doing anything, the function streamVideo is called, so that video will be streamed to the video tag immediately.
The setInterval function repeats its contents at a defined interval. In this case, it’s set at 42 milliseconds, because it corresponds to about 24 repetitions per second. When drawCanvas is called, then, it draws the video at 24 frames per second.
OpenCV is finally used, starting with the imread command, which simply captures the current frame on the streamCanvas. The next line converts the color image into a black and white image. The following line actually applies the Canny edge detection.
Notes the 50 and 100 in the parameters of the Canny function. These are defined as the minimum and maximum values, respectively. Any edges with an intensity above the maximum value are immediately classified as edges. Any edges with an intensity below the minimum value are immediately classified as non-edges. Intensities between the two values are judged based on connectivity to determine its status as an edge.
Finally, the imshow method renders the newly modified frame to the edgeDetectionCanvas and the source of the frame is deleted to free up memory.
A more thorough explanation may be found in the official documentation here.
// main function to clean upfunction main() {    // Hide the video tag    video.style.display = "none";    // Run edge detection    edgeDetection();}// Load mainmain();
In a final clean up, a main function is defined which hides the video tag and runs the edge detection. This will leave the two canvases side-by-side, so that the difference between the regular streamed video and the edge detection may be clearly seen.
OpenCV provides an incredibly powerful toolset for computer vision. With an emphasis on real time application, OpenCV provides a valuable asset to anyone interested in gaining insight from images.
The use of the Canny Edge detection is the basis for many other computer vision applications, such as background removal and object detection and recognition. Understanding the fundamentals will provide a basis for more complex computer vision tasks.