The effortless way to process images with OpenCV Canny algorithm
By: Martynas Pocius
Tags: Neural Networks, Canny Edge Detection, Convolutional Network, Opencv, Image Processing

Simplicity matters in software engineering, when deploying state of the art models in your virtual machine its usually impossible, because of old libraries that do not support your environment. Luckily you can do the task in simple ways of image processing with OpenCV instead of using Neural networks-based models with millions of parameters.

The effortless way to process images is to utilize a canny edge detection algorithm instead of state-of-the-art artificial intelligence methods. The Canny algorithm is mature, it is an unsupervised algorithm that does not require a lot of data to train algorithms, also the results from algorithms are always provable and rigorous. Image processing is used every day from image editing to game making. In image processing, there is a problem like a contour and an edge detection. Usually, this is applicable when cutting out objects from the background. To solve this problem without human intervention it is used a canny edge detection algorithm. It works by finding the intensity gradient of the image, smoothened image is then filtered with a Sobel kernel. This is a popular edge detection algorithm. It was developed by John F. Canny[1]. The Canny algorithm is there to use and it is mature, also there are complex artificial intelligence models which have can have around 25 million [2] parameters that are hard to train such big models. In this case, when training such a big model with so many parameters to prove model convergence is getting harder and harder, that is why kernel-based solutions or unsupervised models work more rigorously in production environments. Unsupervised Machine learning so prominent for its ability to categorize unlabeled data and discover a wide range of unknown patterns within it [3]. To prove non-rule-based algorithms there is always a probability of risk that the algorithm will fail [4]. To fail such a system is very easy there is a one-pixel attack that only needs a one-pixel change in the image. Deep neural networks can be easily altered by adding relatively small changes to the input vector. An attack in an extremely limited scenario where only one pixel can be modified. This attack explores a different take on adversarial machine learning in an extremely limited scenario, showing that current deep neural networks are also vulnerable to such low dimension attacks [5]

Image processing turns to be a very important task to solve in image editors, virtual reality, and visualizations also it is applicable in a lot of fields like medicine or agriculture. The first thing to do for a machine learning problem is to start with data analysis. This is the best way to learn about domain business knowledge. For example, to classify images there are machine learning models like inception v3 [5]. However, there are problems with such models, those models are supervised, it means that it is required to have a lot of data to train, also with so much data to train the model it is required a lot of computation power. That why unsupervised models were invented. Clean, perfectly labeled datasets are not easy to come by. So sometimes, researchers are asking the algorithm questions that do not know the answer to. That is where unsupervised learning comes in. In unsupervised learning, a deep learning model is handed a dataset without explicit instructions on what to do with it. The training dataset is a collection of examples without a specific desired outcome or correct answer. The neural network then attempts to automatically find structure in the data by extracting useful features and analyzing its structure. [6] There is possible that the supervised model slowly converges and to train to get good results it is needed to have a lot of computation to solve a problem. Training such models, it is important to measure model characteristics like inference time, this characteristic says how much time will algorithm computes from start to finish. Talking about artificial intelligence training it should be distinguished two things training and execution time. Training time is usually long and inference execution time should be fast. The model only needs to be trained once and then used multiple times when executing. Simplicity matter in Software engineering algorithms should be implemented following use case requirements, it is stated that by the Occam razor principle the simplest solution is often better.

The Canny edge detection algorithm is well understood it is created in the 1986 year. It also has 14891 citations which prove that this algorithm is tried and true. There are many examples of this method used by developers who need to edit or manipulate images. The second example is based on the idea of a computer vision algorithm based on kernels. This approach works because of the underlying math. We want to remove unwanted pixels that might not be part of an edge. Then happens Thresholding with Hysteresis, accepting pixels as edges if the intensity gradient value exceeds an upper threshold also rejecting pixels as edges if the intensity gradient value is below a lower threshold. If a pixel is between the two thresholds, accept it only if it is adjacent to a pixel that is above the upper threshold [8]. The math is straightforward and distilled to the essence, with such many research made on one canny algorithm we can say that this method is mature. Moreover, we can define Software algorithm maturity with the CMM model. CMM was developed by the Software Engineering Institute (SEI) at Carnegie Mellon University in 1987. 5 maturity levels are initial, repeatable, defined, managed, and optimizing. In each stage, there are requirements for maturity. For example, in the initial stage, the processes are immature and are not well defined. In the repeatable stage project planning, requirement management, and quality assurance are in place. The defined stage it is a well-integrated set of project-specific software engineering and management processes. At the managed stage, quantitative quality goals are set for the organization for software products as well as software processes. And the last stage optimizing is the highest level of process maturity in CMM and focuses on continuous process improvement in the organization using quantitative feedback [9]. The Canny edge detection algorithm is implemented in the OpenCV framework which is open-sourced and well documented and continuously optimized, which means the OpenCV algorithms are in the fifth stage of maturity.

Canny edge detection is an unsupervised method it means that does not need a lot of data. The supervised algorithm learns from its training data, and it tries to converge to the best solution. The algorithm takes a lot of time and resources, which means that will need to use a large amount of computation power. The inference can likely be slow if the dataset was very large because the algorithm starts just memorizing all the possible cases that can happen. Slow convergence can happen when we train models with a lot of parameters, in this case, who can have more computation power can make a better-resulting model. That is why neural networks have one disadvantage when adopting the gradient method by the BP neural network, the problems including a slow learning convergent velocity and easily converging to a local minimum cannot be avoided. Besides, the selection of the learning factor and inertial factor affects the convergence of the BP neural network, which are usually determined by experience. Therefore, the effective application of the BP neural network is limited. A new method in the BP algorithm to avoid a local minimum was proposed using adding gradually training data and hidden units. Also, the paper proposed a new model of a controllable feedforward neural network [10]. In big models inferencing times slowdown that it is impossible to do any mobile or embedded strategies, there are strategies to trim and reduce model complexity, but it will worsen model accuracy. It is very important to test this process before the deployment, otherwise, it could impact business use cases, it could be that the inference times will take from 5 to 10 seconds, but the requirement is a maximum of 1 second. To achieve better results the model should be deployed with GPU or TPU optimizations in mind.

Canny edge detection results are rigorous and provable and can be used for the prediction of images to detect the edges of the object. This is used to cut the background of the object out of the image. By saying rigorously it means when the developer customizing algorithm filter kernels decides how the algorithm will do the task. This is different from ai edge detection algorithms, usually, ai detection algorithms use a lot of data and try to learn features of object edges and lean those patterns. In both cases, there are advantages and disadvantages to doing the task. For example, there is no semantic understanding when using canny edge detection, it means that all edges are treated the same but neural networks algorithms like semantic segmentation work differently, if there is a region it should have a boundary edge. But some boundaries are transparent Cany edge detection algorithm uses a thresholding technique to filter out recognizable edges. If the problem context is simple it is better to use the canny algorithm because of simplicity. There is a simplicity principle like Occam’s razor. Occam’s razor principle dates back to William of Ockham in the 1200s — it is the idea that the simplest and most direct solution should be preferred, or that with different hypotheses, the simplest one or the one with fewest assumptions will be best applied [11]. That means It is easier to prove OpenCV algorithms than Artificial intelligence models, also when deploying models in production environments it should be efficient and cheap ways to execute in the cloud computing servers. To deploy in such environments the complexity of algorithms should be measured by inference times. State-of-the-art deep neural networks (DNNs) have hundreds of millions of connections and are both computationally and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources and power budgets [12].

Simple OpenCV methods like canny edge detection can achieve great results with minimum effort for simple image contextual tasks. This library is used in a lot of applications that means is widely tried and used in production environments. The algorithm is old and acknowledged from many perspectives and can be used in a wide variety of problems. As this algorithm is unsupervised no need to have so much data as neural supervised artificial intelligence models. Take into consideration the Occam razor principle that the simplest solution is often the best choice. No need to try complex supervised artificial intelligence models with millions of parameters if OpenCV algorithms are sufficient to solve the problem. Software maturity can be evaluated by the capability maturity model (CMM). There should be more research done in semi-supervised methods that do not need a lot of data to achieve a state of the art results. Take into consideration that with limited resources it is not possible to deploy complex neural networks that millions of connections. Overall, the most important thing is to look into use cases that meats requirements, this should be the main concert for choosing the best approach to use Artificial intelligence models in business, because a lot of models are state of the art and in production environments, it will not meet certain criteria.

Image source

Edge detection example Image source: https://www.pyimagesearch.com/2019/03/04/holistically-nested-edge-detection-with-opencv-and-deep-learning/

Bibliography

[1]

“Canny,” [Online]. Available: https://docs.opencv.org/master/da/d22/tutorial_py_canny.html.

[2]

“https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/confronting-the-risks-of-artificial-intelligence," [Online]. Available: https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/confronting-the-risks-of-artificial-intelligence.

[3]

“Supervised-vs-unsupervised-learning-what-is-the-difference,” [Online]. Available: https://autome.me/supervised-vs-unsupervised-learning-what-is-the-difference/.

[4]

“Confronting-the-risks-of-artificial-intelligence,” [Online]. Available: https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/confronting-the-risks-of-artificial-intelligence#.

[5]

“https://arxiv.org/abs/1710.08864," [Online].

[6]

“InceptionV3,” [Online]. Available: https://www.tensorflow.org/api_docs/python/tf/keras/applications/InceptionV3.

[7]

“https://blogs.nvidia.com/blog/2018/08/02/supervised-unsupervised-learning," [Online]. Available: https://blogs.nvidia.com/blog/2018/08/02/supervised-unsupervised-learning.

[8]

“https://automaticaddison.com/how-the-canny-edge-detector-works," [Online].

[9]

“https://searchsoftwarequality.techtarget.com/definition/Capability-Maturity-Model," [Online].

[10]

“https://ieeexplore.ieee.org/abstract/document/893417," [Online].

[11]

“https://www.techopedia.com/how-does-occams-razor-apply-to-machine-learning/7/33087," [Online].

[12]

“https://dl.acm.org/doi/abs/10.1145/3007787.3001163," [Online].