# Efficient Implementation of Matrix-Based Image Processing Algorithms for IoT Applications
## 2 The Algorithm Description
Figure 8:0 Matrix multiplication CNN-based algorithm (floating point). The term \(Mt_{1}\) is the computing time for single processor, and the term \(fMt_{2}\) is the computing time for the non-parallelizable fraction in a multiprocessor system with \(N\) processors (cores). The parallelizable fraction \((1-f)Mt_{2}\) is computed on \(N\) processor (cores) so the execution time will be \(\frac{(1-f)Mt_{2}}{N}\). All the computations involved in matrix processing (multiplication, addition) can be implemented using the CNN block in flattened or element wise modes. The above presented CS-RANSAC algorithm illustrated above was implemented using MAX78000 and its CNN accelerator. The numerical precision is similar with numerical precision obtained with previous implementation on Blackfin (the ARM and RISC cores in MAX78000 also use 32-bit fixed-point representation). In the speedup relation, we set \(N=64\) (the number of cores in CNN), \(r=0.13\) (the ratio between the ARM microcontroller speed and Blackfin microcontroller), \(q=0.06\) (the ratio between CNN cores speed and the Blackfin microcontroller), and \(f=0.78\) (the algorithm code that does not contain matrix operations that can be performed in CNN). With this value, the theoretical speedup (between Blackfin implementation and MAX7800 implementation) is \(s=9.84\). The effective speedup (obtained by counting processor cycles by the IDE code profiler) is lower due to the data transfers performed using RISC V. Figure 15 illustrates the execution time obtained with CNN implementation. In this case (software floating point implementation), the speedup obtained is about 7 times for the CS-RANSAC algorithm. The execution time depends proportionally on image size at a constant noise level because the algorithm split the images in blocks of \(16\times 16\) pixel, which are processed individually. The computing time of the algorithm can be improved, and modifying the way to calculate DCT coefficients using original DCT or CS-RANSAC depends on if the block is noisy or not and using the ARM and CNN in parallel. For the first improving method, the original CS-RANSAC algorithm was combined with a noise estimator [16]. Each block is marked as light noised or heavy noised and is processed using simple DCT or CS-RANSAC, respectively. The noise estimator can be implemented in fixed point using the ARM and RISC-V microcontrollers in parallel in the MC78000 chip. Figure 16a shows how the tasks for such implementation can be scheduled. Figure 8:1 The MAX7800x general architecture. The following tasks are defined: _NE_Task_--for noise estimate, _P_Task_--the processing task that implement all the processing for DCT and CS-RANSAC noise removal and compression algorithms excepts the matrix multiplications and matrix additions, which are computed by a matrix processing task--_MP_Task_ and _COMM_task_--a communication task that transfer the information (matrix values) between CNN and ARM using DMA channels. The tasks _NE_Task_ and _P_Task_ are running in ARM core, and the task _COMM_task_ is running in the ARM coprocessor (that acts as a direct memory access--DMA controller). All the matrix manipulations are passed to CNN accelerator (programmed in flatten mode for matrix multiplications or element wise mode for matrix additions or subtractions) and are computed by _MP_Task_. All tasks are synchronized using global semaphores. Depending on the noise level, the execution time can be reduced as illustrated in Figure 17[26]. For an average noise probability of 50%, one can observe that the computation time reduction ratio is about 35%. If the noise estimator is not used, the task _NE_Task_ in the scheduling tasks from Figure 16 is removed. Figure 8:2 Processing time (various noised image sizes) for two 32-bit fixed-point implementations (Blackfin and Max78000 CNN). Figure 8:3 (**a**) Task scheduling for the algorithm; (**b**) algorithm improvement using parallel processing. Figure 8:4 Computing time reduction (in dot line—trendline of ratio).