# Determining Thresholds for Optimal Adaptive Discrete Cosine Transformation
## 2 Materials and Methods
### Proposed Adaptive DCT Algorithm
Figure 4: Examples of image histograms: (**a**) histogram of the original image; (**b**) histogram after increasing the brightness by 40%; (**c**) histogram after normalization, i.e., extending it to the entire 0–255 spectrum width; (**d**) histogram after multiplying the brightness by 2, resulting in a “whiteout” (overexposure). Figure 3: Example of adaptive discrete cosine transform: (**a**) original image; (**b**) quadtree grid on the image, yellow color marks the solid color fragments; (**c**) adaptive discrete cosine transform spectrum of the image (brightened up for clarity). The root mean square error can be calculated for this distribution of 256 tone values. However, a more elegant and efficient approach by calculating tonal distribution variance is illustrated further. For an input image or image fragment consisting of \(N\times N\) pixels, the average tonal distribution (TD) value is \(N\times N/256\), since the tonal values of the \(N\times N\) pixels are arbitrarily distributed among the 256 elements. Therefore, the variance can be calculated as follows:  
\[TDV=\sum_{i=0}^{255}((TD[i]-(L/256))^{2})/256, \tag{5}\]  
where \(i\) is the pixel tone index, \(L=N\times N\) is the total number of pixels in the image or fragment and TD is the tonal distribution array, consisting of 256 counts of pixels of specific brightness. For example, \(TD[0]\) is the number of black pixels and \(TD[255]\) is the number of white pixels in the fragment or image. Examples of this distribution can be observed in \[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]0  
Thus, the variance will be sensitive to the input size since there are more pixel hue values in the distribution. However, by instead dividing the sum of squared errors by \(L(L-L/256)\), a normalized value between 0 and 1 is obtained, regardless of the value of \(L\). A value of 0 means zero variance in the distribution, indicating a completely uniform tonal distribution, and 1 corresponds to the maximum possible variance, where one solid tone fills one tonal distribution array element and leaves other elements empty. Let us prove the abovementioned statement. If \(L/256\) is the mean TD value, there is only one tone array element filled with L pixels and the rest are empty. Then, the sum of squared errors is  
\[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]1  
where the first term is the squared error between the mean and the maximum value of the diapason and the second term is the squared error between the mean and the zeros for the remaining 255 out of 256 tones. This expression is equal to  
\[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]2  
\[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]3  
Therefore, dividing the squared errors' sum by  
\[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]4  
instead of 256 provides a more practically applicable and generalized value. This lies between 0 and 1 and is derived from a sum of squared values. To make the distribution practical and gradual we take a square root from the quotient, without changing the properties of the normalized value. Since the value correlates to variance, we call it tonal distribution variance (TDV). The resulting formula is as follows:  
\[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]5  
For comparison, the RMSE of an image fragment can be calculated with the following formula:  
\[IMG^{\prime}=\widehat{g}_{pm,ps}(x,y). \tag{2}\]6Tonal distribution variance directly correlates with RMSE up to a constant multiplier. However, multiplying or dividing pixel brightness values changes the tonal range and, consequently, the RMSE value. However, the TDV remains constant unless the multiplication makes pixels reach maximum brightness or a division forces individual brightness values to 1. This clearly demonstrates that TDV is more versatile than RMSE due to being independent of the tonal range of the image. Thus, using this metric allows one to safely set the thresholds inside the 0-1 range for arbitrary image size and tonal range.