# An Analytic Transform Kernel Derivation Method for Video Codecs
## 1 Introduction
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Prediction** & **Prediction Tools** & **Block Size Checks** & **Transform Kernels** \\ \hline \multirow{4}{*}{MTS} & \multirow{4}{*}{Intra} & ISP [18] & \multirow{4}{*}{4 \(\times\) 4 up to 16 \(\times\) 16} & DCT-2, DST-7 \\  & & (intra sub-partition) & & \\ \cline{1-1} \cline{3-5}  & \multirow{2}{*}{Normal intra-prediction mode} & \multirow{2}{*}{4 \(\times\) 4 up to \(N\times\) 32/32 \(\times\)\(N\)} & DCT-2, DST-7 \\  & & & SBT [19] & \multirow{2}{*}{4 \(\times\) 4 up to \(N\times\) 32/32 \(\times\)\(N\)} & DST-7, DCT-8 \\  & & (sub-block transform) & & & depends on SBT position \\ \hline \hline \end{tabular}
\end{table} 
Table 1: MTS transform kernels based on prediction and block sizes in VTM-3.0. Although the proposed method provides good gains, it fails to provide the precise 64-point DCT-2 because some of the kernel values of DST-7/DCT-8 are embedded in some rows of 64-point DCT-2. Owing to the mismatch of the proposed 64-point DCT-2 transform kernel with the original DCT-2 transform kernel, coding performance improvement is not significant when only the DCT-2 transform kernel is used. Conversely, the fast computation algorithm [22] for the 64-point DCT-2 transform cannot be used. Similarly, the number of transform kernel matrices is also increased to 5, that is DCT-2, DST-7, DCT-8, DST-4, and DCT-4, even though it was aimed to use only three transform kernels. In [23], a 64-point unified matrix was proposed. It is implemented using the 64 \(\times\) 64 unified matrix with a total of 8180 elements and provides all three types of transform with all block sizes from 4 \(\times\) 4 to 64 \(\times\) 64 by sampling a subset of coefficients and basis vectors of larger transforms. This method is designed with an 8-bit representation. Using simple calculations from the proposed matrix, different kernel types with different sizes were derived. This method uses 131,072 bits amount of kernel elements which account for 16 kilobytes of memory for the total bits amount and is 19% lesser than the total memory of bits used in VTM-3.0. Experimental results show that no gain is observed for all intra (AI) configurations for higher resolution test sequences, that is, class A1 and A2, but they comprise some gain for lower resolution test sequences, that is, classes B, C, D, and E. For random access (RA) and low delay B (LDB) configurations, the overall result shows no gain. The losses are because of the mismatch between the original and derived DST-7 and DCT-8 transform kernels. Similarly, in [24], the adjustment stage concept for memory reduction was introduced. The adjustment stages are defined as sparse block-band orthogonal matrices and are similar to filters with small numbers of taps. It is proposed to approximate different sizes and types of cosine and sine transform such as DCT-5, DCT-8, DST-1, and DST-7 by applying different adjustment matrices (stages). The major drawback of this method is that the number of multiplications increases by using these adjustment stages, and the normalized values also differ for different transform kernels, which results in a mismatch with the original transform kernels. A transform adjustment filter (TAF) was introduced in [25], which is similar to the adjustment stage in [24]. A sparse matrix is used as a preprocessor to the partial butterfly DCT-2 algorithm. It approximates the DCT-8 and DST-7 with an adjustment stage followed by DCT-2. The DCT-8/DST-7 kernels are approximated by the combination of a block band matrix and a DCT-2 kernel. The size-16, size-32, and size-64 adjustment matrices require the overall 356 8-bit coefficients. Also, the adjustment stage is efficiently implemented using the SIMD instructions. Symmetries are applied for the 32-point and 64-point adjustment matrix having the storage requirement of 93 and 189 8-bit coefficients, respectively, whereas, for the 16-point adjustment matrix, 74 coefficients are required. All the adjustment matrix coefficients require 8-bit storage.