An integrated learning and approximation scheme for coding of static or dynamic light fields based on hybrid Tucker–Karhunen–Loève transform‐singular value decomposition via tensor double sketching

IET Signal ProcessingVolume 16, Issue 6 p. 680-694 ORIGINAL RESEARCHOpen Access An integrated learning and approximation scheme for coding of static or dynamic light fields based on hybrid Tucker–Karhunen–Loève transform-singular value decomposition via tensor double sketching Joshitha Ravishankar, Joshitha Ravishankar orcid.org/0000-0001-8252-1827 Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, IndiaSearch for more papers by this authorMansi Sharma, Corresponding Author Mansi Sharma [email protected] orcid.org/0000-0003-3243-3321 Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, India Correspondence Mansi Sharma, Inspire Faculty, Department of Electrical Engineering, Indian Institute of Technology Madras, Room No.: ESB336B, Chennai 600036, India. Email: [email protected]Search for more papers by this author Joshitha Ravishankar, Joshitha Ravishankar orcid.org/0000-0001-8252-1827 Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, IndiaSearch for more papers by this authorMansi Sharma, Corresponding Author Mansi Sharma [email protected] orcid.org/0000-0003-3243-3321 Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, India Correspondence Mansi Sharma, Inspire Faculty, Department of Electrical Engineering, Indian Institute of Technology Madras, Room No.: ESB336B, Chennai 600036, India. Email: [email protected]Search for more papers by this author First published: 29 June 2022 https://doi.org/10.1049/sil2.12141AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This study presents a scheme for efficient representation, coding and streaming of static or dynamic light fields using the authors' novel hybrid Tucker-TensorSketch Karhunen–Loève transform-singular value decomposition via double sketching (HTTS-KLTSVD-DS) algorithm. A deep learning model is employed to obtain acquired images from the light fields by simulating coded aperture patterns. These acquired images can represent the entire light field and are low-rank approximated using HTTS-KLTSVD-DS. Incorporation of double sketching using TensorSketch allows the authors' algorithm to work faster in a single pass itself and there is no need to store large Kronecker products of Tucker decomposition in the memory. This provides an efficient transmission and streaming adaptability of the light field, making it suitable for 3D display applications. Besides, compact representation of factor matrices by KLT-SVD in the authors' proposed model acts as an optimal transform with good energy compaction property. Encoding of low-rank approximated acquired images using HEVC eliminates intra-frame, inter-frame and other intrinsic redundancies in the light field. The authors' complete light field processing pipeline flexibly works for multiple bitrates and is adaptable for a variety of multi-view autostereoscopic platforms. Comparison with state-of-the-art codecs shows reasonable savings and PSNR gains for low and high bitrates, while maintaining good reconstruction quality. 1 INTRODUCTION Computational multi-view displays involving light fields are a fast emerging choice for 3D presentation of real-world scenes [1, 2]. These autostereoscopic or glass-free displays offer a more realistic viewing experience because of their ability to support continuous motion parallax, greater depth of field and a wider field of view [3-8]. Light field displays provide an optimised solution to support direction-dependent outputs simultaneously, without sacrificing the resolution in reproducing real-world 3D scenes. The challenge is to address the storage and transmission requirements that come with handling the large light field data. A dynamic light field can directly be generated from a static light field to account for 3D scene changing or moving over time [9]. This further adds to the coding burden of any system. It is critical to exploit the correlations present in static or dynamic light fields and develop efficient coding and representation solutions that are suitable for display and streaming applications. Existing light field coding approaches are designed either to work directly on the raw lenslet image [10-14] or operate on individual sub-aperture images (SAIs). Further classifications include content-based compression [15], disparity-based [16, 17], epipolar plane image-based and multiplane image-based [18, 19], pseudo sequence-based [20-23], view synthesis-based [24-26], and learning-based compression methods [27-30]. These coding approaches are not explicitly designed to provide a compact representation, flexible coding and streaming of light fields on a variety of 3D displays. Presently available methods usually train a system (or network) to support only specific bitrates during the compression. The current study differs from our earlier models and details the pipeline of a novel hybrid light field representation, coding and streaming scheme, suitable for various computational multi-view autostereoscopic platforms. We first simulate a coded aperture camera 9, 31-34 to generate acquired images. A static or dynamic light field can then be computationally constructed using images acquired through different coded aperture patterns with a deep convolutional neural network (CNN). The next step performs low-rank approximation of acquired images with hybrid Tucker-TensorSketch Karhunen–Loéve transform-singular value decomposition via double sketching (HTTS-KLTSVD-DS). Incorporation of double sketching allows our algorithm to work faster in a single pass itself, and there is no need to store large Kronecker products of Tucker decomposition in the memory. This permits acquired images to be streamed incrementally to the receiving end at every time instant and eliminates the need to store the complete static or dynamic light field. A compact representation of factor matrices by KLT-SVD in our proposed model acts as an optimal transform with good energy compaction property. Additional encoding of low-rank approximated acquired images using High Efficiency Video Coding (HEVC) eliminates intra-frame, inter-frame and other intrinsic redundancies in the static or dynamic light field. Our scheme thus provides efficient representation, coding, transmission and streaming adaptability of static or dynamic light fields, making it suitable for viewing from many angles simultaneously in full resolution on a compact light field display. The critical advantages of the proposed HTTS-KLTSVD-DS scheme are as follows: It is a complete light field representation, coding and streaming scheme. Our model consists of acquiring images from coded aperture patterns and approximation by HTTS-KLTSVD-DS followed by reconstruction of the static or dynamic light field. At every instant, the light field can be streamed incrementally, which eliminates the need to store the complete data. It works with lesser data by compressing the acquired images of the static or dynamic light field, rather than considering a full light field as an input. The deep CNN model produces optimised coded aperture patterns that are sufficient to obtain the acquired images and reconstruct the complete light field of good spatial and angular resolutions. Intrinsic correlations in static or dynamic light field structure are effectively exploited by approximating acquired images obtained through optimised coded aperture patterns. The intra-frame, inter-frame redundancies, and other redundancies are eliminated. Our proposed HTTS-KLTSVD-DS is a one-pass algorithm that employs tensor double sketching to perform low-rank approximation of the acquired images without actually storing them. Besides, local variations in statistics of factor matrices are exploited using KLT-SVD transform, which leads to increased coding efficiency making use of energy compaction property. It is flexible to realise a range of multiple bitrates within a single integrated system for various ranks and sketch dimension parameters. We achieve reasonable bitrate savings and PSNR gains for both low and high bitrates, while maintaining good reconstruction quality. The proposed model complements existing light field coding systems, which support only specific bitrates during the compression, and is adaptable to support other schemes by optimising the bandwidth for a given target bitrate. The rest of this study is organised into four major sections. Related work and gaps in literature are addressed in Section 2. Our proposed HTTS-KLTSVD-DS scheme for light field representation, coding and streaming is discussed in detail in Section 3. We have elaborated our experiments specifying the implementation, results, and analysis in Section 4. Finally, Section 5 concludes this paper findings of our proposed scheme and implications of future work. 2 RELATED WORK The light field image or plenoptic image contains information about the intensity and direction of light rays in space [35, 36] and involves a large volume of data. Thus, the storage and transmission requirements for light fields are tremendous. Efficient compression techniques leveraging the data redundancy in both the spatial and angular domains of light fields are essential. Compression or coding techniques for light fields are often broadly classified into two categories; lenslet-based or approaches based on perspective SAIs. The direct compression using raw light field lenslet image captured by plenoptic camera exploits the spatial redundancy between the microimages [10-14]. The unique light field structural characteristics make it harder to predict the lenslet image regions with complex textures accurately. Moreover, there are numerous microimages in the raw light field image, which need careful handling. The microimages are of low resolution and require customised reshaping before being fed into an encoder. Ideally, the lenslet-based compression solutions need to transmit camera parameters for further processing, and thus increase the coding burden on the compressed data stream. These drawbacks encouraged researchers to extract the light field SAIs from raw plenoptic images and explore compression possibilities on these views. There are various coding methods that directly take light field SAIs as the input. They are broadly categorised as content-based compression [15], disparity-based [16, 17], epipolar plane image-based and multiplane image-based [18, 19], pseudo sequence based [20-23], view synthesis-based 24-26, 37, and learning-based compression methods 27-30, 38, 39. Disparity-based compression methods may apply the homography-based low-rank approximation method called HLRA [16]. This approach depends on how much the disparities across views vary and may not optimally reduce the low-rank approximation error for light fields with large baselines. Dib et al. [17] proposed a novel parametric disparity estimation method to support the low-rank approximation using super rays. A shearlet transform-based method presented in Ref. [18] categorises SAIs into key and decimated views. The scheme performs well under low bitrates. The decimated views are predicted from the compressed key views and a residual bitstream. Chen et al. [19] used multiplane representation and strongly reduced the calculation burden on the decoder. The views are reordered as pseudo sequences in predictive compression methods. Liu et al. [20], in their symmetric 2D hierarchical order, compress the central view first as the I-frame (intra frame) followed by the remaining views as P-frames. Their prediction structure ensures inter-view prediction from adjacent views. Another proposal by Li et al. [21] involves dividing all the light field views into four quadrants and adopting a hierarchical coding structure within each quadrant. Ahmad et al. [22] use hierarchical levels, where views belonging to higher levels are assigned with better quality, to compress the light field in the form of a multi-view sequence. The higher-level views are used as references to predict the lower-level views. Synthesised virtual reference frames are generated from Adaptive Separable Convolution Network in another SAI-based technique [23]. Such frames are considered as extra reference candidates in a hierarchical coding structure for multi-view HEVC (MV-HEVC) to further exploit intrinsic similarities in light field images. The Multi-view Video plus Depth (MVD) structure is adopted for depth image-based rendering to synthesise the intermediate SAIs [24]. A pair of steps to generate noise-refined depth maps for selected perspective views is elaborated in Ref. [25]. Deep learning-based methods encode certain SAIs, and remaining SAIs are synthesised using a CNN from the decoded views 27, 29, 38. However, these methods require large-scale and diverse training samples, and high quality of reconstructed views is only obtained if at least half the SAIs are taken as reference. Liu et al. [30] also adopt a generative adversarial network framework to boost the light field compression. They use an image group-based sampling method to reduce more SAI redundancy and maintain the reconstructed SAI quality. The above-mentioned coding techniques are not explicitly designed for the representation used in multi-layer-based light field displays. They also usually train a system (or network) to support only specific bitrates during the compression. Our previous work on light field coding exploits the spatial and temporal correlations among light field multiplicative layers [40]. We handle inherent redundancies in light fields by approximating multiplicative layers in the image-based spatial domain. In Ref. [41] and its extended version Ref. [42], we have presented a hierarchical light field coding scheme based on hybrid stacked multiplicative layers and Fourier disparity layers. In Ref. [43], we show a Hybrid Tucker-TensorSketch Vector Quantisation algorithm that encodes factor matrices in a codebook form through vector quantisation after Tucker decomposition of acquired images from a dynamic light field. The current study details a pipeline of a novel hybrid representation, coding and streaming scheme for static or dynamic light fields. 3 PROPOSED SCHEME Figure 1 illustrates the pipeline of the proposed light field representation, coding and streaming scheme. The complete work-flow can be divided into three main components. BLOCK I represents convolutional neural networks (CNNs) that reduce static or dynamic light fields into acquired images through a physical imaging process of the coded aperture camera 9, 31. In the second component (BLOCK II), the intrinsic redundancy in acquired images is efficiently removed through our hybrid Tucker-TensorSketch KLT-SVD via double sketching (HTTS-KLTSVD-DS) algorithm. Further encoding of low-rank approximated acquired images using HEVC [44] effectively eliminates inter-frame and intra-frame redundancies. Lastly, in BLOCK III, the light field is reconstructed from the decoded images. Each component of the proposed compression scheme is explained in the following sections. FIGURE 1Open in figure viewerPowerPoint The complete workflow of the proposed light field representation, coding and streaming scheme with three main components Static or dynamic light fields to acquired images The 4D light field L(s, t, u, v) is parameterised by coordinates of the intersections of light rays with two planes [35, 36]. A coded aperture camera can computationally construct the light field using images acquired through different aperture patterns 9, 31-34. As illustrated in Figure 2, such a camera records incoming light rays by adopting a coordinate system, where (s, t) and (u, v) denote the aperture and imaging planes, respectively. FIGURE 2Open in figure viewerPowerPoint Schematic diagram of a coded aperture camera In first component (BLOCK I) of the proposed scheme, we generate acquired images from real light field data through optimised coded aperture patterns. Transmittance for the aperture designs can be controlled for each position and each acquisition of the coded aperture camera, and our goal is to derive optimal aperture patterns that give the best acquired images for static and dynamic light field reconstruction 9, 31. Subsequently, the acquired images can be represented in a low-rank approximation and compressed so that they stored or transmitted instead of the entire light field. The mathematical formulation for obtaining acquired images from static and dynamic light fields through aperture transmittances is as follows: 3.1.1 Static light fields For a 5 × 5 stationary or static light field L(s, t, u, v), s and t are integers limited to a range of [−2, +2]. The light field will hence have 25 SAIs, ls,t(u, v) = L(s, t, u, v). Here, (s, t) corresponds to the position index of a viewpoint on the aperture plane, and (u, v) indicates a discrete pixel position in the SAI [31]. To obtain Nstatic = 2 acquired images, let rn(s, t) be the transmittance at position (s, t) for the n-th acquisition (n = 1 to Nstatic). The corresponding observed/acquired image is formed as a n ( u , v ) = ∬ r n ( s , t ) L ( s , t , u , v ) d s d t = ∬ r n ( s , t ) l s , t ( u , v ) d s d t . $\begin{array}{r}\hfill {a}_{n}(u,v)=\iint {r}_{n}(s,t)L(s,t,u,v)dsdt\\ \hfill =\iint {r}_{n}(s,t){l}_{s,t}(u,v)dsdt.\end{array}$ (1)By considering the discretisation of the aperture plane indexed by (s, t), Equation (1) can be simplified as a n ( u , v ) = ∑ s , t r n ( s , t ) l s , t ( u , v ) . ${a}_{n}(u,v)=\sum\limits _{s,t}{r}_{n}(s,t){l}_{s,t}(u,v).$ (2) 3.1.2 Dynamic light fields When the target 3D scene can change over time, the captured light field is dynamic, that is, corresponding to a moving scene. Sakai et al. [9] have demonstrated that a dynamic light field can also be successfully reconstructed from acquired images using an optimised CNN that can separate disparity information from scene motions. The dynamic light field is constructed such that each sample/camera viewpoint consists of 5 × 5 views over consecutive time frames. In our proposed scheme, we have assigned pseudo motions to a static light field by clipping out five slightly different regions from each of the 25 views (Figure 3). Thus, we obtain five camera viewpoints, Ld−2, Ld−1, Ld, Ld+1 and Ld+2, as temporal sequences, with 25 SAIs each. FIGURE 3Open in figure viewerPowerPoint Creating dynamic light field dataset from a static light field through pseudo motions By considering such scene motions over time d, the light intensity on a 5D space is now L(s, t, u, v, d). The transmittance of the coded aperture at position (s, t) and d is r(s, t, d) and the observed/acquired image is then obtained as a d ( u , v ) = 1 E d ∭ E d × S × T r ( s , t , D ) L ( s , t , u , v , D ) d s d t d D ${a}_{d}(u,v)=\frac{1}{\left\vert {E}_{d}\right\vert }{\iiint }_{{E}_{d}\times S\times T}r(s,t,D)L(s,t,u,v,D)dsdtdD$ (3)where, Ed denotes exposure time around d and S × T is the effective aperture area. Assuming that the light field and aperture pattern are constant during exposure time, L(s, t, u, v, D) = Ld(s, t, u, v) = ls,t,d(u, v) and r(s, t, D) = rd(s, t). Similar to Equation (2), Equation (3) can be re-written as a d ( u , v ) = ∑ s , t r d ( s , t ) l s , t , d ( u , v ) . ${a}_{d}(u,v)=\sum\limits _{s,t}{r}_{d}(s,t){l}_{s,t,d}(u,v).$ (4)Note that each of the five camera viewpoint results in one acquired image and Ndynamic = 1 in the case of dynamic light fields. Another observation is that acquired images for both static and dynamic light fields are the weighted sum of the SAIs (Equations 2 and 4). 3.1.3 Deriving the acquired images If Dd denotes the local temporal window around d, the acquired images from dynamic light field are a d ́ ( u , v ) ${a}_{\stackrel{\check{d}}{d}}(u,v)$ for d ́ ∈ D d $\stackrel{\check{d}}{d}\in {D}_{d}$ . In order to optimise the aperture patterns rd(s, t) that give the best acquired images for light field reconstruction, we can represent the problem in terms of a neural network model. The physical imaging process of the coded aperture camera can be mapped as f 1 : L d → A d , ${f}_{1}:{L}_{d}\to {A}_{d},$ (5)where Ld represents a tensor containing all pixels of ls,t,d(u, v) at a particular time d, and Ad is the tensor containing all pixels of ad(u, v) at d. Next, by considering L d ˆ $\widehat{{L}_{d}}$ as the estimation of light field Ld, we can map the reconstruction as f 2 : A d ́ | d ́ ∈ D d → L d ˆ . ${f}_{2}:\left\{{A}_{\stackrel{\check{d}}{d}}\vert \stackrel{\check{d}}{d}\in {D}_{d}\right\}\to \widehat{{L}_{d}}.$ (6)Thus, the composite mapping f2(f1(Ld)) can be implemented as a deep CNN using 2D convolutional layers that can be trained end-to-end with training samples. The equivalent optimisation goal of the network formulated with the squared loss error is f 1 ˆ , f 2 ˆ = arg min f 1 , f 2 L d − L d ˆ 2 . $\widehat{{f}_{1}},\widehat{{f}_{2}}=\underset{{f}_{1},{f}_{2}}{\mathrm{arg min}\,}{{\Vert}{L}_{d}-\widehat{{L}_{d}}{\Vert}}^{2}.$ (7)Hence, we jointly optimise the learnt parameters f 1 ˆ $\widehat{{f}_{1}}$ and f 2 ˆ $\widehat{{f}_{2}}$ during the CNN training. Consecutively, the learnt optimal transmittance of aperture patterns rd(s, t) generate the required acquired images for the dynamic light field. A similar mapping can be applied for obtaining acquired images from a static light field. The slight difference is that it will not have time d factor and we derive Nstatic = 2 acquired images through two aperture patterns. Dynamic light fields produce Ndynamic = 1 acquired image for every camera viewpoint and network is constructed such that alternating aperture patterns A and B are repeated over d. Figure 4 illustrates the complete CNN architecture to obtain acquired images from both static and dynamic light fields as AIS and AID, respectively. FIGURE 4Open in figure viewerPowerPoint CNN architecture to obtain acquired images from static and dynamic light fields. CNN, convolutional neural network Low-rank representation of static or dynamic acquired images Algorithms that adopt randomised methods are popularly used for tensor decompositions. To perform CANDECOMP/PARAFAC (CP) decomposition, Battaglino et al. [45] and Wang et al. [46] have previously applied sketching techniques. Various randomised methods to compute higher-order singular value decomposition have been developed in Ref. [47-49]. The CUR factorisation for matrices has also been extended for tensors in Ref. [50]. Memory Efficient Tucker (MET) decomposition for large sparse tensors introduced by Kolda and Sun [51] and other distributed methods [52-54] have also been used. The Tucker decomposition is used to decompose a tensor into a core and factor matrices. While incorporating random sketching in Tucker decomposition, it is crucial to consider the large computation due to Kronecker products involved in the algorithm. The standard sketching techniques have design matrices as Kronecker products of the factor matrices. They are quite large to be formed and stored in the RAM, which led to development of TensorSketch that is more suitable for sketching Kronecker products [55, 56]. Malik and Becker [57] have recently developed single-pass algorithms based on TensorSketch that can perform low-rank decomposition of larger tensors and are also capable of handling streamed data. Previous work on incremental tensor analysis [58], multi-aspect-streaming tensor analysis based on histogram approximation [59] and incremental CP decomposition [60] do not correspond to the Tucker decomposition of a tensor whose elements are streamed. The TUCKER-ALS, MET [51] and Higher Order Orthogonal Iteration (HOOI) version of MACH algorithm presented in Ref. [49] run out of memory during decomposition of large tensors and are not one-pass algorithms. Tucker decomposition based on the streaming properties of TensorSketch proposed by Malik and Becker [57] are highly efficient for low-rank decomposition. It can compress data without the need to store it in full by streaming the tensor elements. The algorithm only requires a single pass of the data and performs with good accuracy and a reduced run-time. Motivated by this work, we have presented the proposed light field coding, transmission and streaming scheme that can handle multiple bitrates of data. A static light field produces two acquired images and the dynamic light field produces five acquired images (one for each camera viewpoint) from CNN architecture in the first component of proposed scheme. We take each acquired image as a tensor Y ∈ R I 1 × I 2 × ⋅ ⋅ ⋅ × I N $\mathcal{Y}\in {\mathbb{R}}^{{I}_{1}\times {I}_{2}\times \cdot \cdot \cdot \times {I}_{N}}$ , an array of dimension N. A low-rank representation of the acquired images is constructed using our proposed HTTSVQ algorithm. A standard Tucker decomposition of tensor X $\mathcal{X}$ is defined as X = G × 1 A ( 1 ) × 2 A ( 2 ) ⋅ ⋅ ⋅ × N A ( N ) = : G ; A ( 1 ) , A ( 2 ) , … , A ( N ) , $\mathcal{X}=\mathcal{G}{\times }_{1}{\mathbf{A}}^{(1)}{\times }_{2}{\mathbf{A}}^{(2)}\cdot \cdot \cdot {\times }_{N}{\mathbf{A}}^{(N)}=:\left[\mathcal{G};{\mathbf{A}}^{(1)},{\mathbf{A}}^{(2)},\text{{\ldots}},{\mathbf{A}}^{(N)}\right],$ (8)where G ∈ R R 1 × R 2 × ⋅ ⋅ ⋅ × R N $\mathcal{G}\in {\mathbb{R}}^{{R}_{1}\times {R}_{2}\times \cdot \cdot \cdot \times {R}_{N}}$ is called the core tensor, and every A ( n ) ∈ R I n × R n ${\mathbf{A}}^{(n)}\in {\mathbb{R}}^{{I}_{n}\times {R}_{n}}$ is known as a factor matrix. By assuming that the factor matrices have orthonormal columns, X $\mathcal{X}$ in Equation (8) is a rank-(R1, R2, …, RN) tensor. Decomposing each acquired image as a data tensor Y $\mathcal{Y}$ using Tucker decomposition can be formulated as arg min G , A ( 1 ) , … , A ( N ) Y − G ; A ( 1 ) , A ( 2 ) , … , A ( N ) $\underset{\mathcal{G},{\mathbf{A}}^{(1)},\text{{\ldots}},{\mathbf{A}}^{(N)}}{\mathrm{arg min}\,}{\Vert}\mathcal{Y}-\left[\mathcal{G};{\mathbf{A}}^{(1)},{\mathbf{A}}^{(2)},\text{{\ldots}},{\mathbf{A}}^{(N)}\right]{\Vert}$ (9) Alternating least-squares (ALS) is a standard approach to solve the optimisation problem in Equation (9). It can be rewritten appropriately and solved by repeating the following steps until convergence [61]: For n = 1, …, N, update A ( n ) = arg min A ∈ R I n × R n ⊗ i = N , i ≠ n 1 A ( i ) G ( n ) ⊤ A ⊤ − Y ( n ) ⊤ F 2 ${\mathbf{A}}^{(n)}=\underset{\mathbf{A}\in {\mathbb{R}}^{{I}_{n}\times {R}_{n}}}{\mathrm{arg min}\,}{{\Vert}\left(\underset{i=N,i\ne n{}}{\overset{1}{\otimes }}{\mathbf{A}}^{(i)}\right){\mathbf{G}}_{(n)}^{\top }{\mathbf{A}}^{\top }-{\mathbf{Y}}_{(n)}^{\top }{\Vert}}_{F}^{2}$ (10) Update G = arg min Z ∈ R R 1 × ⋅ ⋅ ⋅ × R N ⊗ i = N 1 A ( i ) z ( : ) − y ( : ) 2 2 $\mathcal{G}=\underset{\mathcal{Z}\in {\mathbb{R}}^{{R}_{1}\times \cdot \cdot \cdot \times {R}_{N}}}{\mathrm{arg min}\,}{{\Vert}\left(\underset{i=N{}}{\overset{1}{\otimes }}{\mathbf{A}}^{(i)}\right){\mathbf{z}}_{(:)}-{\mathbf{y}}_{(:)}{\Vert}}_{2}^{2}$ (11) TensorSketch can be applied on the least-square problems in Equations (10) and (11) to solve the resulting smaller problems efficiently [57, 62]. In our proposed scheme, we randomly initialise core and factor matrices considering each element as i.i.d in U(−1, 1) and orthogonalise them. TensorSketch operators T(n) and T(N+1) are defined by constructing CountSketch [63] operators S 1 ( n ) ∈ R J 1 × I n ${\mathbf{S}}_{1}^{(n)}\in {\mathbb{R}}^{{J}_{1}\times {I}_{n}}$ and S 2 ( n ) ∈ R J 2 × I n ${\mathbf{S}}_{2}^{(n)}\in {\mathbb{R}}^{{J}_{2}\times {I}_{n}}$ , respectively, for n ∈ [N]. Definitions of CountSketch and TensorSketch operators are given in Appendices A and B, respectively. Algorithm 1. Proposed Hybrid Tucker-TensorSketch KLT-SVD via double sketching (HTTS-KLTSVD-DS) For practical implementation, we choose CountSketch operators S 1 ( n ) $\left\{{\mathbf{S}}_{1}^{(n)}\right\}$ and S 2 ( n ) $\left\{{\mathbf{S}}_{2}^{(n)}\right\}$ with different target sketch dimensions J1 = KR(N−1) and J2 = KRN, for sketch dimension parameter K > 4 and rank R as suggested by Malik and Becker [57]. The target sketch dimensions are such that J2 > J1. We use Equation (B2) defined in Appendix B while applying TensorSketch operators to the Kronecker product matrices in lines 10 and 11 of Algorithm 1. Whenever In ≥ J1 + J2 for some n ∈ [N], the size of the least-squares problem on line 10 is automatically reduced by utilising Double Sketching. Only the sketches S 1 ( n ) A ( n ) ${\mathbf{S}}_{1}^{(n)}{\mathbf{A}}^{(n)}$ and S 1 ( n ) A ( n )