Low‐complexity block tree image coder for visual sensor networks

IET Image ProcessingVolume 14, Issue 16 p. 4258-4268 Research ArticleFree Access Low-complexity block tree image coder for visual sensor networks Mohd Rafi Lone, Corresponding Author Mohd Rafi Lone mrlone@bgsbu.ac.in Department of Electronics and Communication Engineering, Baba Gulam Shah Badshah University, Rajouri, J&K, 185234 IndiaSearch for more papers by this authorEkram Khan, Ekram Khan Department of Electronics Engineering, Z. H. College of Engineering and Technology, Aligarh Muslim University, Aligarh, 202002 UP, IndiaSearch for more papers by this author Mohd Rafi Lone, Corresponding Author Mohd Rafi Lone mrlone@bgsbu.ac.in Department of Electronics and Communication Engineering, Baba Gulam Shah Badshah University, Rajouri, J&K, 185234 IndiaSearch for more papers by this authorEkram Khan, Ekram Khan Department of Electronics Engineering, Z. H. College of Engineering and Technology, Aligarh Muslim University, Aligarh, 202002 UP, IndiaSearch for more papers by this author First published: 23 February 2021 https://doi.org/10.1049/iet-ipr.2020.0124AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract The wavelet block tree coding (WBTC) algorithm is an efficient wavelet-based image coder. In this coder, multiple spatial orientation trees are combined together to make a single block tree. It uses three ordered lists to keep track of significant/insignificant coefficients and sets while coding, which increases its memory requirement as well as computational complexity. Also, it uses memory inefficient conventional discrete wavelet transform (DWT) to compute the transformed coefficients. In this study, a Low-Complexity Block Tree Coding (LCBTC) algorithm that uses two state-tables and two very small lists, is proposed. Similar to WBTC, it also uses sorting and refinement passes in each bit-plane. However, it encodes the coefficients in block-tree manner using depth-first search approach to reduce the computational complexity. It uses DWT coefficients obtained from modified fractional wavelet filter (MFrWF) rather than conventional DWT, which further reduces the overall memory and complexity of the image coder. The simulation results show that the memory requirement and computational complexity of LCBTC is much less than WBTC and other state-of-the-art coding algorithms. These features make the image coder a better candidate for compression in memory constrained and real-time visual sensor networks. 1 Introduction 1.1 Overview For surveillance and security monitoring, camera-based networks have been used for a long time now. These cameras would send the video stream to a central facility, where a human observer would monitor and process it further [[1]]. However, with the advancements in networking and image sensor technology, a number of applications have replaced the other sensors with image sensors. This is because the image data provides more precise and reliable information in comparison to one-dimensional signals [[2], [3]]. The image sensors can these days be seen in almost all domestic gadgets like cell phones, computers, toys, robots and vehicles. Due to the miniaturisation of image sensors and their low-power requirements, they are used in networks known as visual sensor networks (VSNs) [[4]]. A VSN consists of a large number of camera nodes. These camera nodes form a distributed network. Each camera node processes the captured data locally, communicates with other camera nodes and provides the user with accurate information [[5], [6]]. These networks are primarily used in military applications. But they can be used in civilian applications like healthcare, automation, traffic control and surveillance, as well [[7]-[9]]. 1.2 Motivation Due to the large number of camera nodes and ever-increasing resolutions in these camera nodes, the huge amount of data are generated and processed. Image compression is used to reduce the size of data required to be transmitted in these networks. Furthermore, these nodes have a very small memory. Most of the image sensor nodes have a typical value of random access memory (RAM) of the order of 64 kB or lesser [[10]-[16]]. This provides the upper limit of memory available for the processing and compression of subsequently captured images. This adds another dimension to the issue of VSNs; complexity of the coder. The coder should be fast enough to encode the current image before the next image is captured, so that the same memory is reused. A typical image coder consists of three stages. The image transformation stage, followed by quantisation and encoder stages. Two typical transforms used in most of the state-of-the-art image coders are: discrete cosine transform (DCT) [[17]] and discrete wavelet transform (DWT) [[18]]. However, several studies have revealed that DWT is more efficient than DCT [[19]]. The algorithm and methodology used by different image coders for quantisation and encoding stages, are different. For memory-limited VSNs, the memory requirement of image coders needs to be minimised. Many image coders have been designed to reduce the memory requirement in order to be used in applications where memory is limited. However, sometimes this has resulted in increased complexity when the memory is reduced beyond certain limit [[20], [21]]. However, this has often resulted in the increased complexity of the encoders and decoders in these image coders. Therefore, a low-complexity image coder needs to be developed, which also requires low on-chip memory. 1.3 Related work 1.3.1 Low-complexity DWT Conventionally convolution method has been used for the implementation of DWT. The computational efficiency of the conventional method is high. Moreover, it requires a large RAM. VSNs and other low-memory devices cannot afford such high memory. In order to reduce the memory, line-based DWT method was first introduced in [[22]]. In line-based approach, a few lines are buffered into the RAM and then processed. The architectures, implementing line-based DWT are proposed in [[23]-[27]]. Other line-based DWT implementations are provided in [[28], [29]] using lifting scheme. Lifting scheme is used to reduce the complexity, by breaking the bigger operations into smaller ones. Block-based implementation was proposed in [[30]-[33]]. In block-based approach, only small blocks are processed, rather than the entire image. The architectures for block-based approach implementation are proposed in [[34]-[36]]. Strip-based approach for DWT implementation was proposed in [[37]-[42]]. The strip-based implementations have mainly been utilised for memory-limited applications like VSNs. DWT architectures using strip-based approach are proposed in [[43]-[45]]. Fractional wavelet filter (FrWF) is another approach for computing DWT coefficients of an image [[46]]. Only one row of the image is buffered into the filter at a time. The memory of FrWF is reduced further in segmented FrWF (SFrWF), proposed in [[47]]. In SFrWF, the input image is segmented in multiple sub-images and only a single segment is read into the memory. However, the complexity of SFrWF is increased in comparison to FrWF. The complexity is reduced in Modified FrWF (MFrWF), proposed in [[48]]. In this approach, the one-dimensional DWT coefficients are obtained the same way as conventional method. However, the column-wise filtering takes place in such a way that the line is accessed only once. 1.3.2 Low-complexity encoders With the improvements in transformation techniques, improvements in image coders have also been witnessed. The DCT based image coder, JPEG [[49]], developed by Joint Photographic Experts Group has been very popular for the last three decades. As DWT provides more efficient energy compaction in comparison to DCT, the researchers have shifted their focus towards DWT based encoders. A number of DWT based encoders have been proposed till date. Embedded block coding with optimised truncation [[50]], used in JPEG2000 [[51]] is considered as the state-of-the-art image coder. Other state-of-the-art algorithms include embedded zerotree wavelet (EZW) coder [[52]], set partitioning embedded block coder (SPECK) [[53]], set partitioning in hierarchical trees (SPIHT) [[54]], embedded zero block coding (EZBC) [[55]], wavelet block tree coding (WBTC) [[56]] and sub-band block hierarchical partitioning [[57]]. None of these image coders are suitable for compression in applications with low on-chip memory or requiring low complexity. This is because, either the computational complexity or the memory requirement of these coders is very high. A number of image coders have been proposed in order to reduce the memory or computational complexity or both. SPIHT has been modified to remove its dependency on long lists in [[58]-[64]]. Modified SPIHT (MSPIHT) [[60]] uses only one list instead of three lists. Also, it merges the two encoding passes; refinement and SP. As a result, the computational complexity and memory requirement of MSPIHT is reduced in comparison to SPIHT. No List SPIHT [[61]] has been very popular as it removes the lists completely and places markers (4 bits per coefficient) instead. Similarly, SPECK has been modified to obtain listless versions in [[20], [21], [65]]. Listless SPECK (LSK) [[20]] uses 2 bits per coefficient while as no list SPECK (NLSK) [[65]] uses only 0.75 bits per coefficient. Zero-memory SPECK (ZM-SPECK) [[21]] is a memory efficient variant of SPECK, eliminating the need for linked lists. Only few registers are used to perform low-level operations. In addition, the computational complexity is also reduced. WBTC is modified to get its listless versions in [[66]-[68]]. Listless block tree coding (LBTC) [[66]] has used markers of fixed size. Low-memory block tree coder (LMBTC) [[68]] uses fractional wavelet filter (FrWF) to meet the memory-constraints of low-cost sensor networks. LMBTC combined with FrWF, with block size 4, requires only 6.24 and 12.49 kB of memory for images with sizes 256 ×256 and 512 ×512, respectively. 1.4 Contribution of this paper The memory in LMBTC [[68]] and ZM-SPECK [[21]] has already been significantly reduced. However, the wavelet transform stage needs some memory, which sets a lower limit on the memory requirement of the overall image coder. Therefore, either the memory of the wavelet transform stage needs to be reduced or an image coder with memory requirement anything lesser than wavelet transform technique is sufficient. In this paper, a low-complexity block-tree coding (LCBTC) algorithm is proposed. The aim is to reduce the complexity of the image coder for real-time applications in VSNs. The coder is combined with low-complexity MFrWF. The memory constraint of VSNs has been considered, for visual sensors with on-chip RAM having typical values of 64 kB and above. The LCBTC uses two state-tables and two very small lists. The encoding takes part in two passes; sorting pass (SP) and refinement pass (RP). In RP, the coefficients are encoded in breadth-first search manner. While as the coefficients in SP are encoded in depth-first search manner. The block-trees are encoded one after the other. As a result of which, only a few block addresses are stored in these lists. The list size is fixed for each decomposition level and does not depend upon the transformed image size. The coefficients are only once checked for significance at a certain threshold. The state-tables are used to store status information of each block. The state-information is used to decide whether a particular block-tree (or block) needs to be checked at certain threshold in a particular pass, or not. The complexity of LCBTC is significantly reduced in comparison to the existing state-of-the-art image coders. Also, the memory requirement of LCBTC is lesser than that of MFrWF. Given the memory constraint of VSNs, the image coder can encode an image with resolution 1024 × 1024. 1.5 Organisation The rest of the paper is organised as follows. Section 2 presents the introduction to FrWF, modified fractional wavelet filter (MFrWF ) and WBTC. Section 3 describes the proposed LCBTC image coder, and the algorithm it uses. Section 4 provides the memory requirement analysis of the proposed LCBTC algorithm. The complexity analysis of the proposed image coder in comparison to WBTC is presented in Section 5. The memory requirement, computational analysis and compression efficiency results are presented in Section 6. Section 7 concludes the paper and also suggests the future work. 2 Background This paper combines the MFrWF with the proposed LCBTC image coder. In this section, overviews of FrWF, MFrWF and WBTC are presented. 2.1 Fractional wavelet filter (FrWF) The FrWF technique of computing the DWT of images has been proposed in [[46]]. Three small buffers are used to store very small number of coefficients. One buffer of size N, for an image size of N × N, stores the input coefficients. Two more buffers of same size each, are used to store coefficients temporarily for destination. For first level of decomposition, the input line is read from an external storage line-by-line. The decomposed coefficients are then stored in a different destination storage in four subbands; LL/HL and LH/HH. For next levels LL band coefficients are read, decomposed and written back into the external storage. The memories (in Bytes) required to compute the wavelet transform of an image with size N × N using fixed and floating point representation are given in (1) and (2), respectively MEM _ FrWF fixed = 5 N , level = 1 ; 6 N level , level > 1 (1) MEM _ FrWF float = 9 N , level = 1 ; 12 N level , level > 1 (2) 2.2 Modified fractional wavelet filter The process of filtering in FrWF is modified in MFrWF [[48]]. For wavelet filter Cohen Daubechies Wavelet–9/7, eight buffers are utilised. For an image size of N × N, one buffer of size 1 × N and seven buffers, each of size 1 ×N/2 are used. The coefficients of one row are read into the first buffer 's'. Three processes are then used to compute the wavelet transform of even and odd rows. Each of the processes works in two stages. In the first stage, the coefficients in buffer s are horizontally filter as in conventional DWT. The values are down-sampled by 2 and then stored in buffer b L / H . In the second stage, the coefficients in buffer b L / H are vertically filtered and stored in the corresponding intermediate filter. Hence, the buffer b L / H is used to store the values for both horizontal and vertical filtering. All the three processes are similar and differ only in coefficients and buffers in which the intermediate values are stored. The MFrWF algorithm is reading each image line only once to compute the complete 2D-DWT. On the other hand, the FrWF would read each odd line image line at least three times and each even line two times to compute 2D-DWT. The memory access time and the hence the computational complexity is reduced. However, the memory requirement by MFrWF is slightly more than FrWF. The memory requirement (in Bytes) is given by (3) MEM _ MFrWF = N ( 4 n l − 5 ) , level = 1 ; 2 N ( 2 n l − 1 ) 2 ( level − 1 ) , level > 1 (3) where n l is the length of low-pass wavelet filter. 2.3 Wavelet block tree coding algorithm Just like WBTC [[56]], the proposed algorithm is also based on spatial orientation block tree coding. It is therefore important to introduce WBTC algorithm here briefly. WBTC is wavelet based encoder, encoding the wavelet coefficients bit-plane wise starting from highest bit-plane. The redundancy among sub-bands at different levels is exploited just like SPIHT. However, unlike SPIHT, the coefficients are grouped into ( m × n ) coefficient blocks and then block-trees are encoded. In this way, multiple spatial orientation trees (SOT) of SPIHT are combined in a single block tree in WBTC. A block tree consists of all the descendant blocks of a root block, which is located at highest sub-band (LL). The block tree nature of encoding helps WBTC to partially exploit the redundancy within the sub-bands as well. Each block has four offspring blocks in next lower decomposition level, except for the sub-bands located at first level and topmost level (LL). The blocks in LL band have only three descendants located in the same level of decomposition in LH, HL and HH sub-bands. Type 'A' consists of set of all descendant blocks, while as Type 'B' consist of set of all grand descendant blocks (all descendant minus offspring). WBTC uses three ordered lists to store the significance information; list of insignificant blocks (LIB), list of insignificant block sets (LIBS) and list of significant pixels (LSP). At initialisation, the blocks in LL band are added to LIB and the blocks in LL band with descendants are added to LIBS as Type 'A' entries. The LSP is initialised as an empty list. The encoding process in WBTC consists of two stages: refinement pass and sorting pass. The coding process starts with SP. The coefficients are encoded from most significant bit-plane to least significant bit-plane. The encoder traverses through LIB first. At current threshold, the blocks are tested for significance. If the block is found to be insignificant, only a '0' bit is outputted to encode a zero block of m × n coefficients. The block remains in LIB to be tested again at next threshold. If the block is significant, it is sub-divided into four equal sized blocks (quad-tree partitioning) and a bit '1' is outputted. This process of quad-tree partitioning continues until the smallest sized block (individual coefficient) is reached. When the smallest block size is reached, the four coefficients are tested for significance. If the coefficient is found to be insignificant against current threshold, a '0' bit is output and the coefficient is moved to LIB. Otherwise, a '1' bit is output followed by the sign bit of the coefficient. The coefficient is then moved to LSP. Once all the four coefficients are tested, the block is deleted from LIB and the process of encoding other blocks in LIB resumes. Once all the blocks in LIB are tested, the encoder traverses through LIBS and examines each set in it for significance. If a set and its descendants are found to be insignificant, it remains in LIBS. Otherwise, the type 'A' set is partitioned into a type 'B' set and four offspring blocks. The type 'B' set is added at the end of LIBS while as the offspring blocks are immediately tested as blocks are tested in LIB. If the type 'B' set is found to be significant, its four offspring type 'A' sets are added to the end of LIBS. The process of encoding continues, until all the sets in LIBS are encoded. After completing the sorting pass, the RP starts. All the coefficients, except the ones that are added in current iteration, are refined with one bit. The threshold is reduced by a factor of 2 and the above mentioned procedure of encoding is repeated. The algorithm is iterated until a desired bitrate is reached. 3 Proposed MFrWF-based LCBTC image coder 3.1 Overview The proposed algorithm provides a modified version of WBTC. However, the organisation and the process of encoding are entirely different from that of WBTC. The input image is first wavelet transformed using Cohen-Daubechies-Wavelet 9/7, which is also used in JPEG2000 in lossy mode. MFrWF approach of transforming is used. Combining MFrWF with the novel compression algorithm reduces the overall complexity for image transformation. During each decomposition level, the low frequency sub-band is filtered using low- and high-pass filters to yield four sub-bands, low-low (LL), low-high (LH), high-low (HL) and high-high (HH). In the subsequent decomposition levels, the LL band is further filtered into four sub-bands and so on, as shown in Fig. 1. There is a spatial correlation among the sub-bands of same type (e.g. LH3, LH2, LH1) at different levels. The coefficients are magnitude-ordered as we move from higher level to lower level. The DWT coefficients are ordered in Morton scan order. There is a correlation among coefficients of like-sub-bands at different decomposition levels. A coefficient has four offspring coefficients that highly correlate with it at one decomposition level lower than the coefficient itself. The transformed image is partitioned into blocks of size ( 2 × 2). Each block in highest level sub-band is the root of a spatial orientation block tree. The addresses of the four offspring of any parent node 'x' are given by: 4 x , 4 x + 1 , 4 x + 2 and 4 x + 3. The lowest level decomposition sub-bands do not have any offspring. Also, the node blocks of the LL band in highest level have only three offspring, one in each sub-band: LH, HL and HH. Fig. 1Open in figure viewerPowerPoint SOT and the parent–children dependency 3.2 Proposed image coding algorithm The proposed low-complexity block tree coder algorithm partitions the DWT transformed image into blocks of size 2 ×2. For an image of size M × N, the number of blocks is given by M × N / 4. The proposed algorithm considers blocks as nodes in the spatial orientation block-tree unlike SPIHT and EZW which takes individual coefficients as nodes. Unlike WBTC, the block size is kept fixed in order to speed up the process of encoding. In Fig. 1, each node consists of a block of four coefficients. It is considered here that the node blocks in LL band have three offspring nodes in the highest decomposition level sub-bands. Two state-table memories, block-significance memory (BSM) and descendant-significance memory (DSM) are used to store the significance information of each node. The proposed algorithms consist of two passes, namely RP and SP. In RP, only the blocks which have become significant at previous threshold, are encoded or refined. However, the blocks, which have not become significant at previous thresholds, are encoded in SP. BSM gives the significance of each block in the image. Block 'x' is considered to be significant if BSM(x) = '1'. The significance information of descendants of all the nodes (that have descendants) is stored in DSM. If all the descendants of node 'x' are significant then DSM(x) = '1'. DSM(x) = '1' means that the whole block tree emerging from node 'x' is encoded in RP and not in SP. Two small lists are being used in addition to the two state-tables, these are: list of child blocks (LCB) and list of parent blocks (LPB). These lists are used to encode in SP. The encoding of the transformed image takes place as shown in Fig. 2. Three phases are used while encoding: Initialisation, RP and SP. The encoder traverses from initialisation to SP via RP. Once all the coefficients are encoded against the current threshold, threshold is updated by dividing it an order of 2. The number of bits in the encoded bit-stream and the bit-budget are continuously compared. Once the bit-budget is reached, the encoding process is terminated and the bit-stream is truncated. Fig. 2Open in figure viewerPowerPoint Flowchart for execution of the proposed algorithm 3.2.1 Initialisation The encoding is performed bit-plane wise starting with the most significant bit-plane 't'. The highest threshold t is computed as t = floor log 2 ⁡ ( Max coeff ) . The magnitude of the threshold is given by T = 2 t . The subsequent thresholds are obtained by decrementing the current threshold by one. The BSM is assigned '1' for the blocks in highest level of DWT (Level L). All other blocks are assigned '0' value in BSM memory. DSM is assigned '0' for all blocks at the beginning of the first RP. Both the lists are initialised to be empty. 3.2.2 Refinement pass The pseudo-code for the complete algorithm is given in Algorithm 1 (see Fig. 3). Those blocks which are significant at previous threshold are encoded in RP. In other words, all those blocks are encoded in RP for which BSM = '1'. Each coefficient is encoded within a significant block. If the coefficient is already significant, only the refinement bit is sent. However, a sign bit is also sent if the coefficient becomes significant at current threshold, as shown in Algorithm 2 (see Fig. 4). The blocks are encoded in breadth-first manner, starting with the blocks in highest decomposition level ( l e v e l = L ) to the lowest decomposition level ( l e v e l = 1 ). The blocks at level L − 1 are encoded only when the encoding of all blocks at level L is completed, and so on. Breadth-first encoding helps the algorithm to efficiently encode the image. This is because most of the information in transformed image is present in higher levels as compared to lower levels. In breadth-first approach, the coefficients which are likely to be more significant are encoded earlier in an encoding pass. Fig. 3Open in figure viewerPowerPoint Algorithm 1: pseudo-code of the proposed image coder Fig. 4Open in figure viewerPowerPoint Algorithm 2: PROCESS: EncodeBlock(x) 3.2.3 Sorting pass Those blocks which are left out in RP are encoded in SP. Encoding in SP takes place block-tree wise. Each block-tree has its roots in LL band. Second block-tree is taken for encoding only when the first block-tree is completely encoded. This helps the algorithm in many ways. Firstly, the list memory required to hold the addressed is significantly reduced. This is because the nodes from only one block-tree are required to be stored. Secondly, computational complexity is decreased because the encoder has not computed the significance of blocks or block-trees again and again in an encoding pass. The complete sequence of steps involved in SP is given in Algorithm 1 (Fig. 3). The significance test of a block 'B', which has been previously insignificant, is done as follows: S n ( B ) = 1 , if T ≤ max ( | c i , j | ) < 2 T ; 0 , if max ( | c i , j | ) < T ; c i , j ∈ B(4) Similarly, the significance test of the block tree 'BT' emerging from the block 'x' is obtained as follows: S n ( d ( x ) ) = 1 , if T ≤ max ( | c i , j | ) < 2 T ; 0 , if max ( | c i , j | ) < T ; c i , j ∈ B T(5) In the algorithm, d(x) and O(x) represent the descendants and offspring of the block 'x'. The SP uses two lists LCB and LPB. The addresses of the offspring block nodes are inserted in LCB, if it is found that the descendants of node 'x' are significant at current threshold. LCB is used to encode coefficients in block-trees and to update the BSM memory. Some highlights of LCB execution are mentioned below: Suppose a block is insignificant. If the whole tree emerging from that block remains insignificant at current threshold, only a bit '0' is output. Otherwise, bit '1' is output followed by encoding bits for the block. If all descendants remain insignificant at current threshold, bit '0' is output. Otherwise, bit '1' is output and the offspring are added to the LCB. If a block is significant, and all its offsprings are insignificant. Then if the descendants remain insignificant at current threshold, only a bit '0' is output. Otherwise a bit '1' is output and the offsprings are added to LCB and parent node is moved to LPB. An offspring block which becomes significant at current threshold is encoded by outputting '1', followed by encoding bits of each coefficient. When a block becomes significant BSM is updated from 0 to 1. When all the entries of LCB are removed, LPB is executed. The only purpose of LPB is to update DSM from the updated BSM. 3.2.4 Threshold update When the SP is completed, the threshold is halved. After updating threshold, if the bit-budget is not reached, the encoder enters RP. The process is repeated until a desired bit-budget is reached. 4 Memory allocation The proposed algorithm consists of two stages, which work one after the another. These stages are wavelet transform and encoding. The effective memory utilised by the overall image coder is the maximum of the memories utilised by the two stages. The memory utilised by the encoder is in the form of two state-tables and two-lists. The state-table memory is a function of image size while as the list memory is a function of number of decomposition levels used. 4.1 Encoder memory 4.1.1 State-table memory The state-table BSM stores the status of each 2 ×2 block of coefficients. One bit is required to hold the status of each block. For an image of size M × N, the size of BSM is given by MN/4 (bits). The second state-table DSM stores the status of descendant blocks. The block nodes in lowest decomposition level ( level = 1 ) do not have any descendants and hence do not require DSM status bits. Therefore, the size of memory required by DSM is MN/16 (bits). As the state-table memory is fixed and does not change while encoding, it is also referred to as static memory. The total memory (bits) required by these state tables is the sum of BSM and DSM memory given by M StateTables = ( 5 / 16 ) × ( M N )(6) 4.1.2 List memory The two lists contain the nodes of a block-tree. Only one block-tree is processed at any given time. Some nodes may be present in both the lists at a time. The number of nodes in LCB and LPB depend upon DWT level, L. As the number of entries in these lists change while encod