Efficient VLSI architecture for the parallel dictionary LZW data compression algorithm

IET Circuits, Devices & SystemsVolume 13, Issue 5 p. 576-583 Research ArticleFree Access Efficient VLSI architecture for the parallel dictionary LZW data compression algorithm Malek Safieh, Malek Safieh Institute for System Dynamics (ISD), HTWG Konstanz, University of Applied Sciences, Konstanz, GermanySearch for more papers by this authorJürgen Freudenberger, Corresponding Author Jürgen Freudenberger jfreuden@htwg-konstanz.de orcid.org/0000-0002-5913-4981 Institute for System Dynamics (ISD), HTWG Konstanz, University of Applied Sciences, Konstanz, GermanySearch for more papers by this author Malek Safieh, Malek Safieh Institute for System Dynamics (ISD), HTWG Konstanz, University of Applied Sciences, Konstanz, GermanySearch for more papers by this authorJürgen Freudenberger, Corresponding Author Jürgen Freudenberger jfreuden@htwg-konstanz.de orcid.org/0000-0002-5913-4981 Institute for System Dynamics (ISD), HTWG Konstanz, University of Applied Sciences, Konstanz, GermanySearch for more papers by this author First published: 01 July 2019 https://doi.org/10.1049/iet-cds.2018.5017Citations: 4AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract The Lempel–Ziv–Welch (LZW) algorithm is an important dictionary-based data compression approach that is used in many communication and storage systems. The parallel dictionary LZW (PDLZW) algorithm speeds up the LZW encoding by using multiple dictionaries. This simplifies the parallel search in the dictionaries. However, the compression gain of the PDLZW depends on the partitioning of the address space, i.e. on the sizes of the parallel dictionaries. This work proposes an address space partitioning technique that optimises the compression rate of the PDLZW. Numerical results for address spaces with 512, 1024, and 2048 entries demonstrate that the proposed address partitioning improves the performance of the PDLZW compared with the original proposal. These address space sizes are suitable for flash storage systems. Moreover, the PDLZW has relative high memory requirements which dominate the costs of a hardware implementation. This work proposes a recursive dictionary structure and a word partitioning technique that significantly reduce the memory size of the parallel dictionaries. 1 Introduction Universal data compression for flash memory systems has recently gained interest [1-9] because data compression can improve the storage capacity and the flash memory lifetime. Compression of redundant data reduces the amount of data transferred from the host to the flash system. This reduces the write amplification in flash memories, which is caused by the fact that the amount of data written to the flash memory is often much larger than the amount intended to be written [1, 10]. Moreover, data compression can be utilised to improve the reliability of the non-volatile storage system [2, 5-7, 9]. Data compression for such applications requires high data throughput that can only be achieved using hardware implementations. Moreover, it requires a universal data compression algorithm that can be used with small data blocks [6, 8]. The Lempel–Ziv–Welch (LZW) algorithm is one of the most important dictionary-based compression and decompression algorithms [11, 12]. The LZW algorithm is an integral part of many data compression systems, e.g. for image compression [13, 14] and storage [15]. It is often combined with other coding techniques to form efficient data compression systems [13, 16]. However, due to the recursive structure of the LZW dictionary, a fast hardware implementation of the LZW encoding is a challenging task [15, 17-19], i.e. several clock cycles are required to process a single input symbol. Hence, the recursive structure of the LZW dictionary limits the achievable data throughput. To speed up the encoding, a simplified LZW architecture, the parallel dictionary LZW (PDLZW), was proposed in [20, 21]. Instead of a single recursive dictionary that stores data strings of different lengths, the PDLZW algorithm uses multiple dictionaries, where each dictionary stores strings of the same length. These parallel dictionaries simplify the search process such that an input symbol can be processed in a single clock cycle. This improves data throughput. But the compression gain of the PDLZW algorithm depends on the partitioning of the address space, i.e. on the sizes of the parallel dictionaries. In fact, the original proposal of the PDLZW presents a number of different address space segmentations [21]. However, it does not advise how the best partitioning can be selected for a particular application or data model. In this paper, we consider the address space partitioning for the PDLZW encoding. In particular, we propose a technique to adjust the partitioning to the data model. We focus on address spaces with 512, 1024, and 2048 entries, which are suitable for compression in current flash storage systems [1, 3, 9]. Numerical results demonstrate that the proposed partitioning improves the performance of the PDLZW compared with the original proposal. These results are based on the Calgary and Canterbury corpora [22, 23]. Both corpora include real-world test files to evaluate lossless compression methods. Furthermore, the PDLZW algorithm requires large dictionaries that dominate the hardware costs of the entire compression system. To reduce the hardware costs for the dictionaries, data compression algorithms that combine the PDLZW algorithm and adaptive Huffman coding were proposed in [18, 24]. In this work, we propose two modifications to the PDLZW algorithm that reduce the size of the dictionaries and reduce the hardware costs. We show that the parallel dictionaries can be organised recursively without sacrificing the performance gain of the parallel dictionaries. This recursive structure significantly reduces the memory requirements for the dictionaries. The second contribution is based on a word partitioning technique for the content-addressable memories (CAMs) that are used to implement the dictionaries. In [24], the authors proposed a PDLZW implementation based on CAM to perform the required parallel search. One disadvantage of this implementation is that the resulting CAM module must be synthesised with circuits based on a large amount of D-type flip-flops because most cell-based libraries or FPGA-based devices do not support these CAM modules. To overcome this issue, we propose a random access memory (RAM)-based implementation of the CAMs. The memory size of the RAM-based implementation presented in [25] increases exponentially with the CAM word size. Hence, such an implementation would not be feasible for the original PDLZW that requires a word size of 64 bit with seven dictionaries. However, the word size required for the recursive PDLZW is significantly reduced compared with the original proposal. The CAM size can be further reduced using word partitioning [26, 27]. We propose a partitioning technique for the PDLZW that significantly reduces the memory requirements for the CAM implementation with only a small loss in compression gain. The proposed recursive structure and the word partitioning enable RAM-based PDLZW implementations. This paper is organised as follows. In Section 2, we discuss the PDLZW and the recursive PDLZW. The partitioning technique for the address space is present in Section 3. The hardware implementation and the partitioning technique for the word size are present in Section 4. In Section 5, we present data compression and synthesis results for the original and the proposed PDLZW algorithm. Finally, we summarise the main results in Section 6. 2 Recursive PDLZW In this section, we discuss and compare the encoding and the decoding process of the PDLZW and the recursive PDLZW. The LZW algorithm builds a dictionary of strings (symbol sequences) that occur in the input sequence. When a string reappears, it will be encoded with the index of the corresponding dictionary entry. The basic idea of the PDLZW is to use different dictionaries to store strings of different lengths. Each dictionary stores only strings of the same length. This improves storage alignment for the strings in the memory. Moreover, the partitioning of the dictionary into sub-dictionaries speeds up the search for strings. The compression of the input data is achieved when stored strings reappear and are represented by their addresses in the dictionary. 2.1 PDLZW encoding The PDLZW encoding is described by Algorithm 1 (see Fig. 1). The integer max denotes the number of existing dictionaries. This value also determines the maximum length of the string s. The actual length of the string s is denoted by the integer l which also determines the number of the currently selected dictionary. That is, the dictionary includes strings of length , the dictionary contains strings of length and so on. The dictionary represents all possible symbols that may appear in the input sequence. It is called the virtual dictionary because no memory is required for this dictionary. The integers old and new contain the addresses of dictionary entries, i.e. the address of the string found in the previous iteration and the address of the current string s in the dictionary , respectively. The character c is the symbol read from the input data in the current iteration. Fig. 1Open in figure viewerPowerPoint Algorithm 1: PDLZW encoding The initial value indicates an empty string s. The encoding loop starts by reading the next input symbol, and the loop terminates when the input is empty. The current string s is extended by appending the symbol c, and l is incremented. This concatenation is denoted by s = [s | c]. The search for the string s in dictionary results either in the new address new or a miss. Note that a miss is only possible for sting length , because the virtual dictionary contains all possible characters. If a miss occurs, the string s is added to the dictionary at the next free index. Moreover, an output codeword is generated. This output is determined by the string s from the previous iteration which is represented by the address stored in the variable old. This address was found in the dictionary . Hence, the output is the concatenation [|old] of the integer and the address old. Finally, the character c is stored in the string s and the values of l and old are updated accordingly. In the case, where the sting s is found in the dictionary , the algorithm depends on the length l of the string. For , the address is stored in the variable old and the algorithm continuous with the next iteration. However, indicates the maximum possible length. Hence, the string s cannot be extended and the corresponding codeword [|new] is written to the output. The algorithm continuous with an empty string s which is indicated by setting . In the final step, the input is empty, but the string s may contain some symbols. The remaining content of the string s is written to the output as [|old]. Fig. 2 illustrates an encoding example, where the input consists of 8-bit values that are interpreted as characters from the ASCII character table. The virtual dictionary represents all 256 possible 8-bit symbols. In this example, only four dictionaries are considered (). Fig. 2Open in figure viewerPowerPoint PDLZW encoding Firstly, the symbol c = ‘A’ is read which corresponds to the index 0 × 41 in the virtual dictionary. Note that this value is just the 8-bit input symbol and no search is required to determine the address in the virtual dictionary. The symbol is stored in the string s, its address is stored in the variable old and l is incremented by 1. Next, the symbol ‘r’ is read and appended to the string s. Dictionary is still empty. Hence, the new sequence s = ‘Ar’ is not found and a miss occurs. Consequently, ‘Ar’ is stored in the first address of the dictionary . The output ‘[0|0 × 41]’ represents the first symbol ‘A’ in the virtual dictionary. Note that two bits are sufficient to denote the unique number of the appropriate dictionary and 8 bits determine its address within the dictionary. When the generated string s already exists in the dictionary and , it is buffered for the next iteration. The address of this string is also stored in the variable old. The encoding of the tenth input symbol in Fig. 2 is an example of this case. The generated string s = ‘ee' exists already in the dictionary 1 at the address 0 × 03, this address is stored in the integer old. In the next iteration, a new string s = ‘ees’ is generated from the buffered string and the next input symbol. This string is stored at the first address in the dictionary because it is still empty and a miss occurs. The output [1|0 × 03] = 0 × 103 consists of the number of the last dictionary , and the stored address old = 0 × 03 of the last found entry. In this manner, all the input data of this example are completely encoded. The output data in Fig. 2 consist of 15 codewords, each of 10 bits length. 2.2 Recursive PDLZW encoding The operating principle of the PDLZW encoding is illustrated as a block diagram in the left side of Fig. 3. This diagram shows an example of the encoding with eight dictionaries. Note that dictionary 0 is the virtual dictionary, which does not require hardware resources and is not drawn. The height of the dictionaries represents its storage capacity. The storage capacity decreases with increasing string length because the probability of occurrence is smaller for longer strings. The output includes the internal address of the dictionary and the dictionary number. Fig. 3Open in figure viewerPowerPoint Comparing the PDLZW As shown in Fig. 3, each dictionary for strings of length extends only strings from the previous dictionary by an additional symbol. For example, dictionary 2 contains strings of length three. The first two symbols of these strings are already stored in the dictionary 1. We exploit this fact to achieve a memory efficient implementation of the PDLZW encoding. This recursive PDLZW encoding is illustrated in the right diagram of Fig. 3. As illustrated in this figure, the memory size for each dictionary for strings of length can be reduced by storing the address of the sub-string from the previous dictionary instead of the sub-string. The memory reduction increases for higher memory numbers because longer sub-strings are represented by addresses. Furthermore, the number of dictionary entries is reduced for longer strings. Hence, fewer bits are required to store the address. The recursive PDLZW encoding is similar to the PDLZW encoding in Algorithm 1 (Fig. 1). The only difference is that the string s is not required, i.e. the string is represented by the last found address in the variable old and the newly received symbol c. Consequently, the string is replaced by the tuple s = [old|c]. For example, the first entries of the dictionaries 1–3 in Fig. 2 will be [0 × 41|r], [0 × 03|s], and [0 × 00|e] instead of the strings ‘Ar’, ‘ees’, and ‘eese’. 2.3 PDLZW decoding The PDLZW decoding is described in Algorithm 2 (see Fig. 4). The decoding process is less complex than the encoding because no search is required. The received codewords contain the dictionary number l and the addresses c of the dictionary entries. Hence, the dictionaries can be implemented using RAM. The integers old and new denote the last and the currently received encoded codewords, respectively. The decoded symbol sequences are stored in the sting s. Fig. 4Open in figure viewerPowerPoint Algorithm 2: PDLZW decoding Note that Algorithm 2 (Fig. 4) describes the decoding of the PDLZW for recursive and non-recursive encoding because both implementations of the PDLZW encoding result in the same codewords. Similarly, we can use a recursive or non-recursive representation of the dictionaries for the decoding. The recursive representation is more memory efficient but requires several read operations for decoding a complete string, because exactly one symbol is decoded per clock cycle. Whereas the non-recursive implementation stores the complete strings in a single RAM, which requires more memory, but provides a faster decoding operation. 3 Address space partitioning The partitioning of the LZW encoding dictionary into multiple dictionaries with the PDLZW algorithm results in a reduced compression gain compared with the original LZW algorithm. This results from the fact that a sub-dictionary of the PDLZW algorithm may be completely filled, whereas other sub-dictionaries still have free entries. In this case, no strings can be added to a filled sub-dictionary, while the original LZW algorithm could store such strings until the complete dictionary is filled. Minimising the compression loss with the PDLZW algorithm requires a suitable partitioning of the address space. In this section, we model statistical dependencies in the input data using Markov chain [28] to drive an address space partitioning technique that optimises the compression rate of the PDLZW. 3.1 Data model The partitioning of the address space depends on the data. Hence, a suitable data model is required to derive a proper partitioning. We use Markov chains to model statistical dependencies in the data [28]. A similar assumption, that there is an underlying Markov model for the data, is made in the original work of Ziv and Lempel [11]. The basic idea of the proposed partitioning method is to use the stationary distribution of the Markov model for the LZW algorithm to derive the address space partitioning for the PDLZW algorithm. We use a Markov chain as depicted in Fig. 5 to model the LZW encoding process. Each state in the Markov chain represents a state of the encoder, i.e. the encoder is in the state i if the current input sequence of length is found in the dictionary. Furthermore, each state supplies a transition probability for the next symbol sequence being found in the dictionary. Fig. 5Open in figure viewerPowerPoint Illustration of a Markov chain for the PDLZW encoding Let be the probability that the Markov chain is in the state i. The stationary distribution is a probability distribution that remains unchanged in the Markov chain as time progresses. Obviously, the probability distribution varies as long as new entries are added to the LZW dictionary. However, once the dictionary is filled, we may assume that the distribution is stationary. Moreover, the probability is proportional to the number of strings of length in the LZW dictionary. Based on the Markov chain model, we can calculate the probabilities for each state with the recursion (1) Assuming a homogeneous Markov chain with , we obtain (2) where can be determined using the normalisation of the probability distribution and the geometric series for (3) resulting in (4) Hence, the homogeneous Markov model results in exponentially decreasing state probabilities, where the stationary distribution is determined by the transition probability q and the number of dictionaries n. To demonstrate that this model can characterise the LZW state distribution for real-world data, we consider data from the Calgary and Canterbury corpora [22, 23]. Fig. 6 illustrates an example of the probability distribution for the complete Calgary corpus. Here an address space with 1024 entries was considered. The probability for the first state (dictionary 0) is not depicted, because the first dictionary has a fixed length of 256 entries and does not require partitioning. The probability distribution is obtained by the following procedure. First, the data is encoded with the LZW algorithm until the dictionary is filled. Then further data is encoded, and the state probability is estimated by the relative frequency of the corresponding encoder state. Fig. 6 depicts the logarithm of the state probabilities (solid line) and a regression line (dashed) for (5) These curves indicate that the exponentially decreasing state probabilities of the homogeneous Markov model approximate the actual distribution, where can be determined from the slope of the regression line. Fig. 6Open in figure viewerPowerPoint Stationary probability distribution for Calgary corpus for an address space with 1024 entries 3.2 Partitioning the PDLZW address space Next, we introduce a new address space partitioning technique for the PDLZW algorithm. With a proper partitioning of the address space, the state probabilities for the LZW should be equal to the probabilities of finding a string in the dictionary i for the PDLZW. Hence, we can use the state distribution to estimate the number of entries in the PDLZW dictionaries. Let M denote the total number of entries in the dictionaries, i.e. M is the difference between the total size of the address space and the size of the virtual dictionary. Typically 256 addresses are reserved for the virtual dictionary. With an address space of 1024, only entries can be stored in the dictionaries. We propose to allocate approximately entries to the dictionary i, because is the expected value of strings of length in the LZW dictionary. Table 1 contains a few examples for the resulting partitioning, where the transition probabilities are obtained by regression for . The file book1 from the Calgary corpus results in the smallest transition probability, where dictionaries would suffice. On the other hand, the file Kennedy has a large transition probability which results in longer strings. Table 1. Dictionary sizes determination using the probability distribution Data q Dictionary sizes (1–9) book1 0.167 640 107 18 3 0 0 0 0 0 0 Calgary 0.379 477 181 68 26 10 4 1 1 0 0 kennedy 0.744 212 157 117 87 65 48 36 27 20 Fig. 7Open in figure viewerPowerPoint Word partitioning Obviously, this partitioning does not result in a universal solution, because the partitioning depends heavily on the transition probability. However, this dependency can be partially resolved by a simple, practical constraint. The PDLZW algorithm is predominantly used for hardware implementations. Consequently the dictionary sizes are typically chosen has powers of two to support RAM-based implementations. Restricting the dictionary sizes to powers of two also limits the number of possible address space segmentations. Table 2 presents the proposals for the dictionary partitioning for different address spaces. Due to the rounding of to powers of two, a large range of different transitions probabilities results in the same dictionary partitioning. The partitioning for 512 addresses is suitable for all files in the Calgary and Canterbury corpora. For an address space with 1024 addresses, we propose two segmentations depending on q, where the partitioning for provides good compression gains for both corpora. Table 2. Proposed dictionary partitioning for different address spaces Address space Dictionary partitioning (1–12) 512 128 64 32 16 8 4 4 0 0 0 0 0 1024 for 512 128 64 32 16 8 4 4 0 0 0 0 1024 for 256 256 128 64 32 16 8 4 4 0 0 0 2048 512 512 256 128 128 64 64 32 32 32 16 16 Table 2 is limited to address spaces up to 2048 addresses. Such addresses are suitable for applications in flash storage systems, where today data blocks of 1, 2, or 4 KB are processed. To determine which address space is best suited for an input block length, the input blocks are compressed using the LZW algorithm. Table 3 lists the mean block length of compressed data using the LZW algorithm depending on the size of the address space. The columns of this table demonstrate different input block lengths up to 8 KB. In each column, the smallest mean block length is marked in boldface, which indicates the best address space for the considered input block length. As shown in this table, the address spaces with 512, 1024, or 2048 entries are suitable for block lengths up to 8 KB. Table 3. Mean compressed block length of LZW for different block lengths Corpus Address space Block length 1024 2048 4096 8192 Calgary 512 638.3 1270.0 2554.7 — Calgary 1024 649.3 1179.3 2295.2 4593.9 Calgary 2048 714.2 1268.7 2262.9 4318.7 Calgary 4096 — — 2462.3 4394.0 Calgary 8192 — — — 4758.8 Canterbury 512 446.4 871.6 1831.5 — Canterbury 1024 470.3 841.4 1599.9 3331.4 Canterbury 2048 517.3 917.2 1648.3 3097.3 Canterbury 4096 — — 1798.2 3259.9 Canterbury 8192 — — — 3531.4 4 Hardware architecture To simplify the search, a PDLZW implementation based on CAMs was proposed in [24]. However, the proposed CAM implementation cannot be realised with RAM due to the large address space. The resulting CAM modules must be synthesised with circuits based on a large amount of D-type flip-flops. In this section, we introduce an efficient CAM-based hardware implementation for the recursive PDLZW algorithm. In [26, 27], the concept of word partitioning was proposed to enable RAM-based implementations of CAMs with larger addresses. We apply this concept to the dictionaries of the PDLZW. In particular, we present a word partitioning that significantly reduces the memory size of the individual dictionaries such that RAM-based implementations become feasible. However, we consider only four dictionaries, because this is the most compact and the simplest implementation. Furthermore, it allows for a comparison with the results from [21]. 4.1 Basic concept Fig. 7 illustrates the basic concept of word partitioning. The input address of C bits width is divided into two sub-addresses. Each sub-address is processed independently in a dedicated layer. Each layer is a small CAM that is used to reduce the length of the address, i.e. we have and . Finally, these bits are concatenated to form the CAM address. If an incoming address appears for the first time in a layer, it is stored in the next free position. The output of the layer is the index of the entry where the input address is found. However, the number of entries in the layers is limited to and , i.e. the first and addresses, respectively. In the case, when all entries are occupied, and a new address appears, a miss occurs, and the address cannot be stored. Note that the miss rate depends on the size of the layer and the statistics of the input addresses. Using the word partitioning for the PDLZW dictionaries may result in a higher miss rate for dictionary entries which could affect the achievable compression gain. However, a low miss rate can be achieved with a proper dimensioning of the layers. 4.2 Dimensioning the layers The miss rate of a layer depends on the statistics of the addresses. To estimate this, we consider again real-world data from the Calgary and Canterbury corpora [22, 23]. Based on the probability of occurrence for the addresses, a suitable word partitioning for the individual dictionaries can be defined. Fig. 8 shows the address distribution for the PDLZW for all the Calgary test files. For this data, we used a PDLZW with four dictionaries. The data was processed in blocks of 1 KB such that each block can be decoded independently. This results in a total of 3190 blocks for the complete Calgary corpus. Fig. 8Open in figure viewerPowerPoint Distribution of the number of unique entries for the different dictionaries Fig. 8 depicts the number of unique address occurrences per block. With four dictionaries, an address space of 1024 addresses and without word partitioning technique, each dictionary contains at most 256 entries. However, not all addresses occur within each encoded block. As can be seen in this figure, the actual number of unique entries is typically much smaller than the maximum value. For instance, consider the virtual dictionary. For this dictionary, an address value is equal to the 8-bit input data. Hence, the address statistics is determined by the input data. As can be seen in Fig. 8, there are data blocks, where more than 200 unique data symbols occur. However, in the great majority of all cases, less than 128 unique symbols occur. This observation can be used to reduce the address space for dictionary 1. The input address of dictionary 1 consists of two entries from the virtual dictionary. Each of these entries is represented by 8 bits. Now consider the structure in Fig. 7 with layers that reduce the output addresses of the virtual dictionary from 8 to 7 bits. These two layers reduce the address space of dictionary 1 from 16 to 14 bits. With a RAM-based CAM implementation, this word partitioning reduces the memory size for dictionary 1 by a factor of four, i.e. from to words. For most of the data blocks in the Calgary corpus, this word partitioning does not affect the compression gain, because the layers are sufficiently large to represent all possible addresses. However, there are some data blocks, where more than 128 unique addresses occur. In such a case, the layers can only represent the first 128 occurrences. An address that is not found in the layer cases a miss. The corresponding symbol will be encoded with a codeword from the virtual dictionary, and no data is lost. But the corresponding layer cannot provide a valid output address. Thus, this symbol cannot be stored in dictionary 1. If this symbol occurs more often, it will always be encoded with a codeword from the virtual dictionary. Consequently, the misses caused by the layers can l