A single‐phase algorithm for mining high utility itemsets using compressed tree structures

ETRI JournalVolume 43, Issue 6 p. 1024-1037 ORIGINAL ARTICLEFree Access A single-phase algorithm for mining high utility itemsets using compressed tree structures Anup Bhat B, Anup Bhat B orcid.org/0000-0002-9910-6758 Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaSearch for more papers by this authorHarish SV, Corresponding Author Harish SV [email protected] Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India Correspondence Harish SV, Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India. Email: [email protected]Search for more papers by this authorGeetha M, Geetha M Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaSearch for more papers by this author Anup Bhat B, Anup Bhat B orcid.org/0000-0002-9910-6758 Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaSearch for more papers by this authorHarish SV, Corresponding Author Harish SV [email protected] Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India Correspondence Harish SV, Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India. Email: [email protected]Search for more papers by this authorGeetha M, Geetha M Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaSearch for more papers by this author First published: 12 November 2021 https://doi.org/10.4218/etrij.2020-0300AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Abstract Mining high utility itemsets (HUIs) from transaction databases considers such factors as the unit profit and quantity of purchased items. Two-phase tree-based algorithms transform a database into compressed tree structures and generate candidate patterns through a recursive pattern-growth procedure. This procedure requires a lot of memory and time to construct conditional pattern trees. To address this issue, this study employs two compressed tree structures, namely, Utility Count Tree and String Utility Tree, to enumerate valid patterns and thus promote fast utility computation. Furthermore, the study presents an algorithm called single-phase utility computation (SPUC) that leverages these two tree structures to mine HUIs in a single phase by incorporating novel pruning strategies. Experiments conducted on both real and synthetic datasets demonstrate the superior performance of SPUC compared with IHUP, UP-Growth, and UP-Growth+ algorithms. 1 INTRODUCTION Transactional databases of supermarket stores serve as a primary source for mining interesting patterns that indicate associations between different items purchased by customers. This mining task is termed as association rule mining (ARM) and used in many applications across various domains, including text mining [1], bioinformatics [2], and pharmacovigilance [3]. To obtain association rules from a huge number of customer transactions, items that co-occur frequently are enumerated to assert the validity and interestingness of associations. Hence, mining such frequent itemsets forms the core phase of the ARM task, which has been extensively researched [4]. More often than not, frequently purchased items do not necessarily contribute adequately to the revenue of a supermarket store. This is due to the inherent nature of the frequent itemset mining (FIM) model that relies only on the presence or absence of an item in a transaction when determining the frequency of the item. Formally, a measure called support is calculated as the ratio of the number of transactions, in which the items of an itemset co-occur, to the total number of transactions to decide whether the itemset is frequent or not based on a threshold set by the user. This measure does not rely on the quantity or unit profit of the items that are essential in determining the revenue incurred. Hence, a framework called the high utility itemset mining (HUIM) has evolved to consider such factors, which can be seen as a generalized form of the FIM task. The utility of an itemset is often measured as the product of the quantity and unit profit of the items that form the itemset. While the algorithms that mine frequent items exploit the downward closure property with respect to the support for effective search space exploration, the measure of utility is not downward closed. This void is filled by various upper bounds on the utility, with the transaction-weighted utility (TWU) being a prominent example. The algorithms for mining HUIs have evolved from two-phase to single-phase. The most efficient two-phase algorithms such as IHUP, UP-Growth, and UP-Growth+ construct tree structures and enumerate candidate patterns via a recursive pattern-growth procedure. This bottom-up procedure constructs conditional pattern trees that consume a lot of memory. This is a severe performance bottleneck. Furthermore, once candidates are outputted from such a procedure, the utility has to be determined from an additional database scan. This paper presents a single-phase algorithm called single-phase utility computation (SPUC) to efficiently utilize the concept of tree structures in mining HUIs. SPUC has the following advantages: It utilizes two new tree structures, namely, Utility Count Tree (UCT) and String Utility Tree (SUT). Both trees are constructed from a single database scan and are complete. While UCT guides the search space exploration for enumerating valid patterns, SUT is a compact tree that provides the utility of these patterns. This completely eliminates the need for rescanning the database to calculate the utility. It executes faster than IHUP, UP-Growth, and UP-Growth+ tree-based algorithms according to experiments on real and synthetic datasets. The rest of this paper is organized as follows. Section 2 provides a formal introduction to the problem of mining HUIs, along with related work. The procedures of constructing the proposed trees, UCT and SUT, and proposed algorithm incorporating novel pruning strategies are outlined in Section 3. The results of performance evaluation are reported in Section 4. Section 5 concludes the paper. 2 BACKGROUND Preliminaries Given a transaction database with distinct items , each transaction in is identified by the transaction identifier and records a collection of items purchased, along with their quantities or internal utility (Table 1B). The ordered pair in each transaction indicates that an item was purchased in a quantity of in that transaction. Each item is also associated with the unit profit or external utility (Table 1A). TABLE 1. Sample database (A) Profit Table Item 1 2 3 4 5 6 7 Profit 5 2 1 2 3 5 1 (B) Transaction Table TID Transaction {(3,1)(5,1)(1,1)(2,5)(4,3)(6,1)} {(3,3)(5,1)(2,4)(4,3)} {(3,1)(1,1)(4,1)} {(3,6)(5,2)(1,2)(7,5)} {(3,2)(5,1)(2,2)(7,2)} Definition 1.(Utility of an item) The utility of an item in a transaction is measured as the product of the quantity and unit profit . Definition 2.(Utility of an itemset) The utility of an itemset in a transaction is defined as Definition 3.(Utility of an itemset in a database). The utility of an itemset in is defined as for example, The utility measure is neither anti-monotone nor monotone. The utility of a few subsets of is compared in Table 2. While the support is strictly increasing across the subsets, the utility is neither increasing nor decreasing. Hence, while the support of an itemset is downward closed, the utility measure is not. TABLE 2. Support vs Utility Itemset Support Utility Definition 4.(Transaction Utility (TU)) The TU of a transaction is defined as the sum of the utilities of all the items in that transaction, that is, Definition 5.(High Utility Itemset (HUI)) An itemset is called a HUI if , where is the minimum utility provided by the user. For instance, if the threshold is set to , then , where is the sum of the TU of all the transactions in the sample database. Then, HUIs for this threshold are , , and , with utilities of 36, 37, and 40, respectively. Definition 6.(Transaction Weighted Utility (TWU)). The TWU of an itemset is defined as the sum of the transaction utility of all the transactions in that contain , that is, . Definition 7.(High-transaction Weighted Utility Itemset (HTWUI)). An itemset is a HTWUI, if . If an itemset is not a HTWUI, then it cannot be a HUI. Property 1.(TWU Downward Closure Property). If an itemset is a HTWUI, then all its subsets are HTWUIs, or if an itemset is not a HTWUI, then none of its supersets can be a HTWUI. For instance, . Hence, higher-order itemsets need not be enumerated from as this property ensures they are neither HTWUI nor HUI. Related work Algorithms for mining frequent itemsets explore the combinatorial search space by employing the downward closure property with respect to the support of an itemset. However, the measure of utility is not downward closed (Definition 3). Yao et al. [5, 6] performed a theoretical analysis of mining HUIs for the first time. In this study, the authors proposed two properties, namely, utility bound property and its extension, support bound property, as heuristic for pruning the search space. While this work formalized the problem of HUIM, the proposed heuristic could not discover a complete set of HUIs. The property of the TWU to be downward closed was proposed by Liu et al. [7-9]. The two-phase algorithm proposed by these authors employed the TWU to enumerate candidate patterns in a level-wise manner analogous to the Apriori algorithm [10]. The candidate generation phase of this algorithms outputs all the -itemsets that are HTWUIs and whose utility calculation is performed by scanning the database again, which is the second phase of the algorithm. Overall, scans are required to output HUIs of length . Another category of algorithms resulting in significant performance gains are tree-based algorithms [11-15]. These algorithms transform the database into a compressed tree structure with at most two database scans. Items that do not have the TWU of at least are discarded, while the remaining items of the transactions are arranged in a predetermined order (for example, descending or ascending) of the item TWU or the lexicographic order. The tree construction is similar to the FP-tree construction procedure [16]. In particular, the TWU [12] or TU [13] is stored instead of storing the utility of items in the node of the tree. To promote access to the nodes of the tree that carry similar items in different branches, a header table is also constructed. This table is scanned from the bottom to obtain conditional pattern trees. Each item in the tree is then appended to the item whose conditional tree is constructed and output as a candidate. Pruning strategies play a vital role in reducing the number of candidates. In this regard, the UP-Growth algorithm [14] discards the items that are unpromising in local conditional trees (the DLU strategy) and decreases the utilities of the remaining items by the utility of the discarded items (the DLN strategy). These strategies are further tightened by using minimal node utilities in the UP-Growth+ algorithm [15]. List-based algorithms such as HUI-Miner [17] and FHM eliminate candidate generation entirely. They adopt the depth-first search strategy and its valid extensions in a single phase to mine patterns by transforming the database information into a utility list (UL) structure. Initially, a UL is constructed for every item whose TWU is at least . Based on a predetermined order, items are extended recursively. Utility information remains intact in the list and is accumulated as the lists are combined during the extension of the item. Hence, unlike the tree-based algorithms, the second phase of rescanning the database for utility calculation is not required. The pruning strategy aims mainly at determining the valid extensions of an itemset. In this regard, U-Prune proposed for HUI-Miner was extended in HUP-Miner [19] by adopting two more pruning strategies, namely, PU-Prune and LA-Prune, that use the utility information stored in partitioned ULs. The FHM algorithm [18] uses the estimated utility co-occurrence structure to store the TWU of two itemsets for faster lookup. Projection-based algorithms have proven to be more efficient than list-based algorithms [20]. EFIM [21] and d2hup [22, 23] are examples of projection-based algorithms. EFIM uses local tree and sub-tree utility pruning strategies in conjunction with a database projection technique for faster exploration of the search space, while d2hup explores search space via a reverse set enumeration tree and makes use of the chain of accurate utility lists for utility computation. Using real and synthetic datasets, the empirical study conducted by Zhang et al. [20] demonstrated the superior performance of EFIM and d2hup on dense and sparse datasets, respectively. However, a recent study modeling the utility measure via subadditivity and monotonicity has revealed that neither list- nor projection-based algorithms perform the best at all times [24]. Pattern-growth tree-based algorithms for HUIM construct conditional pattern trees during recursive enumeration of itemsets. This procedure requires a significant amount of memory, especially for dense datasets. While list structures offer tighter upper bounds on the utility to facilitate effective pruning, the construction of ULs involves significant comparison overhead. Further, when an itemset is extended with any of its valid extensions, it is possible that they do not co-occur in any transactions, thus waste CPU cycles for comparison operations. This can be avoided by employing a prefix tree with bottom-up traversal. Enumerated itemsets are guaranteed to be valid patterns. Nevertheless, candidates are outputted only toward the end of the first phase. Revisiting the transaction database for the utility calculation incurs a significant input/output cost. Several studies highlight the significance of the compact representation of transaction databases for FIM [25, 26]. Hence, this study aims to compress the entire database on a per transaction basis and leverage the advantages of the prefix tree for mining HUIs efficiently. While the prefix tree provides with valid patterns, the compressed tree structure with utility information can be accessed to output the HUIs without rescanning the database, that is, ensuring the utility computation in a single phase. 3 METHODOLOGY A majority of tree-based algorithms eliminate unpromising items during the initial tree construction and require two scans of the database. This section presents two tree structures (UCT and SUT) that are constructed using a single scan by incorporating all items. While both structures ensure prefix sharing, UCT is constructed on per item basis, whereas SUT is constructed on per transaction basis. Subsection 3.1 introduces the tree structures and their respective construction procedures. Subsection 3.2 details the mining procedure using these trees, along with the pruning strategy and SPUC algorithm. Proposed data structures 3.1.1 Utility count tree A node in the UCT has the following fields: item denoting the name of an item; count denoting the count of an item in the given path of the tree; nodeUtility denoting the accumulated utility of an item in the given path of the tree; parent pointing to the parent of the node. Utility Count Tree is constructed without discarding any items during the initial tree construction. The database is scanned, and a node is constructed for every item in a transaction . Algorithm 1 outlines the procedure of inserting transactions into UCT. Initially, is set to the root node of the tree. Items in a transaction are arranged in ascending order and inserted as child nodes of one another. Hence, each path of the tree corresponds to a particular transaction. If a transaction contains a node that is already present in the tree, the procedure updates the count and utility instead of creating a new node in the given path. This ensures prefix sharing. Figure 1 shows UCT for the sample database presented in Table 1. FIGURE 1Open in figure viewerPowerPoint Utility Count Tree (UCT) for the database presented in Table 1 3.1.2 String utility tree Unlike UCT, SUT captures transaction-level information in a node, resulting in a more compact representation of a transaction database compared with FP-tree-like structures. A node in SUT has the following fields: stringItems denoting the concatenation of items purchased in a transaction; TU denoting the TU of a transaction; stringUtilities denoting the concatenation of utilities of corresponding items; parent pointing to the parent of the node. Algorithm 2 outlines the construction procedure of SUT. Once the items of a transaction are arranged in ascending order, they are concatenated using a delimiter such as x and stored in the stringItems field. The utilities are indexed per the order of the items and concatenated in a similar manner. As each node corresponds to a transaction, the tree offers a compact representation without eliminating any items. To ensure prefix sharing, substring comparison is performed to check whether the stringItems of a transaction is present in existing nodes of the tree (line 11). If there is a match, then the new node is appended as the child of this existing node. Figure 2 shows SUT for the sample database presented in Table 1. FIGURE 2Open in figure viewerPowerPoint String Utility Tree (SUT) for the sample database presented in Table 1 Proposed SPUC algorithm A path for a node in UCT is a list containing items from this node to the root. The nodeUtility field of a node stores the cumulated utility value of the item the node represents. This value is the sum of the utilities of the item in different transactions sharing a common prefix. From the header list, if the node link for an item is traversed, then the sum of the nodeUtility fields denotes the utility of the item in the transaction database. With this item as suffix, each path of the node is the prefix path for this item. To mine UCT, the header list is traversed from the bottom. The prefix path(s) for each item is (are) obtained, which forms the conditional pattern base (CPB) for the item. The sum of nodeUtility fields of all items, including the suffix, is also calculated for each prefix path, which denotes the path utility. All possible subsets from prefix paths containing the suffix item are then generated. The rationale here is that these items represent the database conditioned on the suffix item. Hence, it is sufficient to mine only the -itemsets generated from these items. The value of ranges from one to the length of the considered prefix path. Once the different subsets of a prefix path in the CPB of an item are obtained, the actual utility value of them cannot be obtained from UCT. Hence, SUT is employed in mining. The utility of each itemset is accumulated using level-order traversal. As each node in SUT represents a transaction, the accumulated utility value for an itemset post the traversal denotes its real utility value. The following strategy is adopted to enable efficient search of an itemset in SUT. First, a check for the presence of the suffix item in the current node is performed instead of the entire itemset. The node is examined for the presence of the entire itemset only if it contains the suffix item. Subsequently, the current node's children are also examined. If the node does not contain the suffix item, the next sibling is considered without examining the contents of the current node or any of its children. This procedure exploits the characteristic of SUT and knowledge of the suffix item being considered. If a node does not contain the suffix item, then none of its children will contain this item since the children of a node in SUT are substrings completely contained in its parent. Furthermore, all subsets are formed conditioned on this suffix item. Hence, it is sufficient to check for the suffix before examining for the presence of the entire itemset. The following pruning strategy has been proposed by us to improve the efficiency of the mining process: Theorem 1.If the sum of path utilities for an item is lower than , then no itemset from the CPB will be a HUI, and hence, these itemsets need not be generated and evaluated. Case 1 (Isolated path): An isolated path is a non-prefix sharing path in a tree. Hence, for all items is the utility value. For any suffix item in this path, the path utility is the utility of the largest -itemset possible. For any subset of this -itemset that contains as the suffix, the utility cannot exceed the utility of the largest -itemset, that is, the path utility. Hence, if this utility is lower than , the utility of the subsets cannot exceed . Therefore, the subsets for the paths in the CPB do not need to be generated with respect to the suffix. Case 2 (Non-isolated path): In the case of prefix sharing, has the utility value greater than the real utility value, that is, it denotes the sum of the utilities of different transactions along the path for an item. Thus, any subset formed for the suffix item will have a utility lower than the path utility. If two or more prefix paths are present for and a common subset exists, the sum of the utilities of subsets will always be lower than the sum of the path utilities (since individually, every utility is lower than the path utility of the path they occur in). An example of mining employing UCT, SUT, and the proposed pruning strategy is described below. Consider mining HUIs for itemsets with a suffix item . As the header list is traversed, two prefix paths are obtained, namely, and , with path utilities and , respectively (Figure 1). This forms the CPB for item as shown in Table 3. If is set to 30, then item is not pruned as the sum of the path utilities ( ) exceeds . Subsets with suffix can be generated using these prefix paths. Since both paths have a length of 4, the largest possible subset is of length four. When the prefix path is considered from the CPB, the generated subsets are . Next, SUT is traversed to obtain the utilities of these itemsets. The traversal of SUT begins from its node. The first match to the suffix occurs at node . The utilities of all itemsets are obtained from the field of this node and are recorded in the hash table. A snapshot of this table is shown in Table 4. The next sibling found during traversal, that is, , contains the suffix. However, only the itemsets are examined as present in this node, and their utilities are updated in the hash table as shown in Table 5. Once the traversal is complete, the next prefix path, , is selected from the CPB. The subsets generated from this path are . Since the itemsets have already been generated in and examined, only the remaining itemsets, that is, are taken up for mining. The utilities of these itemsets are recorded in as shown in Table 6. The above procedure is repeated for all items of the header list in a bottom-up manner. Eventually, is filtered to retain only those itemsets whose utility is at least . TABLE 3. Conditional pattern base for item Prefix Path Items Path Utility TABLE 4. Utilities of subsets of in after examining the node Itemset Utility 5 15 11 11 21 21 17 27 TABLE 5. Utilities of subsets of in after examining the node Itemset Utility 7 15 15 16 21 21 24 27 TABLE 6. Utilities of subsets of in after examining the node Itemset Utility 7 15 15 16 21 21 24 27 6 8 9 11 3.2.1 Enhancing the mining process As demonstrated in the above example, there is a possibility that the prefix paths within a CPB can generate the same subsets. While the same subsets are not considered during SUT traversal for the utility calculation, they are still generated. To tackle this problem, the CPB is checked for the presence of a path that contains all the different items in the CPB. Such a path has to be the longest path, with the length equal to the suffix item being considered. Such a path will generate all possible subsets, including those generated by the remaining prefix paths in the CPB. Consider the suffix item . The possible prefix paths are , , , and . Here, the longest prefix path with a length equal to the suffix item is . The subsets generated out of include those generated by the paths from to . Hence, it is sufficient to generate subsets from for subsequent mining. The remaining paths can be ignored when generating subsets. Once the subsets corresponding to a prefix path are generated, their utilities can be obtained by traversing SUT. The value stored in UCT is employed to reduce the number of the subsets being evaluated for the utility computation. The values corresponding to every item in the CPB are accumulated as the prefix paths are determined. This results in an overestimated utility value that can be employed to filter itemsets as explained below using the subsets of of . An overestimated utility (OU) table for accumulates the values corresponding to every item in . The OU values are accumulated as and when items are added to the prefix path of the CPB. Table 7 shows the OU values for items that form . Items , are present only in and ; their OUs hold the corresponding values of 20 and 12, respectively. However, the OUs of the other items in both and are also accumulated, that is, . Next, OUs of different subsets formed from are determined using the above-mentioned OU table (Table 8). It can be observed from Table 8 that only the itemsets , , and have their OUs above the of 30. Hence, only the utilities of these itemsets are computed using SUT (which is different from Tables 4 and 5). TABLE 7. Overestimated utility table corresponding to items of Itemset OU 20 12 12 9 7 TABLE 8. Overestimated utility table of subsets in of Itemset OU 7 27 19 16 39 36 28 48 This strategy further enhances the mining process in conjunction with the pruning strategy provided in Theorem 1. Algorithm 3 lists the SPUC algorithm that incorporates the proposed pruning strategy. SPUC takes two trees (UCT and SUT) and the values provided by the user as inputs. A global hash map acts as a table for storing the utilities of subsets generated from different prefix paths as shown in line 1. A bottom-up procedure is then initiated to obtain the CPB of items for subsequent mining (lines 2 to 18). After calculating the path utilities for each prefix path, the pruning strategy (Theorem 3.2) is applied (line 4). The mining procedure considers the next item in the header list if the sum of the path utilities is not at least . For items that satisfy this condition, the longest path is searched to avoid generating subsets from each path in the CPB (lines 5 to 7). Itemsets that are subsets of each prefix path and contain only the suffix item are then generated (line 9). keeps track of different itemsets generated from the prefix paths considered till the current one and thus can be used to eliminate any subsets common to any of the previous paths (lines 10 and 11). As the OUs of the itemsets are determined, only those itemsets whose OU exceeds the threshold are retained in (line 12 to 16). The procedure Mine takes the filtered as input and starts searching for the presence of itemsets from the of SUT (line 23 to 32). Line 24 is the summary of the search strategy explained before. If the suffix is absent in the current SUT node, then none of the itemsets in will be present in the current node. Hence, the search proceeds with the next sibling after the break (line 34). The utility of the itemset is calculated and added to if not present already (lines 26 to 30). Traversal is continued as shown in lines 38 to 40 to mine itemsets from the child of the current node or move to siblings if the suffix item is not present. Complexity analysis The longest path in the CPB of an item is identified using the assumption that its length matches the location of the item in the header list of UCT. If such a path exists, then the number of subsets with this item as prefix is (assuming that the sum of the path