An algebraic semigroup method for discovering maximal frequent itemsets

Abstract Discovering maximal frequent itemsets is an important issue and key technique in many data mining problems such as association rule mining. In the literature, generating maximal frequent itemsets proves either to be NP-hard or to have <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>O</m:mi> <m:mrow> <m:mo>(</m:mo> <m:mrow> <m:msup> <m:mrow> <m:mi>l</m:mi> </m:mrow> <m:mrow> <m:mn>3</m:mn> </m:mrow> </m:msup> <m:msup> <m:mrow> <m:mn>4</m:mn> </m:mrow> <m:mrow> <m:mi>l</m:mi> </m:mrow> </m:msup> <m:mrow> <m:mo>(</m:mo> <m:mrow> <m:mi>m</m:mi> <m:mo>+</m:mo> <m:mi>n</m:mi> </m:mrow> <m:mo>)</m:mo> </m:mrow> </m:mrow> <m:mo>)</m:mo> </m:mrow> </m:math> O\left({l}^{3}{4}^{l}\left(m+n)) complexity in the worst case from the perspective of generating maximal complete bipartite graphs of a bipartite graph, where <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>m</m:mi> </m:math> m , <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>n</m:mi> </m:math> n are the item number and the transaction number, respectively, and <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>l</m:mi> </m:math> l denotes the maximum of <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mo>∣</m:mo> <m:mi>C</m:mi> <m:mo>∣</m:mo> <m:mo>∣</m:mo> <m:mi mathvariant="normal">Ψ</m:mi> <m:mrow> <m:mo>(</m:mo> <m:mrow> <m:mi>C</m:mi> </m:mrow> <m:mo>)</m:mo> </m:mrow> <m:mo>∣</m:mo> <m:mspace width="0.1em" /> <m:mtext>/</m:mtext> <m:mspace width="0.1em" /> <m:mrow> <m:mo>(</m:mo> <m:mrow> <m:mo>∣</m:mo> <m:mi>C</m:mi> <m:mo>∣</m:mo> <m:mo>+</m:mo> <m:mo>∣</m:mo> <m:mi mathvariant="normal">Ψ</m:mi> <m:mrow> <m:mo>(</m:mo> <m:mrow> <m:mi>C</m:mi> </m:mrow> <m:mo>)</m:mo> </m:mrow> <m:mo>∣</m:mo> <m:mo>−</m:mo> <m:mn>1</m:mn> </m:mrow> <m:mo>)</m:mo> </m:mrow> </m:math> | C| | \Psi \left(C)| \hspace{0.1em}\text{/}\hspace{0.1em}\left(| C| +| \Psi \left(C)| -1) , with the maximum taken over all maximal frequent itemsets <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>C</m:mi> </m:math> C . In this article, we put forward a method for discovering maximal frequent itemsets, whose complexity is <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>O</m:mi> <m:mrow> <m:mo>(</m:mo> <m:mrow> <m:mn>3</m:mn> <m:mi>m</m:mi> <m:mi>n</m:mi> <m:msup> <m:mrow> <m:mn>2</m:mn> </m:mrow> <m:mrow> <m:mi>β</m:mi> </m:mrow> </m:msup> <m:mo>+</m:mo> <m:msup> <m:mrow> <m:mn>4</m:mn> </m:mrow> <m:mrow> <m:mi>β</m:mi> </m:mrow> </m:msup> <m:mi>n</m:mi> </m:mrow> <m:mo>)</m:mo> </m:mrow> </m:math> O\left(3mn{2}^{\beta }+{4}^{\beta }n) , lower than the known complexity both in the worst case, from the perspective of semigroup algebra, where <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>β</m:mi> </m:math> \beta is the number of items whose support is more than the minimum support threshold. Experiments also show that an algorithm based on the algebraic method performs better than the other three well-known algorithms. Meanwhile, we explore some algebraic properties with respect to items and transactions, prove that the maximal frequent itemsets are exactly the simplified generators of frequent itemsets, give a necessary and sufficient condition for a maximal <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>i</m:mi> <m:mo>+</m:mo> <m:mn>1</m:mn> </m:math> i+1 -frequent itemset being a subset of a closed <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mi>i</m:mi> </m:math> i -frequent itemset, and provide a recurrence formula of maximal frequent itemsets.