The wide window string matching algorithm Longtao He, Binxing Fang, Jie Sui Theoretical Computer Science Volume: 332, Issue: 1-3, February 28, 2005, pp. 391-404 Professor R.C.T Lee Speaker K.W. Liu Department of Computer Science National Chi Nan University 1 String Matching Problem Given : a text string T = t1t2t3…tn . a pattern string P = p1p2p3…pm. where |P|≤|T|. Output: All occurrence(s) of the pattern string within the text string. Example T = ababababaabbaabbabababa P = aabbaabb T= a b a b a b a b a a b b a a a b b a b a b a b P= a a b b a a a b 2 Traditional Method Example T = ababababaabbaabbabababa P = aabbaabb a b a b a b a b a a b b a a a b b a b a b a b a a b b a a a b a a b b a a a b a a b b a a a b a a b b a a a b 3 In this talk, we shall provide three ideas: 1. The wide-window method 2. The convolution method 3. The bit pattern (modified convolution) method. 4 Basic Idea of the Wide Window Approach ?Open a window with size 2|P|-1. ?Divide it into two parts: ?The first one denoted as T1 is with size |P|-1 ?The second part denoted as T2 is with size |P| 2|P|-1 |P|-1 |P| T T1 T2 Since |T1|<|P| , some suffix of P must be in T2 if it exists. 5 ?Find all prefixes of T2 which are also suffixes of P. ?Let r denote the length of such a longest prefix. ?We can be sure that one part of T2 can be ignored as shown. 2|P|-1 |P|-1 |P| T r T1 T2 Can be ignored. 6 ?For every prefix of T2 which is a suffix of P, we should find whether there exists a suffix in T1 which is also a prefix of P. 2|P|-1 |P|-1 |P| T r T1 T2 7 Definition: n-suffix : Given a string S, n-suffix of S is the suffix of S whose length is n. -1< n < |S|+1 Example: S = abcde 0-suffix of S = ε 1-suffix of S = e 2-suffix of S = de 3-suffix of S = cde n-prefix : Given a string S, n-prefix of S is the prefix of S whose length is n. -1< n < |S|+1 Example: S = abcde 0-prefix of S = ε 1-prefix of S = a 2-prefix of S = ab 3-prefix of S = abc 8 An Example of the Wide Window Approach Given: T = aababcbdcea P = abcbd Let us produce a wide window whose length is |P| - 1 + |P| = 2|P| - 1 In this case, |P|=5 , 2|P| - 1 = 9 T =aababcbdcea |P|-1 T= |P| a a b a b c b d c e a T1 T2 9 We first find all prefixes of T2 which are equal to some suffixes of P. In this case, we obtain bcbd whose length is 4. |P|-4 = 5-4 = 1 If the 1-suffix of T1 is the 1-prefix of P, we have found a matching. 1-suffix of T1 = a 1-prefix of P = a ∴1-suffix of T1 = 1-prefix of P. Thus we conclude that a matching is found. T1 T= P= T2 a a b a b c b d c e a a b c b d 10 Another Example Given: T = ababa P = aba Let us produce a wide window whose length is |P| - 1 + |P| = 2|P| - 1 In this case, |P| = 3 , 2|P| - 1 = 5 T = ababa |P|-1 T= |P| a T1 b a b T2 a 11 We first find all prefixes of T2 which are equal to some suffixes of P. In our case, we obtain aba and a where lengths are 3 and 1. |P| - 3 = 3 - 3 = 0 (? the whole P is equal to T2 ? one matching is found ) |P| - 1 = 3 – 1 = 2 If the 2-suffix of T1 is the 1prefix of P, we have found a matching. 2-suffix of T1 = ab 2-prefix of P = ab ∴2-suffix of T1 = 2-prefix of P. Thus we conclude that two matchings are found. |P|-1 T= P= P= a b a b a |P| b a T1 a a T2 b a 12 Question: How can we find a suffix of a string S1 to be a prefix of S2? Answer : We use the convolution method. 13 Convolution Method T = aabc , P = ab P = ba a 1 0 1 a 1 1 2 0 0 b b 0 0 0 c a 0 0 T= P= T= P= T= P= T= P= a a b c a b 0 c b 0 c a a b a 0 b b 1 b a a a 1 a b 0 a a a 1 a b 0 c T= P= a b c 14 We may use the convolution method to find all prefixes of T2 which are equal to some suffixes of P. T2 = bcbdc , P = abcbd T 2 = cdbcb P = T2= a b c b d c d b c b 0 1 0 1 0 0 0 1 0 0 0 1 0 1 0 If any zero appears in the column, we can not get a matching. + 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 4 0 1 0 The unused region to find matching! A 4-suffix of P equal to a prefix of 152. T P a b c b d T2 b c b d c 0 P = T2 = a b c b d c d b c b 0 1 0 1 0 0 0 1 0 0 0 1 0 1 0 P a b c b d T2 b c b d c 0 0 0 P a b c b d T2 b c b d c 1 1 1 1 P a b c b T2 b c b d 0 0 0 0 d c 0 16 P a b c b d T2 b c b d c 1 0 + 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 4 0 1 0 May be ignored. No further sliding to the left is needed. We may also use the logic operator (AND &) to find all prefixes of T2 which are equal to some suffixes of P. T2 = bcbdc P = abcbd a b c b d c d b c b 0 1 0 1 0 0 0 1 0 0 0 1 0 1 0 0 1 0 1 0 0 1 0 0 0 1 0 0 1 P = T2 = + 0 0 0 0 1 0 0 1 0 0 0 0 1 1 0 4 0 1 0 & 0 0 1 0 0 0 The unused region to find matching! A 4-suffix of P equal to a 17 prefix of T2 We may use the convolution method to find all suffixes of T1 which are equal to some prefixes of P. T1 = aaba , P = abcbd P = dbcba P = d b c b a a a b a 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 T1 = If any zero appears in the column, we can not get a matching. + The unused region to find matching! 0 0 0 0 1 0 0 0 1 1 2 0 1 A 1-prefix of P equal to a suffix of T1. 18 P = T1 = d b c b a a a b a 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 P T1 P T1 P T1 P T1 a b c b d a a b a 1 a b c b d a a b a 0 0 a b c b d a a b a 1 1 0 a b c b d a a b a 1 0 0 0 + 0 0 0 0 1 0 0 0 1 1 2 0 1 May be ignored. No further sliding to the right is needed. 19 We may use the logic operator (AND &) to find all suffixes of T1 which are equal to some prefixes of P. T1 = aaba P = abcbd P = T1 = d b c b a a a b a 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0 1 0 0 1 + 0 0 0 0 1 0 0 0 1 1 2 0 1 & 1 0 0 0 1 The unused region to find matching! A 1-prefix of P equals to a suffix of T1. 20 ∴1-suffix of T1 = 1-prefix of P. Thus we conclude that a matching is found. The Bit Pattern Approach Let us consider the following case: T = bcbdc P = abcbd Our job is to determine whether there is a prefix in T which is a suffix of P. Indeed, in this case, we have 4-prefix of T (bcbd) which is also the 4-suffix of P. As indicated before , we may use convolution. 21 Convolution P= a c 0 0 AND OPERATION b d 1 1 1 1 1 c b 0 0 0 b c 1 0 d b 0 V1 V2 V3 V4 V5 T= 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 A 4-suffix of T is a 4-prefix of P. What are the vectors V1,V2,…,V5? 22 Given a string S = s1s2…sn and a character α, the α-bit pattern of S is defined as b1b2…bn where bi=1 if si = α and bi=0 if otherwise. Example: S = abcbd a-bit pattern of S = 1 0 0 0 0 b-bit pattern of S = 0 1 0 1 0 c-bit pattern of S = 0 0 1 0 0 d-bit pattern of S = 0 0 0 0 1 23 T=bcbdc ,P=abcbd P= T= a c 0 0 0 0 0 0 b d 1 1 1 1 1 c b 0 0 0 b c 1 0 d b 0 AND OPERATION 0 0 0 0 0 0 0 1 0 0 1 0 0 0 V1 V2 V3 V4 V5 0 0 0 We can now observe that 1. V1 = b-bit pattern of P as we are comparing T[1] = b with P, 2. V2 = c-bit pattern of P as we are comparing T[2] = c with P, 3. V3 = b-bit pattern of P as we are comparing T[3] = b with P, 4. V4 = d-bit pattern of P as we are comparing T[4] = d with P, and 5. V5 = c-bit pattern of P as we are comparing T[5] = c with P. 24 T = bcbdc P = abcbd (1)T[1]=b. We want to decide whether P[5] = T[1] = b. b-bit vector of P = 0 1 0 1 0 The last bit is 0 ≠1. T[1] ≠ P[5] Besides, we know that T[1] = P[2] = P[4] 25 T =bcbdc P =abcbd (2) T[2] = c. We want to decide whether T[1]T[2] = bc= P[4]P[5]. c-bit pattern of P = 0 0 1 0 0 AND-operation of T[1]-bit pattern of P and T[2]-bit pattern of P in the following way: T[2] = P[3] T[1] = P[2] ignore 0 0 0 0 0 1 1 1 0 0 0 1 0 0 Last bit 0 0 ignore The result of comparing T[1] and P[5] can be ignored from now on. The last bit is 0. The 2-prefix of T ≠ the 2-suffix of P. What does 0 1 0 0 mean ? It means that T[1] = P[2] = b and T[2] = P[3] = c. We keep this result 0 1 0 0. 26 T =bcbdc P =abcbd Resulting vector = 0 1 0 0 (3) T[3] = b. We want to decide whether T[1]T[2]T[3] = P[3]P[4]P[5]. b-bit pattern of P = 0 1 0 1 0 We only take the last 3 bits, namely 0 1 0 because we are interested in P[3]P[4]P[5]. ignore AND-operation 0 0 0 1 1 1 Result 0 0 0 Last bit 0 0 The last bit is 0. The 3-prefix of T ≠ the 3-suffix of P. 0 1 0 means that T[1] = P[2] = b , T[2] = P[3] (previously obtained) and T[3] = P[4] 27 We keep the resulting vector 010. T =bcbdc P =abcbd Resulting vector = 0 1 0 (4) T[4] = d. We want to decide whether T[1]T[2]T[3]T[4] = P[2]P[3]P[4]P[5]. d-bit pattern of P = 0 0 0 0 1 We only take the last 2 bits, namely 0 1. ignore AND-operation 0 0 0 Result 1 1 1 Last bit 0 0 The last bit is 1. The 4-prefix of T = the 4-suffix of P. 0 1 means that T[1] = P[2] = b , T[2] = P[3]=c , T[3] = P[4]=b (previously obtained) 28 and T[4] = P[5] = d T =bcbdc P =abcbd Resulting vector = 0 1 (4) T[5] = c. We want to decide whether T[1]T[2]T[3]T[4]T[5] = P[1]P[2]P[3]P[4]P[5]. c-bit pattern of P = 0 0 1 0 0 We only take the last 1 bits, namely 0. AND-operation 0 0 0 Result 1 1 ignore The last bit is 0. The 5-prefix of T ≠ the 5-suffix of P. 0 means that T[1] = P[2] = b , T[2] = P[3] , T[3] = P[4] ,T[4] = P[5] (previously obtained) 29 Definition: The Logic Operator (AND &) 1&1=1 1&0=0 0&1=0 0&0=0 Bit Pattern Of String - BPS Given a string S which is composed of n characters. S = abcabcabc S is composed of 3 characters which are a, b and c. BPS means to make bit patterns where each pattern represents each character appeared position in string. a-bit pattern = 1 0 0 1 0 0 1 0 0 b-bit pattern = 0 1 0 0 1 0 0 1 0 c-bit pattern = 0 0 1 0 0 1 0 0 1 30 The Algorithm ww( T =t1t2…tm , P=p1p2…pn) Preprocessing Find the character set of P Build the character_bit pattern of P the character_rbit pattern of inversed P Search For k ?n? ∈ 1...? ? do ?m? Open a wide window whose length is 2m-1 and its center point is at km Let the window be denoted as a1a2…a2m-1 Let a1a2…am-1 be denoted as T1 Let amam+1…a2m-1 be denoted as T2 /*we use modified convolution method to find out the matching*/ Find out all prefixes of T2 which are the suffix of P. (page 33-34) state 1: Find out the corresponding prefixes of P which are the suffix of T1 .(page 35-36) /*each time we can jump the wide window |P|*/ state 2: End For 31 Having constructed the character bit pattern of P, we may use the character bit pattern of P to find whether the prefix of T2 is equal to the suffix of P. Having constructed the character bit pattern of reversed P, we may use the character bit pattern of reversed P to find whether the suffix of T1 is equal to the prefix of P. 32 Find out all prefixes of T2 which are the suffix of P. Suf_bit = 1 … 1 |Suf_bit| = | T2 | x=1 Read T2 from left to right //if the reading character of T2 is one of the characters of P. //temporary space for storing the result //the length of tem space is equal to the length of T2 // x is the index for reading T2 if the character which belongs to the character set of P //We use AND-operation to simulate the convolution method //After each simulation, we store the result into the temporary space Suf_bit[|P|…x] = Suf_bit[|P|….x] & character_bit[(|P|-x+1)...1] /*check whether the |P|th to xth bit of Suf_bit are zeros, they are all zeros means no more prefix of T2 will be equal to the suffix of P. Therefore we can skip the remaining reading character from T2. */ if the |P|th to xth bit of Suf_bit are all zero goto state 1 end if else //if the reading character of T2 is not one of the characters of P. Set the |P|th to xth bit of Suf_bit to zero. goto state 2 // finish the reading from T2 end if 33 //if the xth bit of suf_bit is 1, x-suffix of T2 is equal to the x-prefix of P if Suf_bit[x] == 1 if x == |P| , //if the length matching of suffix of T2 is equal to the length of P , we found a we found a matching at km else we found x-suffix end if end if x++ Read next character // increase the index for reading next character Fig :: Find out all prefixes of T2 which are the suffix of P. 34 Having constructed the character bit pattern of reversed P, we may use the character bit pattern of reversed P to find whether the suffix of T1 is equal to the prefix of P. 35 Find out the corresponding prefixes of P which are the suffix of T1 . state 1: //if in the previous processing, we did not find any prefix of T2 is equal to the suffix of P //We do not need to find any corresponding suffix of T1 is equal to the prefix of P if the |P|th to 1st bit of Suf_bit are all zero goto state 2 else Pre_bit = 1 …