Talk:Boyer–Moore string-search algorithm

message: The first table (\delta_1) gives the next offset at which a match could possibly occur given the last (nonmatching) character encountered while comparing the pattern with the text; the second (\delta_2) gives a similar offset based on the number of characters that matched correctly. When a mismatch occurs, the largest of these two values is used to skip ahead to the next position where a match could potentially occur.
:Does anyone know the reason that the somewhat obscure references of \delta_1 and \delta_2 are used here?  Were those perhaps the way they were referenced in the original publications about the algorithm?  I'm replacing this text with a (hopefully!) more straightforward explanation but I'm wondering why they were referenced that way. -- Antaeus Feldspar 05:04, 19 Dec 2004 (UTC)
replies: text: Yes \delta_1 and \delta_2 are the names a lot of technical papers give to these functions. -- 14:03, 13 Feb 2007 (UTC)
thread_title:  First Table 

message: :Does anyone know when the Boyer-Moore algorithm was first published, so we can state it?  I added the full names of Boyer and Moore. RJFJR 03:16, Jan 12, 2005 (UTC)
Yes, 1977.  See R. BOYER AND S. MOORE, A fast string matching algorithm, CACM, 20 (1977), pp. 762-772.
What about Gosper? Sedgewick, in "Algorithms" says Gosper independently invented it at about the same time. I've occasionally heard the algorithm called Boyer-Moore-Gosper. Anyone know more? 203.164.223.88 11:05, 17 December 2006 (UTC)
thread_title: who and when

message: * Please refer to http://www.movsd.com/bm.htm  -- 83.237.28.135 2005-05-02 16:33:20
** The version without the second table is the Boyer-Moore-Horspool algorithm.  The real Boyer-Moore algorithm requires both tables. --Bisqwit 11:01, 22 May 2006 (UTC)
thread_title:  No need for the second jump table. 

message: I'm thinking that "WIKIPEDIA" should not be used as an example string, for the purpose of avoiding self-references. ~ Booyabazooka 20:06, 4 April 2006 (UTC)
replies: text: I agree that WIKIPEDIA should not be used as an example word, but that we should use the same word throughout. I'll fix it. Deco 06:03, 28 May 2006 (UTC)
thread_title:  Wikipedia self-reference 

message: The example code I wrote on this page (the one with boyermoore_needlematch function) is quite slow.
I created it by literally translating the specs (the explanation how the algorithm works) into C code. I'm afraid it does not satisfy readers who are in search of a fast search algorithm example code. I would like it to be replaced with a code that is fast, but that is not unreadable.
The suffixes and preBmGs functions at  are very obfuscated in my opinion; they (and derivatives thereof) are not suitable for example code. 
Is it possible to come up with a reasonably fast example that is not obfuscated?
For comparison, the code at Boyer-Moore-Horspool algorithm is quite simple. It is this extra step, i.e. calculation of the "skip" array that is at question.
--Bisqwit 08:59, 11 December 2006 (UTC)
thread_title:  About the example code 

message: I think the Boyer-Moore algorithm might be easier to understand if we:
* explain how the first table works all by itself, Skip1 -- the Boyer-Moore-Horspool algorithm
* explain how the second table works all by itself, Skip2 -- (or would it work all by itself? Is there a name for this algorithm? I suspect it might work well for searching bit-sequences or DNA base-pair sequences)
And finally summarize it by
* the Boyer-Moore algorithm, whenever it hits a mismatch, skips ahead the *maximum* of Skip1 and Skip2 (since all shorter skips have been ruled out).
--68.0.120.35 14:49, 22 March 2007 (UTC)
thread_title:  better way to explain 

message: I revised the example functions for the Boyer-Moore and the Boyer-Moore-Horspool search algorithms and added a Turbo-Boyer-Moore example. However, I believe this code requires some refitting before it is suitable for an educational example. These are written in C++, and are fairly optimal. I have tested them extensively to ensure they produce correct results. --Bisqwit 08:27, 21 August 2007 (UTC)
thread_title:  Revised example programs 

message: It appears that the behavior of memset from the C standard library isn't well defined across platforms except
# when filling memory with a value of 0,
# when filling arrays of signed char or unsigned char, or
# when your platform's ABI guarantees that a type has a specific representation.
 edited the sample code to use memset to fill an array of ssize_t objects with -1 values. This is not portable, and  reverted the edit. Two months later, Thegeneralguy made the same change again, and I reverted it again. I guess Thegeneralguy's rationale for the changes is option 3: memset with fill value of -1 works on most widespread 16-, 32-, and 64-bit platforms, all of which use a two's complement representation for ssize_t.
So here's my question: Do we require strictly conforming C, or do we relax C and assume two's complement? I propose we use ISO C, which works everywhere and is easier to understand, as opposed to what I view as premature optimization. --Damian Yerrick (talk | stalk) 23:23, 7 March 2008 (UTC)
thread_title:  memset 

message: Is this code actually correct? It looks pretty buggy (or, at best, relies on a lucky bug to be functional). It isn't referenced--isn't it WP:OR and/or WP:NOT#HOWTO, then? -- Mikeblas (talk) 15:38, 4 May 2008 (UTC)
thread_title:  Correct code? 

message: This looks wrong to me (am preparing for an exam, but it goes against what I've read elsewhere).
Could someone check it? 131.111.1.66 (talk) 16:55, 4 May 2009 (UTC)
thread_title:  First table...? 

message: The article should use the same term throughout, now it uses key in the introduction and later uses needle without introducing the term. (I like needle better)  —Preceding unsigned comment added by 85.230.77.206 (talk) 09:30, 30 March 2010 (UTC)
thread_title:  Key or Needle 

message: O(n/m), where `n` is the length of the haystack and `m` is the length of the needle, is still linear in `n`.  Rewording the initial paragraph so as not to be wrong.  —Preceding unsigned comment added by Jemfinch (talk • contribs) 17:09, 23 August 2010 (UTC)
thread_title:  O(n/m) != "sub-linear" 

message: I am doing benchmark of substring search algorithms (at volnitsky.com/project/str_search) and BM looks quite slow.  Does anyone knows where to find faster BM implementation(s)?  BTW, BM C source posted here is incorrect - returns wrong results if compiled with optimization ( gcc-4.5.1).  — Preceding unsigned comment added by Leonid Volnitsky (talk • contribs) 09:36, 2 February 2011 (UTC)
thread_title:  Fast BM implementation (including closed source and commercial).  

message: Can be found at:
http://code.activestate.com/recipes/117223-boyer-moore-horspool-string-searching/
===Python===
# bmh.py
#
# An implementation of Boyer-Moore-Horspool string searching.
#
# This code is Public Domain.
#
def BoyerMooreHorspool(pattern, text):
    m = len(pattern)
    n = len(text)
    if m > n: return -1
    skip = []
    for k in range(256): skip.append(m)
    for k in range(m - 1): skip[ord(pattern[k])] = m - k - 1
    skip = tuple(skip)
    k = m - 1
    while k < n:
        j = m - 1; i = k
        while j >= 0 and text[i] == pattern[j]:
            j -= 1; i -= 1
        if j == -1: return i + 1
        k += skip[ord(text[k])]
    return -1
if __name__ == '__main__':
    text = "this is the string to search in"
    pattern = "the"
    s = BoyerMooreHorspool(pattern, text)
    print 'Text:',text
    print 'Pattern:',pattern
    if s > -1:
        print 'Pattern \"' + pattern + '\" found at position',s
## end of http://code.activestate.com/recipes/117223/ }}}
(code is released under the PSF license (http://www.opensource.org/licenses/Python-2.0)
Ooops, sorry this is the Boyer Moore Horspool algorithm.  Will move this to that page.  — Preceding unsigned comment added by 64.251.74.12 (talk) 21:14, 21 July 2011 (UTC)
thread_title:  Python version 

message: # bmh.py
#
# An implementation of Boyer-Moore-Horspool string searching.
#
# This code is Public Domain.
#
def BoyerMooreHorspool(pattern, text):
    m = len(pattern)
    n = len(text)
    if m > n: return -1
    skip = []
    for k in range(256): skip.append(m)
    for k in range(m - 1): skip[ord(pattern[k])] = m - k - 1
    skip = tuple(skip)
    k = m - 1
    while k < n:
        j = m - 1; i = k
        while j >= 0 and text[i] == pattern[j]:
            j -= 1; i -= 1
        if j == -1: return i + 1
        k += skip[ord(text[k])]
    return -1
if __name__ == '__main__':
    text = "this is the string to search in"
    pattern = "the"
    s = BoyerMooreHorspool(pattern, text)
    print 'Text:',text
    print 'Pattern:',pattern
    if s > -1:
        print 'Pattern \"' + pattern + '\" found at position',s
## end of http://code.activestate.com/recipes/117223/ }}}
(code is released under the PSF license (http://www.opensource.org/licenses/Python-2.0)
Ooops, sorry this is the Boyer Moore Horspool algorithm.  Will move this to that page.  — Preceding unsigned comment added by 64.251.74.12 (talk) 21:14, 21 July 2011 (UTC)
thread_title: Python

message: The 'fast' implementation outlined in the original paper really should be mentioned on this page, as well as Hume and Sunday's improvements to delta1 in "Fast String Searching". The code from the October edit is mine, by the way. I'll be slowly merging the comments into the main body (I think that needs an overhaul, too). I just thought the page urgently needed something...erm, correct on it.
The previous algorithm was
a) confusing and inefficient. Why was the search pattern being reversed? Was the original author trying to reuse code from a KMP implementation? There's a comment in the middle that indicates confusion over how (if?) the algorithm works. Does it?
b) flat out wrong in at least one place. The delta1 offset is relative to the *offset* (in the old implementation, s+j or whatever), not the start of the potential match (s). The implementation in the original paper doesn't even bother keeping track of the start - it's not necessary.
Consider the example search text .....LTT-THAT and search pattern TT-THAT
TT-THAT
.....LTT-THAT
T will match, then A != L, and L does not occur in the pattern, so delta1[L] is 7 (length of TT-THAT).
The algorithm should move the text 7 relative to the mismatched character (i.e. 6 relative to the start of the block), not the start of the block. You can see here that moving 7 relative to the start will move past a valid match.
Source 2 (movsd) seems...extremely seedy. I don't trust a website that can mix up the names for the bad character rule and good suffix rule. The site also doesn't adequately explain what you are supposed to do if your search pattern has repeated sequences where the repeated block is more than one character long (surely a very, very, very rare case). In fact, I'm tempted to write off the entire page as wholly unreliable, and carefully review any content on this page that seemed to be derived from it (so far, it seems like most of the wiki article). Good grief.
Wikipedia is an encyclopedia, not an algorithms cookbook. I request that people looking to edit the code make sure they don't break anything in the name of "efficiency". Also, if the edit is for efficiency, make sure it's actually, well, more efficient. At the very least, make sure your edits increase or at least maintain clarity.
Reverted 3 changes in the delta2 table construction from user Snowgene. is_prefix() does not always return true (obviously, not every suffix of the pattern is a prefix), so last_prefix_index may very well get "fixed" at a previous value p. 
That is, the p in last_prefix_index = p+1 is not necessarily the p where delta2[p] = last_prefix_index + (patlen-1 - p).
Jy2wong (talk) 12:18, 17 October 2011 (UTC)
thread_title:  Miscellaneous gripes 

message: What are your thoughts? I think it is a bit more clear now. I used ideas from Dan Gusfield's book Algorithms on Strings, Trees, and Sequences. I also added more citations, and put the Performance section on strong theoretical ground - previously there were some pretty big misconceptions about runtimes vis a vis pattern occurring in the text or not.
Any suggestions, criticism? This is my first large-scale edit! Andrew Helwer (talk) 03:56, 8 April 2012 (UTC)
thread_title:  I rewrote the majority of the article 

message: I added a minor tweak for correctness to the Java section.  In theory, if you only consider the 'bad character table,' you have the Boyer-Moore-Horspool algorithm, however as it stands, the Java section is incomplete, and even enters infinite loops on some trivial examples.  This stems from using the variable i both as the alignment index and as the cursor for checking characters. For example, if the haystack is "ab ab abc def" and the needle is "xy abc" the program gets caught infinitely looping, because it forgets that it has already checked the potential match ending at abc and it tries to match it again, succeeds for 4 characters, fails, then hops back to where it started.  The k in the outer loop retains this information that is otherwise lost by decrementing the variable.
I am almost certain that the shift from the offset table will always be larger in such cases, but it seems to me that for correctness sake, that shouldn't be assumed unless it is explicitly stated in comments or in the article.  For instance, I was trying to build up my understanding of the algorithm by programming it piecemeal and testing as I went along.  I was quite puzzled when my incomplete BM algorithm (though complete BMH, as far as I knew) worked in some cases, but froze in others.  For that reason, I decided to fix this code (At least in the Java section.)
Bill Lava (talk) 04:46, 20 August 2012 (UTC)
thread_title:  Algorithm tweak in Java section 

message: The choice of data types for the C implementation is rather odd, to say the least.  There is no reason to be using exact-bit-width types here; the architecturally natural (fundamental) integer types should be used instead.  121a0012 (talk) 07:36, 18 January 2013 (UTC)
replies: text: The C and Java implementations were present before I came across this article, so I just left them there. They are untested (or don't work at all) for all I know. I personally wrote (and extensively tested, see here: https://github.com/ahelwer/IvoryTower) the Python implementation, so I'm available to answer any questions you might have about it. Andrew Helwer (talk) 06:20, 23 January 2013 (UTC)

text: Actually from a quick look at both the C and Java implementations, the preprocessing is not done in steps linear in the length of the pattern, but rather quadratic. This is due to use of the suffix length function... it isn't canonical Boyer-Moore, which preprocesses the pattern in linear time. On other other hand if your pattern is sufficiently short you may not care too much, as making the preprocess run in linear time adds a ton of complexity (see the Python implementation). Preprocessing for the bad suffix rule in linear time is so complicated I left the description out of the article, as it could pretty much have an article to itself. Andrew Helwer (talk) 06:30, 23 January 2013 (UTC)
thread_title:  C data types 

message: In the last section (second to last sentence) of the 'Description' subheading it is stated that:
"If an occurrence of P is found, then shift P by the least amount so that a proper prefix of the shifted P matches a suffix of the occurrence of P in T."
The word 'proper' is italicised; so I assume a 'proper prefix' is different in some way to your standard, run of the mill prefix. The article does not specify what this difference might be or if indeed there is a difference at all. Could this be clarified?
JPAIvan (talk) 23:22, 19 July 2014 (UTC)
replies: text: The proper prefix of a string is a prefix which is not equal to the string itself, similar to the definition of a proper subset. This should perhaps be defined in-article. Andrew Helwer (talk) 00:56, 2 August 2014 (UTC)
thread_title:  The Good Suffix Rule 

message: See Alpha Skip Search, Combinatorial Pattern Matching 1998, for an algorithm that also works on base two, text length N, pattern length M, with average runtime of O( (N/M + M)logM ), with log base alphabet size. 
Preprocess the pattern by creating a lookup table of M-logM+1 keys of positions in the pattern, where each key is a sequence of logM letters in the pattern.  time O(MlogM).
Main: Skip through the text by length M-logM+1, trying to match key of size logM to the lookup table. If it is in the lookup table, try matching the pattern naively, at the indicated start position.  
Often the key will not even be in the lookup table, in which case the algorithm just skips over M-logM+1 letters of the text and tries again.  The expected running time is O( (N/M)logM ), even for alphabets of size 2.
Daniel Pehoushek 24.33.70.144 (talk) 19:10, 24 August 2014 (UTC)
thread_title:  Does this algorithm work for alphabet size 2? 

message: The description says "... It is the reason comparisons begin at the end of the pattern rather than the start ..."
IMHO this is misleading. The comparison also begins at the end rather than the start to enable the 'Bad Character Rule' to work. So, IMHO, it is misleading to say "It is the reason comparisons ..." in the context of "Good Suffix"; the entire algorithm would be much less efficient, and arguably not  recognisable as Boyer-Moore without "comparisons begin at the end". 
As further evidence, the Boyer–Moore–Horspool algorithm _only_ uses the 'Bad Character" rule and its table.
It might be reasonable to write something like:  
"The good suffix rule is markedly more complex in both concept and implementation than the bad character rule. Like the 'Bad Character Rule', it also exploits the algorithm's feature of comparisons beginning at the end of the pattern and proceeding towards the pattern's start ..."
Gbulmeruk (talk) 11:29, 17 November 2015 (UTC)
thread_title:  [[Boyer%E2%80%93Moore_string_search_algorithm#Description_3|Good Suffix Rule Description]] 

message: The article describes what is called "extended bad character rule", not the basic "bad character rule" (see the original paper, and the book https://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string_search_algorithm#cite_note-ASTS-3 cited in the article). Perhaps this should be clarified.
BartoszKP (talk) 11:29, 4 December 2016 (UTC)
thread_title:  Misleading description of bad character rule 

message: Why? And what does it mean "past k1"? Aren't we scanning from the end? This whole section is mighty confusing... Can anyone explain it clearly without leaving room for imagination? --84.245.121.159 (talk) 23:56, 18 November 2020 (UTC)
thread_title:  The Galil rule 

message: The Python Code for this algo is incorrect. 
```>>> from Wiki import string_search
>>> from BMAlgo import BMA           
>>> with open("test2.txt", 'r', errors = 'ignore') as f:
...     strs = f.read()
...     A = BMA(strs, "caba") 
...     B = string_search('caba', strs)
...     print(A)
...     print(B)
...     print(A == B)
...               
[12168657]
[12168657, 12781460]
False
>>> strs[12168657: 12168657 + 4]
'caba'
>>> strs[12781460: 12781460 + 4]
'aaba'
>>>``` Bamgm14 (talk) 03:47, 5 April 2024 (UTC)
thread_title:  Python Code is incorrect