Repository: ashvardanian/StringZilla (language: rust)

Issue:
Bug: `sz_find_neon_too_smart` disagrees with `sz_find_neon`/`sz_find_serial`
### Describe the bug

Hi,

`sz_find_neon_too_smart` is disagreening with `sz_find_neon`/`sz_find_serial`
this happens in C, **only** with `n_length=2` (tested with lengths 1-32, and only "2" has this issue)

I noticed the first byte is equal where `sz_find_neon_too_smart` flagged, but the 2nd wasn't.

I used random strings of length `128` to compare the 2 functions.
```c
n[0]: 0xcb
n[1]: 0x7
n_length: 2
sz_find_neon: 64.            h[found_offset0]: 0xcb, h[found_offset0 + 1]: 0x7  <-- first and 2nd bytes good
sz_find_neon_too_smart: 60.  h[found_offset1]: 0xcb, h[found_offset1 + 1]: 0x30 <-- first byte good, 2nd not
```

Thanks!


### Steps to reproduce


haystack mem dump:
```c
0x80:     0x93    0xd9    0x96    0x3f    0xce    0xad    0xcf    0x48
0x88:     0xd8    0x2b    0x3     0x4a    0x8     0x5     0xe     0x8c
0x90:     0xf9    0xdc    0x6     0x3a    0x1     0x99    0x72    0x1e
0x98:     0xc6    0x5e    0xc1    0x23    0x5e    0x7c    0x98    0xdf
0xa0:     0xcf    0xe2    0x3f    0x28    0xc6    0x28    0xe1    0xec
0xa8:     0x64    0x8f    0x3a    0x3b    0xb4    0x7     0xfe    0xc8
0xb0:     0x95    0xd0    0xa1    0x9b    0x4b    0x24    0x90    0x50
0xb8:     0x32    0x95    0xa4    0x44    0xcb    0x30    0xf2    0xa   <-- 0xcb here, but not followed by 0x7
0xc0:     0xcb    0x7     0xc     0x65    0xa8    0x1c    0x6b    0x25  <-- substring was selected from this line
0xc8:     0xb0    0x13    0x22    0x88    0xba    0xd0    0xd3    0xdc
0xd0:     0xf1    0xc5    0x31    0x8b    0xf0    0xc9    0x34    0xe6
0xd8:     0x5e    0x5a    0x37    0xea    0x6e    0x1e    0x74    0xfb
0xe0:     0x87    0x84    0x45    0x22    0x42    0x9b    0x33    0x48
0xe8:     0xbc    0x44    0x8a    0xb9    0x29    0xa7    0x2     0xea
0xf0:     0xd     0x59    0xd3    0x6     0x4f    0x64    0x10    0xe8
0xf8:     0x4a    0x1a    0xe5    0xbc    0x2     0x99    0xce    0xd
```
needle mem dump:
```c
0xcb    0x7
```


### Expected behavior

`sz_find_neon_too_smart` should look at all needle bytes, not just the 1st one

### StringZilla version

v3.9.0

### Operating System

Ubuntu 22

### Hardware architecture

Arm

### Which interface are you using?

C implementation

### Contact Details

_No response_

### Are you open to being tagged as a contributor?

- [ ] I am open to being mentioned in the project `.git` history as a contributor

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion / hints:
@hillelme, the experimental kernels are not supposed to be correct, they are just idea drafts. Poor naming in my side. Is the sz_find reporting a correct result?

Fix patch:
diff --git a/.github/workflows/prerelease.yml b/.github/workflows/prerelease.yml
index 6efbdef6..777772a4 100644
--- a/.github/workflows/prerelease.yml
+++ b/.github/workflows/prerelease.yml
@@ -28,7 +28,7 @@ jobs:
           fetch-depth: 0
           persist-credentials: false
       - name: Run TinySemVer
-        uses: ashvardanian/tinysemver@v2.0.1
+        uses: ashvardanian/tinysemver@v2.0.7
         with:
           verbose: "true"
           version-file: "VERSION"
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index a7b42e9e..ed18e1f0 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -26,18 +26,14 @@ jobs:
         with:
           fetch-depth: 0
           persist-credentials: false
-      - name: Set up Cargo
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          override: true
       - name: Run TinySemVer
-        uses: ashvardanian/tinysemver@v2.0.1
+        uses: ashvardanian/tinysemver@v2.0.7
         with:
           verbose: "true"
           version-file: "VERSION"
           update-version-in: |
             Cargo.toml:^version = "(\d+\.\d+\.\d+)"
+            Cargo.lock:name = "stringzilla"\nversion = "(\d+\.\d+\.\d+)"
             package.json:"version": "(\d+\.\d+\.\d+)"
             CMakeLists.txt:VERSION (\d+\.\d+\.\d+)
           update-major-version-in: |
diff --git a/.vscode/settings.json b/.vscode/settings.json
index e7194e7a..2739b54a 100644
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -39,6 +39,7 @@
     "cibuildwheel",
     "CONCAT",
     "copydoc",
+    "Corasick",
     "cptr",
     "endregion",
     "endswith",
@@ -68,12 +69,14 @@
     "kwnames",
     "Lemire",
     "Levenshtein",
+    "LIBC",
     "libdivide",
     "lstrip",
     "Manber",
     "maxsplit",
     "memcpy",
     "Merkle-Damgård",
+    "Mersenne",
     "MODINIT",
     "napi",
     "nargsf",
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index c9d6a950..bd903210 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -57,8 +57,9 @@ unzip enwik9.zip && rm enwik9.zip && mv enwik9 enwik9.txt
 wget --no-clobber -O xlsum.csv.gz https://github.com/ashvardanian/xl-sum/releases/download/v1.0.0/xlsum.csv.gz
 gzip -d xlsum.csv.gz
 
-# Human chromosome generator dataset generated by https://github.com/rghilduta/human-chromosome-data-generator/blob/main/generate_chromosome_data.sh
+# Human chromosome generator dataset generated by:
 # 1200 rows, each 800 characters long (939K)
+# https://github.com/rghilduta/human-chromosome-data-generator/blob/main/generate_chromosome_data.sh
 wget --no-clobber -O human_protein_1200row_800len.txt https://media.githubusercontent.com/media/rghilduta/human-chromosome-data-generator/main/examples/human_protein_1200row_800len.txt
 ```
 
diff --git a/Cargo.lock b/Cargo.lock
index b627c249..5fe90449 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -10,7 +10,7 @@ checksum = "d32a725bc159af97c3e629873bb9f88fb8cf8a4867175f76dc987815ea07c83b"
 
 [[package]]
 name = "stringzilla"
-version = "3.8.4"
+version = "3.9.0"
 dependencies = [
  "cc",
 ]
diff --git a/README.md b/README.md
index e68a130b..8fd3406b 100644
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@ A typical codebase processes strings character by character, resulting in too ma
 LibC is different.
 It attempts to leverage SIMD instructions to boost some operations, and is often used by higher-level languages, runtimes, and databases.
 But it isn't perfect.
-1️⃣ First, even on common hardware, including over a billion 64-bit ARM CPUs, common functions like `strstr` and `memmem` only achieve 1/3 of the CPU's thoughput.
+1️⃣ First, even on common hardware, including over a billion 64-bit ARM CPUs, common functions like `strstr` and `memmem` only achieve 1/3 of the CPU's throughput.
 2️⃣ Second, SIMD coverage is inconsistent: acceleration in forward scans does not guarantee speed in the reverse-order search.
 3️⃣ At last, most high-level languages can't always use LibC, as the strings are often not NULL-terminated or may contain the Unicode "Zero" character in the middle of the string.
 That's why StringZilla was created.
@@ -254,7 +254,7 @@ StringZilla has a lot of functionality, most of which is covered by benchmarks a
 You can find those in the `./scripts` directory, with usage notes listed in the [`CONTRIBUTING.md`](CONTRIBUTING.md) file.
 Notably, if the CPU supports misaligned loads, even the 64-bit SWAR backends are faster than either standard library.
 
-> Most benchmarks were conducted on a 1 GB English text corpus, with an average word length of 5 characters.
+> Most benchmarks were conducted on a 1 GB English text corpus, with an average word length of 6 characters.
 > The code was compiled with GCC 12, using `glibc` v2.35.
 > The benchmarks performed on Arm-based Graviton3 AWS `c7g` instances and `r7iz` Intel Sapphire Rapids.
 > Most modern Arm-based 64-bit CPUs will have similar relative speedups.
@@ -521,10 +521,10 @@ Similar to how `File` can be used to read a large file, other interfaces can be
 The `Str` class has `write_to` to write the string to a file, and `offset_within` to obtain integer offsets of substring view in larger string for navigation.
 
 ```py
-web_archieve = Str("<html>...</html><html>...</html>")
-_, end_tag, next_doc = web_archieve.partition("</html>") # or use `find`
-next_doc_offset = next_doc.offset_within(web_archieve)
-web_archieve.write_to("next_doc.html") # no GIL, no copies, just a view
+web_archive = Str("<html>...</html><html>...</html>")
+_, end_tag, next_doc = web_archive.partition("</html>") # or use `find`
+next_doc_offset = next_doc.offset_within(web_archive)
+web_archive.write_to("next_doc.html") # no GIL, no copies, just a view
 ```
 
 #### PyArrow
@@ -980,8 +980,8 @@ The STL provides `std::generate` and `std::random_device`, that can be used with
 ```cpp
 sz::string random_string(std::size_t length, char const *alphabet, std::size_t cardinality) {
     sz::string result(length, '\0');
-    static std::random_device seed_source; // Too expensive to construct every time
-    std::mt19937 generator(seed_source());
+    static std::random_device seed_source; // Expensive to construct - due to system calls
+    static std::mt19937 generator(seed_source()); // Also expensive - due to the state size
     std::uniform_int_distribution<std::size_t> distribution(0, cardinality);
     std::generate(result.begin(), result.end(), [&]() { return alphabet[distribution(generator)]; });
     return result;
@@ -1464,6 +1464,11 @@ If you like strings and value efficiency, you may also enjoy the following proje
 - [pyahocorasick](https://github.com/WojciechMula/pyahocorasick) - Aho-Corasick algorithm in Python.
 - [rapidfuzz](https://github.com/rapidfuzz/RapidFuzz) - fast string matching in C++ and Python.
 
+If you are looking for more reading materials on this topic, consider the following:
+
+- [5x faster strings with SIMD & SWAR](https://ashvardanian.com/posts/stringzilla/).
+- [The Painful Pitfalls of C++ STL Strings](https://ashvardanian.com/posts/painful-strings/).
+
 ## License 📜
 
 Feel free to use the project under Apache 2.0 or the Three-clause BSD license at your preference.
diff --git a/include/stringzilla/experimental.h b/include/stringzilla/drafts.h
similarity index 99%
rename from include/stringzilla/experimental.h
rename to include/stringzilla/drafts.h
index b769c0f5..17147f7e 100644
--- a/include/stringzilla/experimental.h
+++ b/include/stringzilla/drafts.h
@@ -1,6 +1,6 @@
 /**
  *  @brief  Experimental kernels for StringZilla.
- *  @file   experimental.h
+ *  @file   drafts.h
  *  @author Ash Vardanian
  */
 #ifndef STRINGZILLA_EXPERIMENTAL_H_
diff --git a/scripts/bench_token.cpp b/scripts/bench_token.cpp
index 0a57ff69..e1f9d15e 100644
--- a/scripts/bench_token.cpp
+++ b/scripts/bench_token.cpp
@@ -7,7 +7,7 @@
 #include <bench.hpp>
 #include <test.hpp> // `random_string`
 
-#include <stringzilla/experimental.h> // `sz_hashes_neon`
+#include <stringzilla/drafts.h> // `sz_hashes_neon`
 
 using namespace ashvardanian::stringzilla::scripts;
 
diff --git a/setup.py b/setup.py
index e2b3e09f..5573a59a 100644
--- a/setup.py
+++ b/setup.py
@@ -28,7 +28,7 @@ def is_64bit_arm() -> bool:
     if using_cibuildwheels:
         return "SZ_ARM64" in os.environ
     arch = platform.machine()
-    return  arch in ["arm64", "aarch64", "ARM64"]
+    return arch in ["arm64", "aarch64", "ARM64"]
 
 
 def is_big_endian() -> bool:
@@ -150,8 +150,10 @@ def windows_settings() -> Tuple[List[str], List[str], List[Tuple[str]]]:
 setup(
     name=__lib_name__,
     version=__version__,
-    author="Ash Vardanian",
     description="SIMD-accelerated string search, sort, hashes, fingerprints, & edit distances",
+    author="Ash Vardanian",
+    author_email="1983160+ashvardanian@users.noreply.github.com",
+    url="https://github.com/ashvardanian/stringzilla",
     long_description=long_description,
     long_description_content_type="text/markdown",
     license="Apache-2.0",


Test patch: