Repository: PaulWoitaschek/Voice (language: kotlin)

Issue:
Search seems to rely on a strange query preparation, that does not seem right?!
### Overview

I stumbled over this while trying to search for the new `series` and `part` fields recently added, e.g `Benjamin Blümchen 124`

When searching a book, the search query is rewritten prepared, to use the `MATCH` keyword in an sqlite full text search using `FTS4` here:

https://github.com/PaulWoitaschek/Voice/blob/main/core/search/src/main/kotlin/voice/core/search/BookSearch.kt

This is the relevant code snippet:

```kotlin
  suspend fun search(query: String): List<Book> {
    return dao.search(
      buildString {
        append("\"")
        append('*')
        append(query.trim().replace("\"", "\"\""))
        append('*')
        append("\"")
      },
    ).mapNotNull { repo.get(it) }
  }
```

So searching for `Harry Potter 1` will become `"*Harry Potter 1*"`.

After doing a bit of research, I think a few things are strange:

- Enclosing the whole query in `"` (double quotes) results in searching for an `exact phrase` query, which should not work for wildcards at all (I still wonder why this still seems to work somehow, probably the asterisks and quotes are just ignored?)
- Prepending and appending an asterisk (`*`) seems strange, since sqlite only supports prefix matching (e.g. `hel* worl*`) - a leading asterisk `*arry *otter` seems not to be supported at all (is this an Android specific extension?)
- Replacing `"` (double quotes) within the query would search for the real existence of double quotes in the target string, which seems to have the same effect than removing it completely, because it does not even seem to be stored in the index


So I think this could be changed to something like this:

```kotlin
  suspend fun search(query: String): List<Book> {
    val searchString = buildString {
      append(
        query.trim()
          // replace all special chars except umlauts and other language specific letters
          .replace("[^\\p{L}0-9\\s]".toRegex(), " ")
          // replace all whitespaces with *
          .split("\\s+".toRegex()).joinToString("*")
      )
      append('*')
    }
    val results = dao.search(searchString)

    return results.mapNotNull { repo.get(it) }
  }
```

Now searching for `Harry Potter 1` will become `Harry*Potter*1*`.

`the wise man's fear 2.1` becomes `the*wise*man*s*fear*2*1*`, which should pretty likely result in what would be expected...

Could you elaborate on your decision on how to prepare the query for using it in the MATCH part?

### How to reproduce

Search for `Harry Potter 1`, when the test data contains 

### Version

Latest

### Android Version

13

### Device Model

Pixel 4a


Discussion hints:
Tbh I don't remember. I also recently looked at the implementation and was rather puzzled. 

Feel free to drop a few test cases and improve the implementation!

Pull request: Improve the Search Logic
Fixes #3132.

While the copilot pull request is a good approach, this one includes more sophisticated unit tests together with `BookFactory` adjustments to search for the new fields `genre`, `narrator`, `series` and `part`. The production code is basically the same.

@PaulWoitaschek 
A 2 minute review should already be enough to decide which one you'd prefer :-)

Test command: ./gradlew :core:search:testDebugUnitTest --tests "voice.core.search.BookSearchTest"

Fix patch:
diff --git a/core/search/src/main/kotlin/voice/core/search/BookSearch.kt b/core/search/src/main/kotlin/voice/core/search/BookSearch.kt
index 8bca83d616..cd1dd62cac 100644
--- a/core/search/src/main/kotlin/voice/core/search/BookSearch.kt
+++ b/core/search/src/main/kotlin/voice/core/search/BookSearch.kt
@@ -4,22 +4,25 @@ import dev.zacsweers.metro.Inject
 import voice.core.data.Book
 import voice.core.data.repo.BookRepository
 import voice.core.data.repo.internals.dao.BookContentDao
+import voice.core.logging.api.Logger
 
 @Inject
 class BookSearch(
   private val dao: BookContentDao,
   private val repo: BookRepository,
 ) {
-
   suspend fun search(query: String): List<Book> {
-    return dao.search(
-      buildString {
-        append("\"")
-        append('*')
-        append(query.trim().replace("\"", "\"\""))
-        append('*')
-        append("\"")
-      },
-    ).mapNotNull { repo.get(it) }
+    val matchQuery = buildString {
+      append(
+        query.trim()
+          // replace all special chars except umlauts and other language specific letters
+          .replace("[^\\p{L}0-9\\s]".toRegex(), " ")
+          // replace all whitespaces with *
+          .split("\\s+".toRegex()).filter { it.isNotEmpty() }.joinToString("*"),
+      )
+      append('*')
+    }
+    Logger.d("search with MATCH: $matchQuery")
+    return dao.search(matchQuery).mapNotNull { repo.get(it) }
   }
 }


Test patch:
diff --git a/core/search/src/test/kotlin/voice/core/search/BookFactory.kt b/core/search/src/test/kotlin/voice/core/search/BookFactory.kt
index 965e02226c..818c8ec3c5 100644
--- a/core/search/src/test/kotlin/voice/core/search/BookFactory.kt
+++ b/core/search/src/test/kotlin/voice/core/search/BookFactory.kt
@@ -14,6 +14,10 @@ fun book(
   currentChapter: ChapterId = chapters.first().id,
   name: String = UUID.randomUUID().toString(),
   author: String? = UUID.randomUUID().toString(),
+  genre: String? = "Genre",
+  narrator: String? = "Narrator",
+  series: String? = null,
+  part: String? = null,
 ): Book {
   return Book(
     content = BookContent(
@@ -30,10 +34,10 @@ fun book(
       skipSilence = false,
       id = BookId(UUID.randomUUID().toString()),
       gain = 0F,
-      genre = null,
-      narrator = null,
-      series = null,
-      part = null,
+      genre = genre,
+      narrator = narrator,
+      series = series,
+      part = part,
     ),
     chapters = chapters,
   )
diff --git a/core/search/src/test/kotlin/voice/core/search/BookSearchTest.kt b/core/search/src/test/kotlin/voice/core/search/BookSearchTest.kt
index a82b7ea29b..be8b93e95d 100644
--- a/core/search/src/test/kotlin/voice/core/search/BookSearchTest.kt
+++ b/core/search/src/test/kotlin/voice/core/search/BookSearchTest.kt
@@ -34,6 +34,40 @@ class BookSearchTest {
     author = "Jimi Hendrix",
   )
 
+  private val harryPotter1 = book(
+    name = "Harry Potter and the Philosopher's Stone",
+    author = "J. K. Rowling",
+    genre = "Fantasy",
+    narrator = "Stephen Fry",
+    series = "Harry Potter",
+    part = "1",
+  )
+
+  private val harryPotter2 = book(
+    name = "Harry Potter and the Chamber of Secrets",
+    author = "J. K. Rowling",
+    genre = "Fantasy",
+    narrator = "Stephen Fry",
+    series = "Harry Potter",
+    part = "2",
+  )
+  private val kingkiller1 = book(
+    name = "The Name of the Wind",
+    author = "Patrick Rothfuss",
+    genre = "Fantasy",
+    narrator = "Rupert Degas",
+    series = "Kingkiller Chronicle",
+    part = "1",
+  )
+  private val kingkiller25 = book(
+    name = "The Slow Regard of Silent Things",
+    author = "Patrick Rothfuss",
+    genre = "Fantasy",
+    narrator = "Patrick Rothfuss",
+    series = "Kingkiller Chronicle",
+    part = "2.5",
+  )
+
   @Test
   fun `search complete name`() = test {
     expectSearchResult("Die Ermordung des Commendatore", commendatore)
@@ -78,6 +112,19 @@ class BookSearchTest {
     search("-*\"--*\"\"")
   }
 
+  @Test
+  fun `search for narrator, series and part`() = test {
+    expectSearchResult("Harry Potter 1", harryPotter1)
+    expectSearchResult("harry potter 1", harryPotter1)
+    expectSearchResult("harry potter", harryPotter1, harryPotter2)
+    expectSearchResult("kingkiller 1", kingkiller1)
+    expectSearchResult("rupert degas 1", kingkiller1)
+    expectSearchResult("kingkiller 2.5", kingkiller25)
+    expectSearchResult("slow regard 2.5", kingkiller25)
+    expectSearchResult("rothfuss 2.5", kingkiller25)
+    expectSearchResult("kingkiller", kingkiller1, kingkiller25)
+  }
+
   private fun test(run: suspend TestBase.() -> Unit) {
     val db = Room.inMemoryDatabaseBuilder(ApplicationProvider.getApplicationContext(), AppDb::class.java)
       .build()
@@ -103,11 +150,19 @@ class BookSearchTest {
       addBook(commendatore)
       addBook(watchtower)
       addBook(unicorns)
+      addBook(harryPotter1)
+      addBook(harryPotter2)
+      addBook(kingkiller1)
+      addBook(kingkiller25)
 
       // this ensures that inactive books are never accounted for
       addBook(commendatore.withNewIdAndInactive())
       addBook(watchtower.withNewIdAndInactive())
       addBook(unicorns.withNewIdAndInactive())
+      addBook(harryPotter1.withNewIdAndInactive())
+      addBook(harryPotter2.withNewIdAndInactive())
+      addBook(kingkiller1.withNewIdAndInactive())
+      addBook(kingkiller25.withNewIdAndInactive())
 
       testBase.run()
       db.close()