Repository: beetbox/beets

Issue:
Genius backend of lyrics plugin fails occasionally when artist contains special characters.
<!--
When searching for lyrics using genius as a backend, the plugin sometimes fails to match artists that contain special characters.
-->

### Problem

Running this command in verbose (`-vv`) mode:

```
$ beet --version
beets version 1.4.9
Python version 3.6.9
plugins: duplicates, edit, fetchart, info, lyrics, mbcollection, mbsubmit, mbsync, replaygain, scrub, zero
```

```
$ beet -vv lyrics watch the throne
user configuration: /home/beets/.config/beets/config.yaml
data directory: /home/beets/.config/beets
plugin paths: /root/plugins
Sending event: pluginload
library database: /home/beets/.config/beets/library.db
library directory: /tank/music
Sending event: library_opened
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - Niggas in Paris
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - Gotta Have It
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - New Day
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - That’s My Bitch
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - Welcome to the Jungle
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - Who Gon Stop Me
lyrics: lyrics not found: JAY‐Z & Kanye West - Watch the Throne - Murder to Excellence
lyrics: lyrics not found: JAY‐Z & Kanye West feat. Beyoncé - Watch the Throne - Lift Off
lyrics: lyrics not found: JAY‐Z & Kanye West feat. Mr Hudson - Watch the Throne - Why I Love You
lyrics: lyrics not found: JAY‐Z & Kanye West feat. Frank Ocean - Watch the Throne - Made in America
lyrics: lyrics not found: JAY‐Z & Kanye West feat. Frank Ocean & The‐Dream - Watch the Throne - No Church in the Wild
lyrics: lyrics not found: JAY‐Z & Kanye West feat. Otis Redding - Watch the Throne - Otis
Sending event: cli_exit
```

I also did some testing on the current version (1.5)
```
$ beet --version
beets version 1.5.0
Python version 3.8.2
plugins: lyrics
```
config:
```
plugins: lyrics

lyrics:
  force: yes
  sources: genius
~                    
```
It seems the string comparison between the artist in beets and the artist on the genius page might be causing the issue.

https://github.com/beetbox/beets/blob/31855a93949a7b2e2636f709c2678d0efef1dcba/beetsplug/lyrics.py#L427-L430
I added a debug line to see what was getting compared...
```
            if hit_artist == artist.lower():
                return self.lyrics_from_song_page(hit["result"]["url"])
            else:
                self._log.debug(u'hit artist: {0} != {1}', hit_artist, artist.lower())
```


```
$ beet -vv lyrics request denied
user configuration: /home/jacob/.config/beets/config.yaml
data directory: /home/jacob/.config/beets
plugin paths: 
Sending event: pluginload
library database: /home/jacob/.config/beets/library.db
library directory: /home/jacob/Music
Sending event: library_opened
lyrics: hit artist: el-p != el‐p
lyrics: hit artist: el-p != el‐p
lyrics: hit artist: leftöver crack != el‐p
lyrics: hit artist: intro5pect != el‐p
lyrics: hit artist: u.s. court of appeals, fifth circuit != el‐p
lyrics: hit artist: noam chomsky != el‐p
lyrics: hit artist: jack london != el‐p
lyrics: hit artist: naguib mahfouz != el‐p
lyrics: hit artist: james joyce != el‐p
lyrics: hit artist: nosticthepoet != el‐p
lyrics: genius: no matching artist
lyrics: lyrics not found: El‐P - Cancer 4 Cure - Request Denied
Sending event: cli_exit
```


Discussion:
Sounds good! Maybe we should ASCIIfy artists when comparing them for a match. See also https://github.com/beetbox/beets/issues/3446 where we recently dealt with another special case of a Unicode character that was preventing matches.
Yeah, it [seems genius uses the hyphen-minus rather than the unicode hyphen that beets uses](https://www.babelstone.co.uk/Unicode/whatisit.html?string=el-p%20el%E2%80%90p). According to [stack overflow](https://stackoverflow.com/questions/10294032/python-replace-typographical-quotes-dashes-etc-with-their-ascii-counterparts), our best bet may be to just translate any differences found on genius to whatever unicode equivalent beets uses.

I'm not sure if this issue only deals with hyphens or not, but we can just start by translating those and work from there if anything else pops up...
Indeed. It may be worth taking a look at the lyrics plugin, which also does some punctuation normalization for artist matching.
I'll tackle this this weekend when I get some more free time. 

Resolving patch:
diff --git a/beetsplug/lyrics.py b/beetsplug/lyrics.py
--- a/beetsplug/lyrics.py
+++ b/beetsplug/lyrics.py
@@ -419,15 +419,13 @@ def fetch(self, artist, title):
             return None
 
         for hit in json["response"]["hits"]:
-            # Genius uses zero-width characters to denote lowercase
-            # artist names.
-            hit_artist = hit["result"]["primary_artist"]["name"]. \
-                strip(u'\u200b').lower()
+            hit_artist = hit["result"]["primary_artist"]["name"]
 
-            if hit_artist == artist.lower():
+            if slug(hit_artist) == slug(artist):
                 return self.lyrics_from_song_page(hit["result"]["url"])
 
-        self._log.debug(u'genius: no matching artist')
+        self._log.debug(u'Genius failed to find a matching artist for \'{0}\'',
+                        artist)
 
 
 class LyricsWiki(SymbolsReplaced):


Test patch:
diff --git a/test/test_lyrics.py b/test/test_lyrics.py
--- a/test/test_lyrics.py
+++ b/test/test_lyrics.py
@@ -17,6 +17,7 @@
 
 from __future__ import absolute_import, division, print_function
 
+import itertools
 import os
 import re
 import six
@@ -485,18 +486,28 @@ def test_slug(self):
         # plain ascii passthrough
         text = u"test"
         self.assertEqual(lyrics.slug(text), 'test')
+
         # german unicode and capitals
         text = u"Mørdag"
         self.assertEqual(lyrics.slug(text), 'mordag')
+
         # more accents and quotes
         text = u"l'été c'est fait pour jouer"
         self.assertEqual(lyrics.slug(text), 'l-ete-c-est-fait-pour-jouer')
+
         # accents, parens and spaces
         text = u"\xe7afe au lait (boisson)"
         self.assertEqual(lyrics.slug(text), 'cafe-au-lait-boisson')
         text = u"Multiple  spaces -- and symbols! -- merged"
         self.assertEqual(lyrics.slug(text),
                          'multiple-spaces-and-symbols-merged')
+        text = u"\u200Bno-width-space"
+        self.assertEqual(lyrics.slug(text), 'no-width-space')
+
+        # variations of dashes should get standardized
+        dashes = [u'\u200D', u'\u2010']
+        for dash1, dash2 in itertools.combinations(dashes, 2):
+            self.assertEqual(lyrics.slug(dash1), lyrics.slug(dash2))
 
 
 def suite():