Repository: beetbox/beets

Issue:
musixmatch source is broken on lyrics plugin (404)
### Problem

```
lyrics:
    auto: yes
    force: yes
    sources: musixmatch
    fallback: ''
    import.write: yes
```

is giving me no results for any query that I give the plugin. For instance:

```
beet lyrics keren ann
lyrics: lyrics not found: Keren Ann - 101 - My Name Is Trouble
lyrics: lyrics not found: Keren Ann - 101 - Run With You
lyrics: lyrics not found: Keren Ann - 101 - All the Beautiful Girls
lyrics: lyrics not found: Keren Ann - 101 - Sugar Mama
lyrics: lyrics not found: Keren Ann - 101 - She Won't Trade It for Nothing
lyrics: lyrics not found: Keren Ann - 101 - You Were on Fire
lyrics: lyrics not found: Keren Ann - 101 - Blood on My Hands
lyrics: lyrics not found: Keren Ann - 101 - Song From a Tour Bus
lyrics: lyrics not found: Keren Ann - 101 - Strange Weather
lyrics: lyrics not found: Keren Ann - 101 - 101
lyrics: lyrics not found: Keren Ann - 101 - Daddy, You Been On My Mind
lyrics: lyrics not found: Keren Ann - Bleue - Les Jours heureux
lyrics: lyrics not found: Keren Ann - Bleue - Bleu
lyrics: lyrics not found: Keren Ann - Bleue - Le Fleuve doux
lyrics: lyrics not found: Keren Ann - Bleue - Nager la nuit
lyrics: lyrics not found: Keren Ann - Bleue - Sous l'eau
lyrics: lyrics not found: Keren Ann - Bleue - Ton île prison
lyrics: lyrics not found: Keren Ann - Bleue - Odessa, odyssée
lyrics: lyrics not found: Keren Ann - Bleue - La Mauvaise Fortune
lyrics: lyrics not found: Keren Ann - Bleue - Le goût était acide
lyrics: lyrics not found: Keren Ann - Keren Ann - It's All a Lie
lyrics: lyrics not found: Keren Ann - Keren Ann - Lay Your Head Down
lyrics: lyrics not found: Keren Ann - Keren Ann - In Your Back
lyrics: lyrics not found: Keren Ann - Keren Ann - The Harder Ships of the World
lyrics: lyrics not found: Keren Ann - Keren Ann - It Ain't No Crime
lyrics: lyrics not found: Keren Ann - Keren Ann - Where No Endings End
lyrics: lyrics not found: Keren Ann - Keren Ann - Liberty
lyrics: lyrics not found: Keren Ann - Keren Ann - Between the Flatland and the Caspian Sea
lyrics: lyrics not found: Keren Ann - Keren Ann - Caspia
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Dimanche en hiver
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Dans ma ville
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Seule
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - On est loin
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Sur le fil
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Peut-être
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Reste là
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Décrocher les étoiles
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Jardin d'hiver
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Aéroplane
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Deux
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Les mercenaires
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Autour de l'arbre
lyrics: lyrics not found: Keren Ann - La disparition - Au coin du monde
lyrics: lyrics not found: Keren Ann - La disparition - Le sable mouvant
lyrics: lyrics not found: Keren Ann - La disparition - Les rivières de janvier
lyrics: lyrics not found: Keren Ann - La disparition - La corde et les chaussons
lyrics: lyrics not found: Keren Ann - La disparition - Surannée
lyrics: lyrics not found: Keren Ann - La disparition - Ailleurs
lyrics: lyrics not found: Keren Ann - La disparition - L'illusioniste
lyrics: lyrics not found: Keren Ann - La disparition - La tentation
lyrics: lyrics not found: Keren Ann - La disparition - Mes pas dans la neige
lyrics: lyrics not found: Keren Ann - La disparition - Le chien d'avant-garde
lyrics: lyrics not found: Keren Ann - La disparition - La disparition
lyrics: lyrics not found: Keren Ann - Nolita - Que n'ai-je?
lyrics: lyrics not found: Keren Ann - Nolita - Greatest You Can Find
lyrics: lyrics not found: Keren Ann - Nolita - Chelsea Burns
lyrics: lyrics not found: Keren Ann - Nolita - One Day Without
lyrics: lyrics not found: Keren Ann - Nolita - La forme et le fond
lyrics: lyrics not found: Keren Ann - Nolita - Nolita
lyrics: lyrics not found: Keren Ann - Nolita - Roses & Hips
lyrics: lyrics not found: Keren Ann - Nolita - Midi dans le salon de la Duchesse
lyrics: lyrics not found: Keren Ann - Nolita - L'onde amère
lyrics: lyrics not found: Keren Ann - Nolita - For You and I
lyrics: lyrics not found: Keren Ann - Nolita - Song of Alice (feat. Sean Gullette)
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Not Going Anywhere
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Polly
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Road Bin
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - End of May
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Sailor & Widow
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Sit in the Sun
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Right Now & Right Here
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Seventeen
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Spanish Song Bird
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - By the Cathedral
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - You're Gonna Get Love
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Bring Back
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - The Separated Twin
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Insensible World
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Where Did You Go?
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Easy Money
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - My Man Is Wanted but I Ain't Gonna Turn Him In
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - You Knew Me Then
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Again and Again
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - The River That Swallows All the Rivers
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - You Have It All to Lose
lyrics: lyrics not found: Keren Ann feat. Barði Jóhannsson - Not Going Anywhere - Ending Song
lyrics: lyrics not found: Keren Ann feat. David Byrne - Bleue - Le Goût d'inachevé
```

```sh
plugin paths: 
artresizer: method is (2, (7, 0, 10), False)
Sending event: pluginload
library database: /Users/redacted/.config/beets/library.db
library directory: /Users/redacted/Music
Sending event: library_opened
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Que-N%27Ai-Je%3F (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Que-N%27Ai-Je%3F (404)
lyrics: lyrics not found: Keren Ann - Nolita - Que n'ai-je?
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Greatest-You-Can-Find (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Greatest-You-Can-Find (404)
lyrics: lyrics not found: Keren Ann - Nolita - Greatest You Can Find
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Chelsea-Burns (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Chelsea-Burns (404)
lyrics: lyrics not found: Keren Ann - Nolita - Chelsea Burns
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/One-Day-Without (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//One-Day-Without (404)
lyrics: lyrics not found: Keren Ann - Nolita - One Day Without
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/La-Forme-Et-Le-Fond (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//La-Forme-Et-Le-Fond (404)
lyrics: lyrics not found: Keren Ann - Nolita - La forme et le fond
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Nolita (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Nolita (404)
lyrics: lyrics not found: Keren Ann - Nolita - Nolita
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Roses-%26-Hips (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Roses-%26-Hips (404)
lyrics: lyrics not found: Keren Ann - Nolita - Roses & Hips
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Midi-Dans-Le-Salon-De-La-Duchesse (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Midi-Dans-Le-Salon-De-La-Duchesse (404)
lyrics: lyrics not found: Keren Ann - Nolita - Midi dans le salon de la Duchesse
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/L%27Onde-Am%C3%A8re (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//L%27Onde-Am%C3%A8re (404)
lyrics: lyrics not found: Keren Ann - Nolita - L'onde amère
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/For-You-And-I (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//For-You-And-I (404)
lyrics: lyrics not found: Keren Ann - Nolita - For You and I
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Song-Of-Alice-%28Feat.-Sean-Gullette%29 (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Song-Of-Alice (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Song-Of-Alice-%28Feat.-Sean-Gullette%29 (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Song-Of-Alice (404)
lyrics: lyrics not found: Keren Ann - Nolita - Song of Alice (feat. Sean Gullette)
```

### Setup

* OS:  macOS 10.13.6 
* Python version: 3
* beets version: 1.6.0

My configuration (output of `beet config`) is:

```yaml
lyrics:
    bing_lang_from: []
    auto: yes
    force: yes
    sources: musixmatch
    fallback: ''
    import.write: yes
    bing_client_secret: REDACTED
    bing_lang_to:
    google_API_key: REDACTED
    google_engine_ID: REDACTED
    genius_api_key: REDACTED
    local: no

plugins: embedart fetchart lyrics
autotag: yes
ignore_hidden: yes
original_date: yes
languages: en
va_name: Compilation

import:
    copy: no
    write: yes
    move: no
convert:
    tmpdir: /Volumes/Expansion-1/Media/Audio/
    auto: yes
    delete_originals: no
    no_convert: path::\.(m4a)$
    command: ffmpeg -y -i $source -vsync 0 -q:a 0 -map_metadata 0 $dest
    extension: m4a
discogs:
    user_token: yhRjvGwdihHSyuSMebbwfkDCovmrNHkodhTEdOuk
fetchart:
    auto: yes
    sources: fanarttv itunes amazon
    fanarttv_key: REDACTED
    minwidth: 0
    maxwidth: 0
    quality: 0
    max_filesize: 0
    enforce_ratio: no
    cautious: no
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    lastfm_key: REDACTED
    store_source: no
    high_resolution: no
    deinterlace: no
    cover_format:
embedart:
    auto: yes
    maxwidth: 1000
    remove_art_file: yes
    compare_threshold: 0
    ifempty: no
    quality: 0
zero:
    auto: yes
    update_database: yes
    keep_fields:
    - artist
    - title
    - album
    - year
    - albumartist
    - track
    - tracktotal
    - disc
    - disctotal
    - label
    - unsyncedlyrics
    - images
    - rating
    - first played
    - last played
    - play count
```



Discussion:
Broken on 1.6.1 also.
It seems like Musixmatch have blocked the `beets/...` user agent (actually any UA containing `beets`):

```
$ curl -H "User-Agent: beets/1.6.1" 'https://www.musixmatch.com/lyrics/Disclosure/White-Noise'
Not Allowed
```

```
$ curl -H "User-Agent: boots/1.6.1" 'https://www.musixmatch.com/lyrics/Disclosure/White-Noise'
<normal response here>
```

Musixmatch do offer an API but their free tier is for evaluation only and doesn't give full access to lyrics. We may have to just remove Musixmatch as a backend, which is a bit of a shame.
Man, that is indeed a shame 😞 They had the best lyrics from what I've gathered, I know Spotify uses them and just doing a quick comparison Genius has a lot of straight up garbage on their lyrics that Musixmatch doesn't ([Publishers: ©Sony/ATV Music Publishing LLC Popularity : 0 users have visited this page Length: 3:46 Sponsored links](https://genius.com/Keren-ann-autour-de-larbre-lyrics), repeating the [artist name] Lyrics header into the actual lyrics, [messed up unicode](https://genius.com/Keren-ann-ailleurs-lyrics) etc)
Indeed; this is too bad. I think the right solution is probably to leave the code in place, in case something changes in the future, but disable it by default and remove it from the documentation?
Would it be possible to change the UA identifier like you can in a browser, perhaps at the user's side only if not on the actual beets bundle? Maybe adding some function to slow the requests per second. At some point between those two it wouldn't be any more specific than normal/accepted behaviour of opening up their site from a browser and looking up lyrics.

Resolving patch:
diff --git a/beetsplug/lyrics.py b/beetsplug/lyrics.py
--- a/beetsplug/lyrics.py
+++ b/beetsplug/lyrics.py
@@ -743,7 +743,9 @@ def __init__(self):
             'fallback': None,
             'force': False,
             'local': False,
-            'sources': self.SOURCES,
+            # Musixmatch is disabled by default as they are currently blocking
+            # requests with the beets user agent.
+            'sources': [s for s in self.SOURCES if s != "musixmatch"],
             'dist_thresh': 0.1,
         })
         self.config['bing_client_secret'].redact = True


Test patch: