Repository: beetbox/beets

Issue:
Follow MusicBrainz pseudo-releases
The MB database uses "pseudo-releases" to provide translations of track names when there was, in fact, never a translated release. Apparently, [the correct way to handle these](http://tickets.musicbrainz.org/browse/PICARD-145) is to use the original release's IDs but the pseudo-release's human-readable metadata. The autotagger would need to follow the [translation/transliteration relationship](http://musicbrainz.org/relationship/fc399d47-23a7-4c28-bfcf-0607a562b644) to the original release.



Discussion:
More precisely, the division goes like this:
- Release title, release artist credit : from PR
- Track titles, track artists credits : from PR
- Artist IDs, Recording IDs, Release group ID : from either (should match), probably best to use
     real release and warn on a mismatch.
- Relationships (other than the trans(li)teration relationship) : from real release
- Cover art and URLs : from real release
- Medium titles : from PR
- Medium formats : from real release
- Release dates, release countries, label/imprint, catalog number : from real release
- Disambiguation : from PR if it has one, else from real release if it has one
- Language and script : from PR
- Barcode and other machine-readable attributes : from real release
- Release ID : from real release, PR's ID is stored in a different field as described in the Picard ticket.

Keep in mind that the tracklist should be derived per the relationship, not based on whether a release is a pseudo-release or not: the relationship may well be between two "official" releases, e.g., an original Japanese one and one with the track titles translated to English for a US/international market.

Just a heads up/note: https://tickets.musicbrainz.org/browse/MBS-8938 was part of the last schema change, so once the UI for this actually gets written and put to use, this will replace the use of pseudo-releases for transl(iter)ated tracklists.

Nice! This seems like it should be easier for us to consume.

I can't find information in MBS-8938 that alternative transl(iter)ated release titles are included in the schema as well; it still seems pseudo-releases are the only things storing transl(iter)ated release titles?
Hmm, no, I don't know how (or if) the data is exposed in the web service. But perhaps @Freso happens to know?
[This ticket](https://tickets.metabrainz.org/browse/MBS-4501) (linked to the one that @Freso mentioned) suggests that there's no actual interface for alternative tracklists yet. I assume that means there's no webface or API access to the functionality, so there's no way for beets to support them right now.

It looks to me like pseudo-releases are being deprecated in favor of alternative tracklists... eventually. (Which I personally support, having had awkward experiences with pseudo-releases in my own contributions to musicbrainz.) Given that, are we still interested in implementing the pseudo-release handling in beets? I might have a chance to work on this in the next few months, if it's still something that maintainers would accept.
Yeah, I'd support it! Unless we get word that pseudo-releases are really going away sooner rather than later, it still seems like a good idea to support them.
Hey, is there any movement on this?

It seems to me like this would (at least mostly) solve Issue #1916. Auto-transliteration sounds cool but also seems like a *way* harder problem than making beets aware of the transliterated entries that are available in the primary data-source we already use. Yes?

As a "usable now" solution, would it be possible to implement something like "search for track under a Release ID that I specify"...? Maybe under the S for Search option during import?

As it stands right now, I seem to have hit a wall in terms of importing Japanese and Chinese tracks into beets. As an example, I have a handful of singleton tracks by the Japanese idol-pop ensemble AKB48. I can find the tracks' parent Release entries  on the MusicBrainz website, open the transliterated pseudo-releases and see the Romaji track titles there, but the *recordings* don't seem to have alternate MBIDs or anything (the transliterated track names under the pseudorelease are links, but they seem to just link to the original Japanese-language entry for each track). So I can see that somebody has already done the work of creating this entity in the MB database with all the track names transliterated, but I can find no way to tell beets to use them... which AFAICT leaves me no option but to set up the tags manually in a different tool, then come back and import them to beets with the "Use as is" option (and/or set some sort of custom tag or flexible-attribute to add to the "ignore" list in my config so they don't get mangled if I do a re-import some day) to make sure beets *doesn't* connect them to the MB database entries. This workaround feels kind of counter to the whole point of beets.

There seems to have been no news in over a year on MB's "alternate tracklists" schema update, and I like to think that if beets implemented support for pseudo-releases, the change to supporting alternate tracklists could re-use a fair amount of the same code and would be only a modest amount of work.
I'm confused about what isn't working here. For example, I have [this album](https://musicbrainz.org/release-group/51301eed-a453-3bad-aff4-3e2308c1c27f) in my collection.

There is [a release](https://musicbrainz.org/release/eddd4819-f92c-48e0-a91b-b8b05fc9d1ff) with some tracks in Japanese, as well as [a pseudo-release](https://musicbrainz.org/release/e69b29a3-b20f-4e71-a0d9-05c9016fe511) with those tracks transliterated to Romaji. In my beets database, I see (some rows omitted for brevity):

```
$ beet info language:jpn script:Latn candlize track:8
       acoustid_id: 22666262-5e64-44f6-ae83-6eae9b812a60
             album: Candlize
       albumartist: 矢井田瞳
albumartist_credit: 矢井田瞳
  albumartist_sort: Yaida, Hitomi
       albumstatus: Pseudo-Release
         albumtype: album
            artist: 矢井田瞳
     artist_credit: 矢井田瞳
       artist_sort: Yaida, Hitomi
           country: JP
          language: jpn
  mb_albumartistid: eee58210-1dca-4c05-8cb8-5ea51e3edc07
        mb_albumid: e69b29a3-b20f-4e71-a0d9-05c9016fe511
       mb_artistid: eee58210-1dca-4c05-8cb8-5ea51e3edc07
 mb_releasegroupid: 51301eed-a453-3bad-aff4-3e2308c1c27f
 mb_releasetrackid: 8c2d4f27-7dbb-3a88-bd66-3b73d6e14312
        mb_trackid: 8536c2a2-3b24-48fa-8d38-86fd89541180
            script: Latn
             title: Zeitaku na Sekai
             track: 8
        tracktotal: 11
```

In other words, it's tagged using the pseudo-release, and has the transliterated title just fine.

I'll admit that my workflow is often to tag things first in MusicBrainz Picard, and then import into beets afterward. Is this issue about what happens if you _don't_ do that first? Or has the situation in beets improved in recent times?

Regardless, I don't think addressing this issue (if indeed there is anything to be done) would fully solve #1916—at least, I personally have quite a few albums in my collection without transliterated pseudo-releases on MB. I'd love to see beets adopt the [unihandecode](https://pypi.org/project/unihandecode/) library to improve the quality of its own transliterations.
Not sure if this applies to the scope of this issue but, is it possible to prefer "Official" to "Pseudo-Release" as seen in the `albumstatus` tag? I want imported tracks to retain the original language where possible without any kind of conversion. Seems like now beets selects the release type at random or based on weight.

Is there an option to drop unofficial releases like this? If no, is it possible to implement it? 

The [languages](https://beets.readthedocs.io/en/stable/reference/config.html#languages) option doesn't seem to help here. I have to reimport and manually provide ID of an official release each time autotagger decides that album is a PR.
@azahi maybe distance weights?

https://github.com/beetbox/beets/blob/master/beets/config_default.yaml#L119
I got kinda fed up about beets not handling this, especially things like the catalogue number or label missing for albums tagged with pseudo-releases. So i implemented the behaviour as a [plugin](https://github.com/TypicalFence/beets-resolve-pesudo-releases). I am not 100% sure how well it actually works just yet.

It still tags the pseudo-release's  id and status, because the ticket about this over on the picard issue tracker hasn't moved in years and that way other things like mbsync can update stuff properly.

I think it would be better if beets handled this internally.

If the maintainers would still support this (arguably) "edge-case", then I am happy to implement it. I am just really unsure on how to handle the id's.


That's awesome, even if the plugin only kinda works!!

It would indeed be good to handle this automatically inside beets. I also am not sure how to handle IDs, but maybe it would be a good strategy to try implementing it with the easiest/most obvious approach (using the pseudo-release's IDs, like you alluded to?) and see how many problems that causes for folks?

Resolving patch:
diff --git a/beets/autotag/mb.py b/beets/autotag/mb.py
--- a/beets/autotag/mb.py
+++ b/beets/autotag/mb.py
@@ -73,7 +73,8 @@ def get_message(self):
 RELEASE_INCLUDES = ['artists', 'media', 'recordings', 'release-groups',
                     'labels', 'artist-credits', 'aliases',
                     'recording-level-rels', 'work-rels',
-                    'work-level-rels', 'artist-rels', 'isrcs', 'url-rels']
+                    'work-level-rels', 'artist-rels', 'isrcs',
+                    'url-rels', 'release-rels']
 BROWSE_INCLUDES = ['artist-credits', 'work-rels',
                    'artist-rels', 'recording-rels', 'release-rels']
 if "work-level-rels" in musicbrainzngs.VALID_BROWSE_INCLUDES['recording']:
@@ -133,7 +134,7 @@ def _preferred_alias(aliases: List):
         matches = []
         for a in aliases:
             if a['locale'] == locale and 'primary' in a and \
-               a.get('type', '').lower() not in ignored_alias_types:
+                    a.get('type', '').lower() not in ignored_alias_types:
                 matches.append(a)
 
         # Skip to the next locale if we have no matches
@@ -583,10 +584,10 @@ def album_info(release: Dict) -> beets.autotag.hooks.AlbumInfo:
 
 
 def match_album(
-    artist: str,
-    album: str,
-    tracks: Optional[int] = None,
-    extra_tags: Optional[Dict[str, Any]] = None,
+        artist: str,
+        album: str,
+        tracks: Optional[int] = None,
+        extra_tags: Optional[Dict[str, Any]] = None,
 ) -> Iterator[beets.autotag.hooks.AlbumInfo]:
     """Searches for a single album ("release" in MusicBrainz parlance)
     and returns an iterator over AlbumInfo objects. May raise a
@@ -670,6 +671,64 @@ def _parse_id(s: str) -> Optional[str]:
     return None
 
 
+def _is_translation(r):
+    _trans_key = 'transl-tracklisting'
+    return r['type'] == _trans_key and r['direction'] == "backward"
+
+
+def _find_actual_release_from_pseudo_release(pseudo_rel: Dict) \
+        -> Optional[Dict]:
+    relations = pseudo_rel['release']["release-relation-list"]
+
+    # currently we only support trans(liter)ation's
+    translations = [r for r in relations if _is_translation(r)]
+
+    if not translations:
+        return None
+
+    actual_id = translations[0]['target']
+
+    return musicbrainzngs.get_release_by_id(actual_id,
+                                            RELEASE_INCLUDES)
+
+
+def _merge_pseudo_and_actual_album(
+        pseudo: beets.autotag.hooks.AlbumInfo,
+        actual: beets.autotag.hooks.AlbumInfo
+) -> Optional[beets.autotag.hooks.AlbumInfo]:
+    """
+    Merges a pseudo release with its actual release.
+
+    This implementation is naive, it doesn't overwrite fields,
+    like status or ids.
+
+    According to the ticket PICARD-145, the main release id should be used.
+    But the ticket has been in limbo since over a decade now.
+    It also suggests the introduction of the tag `musicbrainz_pseudoreleaseid`,
+    but as of this field can't be found in any offical Picard docs,
+    hence why we did not implement that for now.
+    """
+    merged = pseudo.copy()
+    from_actual = {k: actual[k] for k in [
+        "media",
+        "mediums",
+        "country",
+        "catalognum",
+        "year",
+        "month",
+        "day",
+        "original_year",
+        "original_month",
+        "original_day",
+        "label",
+        "asin",
+        "style",
+        "genre"
+    ]}
+    merged.update(from_actual)
+    return merged
+
+
 def album_for_id(releaseid: str) -> Optional[beets.autotag.hooks.AlbumInfo]:
     """Fetches an album by its MusicBrainz ID and returns an AlbumInfo
     object or None if the album is not found. May raise a
@@ -683,13 +742,29 @@ def album_for_id(releaseid: str) -> Optional[beets.autotag.hooks.AlbumInfo]:
     try:
         res = musicbrainzngs.get_release_by_id(albumid,
                                                RELEASE_INCLUDES)
+
+        # resolve linked release relations
+        actual_res = None
+
+        if res['release']['status'] == 'Pseudo-Release':
+            actual_res = _find_actual_release_from_pseudo_release(res)
+
     except musicbrainzngs.ResponseError:
         log.debug('Album ID match failed.')
         return None
     except musicbrainzngs.MusicBrainzError as exc:
         raise MusicBrainzAPIError(exc, 'get release by ID', albumid,
                                   traceback.format_exc())
-    return album_info(res['release'])
+
+    # release is potentially a pseudo release
+    release = album_info(res['release'])
+
+    # should be None unless we're dealing with a pseudo release
+    if actual_res is not None:
+        actual_release = album_info(actual_res['release'])
+        return _merge_pseudo_and_actual_album(release, actual_release)
+    else:
+        return release
 
 
 def track_for_id(releaseid: str) -> Optional[beets.autotag.hooks.TrackInfo]:
diff --git a/beets/ui/commands.py b/beets/ui/commands.py
--- a/beets/ui/commands.py
+++ b/beets/ui/commands.py
@@ -211,6 +211,9 @@ def disambig_string(info):
             disambig.append(info.catalognum)
         if info.albumdisambig:
             disambig.append(info.albumdisambig)
+        # Let the user differentiate between pseudo and actual releases.
+        if info.albumstatus == 'Pseudo-Release':
+            disambig.append(info.albumstatus)
 
     if disambig:
         return ', '.join(disambig)


Test patch:
diff --git a/test/test_importer.py b/test/test_importer.py
--- a/test/test_importer.py
+++ b/test/test_importer.py
@@ -1931,7 +1931,8 @@ def mocked_get_release_by_id(id_, includes=[], release_status=[],
             }],
             'release-group': {
                 'id': 'another-id',
-            }
+            },
+            'status': 'Official',
         }
     }
 
diff --git a/test/test_mb.py b/test/test_mb.py
--- a/test/test_mb.py
+++ b/test/test_mb.py
@@ -604,6 +604,7 @@ def test_match_album(self):
                     'release': {
                         'title': 'hi',
                         'id': mbid,
+                        'status': 'status',
                         'medium-list': [{
                             'track-list': [{
                                 'id': 'baz',
@@ -648,6 +649,164 @@ def test_match_album_empty(self):
             self.assertFalse(p.called)
             self.assertEqual(ail, [])
 
+    def test_follow_pseudo_releases(self):
+        side_effect = [
+            {
+                'release': {
+                    'title': 'pseudo',
+                    'id': 'd2a6f856-b553-40a0-ac54-a321e8e2da02',
+                    'status': 'Pseudo-Release',
+                    'medium-list': [{
+                        'track-list': [{
+                            'id': 'baz',
+                            'recording': {
+                                'title': 'translated title',
+                                'id': 'bar',
+                                'length': 42,
+                            },
+                            'position': 9,
+                            'number': 'A1',
+                        }],
+                        'position': 5,
+                    }],
+                    'artist-credit': [{
+                        'artist': {
+                            'name': 'some-artist',
+                            'id': 'some-id',
+                        },
+                    }],
+                    'release-group': {
+                        'id': 'another-id',
+                    },
+                    'release-relation-list': [
+                        {
+                            'type': 'transl-tracklisting',
+                            'target': 'd2a6f856-b553-40a0-ac54-a321e8e2da01',
+                            'direction': 'backward'
+                        }
+                    ]
+                }
+            },
+            {
+                    'release': {
+                        'title': 'actual',
+                        'id': 'd2a6f856-b553-40a0-ac54-a321e8e2da01',
+                        'status': 'Offical',
+                        'medium-list': [{
+                            'track-list': [{
+                                'id': 'baz',
+                                'recording': {
+                                    'title': 'original title',
+                                    'id': 'bar',
+                                    'length': 42,
+                                },
+                                'position': 9,
+                                'number': 'A1',
+                            }],
+                            'position': 5,
+                        }],
+                        'artist-credit': [{
+                            'artist': {
+                                'name': 'some-artist',
+                                'id': 'some-id',
+                            },
+                        }],
+                        'release-group': {
+                            'id': 'another-id',
+                        },
+                        'country': 'COUNTRY',
+                    }
+                }
+        ]
+
+        with mock.patch('musicbrainzngs.get_release_by_id') as gp:
+            gp.side_effect = side_effect
+            album = mb.album_for_id('d2a6f856-b553-40a0-ac54-a321e8e2da02')
+            self.assertEqual(album.country, 'COUNTRY')
+
+    def test_pseudo_releases_without_links(self):
+        side_effect = [{
+                    'release': {
+                        'title': 'pseudo',
+                        'id': 'd2a6f856-b553-40a0-ac54-a321e8e2da02',
+                        'status': 'Pseudo-Release',
+                        'medium-list': [{
+                            'track-list': [{
+                                'id': 'baz',
+                                'recording': {
+                                    'title': 'translated title',
+                                    'id': 'bar',
+                                    'length': 42,
+                                },
+                                'position': 9,
+                                'number': 'A1',
+                            }],
+                            'position': 5,
+                        }],
+                        'artist-credit': [{
+                            'artist': {
+                                'name': 'some-artist',
+                                'id': 'some-id',
+                            },
+                        }],
+                        'release-group': {
+                            'id': 'another-id',
+                        },
+                        'release-relation-list': []
+                    }
+                },
+        ]
+
+        with mock.patch('musicbrainzngs.get_release_by_id') as gp:
+            gp.side_effect = side_effect
+            album = mb.album_for_id('d2a6f856-b553-40a0-ac54-a321e8e2da02')
+            self.assertEqual(album.country, None)
+
+    def test_pseudo_releases_with_unsupported_links(self):
+        side_effect = [
+            {
+                'release': {
+                    'title': 'pseudo',
+                    'id': 'd2a6f856-b553-40a0-ac54-a321e8e2da02',
+                    'status': 'Pseudo-Release',
+                    'medium-list': [{
+                        'track-list': [{
+                            'id': 'baz',
+                            'recording': {
+                                'title': 'translated title',
+                                'id': 'bar',
+                                'length': 42,
+                            },
+                            'position': 9,
+                            'number': 'A1',
+                        }],
+                        'position': 5,
+                    }],
+                    'artist-credit': [{
+                        'artist': {
+                            'name': 'some-artist',
+                            'id': 'some-id',
+                        },
+                    }],
+                    'release-group': {
+                        'id': 'another-id',
+                    },
+                    'release-relation-list': [
+                        {
+                            'type': 'remaster',
+                            'target': 'd2a6f856-b553-40a0-ac54-a321e8e2da01',
+                            'direction': 'backward'
+                        }
+                    ]
+                }
+            },
+        ]
+
+        with mock.patch('musicbrainzngs.get_release_by_id') as gp:
+            gp.side_effect = side_effect
+            album = mb.album_for_id('d2a6f856-b553-40a0-ac54-a321e8e2da02')
+            self.assertEqual(album.country, None)
+
 
 def suite():
     return unittest.TestLoader().loadTestsFromName(__name__)