Repository: beetbox/beets

Issue:
mbsync: Double output
Recently the `mbsync` plugin has started to produce double output.
 ```
$ beet mbsync -p black sabbath vol 4
Black Sabbath - Vol 4 - Wheels of Confusion
  script:  -> Latn
  language:  -> eng
  script:  -> Latn
  language:  -> eng

...snip...

Black Sabbath - Vol 4 - Under the Sun
  script:  -> Latn
  language:  -> eng
  script:  -> Latn
  language:  -> eng
```

A quick `git bisect` reveals this was introduced in 72515448ad505ef0b73c0d0aa4aa424419b55755.

It looks like this could just be a cosmetic issue but I haven't had time to dive into the new code from #2988 to confirm yet.


Discussion:
I'd guess that in

https://github.com/beetbox/beets/blob/c4347960eab78be2ad54f4e35b32928c47e287d1/beets/library.py#L620-L627

the keys should be round-tripped through a set, i.e. `return list(set(keys))`. Or maybe there's a way to deduplicate lists in-place?
Ah, yes, that does look like the issue. Another simple way would be to just have `keys` be a set in the first place… i.e., something like `keys = set(super(...).keys(...)`. Perhaps the consumer doesn't care whether the return value is a set or a list, in which case it can just stay that way, or else we can do `return list(keys)` at the end.
> Ah, yes, that does look like the issue. Another simple way would be to just have `keys` be a set in the first place… i.e., something like `keys = set(super(...).keys(...)`. Perhaps the consumer doesn't care whether the return value is a set or a list, in which case it can just stay that way, or else we can do `return list(keys)` at the end.

Yeah, I proposed wrapping in a `list` in order not to change the return type. This method very much resembles `dict.keys()`, so I'd say that consumers shouldn't rely on any properties of the result except for being an iterator. Whether that is really the case in our codebase, I don't know. We might just try returning a set, and see whether the tests succeed.
So, locally on Python 3.9 at least, returning a `set` seems to work, i.e. the tests suceed. @FichteFoll does that look correct to you? I haven't followed the album field fallback PR, so I might be missing something.
As set definitely is the proper data structure for this task. I just didn't consider the keys to be iterated over when implementing fallback and thus the potential for duplicates.

I can't comment on how willing you are to potentially break backwards compatibility for any API users that rely on the return value being a list since *that's clearly documented in the docstring*, though.
> As set definitely is the proper data structure for this task. I just didn't consider the keys to be iterated over when implementing fallback and thus the potential for duplicates.

Ok, I was mostly wondering whether this issue would point at any other oversights with the fallback code. But I guess that's not the case then, thanks!

> I can't comment on how willing you are to potentially break backwards compatibility for any API users that rely on the return value being a list since _that's clearly documented in the docstring_, though.

That's a good point. I noticed the docstring, but I was a little curious whether anything really depends on it, since I wouldn't guarantee returning a list if designing the API anew.
I wanted to find out where exactly in the code path the mbsync plugin triggers an iteration over `keys` so that I could ensure I'm really not missing anything here, but wasn't able to find it after a brief look through the code base. Could anyone give me a pointer?
@FichteFoll it looks like this:

https://github.com/beetbox/beets/blob/c4347960eab78be2ad54f4e35b32928c47e287d1/beetsplug/mbsync.py#L161

Calls `show_model_changes`, which iterates over fields:

https://github.com/beetbox/beets/blob/c4347960eab78be2ad54f4e35b32928c47e287d1/beets/ui/__init__.py#L713

Which are returned from `__iter__`:

https://github.com/beetbox/beets/blob/c4347960eab78be2ad54f4e35b32928c47e287d1/beets/library.py#L432-L433

And it seems like `all_keys` is an amalgamation of `model_keys` and `album_keys`:

https://github.com/beetbox/beets/blob/c4347960eab78be2ad54f4e35b32928c47e287d1/beets/library.py#L385-L387

And `model_keys` is populated from the `keys` method:

https://github.com/beetbox/beets/blob/c4347960eab78be2ad54f4e35b32928c47e287d1/beets/library.py#L382

I am a bit confused how this is an issue though, as `all_keys` already converts `model_keys` to a set...
> I am a bit confused how this is an issue though, as `all_keys` already converts `model_keys` to a set...

The call trace is actually slightly different, you looked at `beets.library.FormattedItemMapping.__iter__`, what's actually being iterated over in `show_model_changes` is not this formatted view, but the `beets.library.Item` directly. That in turn inherits `beets.dbcore.db.Model.__iter__`, which just returns `iter(self.keys())`.

So it appears to be correct that `Item.keys()` should always return a list, but if `with_album=True`, it should remove duplicates first.

In addition, I have the impression that `show_model_changes` can be improved performance-wise. I guess it's correct that it iterates over the item rather than the formatted view, since it should compare the actual values, not any formatted output. However, it creates a new view for each key by calling `Item.formatted()`. It should keep this view around instead.

Resolving patch:
diff --git a/beets/library.py b/beets/library.py
--- a/beets/library.py
+++ b/beets/library.py
@@ -623,7 +623,9 @@ def keys(self, computed=False, with_album=True):
         """
         keys = super(Item, self).keys(computed=computed)
         if with_album and self._cached_album:
-            keys += self._cached_album.keys(computed=computed)
+            keys = set(keys)
+            keys.update(self._cached_album.keys(computed=computed))
+            keys = list(keys)
         return keys
 
     def get(self, key, default=None, with_album=True):


Test patch: