I just tried out using enumerate() instead of enumerateAll(), and even without batching it makes a better experience. It still takes a long time to show all my music, but I see something right away.  And I don't get the OOM.

You should experiment with batches to see if that improves the speed.