Repository: Cog-Creators/Red-DiscordBot

Issue:
Localtracks will not show symlinked folders in Red 3.2.2 on Ubuntu 18.04
# Command bugs

<!-- 
Did you find a bug with a command? Fill out the following:
-->

#### Command name

local play

#### What cog is this command from?

Audio

#### What were you expecting to happen?

Listing all the folders and tracks that are in localtracks.

#### What actually happened?

Symlinked folders were not showing up in the bot reply. Only local folders and files and symlinks of files show up.

#### How can we reproduce this issue?

1. Create a symlink to a folder containing music in localtracks:
`ln -s path/to/local/music/folder path/to/reds/localtracks/`
2. Initiate playing of local music in redbot:
`[p]local play`
3. Red will list folders and tracks for you to choose from. I observed that any local folders and files show up as well as symlinked files but the symlinked folder will not show up in the selection.

#### Additional context
This happens with Red 3.2.2, on Ubuntu 18.04 64 Bit. It has been working as expected prior to updating, which was Red 3.1.8.
Localtracks will not show symlinked folders in Red 3.2.2 on Ubuntu 18.04
# Command bugs

<!-- 
Did you find a bug with a command? Fill out the following:
-->

#### Command name

local play

#### What cog is this command from?

Audio

#### What were you expecting to happen?

Listing all the folders and tracks that are in localtracks.

#### What actually happened?

Symlinked folders were not showing up in the bot reply. Only local folders and files and symlinks of files show up.

#### How can we reproduce this issue?

1. Create a symlink to a folder containing music in localtracks:
`ln -s path/to/local/music/folder path/to/reds/localtracks/`
2. Initiate playing of local music in redbot:
`[p]local play`
3. Red will list folders and tracks for you to choose from. I observed that any local folders and files show up as well as symlinked files but the symlinked folder will not show up in the selection.

#### Additional context
This happens with Red 3.2.2, on Ubuntu 18.04 64 Bit. It has been working as expected prior to updating, which was Red 3.1.8.


Discussion:
Can be assigned to me, will work on it in the next few days .... also fuck localtracks
Can be assigned to me, will work on it in the next few days .... also fuck localtracks

Resolving patch:
diff --git a/redbot/cogs/audio/audio.py b/redbot/cogs/audio/audio.py
--- a/redbot/cogs/audio/audio.py
+++ b/redbot/cogs/audio/audio.py
@@ -2452,7 +2452,11 @@ async def _localtracks_folders(
         if not await self._localtracks_check(ctx):
             return
 
-        return audio_data.subfolders_in_tree() if search_subfolders else audio_data.subfolders()
+        return (
+            await audio_data.subfolders_in_tree()
+            if search_subfolders
+            else await audio_data.subfolders()
+        )
 
     async def _folder_list(
         self, ctx: commands.Context, query: audio_dataclasses.Query
@@ -2463,9 +2467,9 @@ async def _folder_list(
         if not query.track.exists():
             return
         return (
-            query.track.tracks_in_tree()
+            await query.track.tracks_in_tree()
             if query.search_subfolders
-            else query.track.tracks_in_folder()
+            else await query.track.tracks_in_folder()
         )
 
     async def _folder_tracks(
@@ -2504,9 +2508,9 @@ async def _all_folder_tracks(
             return
 
         return (
-            query.track.tracks_in_tree()
+            await query.track.tracks_in_tree()
             if query.search_subfolders
-            else query.track.tracks_in_folder()
+            else await query.track.tracks_in_folder()
         )
 
     async def _localtracks_check(self, ctx: commands.Context) -> bool:
diff --git a/redbot/cogs/audio/audio_dataclasses.py b/redbot/cogs/audio/audio_dataclasses.py
--- a/redbot/cogs/audio/audio_dataclasses.py
+++ b/redbot/cogs/audio/audio_dataclasses.py
@@ -1,9 +1,12 @@
+import asyncio
+import contextlib
+import glob
 import ntpath
 import os
 import posixpath
 import re
 from pathlib import Path, PosixPath, WindowsPath
-from typing import List, Optional, Union, MutableMapping
+from typing import List, Optional, Union, MutableMapping, Iterator, AsyncIterator
 from urllib.parse import urlparse
 
 import lavalink
@@ -167,29 +170,48 @@ def joinpath(cls, *args):
         modified.path = modified.path.joinpath(*args)
         return modified
 
-    def multiglob(self, *patterns):
-        paths = []
-        for p in patterns:
-            paths.extend(list(self.path.glob(p)))
-        for p in self._filtered(paths):
-            yield p
+    def rglob(self, pattern, folder=False) -> Iterator[str]:
+        if folder:
+            return glob.iglob(f"{self.path}{os.sep}**{os.sep}", recursive=True)
+        else:
+            return glob.iglob(f"{self.path}{os.sep}**{os.sep}{pattern}", recursive=True)
 
-    def multirglob(self, *patterns):
-        paths = []
-        for p in patterns:
-            paths.extend(list(self.path.rglob(p)))
+    def glob(self, pattern, folder=False) -> Iterator[str]:
+        if folder:
+            return glob.iglob(f"{self.path}{os.sep}*{os.sep}", recursive=False)
+        else:
+            return glob.iglob(f"{self.path}{os.sep}*{pattern}", recursive=False)
 
-        for p in self._filtered(paths):
-            yield p
+    async def multiglob(self, *patterns, folder=False) -> AsyncIterator["LocalPath"]:
+        for p in patterns:
+            for rp in self.glob(p):
+                rp = LocalPath(rp)
+                if folder and rp.is_dir() and rp.exists():
+                    yield rp
+                    await asyncio.sleep(0)
+                else:
+                    if rp.suffix in self._all_music_ext and rp.is_file() and rp.exists():
+                        yield rp
+                        await asyncio.sleep(0)
 
-    def _filtered(self, paths: List[Path]):
-        for p in paths:
-            if p.suffix in self._all_music_ext:
-                yield p
+    async def multirglob(self, *patterns, folder=False) -> AsyncIterator["LocalPath"]:
+        for p in patterns:
+            for rp in self.rglob(p):
+                rp = LocalPath(rp)
+                if folder and rp.is_dir() and rp.exists():
+                    yield rp
+                    await asyncio.sleep(0)
+                else:
+                    if rp.suffix in self._all_music_ext and rp.is_file() and rp.exists():
+                        yield rp
+                        await asyncio.sleep(0)
 
     def __str__(self):
         return self.to_string()
 
+    def __repr__(self):
+        return str(self)
+
     def to_string(self):
         try:
             return str(self.path.absolute())
@@ -209,48 +231,56 @@ def to_string_user(self, arg: str = None):
             string = f"...{os.sep}{string}"
         return string
 
-    def tracks_in_tree(self):
+    async def tracks_in_tree(self):
         tracks = []
-        for track in self.multirglob(*[f"{ext}" for ext in self._all_music_ext]):
-            if track.exists() and track.is_file() and track.parent != self.localtrack_folder:
-                tracks.append(Query.process_input(LocalPath(str(track.absolute()))))
+        async for track in self.multirglob(*[f"{ext}" for ext in self._all_music_ext]):
+            with contextlib.suppress(ValueError):
+                if track.path.parent != self.localtrack_folder and track.path.relative_to(
+                    self.path
+                ):
+                    tracks.append(Query.process_input(track))
         return sorted(tracks, key=lambda x: x.to_string_user().lower())
 
-    def subfolders_in_tree(self):
-        files = list(self.multirglob(*[f"*{ext}" for ext in self._all_music_ext]))
-        folders = []
-        for f in files:
-            if f.exists() and f.parent not in folders and f.parent != self.localtrack_folder:
-                folders.append(f.parent)
+    async def subfolders_in_tree(self):
         return_folders = []
-        for folder in folders:
-            if folder.exists() and folder.is_dir():
-                return_folders.append(LocalPath(str(folder.absolute())))
+        async for f in self.multirglob("", folder=True):
+            with contextlib.suppress(ValueError):
+                if (
+                    f not in return_folders
+                    and f.path != self.localtrack_folder
+                    and f.path.relative_to(self.path)
+                ):
+                    return_folders.append(f)
         return sorted(return_folders, key=lambda x: x.to_string_user().lower())
 
-    def tracks_in_folder(self):
+    async def tracks_in_folder(self):
         tracks = []
-        for track in self.multiglob(*[f"*{ext}" for ext in self._all_music_ext]):
-            if track.exists() and track.is_file() and track.parent != self.localtrack_folder:
-                tracks.append(Query.process_input(LocalPath(str(track.absolute()))))
+        async for track in self.multiglob(*[f"{ext}" for ext in self._all_music_ext]):
+            with contextlib.suppress(ValueError):
+                if track.path.parent != self.localtrack_folder and track.path.relative_to(
+                    self.path
+                ):
+                    tracks.append(Query.process_input(track))
         return sorted(tracks, key=lambda x: x.to_string_user().lower())
 
-    def subfolders(self):
-        files = list(self.multiglob(*[f"*{ext}" for ext in self._all_music_ext]))
-        folders = []
-        for f in files:
-            if f.exists() and f.parent not in folders and f.parent != self.localtrack_folder:
-                folders.append(f.parent)
+    async def subfolders(self):
         return_folders = []
-        for folder in folders:
-            if folder.exists() and folder.is_dir():
-                return_folders.append(LocalPath(str(folder.absolute())))
+        async for f in self.multiglob("", folder=True):
+            with contextlib.suppress(ValueError):
+                if (
+                    f not in return_folders
+                    and f.path != self.localtrack_folder
+                    and f.path.relative_to(self.path)
+                ):
+                    return_folders.append(f)
         return sorted(return_folders, key=lambda x: x.to_string_user().lower())
 
     def __eq__(self, other):
-        if not isinstance(other, LocalPath):
-            return NotImplemented
-        return self.path._cparts == other.path._cparts
+        if isinstance(other, LocalPath):
+            return self.path._cparts == other.path._cparts
+        elif isinstance(other, Path):
+            return self.path._cparts == other._cpart
+        return NotImplemented
 
     def __hash__(self):
         try:
@@ -260,24 +290,32 @@ def __hash__(self):
             return self._hash
 
     def __lt__(self, other):
-        if not isinstance(other, LocalPath):
-            return NotImplemented
-        return self.path._cparts < other.path._cparts
+        if isinstance(other, LocalPath):
+            return self.path._cparts < other.path._cparts
+        elif isinstance(other, Path):
+            return self.path._cparts < other._cpart
+        return NotImplemented
 
     def __le__(self, other):
-        if not isinstance(other, LocalPath):
-            return NotImplemented
-        return self.path._cparts <= other.path._cparts
+        if isinstance(other, LocalPath):
+            return self.path._cparts <= other.path._cparts
+        elif isinstance(other, Path):
+            return self.path._cparts <= other._cpart
+        return NotImplemented
 
     def __gt__(self, other):
-        if not isinstance(other, LocalPath):
-            return NotImplemented
-        return self.path._cparts > other.path._cparts
+        if isinstance(other, LocalPath):
+            return self.path._cparts > other.path._cparts
+        elif isinstance(other, Path):
+            return self.path._cparts > other._cpart
+        return NotImplemented
 
     def __ge__(self, other):
-        if not isinstance(other, LocalPath):
-            return NotImplemented
-        return self.path._cparts >= other.path._cparts
+        if isinstance(other, LocalPath):
+            return self.path._cparts >= other.path._cparts
+        elif isinstance(other, Path):
+            return self.path._cparts >= other._cpart
+        return NotImplemented
 
 
 class Query:


Test patch: