Repository: sunpy/sunpy

Issue:
Passing glob pattern to Map returns unsorted list of Maps
When passing a glob pattern to `sunpy.map.Map`, the list of maps that it returns is not sorted, even if the filenames in the glob can be sorted, e.g. by time.

For example,

```python
from sunpy.map import Map
maps = Map('/path/to/my/files/*.fits')  # these maps are not sorted!
```

where all of the filenames in `'/path/to/my/files/'` include a timestamp such that they can be sorted sequentially. The resulting list of files, `maps`, is not sorted.

The reason for this is that [we are using `glob.glob()` to get the list of files in the case that the user passes in a pattern](https://github.com/sunpy/sunpy/blob/master/sunpy/map/map_factory.py#L214) and `glob()` does not sort the resulting list of files by default.

This can easily be solved by just wrapping a `sorted` argound `glob.glob(...)`. The question is do we want to make the choice to sort the files for the user? I find it a bit unexpected that the resulting Maps are not sorted if the filenames allow for it.


Discussion:
I think this is probably a good idea, although I would suggest it should be thought of as a new feature rather than a bug.
Hmm.  What should be the expected behavior if the user calls:
```
maps = Map(['hmi*.fits', 'aia*.fits'])
```
Should it return a list of maps that have the HMI maps sorted by filename followed by the AIA maps sorted by filename, or should it return a more comprehensively sorted list of the AIA maps followed by the HMI maps?
> Hmm. What should be the expected behavior if the user calls:
> 
> ```
> maps = Map(['hmi*.fits', 'aia*.fits'])
> ```
> 
> Should it return a list of maps that have the HMI maps sorted by filename followed by the AIA maps sorted by filename, or should it return a more comprehensively sorted list of the AIA maps followed by the HMI maps?

I don't think I understand the difference between your two proposed outcomes. As a user, I would naively expect it to a return a list of HMI files (sorted according to their filename) followed by a list of AIA filenames (sorted according to their filenames).

In its current state, how does the MapFactory handle this pattern?
The second outcome is different from the first outcome because it would have the sorted AIA maps come before the HMI maps.  The easiest fix, as you suggest in the first post, would result in the first outcome because the loop processes each list element in order, so that matches your expectation.  I was just musing about whether a user who expects some sorting may also expect sorting across multiple inputs with wildcards.

As an aside, it looks like the code that passes off to `glob()` checks only for `*`, so apparently we don't support `?` or `[<character range>]`.
Just going to leave [this](https://www.vice.com/en_us/article/zmjwda/a-code-glitch-may-have-caused-errors-in-more-than-100-published-studies) here...
Just to add to this, apparently whether the result of `glob.glob` is sorted or not is OS-dependent so currently the order of the Maps you get back from `Map('*')` may vary depending on whether you're on Linux or Windows...
I think we should at least be consistent, so adding a `sorted` around the glob call would be a good place to start. 
I also think that it would be counter intuitive to sort over different positional arguments to `Map`, I would expect them to come back in that order if they were just filenames.

Resolving patch:
diff --git a/sunpy/map/map_factory.py b/sunpy/map/map_factory.py
--- a/sunpy/map/map_factory.py
+++ b/sunpy/map/map_factory.py
@@ -1,13 +1,15 @@
 import os
-import glob
 from collections import OrderedDict
 import warnings
+import pathlib
+import glob
 
 import numpy as np
 import astropy.io.fits
 from astropy.wcs import WCS
 
 import sunpy
+from sunpy import log
 from sunpy.map.mapbase import GenericMap, MapMetaValidationError
 from sunpy.map.compositemap import CompositeMap
 from sunpy.map.mapsequence import MapSequence
@@ -97,6 +99,14 @@ class MapFactory(BasicRegistrationFactory):
 
     >>> mymap = sunpy.map.Map('local_dir/sub_dir')   # doctest: +SKIP
 
+    * A filesystem path expressed as a `pathlib.Path`
+
+    >>> import pathlib
+    >>> mymap = sunpy.map.Map(pathlib.Path('file1.fits'))   # doctest: +SKIP
+    >>> sub_dir = pathlib.Path('local_dir/sub_dir')
+    >>> mymap = sunpy.map.Map(sub_dir)   # doctest: +SKIP
+    >>> mymap = sunpy.map.Map(sub_dir / 'file3.fits')   # doctest: +SKIP
+
     * Some regex globs
 
     >>> mymap = sunpy.map.Map('eit_*.fits')   # doctest: +SKIP
@@ -124,7 +134,10 @@ def _read_file(self, fname, **kwargs):
 
         # File gets read here.  This needs to be generic enough to seamlessly
         # call a fits file or a jpeg2k file, etc
-        pairs = read_file(fname, **kwargs)
+        # NOTE: use os.fspath so that fname can be either a str or pathlib.Path
+        # This can be removed once read_file supports pathlib.Path
+        log.debug(f'Reading {fname}')
+        pairs = read_file(os.fspath(fname), **kwargs)
 
         new_pairs = []
         for pair in pairs:
@@ -156,8 +169,8 @@ def _parse_args(self, *args, **kwargs):
         * data, header not in a tuple
         * data, wcs object in a tuple
         * data, wcs object not in a tuple
-        * filename, which will be read
-        * directory, from which all files will be read
+        * filename, as a str or pathlib.Path, which will be read
+        * directory, as a str or pathlib.Path, from which all files will be read
         * glob, from which all files will be read
         * url, which will be downloaded and read
         * lists containing any of the above.
@@ -196,34 +209,29 @@ def _parse_args(self, *args, **kwargs):
                     data_header_pairs.append(pair)
                     i += 1    # an extra increment to account for the data-header pairing
 
-            # File name
-            elif (isinstance(arg, str) and
-                  os.path.isfile(os.path.expanduser(arg))):
-                path = os.path.expanduser(arg)
-                pairs = self._read_file(path, **kwargs)
-                data_header_pairs += pairs
-
-            # Directory
-            elif (isinstance(arg, str) and
-                  os.path.isdir(os.path.expanduser(arg))):
-                path = os.path.expanduser(arg)
-                files = [os.path.join(path, elem) for elem in os.listdir(path)]
-                for afile in files:
-                    data_header_pairs += self._read_file(afile, **kwargs)
+            # File system path (file or directory)
+            elif _is_path(arg):
+                path = pathlib.Path(arg).expanduser()
+                if path.is_file():
+                    pairs = self._read_file(path, **kwargs)
+                    data_header_pairs += pairs
+                elif path.is_dir():
+                    for afile in sorted(path.glob('*')):
+                        data_header_pairs += self._read_file(afile, **kwargs)
+                else:
+                    raise ValueError(f'{path} is neither a file nor a directory')
 
             # Glob
-            elif (isinstance(arg, str) and '*' in arg):
-                files = glob.glob(os.path.expanduser(arg))
-                for afile in files:
+            elif isinstance(arg, str) and glob.glob(arg):
+                for afile in sorted(glob.glob(arg)):
                     data_header_pairs += self._read_file(afile, **kwargs)
 
             # Already a Map
             elif isinstance(arg, GenericMap):
                 already_maps.append(arg)
 
-            # A URL
-            elif (isinstance(arg, str) and
-                  _is_url(arg)):
+            # URL
+            elif isinstance(arg, str) and _is_url(arg):
                 url = arg
                 path = download_file(url, get_and_create_download_dir())
                 pairs = self._read_file(path, **kwargs)
@@ -347,6 +355,15 @@ def _is_url(arg):
     return True
 
 
+def _is_path(arg):
+    try:
+        is_path = pathlib.Path(arg).expanduser().exists()
+    except Exception:
+        return False
+    else:
+        return is_path
+
+
 class InvalidMapInput(ValueError):
     """Exception to raise when input variable is not a Map instance and does
     not point to a valid Map input file."""


Test patch:
diff --git a/sunpy/map/tests/test_map_factory.py b/sunpy/map/tests/test_map_factory.py
--- a/sunpy/map/tests/test_map_factory.py
+++ b/sunpy/map/tests/test_map_factory.py
@@ -5,8 +5,8 @@
 @author: stuart
 """
 import os
-import glob
 import tempfile
+import pathlib
 
 import pytest
 import numpy as np
@@ -19,7 +19,7 @@
 
 
 filepath = sunpy.data.test.rootdir
-a_list_of_many = glob.glob(os.path.join(filepath, "EIT", "*"))
+a_list_of_many = [os.fspath(f) for f in pathlib.Path(filepath, "EIT").glob("*")]
 a_fname = a_list_of_many[0]
 
 AIA_171_IMAGE = os.path.join(filepath, 'aia_171_level1.fits')
@@ -31,14 +31,13 @@
 #==============================================================================
 class TestMap:
     def test_mapsequence(self):
-        #Test making a MapSequence
+        # Test making a MapSequence
         sequence = sunpy.map.Map(a_list_of_many, sequence=True)
         assert isinstance(sequence, sunpy.map.MapSequence)
 
     def test_composite(self):
-        #Test making a CompositeMap
-        comp = sunpy.map.Map(AIA_171_IMAGE, RHESSI_IMAGE,
-                         composite=True)
+        # Test making a CompositeMap
+        comp = sunpy.map.Map(AIA_171_IMAGE, RHESSI_IMAGE, composite=True)
         assert isinstance(comp, sunpy.map.CompositeMap)
 
     def test_patterns(self):
@@ -49,14 +48,44 @@ def test_patterns(self):
         assert isinstance(eitmap, sunpy.map.GenericMap)
 
         # Directory
-        maps = sunpy.map.Map(os.path.join(filepath, "EIT"))
+        directory = pathlib.Path(filepath, "EIT")
+        maps = sunpy.map.Map(os.fspath(directory))
         assert isinstance(maps, list)
-        assert ([isinstance(amap,sunpy.map.GenericMap) for amap in maps])
+        assert ([isinstance(amap, sunpy.map.GenericMap) for amap in maps])
+        # Test that returned maps are sorted
+        files_sorted = sorted(list(directory.glob('*')))
+        maps_sorted = [sunpy.map.Map(os.fspath(f)) for f in files_sorted]
+        assert all([m.date == m_s.date for m, m_s in zip(maps, maps_sorted)])
+
+        # Pathlib
+        path = pathlib.Path(a_fname)
+        eitmap = sunpy.map.Map(path)
+        assert isinstance(eitmap, sunpy.map.GenericMap)
+        maps = sunpy.map.Map(directory)
+        assert isinstance(maps, list)
+        assert ([isinstance(amap, sunpy.map.GenericMap) for amap in maps])
 
         # Glob
-        maps = sunpy.map.Map(os.path.join(filepath, "EIT", "*"))
+        pattern = os.path.join(filepath, "EIT", "*")
+        maps = sunpy.map.Map(pattern)
+        assert isinstance(maps, list)
+        assert ([isinstance(amap, sunpy.map.GenericMap) for amap in maps])
+        # Test that returned maps are sorted
+        files_sorted = sorted(list(pathlib.Path(pattern).parent.glob('*')))
+        maps_sorted = [sunpy.map.Map(os.fspath(f)) for f in files_sorted]
+        assert all([m.date == m_s.date for m, m_s in zip(maps, maps_sorted)])
+        # Single character wildcard (?)
+        pattern = os.path.join(filepath, "EIT", "efz20040301.0?0010_s.fits")
+        maps = sunpy.map.Map(pattern)
+        assert isinstance(maps, list)
+        assert len(maps) == 7
+        assert ([isinstance(amap, sunpy.map.GenericMap) for amap in maps])
+        # Character ranges
+        pattern = os.path.join(filepath, "EIT", "efz20040301.0[2-6]0010_s.fits")
+        maps = sunpy.map.Map(pattern)
         assert isinstance(maps, list)
-        assert ([isinstance(amap,sunpy.map.GenericMap) for amap in maps])
+        assert len(maps) == 4
+        assert ([isinstance(amap, sunpy.map.GenericMap) for amap in maps])
 
         # Already a Map
         amap = sunpy.map.Map(maps[0])
@@ -65,7 +94,7 @@ def test_patterns(self):
         # A list of filenames
         maps = sunpy.map.Map(a_list_of_many)
         assert isinstance(maps, list)
-        assert ([isinstance(amap,sunpy.map.GenericMap) for amap in maps])
+        assert ([isinstance(amap, sunpy.map.GenericMap) for amap in maps])
 
         # Data-header pair in a tuple
         pair_map = sunpy.map.Map((amap.data, amap.meta))
@@ -127,7 +156,7 @@ def test_url_pattern(self):
         assert isinstance(amap, sunpy.map.GenericMap)
 
     def test_save(self):
-        #Test save out
+        # Test save out
         eitmap = sunpy.map.Map(a_fname)
         afilename = tempfile.NamedTemporaryFile(suffix='fits').name
         eitmap.save(afilename, filetype='fits', overwrite=True)
@@ -138,48 +167,54 @@ def test_save(self):
 # Sources Tests
 #==============================================================================
     def test_sdo(self):
-        #Test an AIAMap
+        # Test an AIAMap
         aia = sunpy.map.Map(AIA_171_IMAGE)
-        assert isinstance(aia,sunpy.map.sources.AIAMap)
-        #Test a HMIMap
+        assert isinstance(aia, sunpy.map.sources.AIAMap)
+        # TODO: Test a HMIMap
 
     def test_soho(self):
-        #Test EITMap, LASCOMap & MDIMap
+        # Test EITMap, LASCOMap & MDIMap
         eit = sunpy.map.Map(os.path.join(filepath, "EIT", "efz20040301.000010_s.fits"))
-        assert isinstance(eit,sunpy.map.sources.EITMap)
+        assert isinstance(eit, sunpy.map.sources.EITMap)
 
         lasco = sunpy.map.Map(os.path.join(filepath, "lasco_c2_25299383_s.fts"))
-        assert isinstance(lasco,sunpy.map.sources.LASCOMap)
+        assert isinstance(lasco, sunpy.map.sources.LASCOMap)
 
         mdi_c = sunpy.map.Map(os.path.join(filepath, "mdi_fd_Ic_6h_01d.5871.0000_s.fits"))
-        assert isinstance(mdi_c,sunpy.map.sources.MDIMap)
+        assert isinstance(mdi_c, sunpy.map.sources.MDIMap)
 
         mdi_m = sunpy.map.Map(os.path.join(filepath, "mdi_fd_M_96m_01d.5874.0005_s.fits"))
-        assert isinstance(mdi_m,sunpy.map.sources.MDIMap)
+        assert isinstance(mdi_m, sunpy.map.sources.MDIMap)
 
     def test_stereo(self):
-        #Test EUVIMap & CORMap & HIMap
+        # Test EUVIMap & CORMap & HIMap
         euvi = sunpy.map.Map(os.path.join(filepath, "euvi_20090615_000900_n4euA_s.fts"))
-        assert isinstance(euvi,sunpy.map.sources.EUVIMap)
+        assert isinstance(euvi, sunpy.map.sources.EUVIMap)
 
         cor = sunpy.map.Map(os.path.join(filepath, "cor1_20090615_000500_s4c1A.fts"))
-        assert isinstance(cor,sunpy.map.sources.CORMap)
+        assert isinstance(cor, sunpy.map.sources.CORMap)
 
-        hi = sunpy.map.Map(os.path.join(filepath,"hi_20110910_114721_s7h2A.fts"))
-        assert isinstance(hi,sunpy.map.sources.HIMap)
+        hi = sunpy.map.Map(os.path.join(filepath, "hi_20110910_114721_s7h2A.fts"))
+        assert isinstance(hi, sunpy.map.sources.HIMap)
 
     def test_rhessi(self):
-        #Test RHESSIMap
+        # Test RHESSIMap
         rhessi = sunpy.map.Map(RHESSI_IMAGE)
-        assert isinstance(rhessi,sunpy.map.sources.RHESSIMap)
+        assert isinstance(rhessi, sunpy.map.sources.RHESSIMap)
 
     def test_sot(self):
-        #Test SOTMap
-        sot = sunpy.map.Map(os.path.join(filepath , "FGMG4_20110214_030443.7.fits"))
-        assert isinstance(sot,sunpy.map.sources.SOTMap)
+        # Test SOTMap
+        sot = sunpy.map.Map(os.path.join(filepath, "FGMG4_20110214_030443.7.fits"))
+        assert isinstance(sot, sunpy.map.sources.SOTMap)
 
-        #Test SWAPMap
+    def test_swap(self):
+        # Test SWAPMap
+        swap = sunpy.map.Map(os.path.join(filepath, "swap_lv1_20140606_000113.fits"))
+        assert isinstance(swap, sunpy.map.sources.SWAPMap)
 
-        #Test XRTMap
+    def test_xrt(self):
+        # Test XRTMap
+        xrt = sunpy.map.Map(os.path.join(filepath, "HinodeXRT.fits"))
+        assert isinstance(xrt, sunpy.map.sources.XRTMap)
 
-        #Test SXTMap
+    # TODO: Test SXTMap