Repository: rasterio/rasterio

Issue:
Unable to open EPSG support file gcs.csv
In case I install rasterio from github:

```
$ pip install git+https://github.com/mapbox/rasterio.git
```

the following code works flawlessly:

```
>>> from rasterio.crs import CRS
>>> CRS({'init': 'epsg:4326'}).is_geographic
True
```

But if  I install rasterio from PyPI I get such error:

```
>>> from rasterio.crs import CRS
>>> CRS({'init': 'epsg:4326'}).is_geographic
ERROR 4: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 1310, in rasterio._base._osr_from_crs
  File "rasterio/_err.pyx", line 182, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_OpenFailedError: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "rasterio/_crs.pyx", line 37, in rasterio._crs._CRS.is_geographic
  File "rasterio/_base.pyx", line 1316, in rasterio._base._osr_from_crs
rasterio.errors.CRSError: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
```

To overcome this issue we've wrapped the code that need access to `GDAL_DATA` with `rasterio.Env()`:

```
>>> import rasterio
>>> from rasterio.crs import CRS
>>> with rasterio.Env():
...     CRS({'init': 'epsg:4326'}).is_geographic
... 
True
>>> CRS({'init': 'epsg:3857'}).is_geographic
False
```

Is it expected behavior? It looks strange that subsequent calls of the same code works fine without wrapping. Another interesting case is when we call `is_geographic` without wrapping then the next call with wrapping doesn't work:

```
>>> import rasterio
>>> from rasterio.crs import CRS
>>> CRS({'init': 'epsg:4326'}).is_geographic
ERROR 4: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 1310, in rasterio._base._osr_from_crs
  File "rasterio/_err.pyx", line 182, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_OpenFailedError: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "rasterio/_crs.pyx", line 37, in rasterio._crs._CRS.is_geographic
  File "rasterio/_base.pyx", line 1316, in rasterio._base._osr_from_crs
rasterio.errors.CRSError: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
>>> with rasterio.Env():
...     CRS({'init': 'epsg:4326'}).is_geographic
... 
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 1310, in rasterio._base._osr_from_crs
  File "rasterio/_err.pyx", line 182, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_OpenFailedError: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "rasterio/_crs.pyx", line 37, in rasterio._crs._CRS.is_geographic
  File "rasterio/_base.pyx", line 1316, in rasterio._base._osr_from_crs
rasterio.errors.CRSError: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
```

@sgillies please take a look.


Discussion:
@drnextgis calling `Env()` sets the GDAL_DATA and PROJ_LIB environment variables in code that is executed only for packages installed from PyPI, which is why you see subsequent success, Env or not. It's a bit of a hack, and I'm sorry that it has tripped you up.

We could use the @ensure_env decorator to make sure that `CRS.is_geographic` (etc) always execute within a configured GDAL and PROJ environment. What would you think about that? There would be some details to solve, but it is very feasible.
Thanks for quick response. Your suggestion to decorate all methods that need access to `GDAL_DATA` looks good to me. First of all we need to create a list of such methods. But it is not clear for me how can I test changes? Because as I mentioned above if install rasterio from source code all works without any modifications.
@drnextgis I'll take of this issue before the end of the week.
Confirmed: some methods of the crs module can't execute outside of a configured environment. I think that decorating it is going to work well. I don't see any problem with checking the crs property of a dataset opened by rasterio.open() (which is already decorated).
Is it possible to catch these cases with tests?

Resolving patch:
diff --git a/rasterio/env.py b/rasterio/env.py
--- a/rasterio/env.py
+++ b/rasterio/env.py
@@ -338,6 +338,29 @@ def delenv():
     local._env = None
 
 
+class NullContextManager(object):
+    def __init__(self):
+        pass
+    def __enter__(self):
+        return self
+    def __exit__(self, *args):
+        pass
+
+
+def env_ctx_if_needed():
+    """Return an Env if one does not exist
+
+    Returns
+    -------
+    Env or a do-nothing context manager
+
+    """
+    if local._env:
+        return NullContextManager()
+    else:
+        return Env.from_defaults()
+
+
 def ensure_env(f):
     """A decorator that ensures an env exists before a function
     calls any GDAL C functions."""


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -48,18 +48,16 @@ def runner():
 
 
 @pytest.fixture(scope='function')
-def data():
+def data(tmpdir):
     """A temporary directory containing a copy of the files in data."""
-    tmpdir = py.test.ensuretemp('tests/data')
     for filename in test_files:
         shutil.copy(filename, str(tmpdir))
     return tmpdir
 
 
 @pytest.fixture(scope='function')
-def red_green():
+def red_green(tmpdir):
     """A temporary directory containing copies of red.tif, green.tif."""
-    tmpdir = py.test.ensuretemp('tests/data')
     for filename in ['tests/data/red.tif', 'tests/data/red.tif.ovr', 'tests/data/green.tif', 'tests/data/green.tif.ovr']:
         shutil.copy(filename, str(tmpdir))
     return tmpdir
@@ -560,15 +558,15 @@ def path_alpha_tif(data_dir):
 
 
 @pytest.fixture(scope='session')
-def path_zip_file():
+def path_zip_file(data_dir):
     """Creates ``coutwildrnp.zip`` if it does not exist and returns
     the absolute file path."""
-    path = '{}/white-gemini-iv.zip'.format(data_dir())
+    path = '{}/white-gemini-iv.zip'.format(data_dir)
     if not os.path.exists(path):
         with zipfile.ZipFile(path, 'w') as zip:
             for filename in ['white-gemini-iv.vrt',
                              '389225main_sw_1965_1024.jpg']:
-                zip.write(os.path.join(data_dir(), filename), filename)
+                zip.write(os.path.join(data_dir, filename), filename)
     return path
 
 
diff --git a/tests/data/compdcs.vrt b/tests/data/compdcs.vrt
new file mode 100644
--- /dev/null
+++ b/tests/data/compdcs.vrt
@@ -0,0 +1,37 @@
+<VRTDataset rasterXSize="1024" rasterYSize="768">
+<SRS>COMPD_CS[&quot;unknown&quot;,GEOGCS[&quot;WGS 84&quot;,DATUM[&quot;WGS_1984&quot;,SPHEROID[&quot;WGS 84&quot;,6378137,298.257223563,AUTHORITY[&quot;EPSG&quot;,&quot;7030&quot;]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[&quot;EPSG&quot;,&quot;6326&quot;]],PRIMEM[&quot;Greenwich&quot;,0],UNIT[&quot;degree&quot;,0.0174532925199433],AUTHORITY[&quot;EPSG&quot;,&quot;4326&quot;]],VERT_CS[&quot;unknown&quot;,VERT_DATUM[&quot;unknown&quot;,2005],UNIT[&quot;metre&quot;,1.0,AUTHORITY[&quot;EPSG&quot;,&quot;9001&quot;]],AXIS[&quot;Up&quot;,UP]]]</SRS>
+<GCPList Projection="COMPD_CS[&quot;unknown&quot;,GEOGCS[&quot;WGS 84&quot;,DATUM[&quot;WGS_1984&quot;,SPHEROID[&quot;WGS 84&quot;,6378137,298.257223563,AUTHORITY[&quot;EPSG&quot;,&quot;7030&quot;]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[&quot;EPSG&quot;,&quot;6326&quot;]],PRIMEM[&quot;Greenwich&quot;,0],UNIT[&quot;degree&quot;,0.0174532925199433],AUTHORITY[&quot;EPSG&quot;,&quot;4326&quot;]],VERT_CS[&quot;unknown&quot;,VERT_DATUM[&quot;unknown&quot;,2005],UNIT[&quot;metre&quot;,1.0,AUTHORITY[&quot;EPSG&quot;,&quot;9001&quot;]],AXIS[&quot;Up&quot;,UP]]]">
+  <GCP Id="1" Info="a" Pixel="0.0" Line="0.0" X="157168.0" Y="2818194.0" Z="0.0" />
+  <GCP Id="2" Info="b" Pixel="1024.0" Line="0.0" X="338615.0" Y="2786088.0" Z="0.0" />
+  <GCP Id="3" Info="c" Pixel="0.0" Line="768.0" X="116792" Y="2651340.0" Z="0.0" />
+</GCPList>
+  <VRTRasterBand dataType="Byte" band="1">
+    <ColorInterp>Gray</ColorInterp>
+    <SimpleSource>
+      <SourceFilename relativeToVRT="1">389225main_sw_1965_1024.jpg</SourceFilename>
+      <SourceBand>1</SourceBand>
+      <SrcRect xOff="0" yOff="0" xSize="1024" ySize="768"/>
+      <DstRect xOff="0" yOff="0" xSize="1024" ySize="768"/>
+    </SimpleSource>
+  </VRTRasterBand>
+  <VRTRasterBand dataType="Byte" band="2">
+    <ColorInterp>Gray</ColorInterp>
+    <SimpleSource>
+      <SourceFilename relativeToVRT="1">389225main_sw_1965_1024.jpg</SourceFilename>
+      <SourceBand>2</SourceBand>
+      <SrcRect xOff="0" yOff="0" xSize="1024" ySize="768"/>
+      <DstRect xOff="0" yOff="0" xSize="1024" ySize="768"/>
+    </SimpleSource>
+  </VRTRasterBand>
+  <VRTRasterBand dataType="Byte" band="3">
+    <ColorInterp>Gray</ColorInterp>
+    <SimpleSource>
+      <SourceFilename relativeToVRT="1">389225main_sw_1965_1024.jpg</SourceFilename>
+      <SourceBand>3</SourceBand>
+      <SrcRect xOff="0" yOff="0" xSize="1024" ySize="768"/>
+      <DstRect xOff="0" yOff="0" xSize="1024" ySize="768"/>
+    </SimpleSource>
+  </VRTRasterBand>
+
+</VRTDataset>
+
diff --git a/tests/data/test_esri_wkt.tif b/tests/data/test_esri_wkt.tif
new file mode 100644
Binary files /dev/null and b/tests/data/test_esri_wkt.tif differ
diff --git a/tests/test__env.py b/tests/test__env.py
new file mode 100644
--- /dev/null
+++ b/tests/test__env.py
@@ -0,0 +1,125 @@
+"""Tests of _env util module"""
+
+import pytest
+
+from rasterio._env import GDALDataFinder, PROJDataFinder
+
+from .conftest import gdal_version
+
+
+@pytest.fixture
+def mock_wheel(tmpdir):
+    """A fake rasterio wheel"""
+    moduledir = tmpdir.mkdir("rasterio")
+    moduledir.ensure("__init__,py")
+    moduledir.ensure("_env.py")
+    moduledir.ensure("gdal_data/pcs.csv")
+    moduledir.ensure("proj_data/epsg")
+    return moduledir
+
+
+@pytest.fixture
+def mock_fhs(tmpdir):
+    """A fake FHS system"""
+    tmpdir.ensure("share/gdal/pcs.csv")
+    tmpdir.ensure("share/proj/epsg")
+    return tmpdir
+
+
+@pytest.fixture
+def mock_debian(tmpdir):
+    """A fake Debian multi-install system"""
+    tmpdir.ensure("share/gdal/1.11/pcs.csv")
+    tmpdir.ensure("share/gdal/2.0/pcs.csv")
+    tmpdir.ensure("share/gdal/2.1/pcs.csv")
+    tmpdir.ensure("share/gdal/2.2/pcs.csv")
+    tmpdir.ensure("share/gdal/2.3/pcs.csv")
+    tmpdir.ensure("share/gdal/2.4/pcs.csv")
+    tmpdir.ensure("share/proj/epsg")
+    return tmpdir
+
+
+def test_search_wheel_gdal_data_failure(tmpdir):
+    """Fail to find GDAL data in a non-wheel"""
+    finder = GDALDataFinder()
+    assert not finder.search_wheel(str(tmpdir))
+
+
+def test_search_wheel_gdal_data(mock_wheel):
+    """Find GDAL data in a wheel"""
+    finder = GDALDataFinder()
+    assert finder.search_wheel(str(mock_wheel.join("_env.py"))) == str(mock_wheel.join("gdal_data"))
+
+
+def test_search_prefix_gdal_data_failure(tmpdir):
+    """Fail to find GDAL data in a bogus prefix"""
+    finder = GDALDataFinder()
+    assert not finder.search_prefix(str(tmpdir))
+
+
+def test_search_prefix_gdal_data(mock_fhs):
+    """Find GDAL data under prefix"""
+    finder = GDALDataFinder()
+    assert finder.search_prefix(str(mock_fhs)) == str(mock_fhs.join("share/gdal"))
+
+
+def test_search_debian_gdal_data_failure(tmpdir):
+    """Fail to find GDAL data in a bogus Debian location"""
+    finder = GDALDataFinder()
+    assert not finder.search_debian(str(tmpdir))
+
+
+def test_search_debian_gdal_data(mock_debian):
+    """Find GDAL data under Debian locations"""
+    finder = GDALDataFinder()
+    assert finder.search_debian(str(mock_debian)) == str(mock_debian.join("share/gdal/{}".format(str(gdal_version))))
+
+
+def test_search_gdal_data_wheel(mock_wheel):
+    finder = GDALDataFinder()
+    assert finder.search(str(mock_wheel.join("_env.py"))) == str(mock_wheel.join("gdal_data"))
+
+
+def test_search_gdal_data_fhs(mock_fhs):
+    finder = GDALDataFinder()
+    assert finder.search(str(mock_fhs)) == str(mock_fhs.join("share/gdal"))
+
+
+def test_search_gdal_data_debian(mock_debian):
+    """Find GDAL data under Debian locations"""
+    finder = GDALDataFinder()
+    assert finder.search(str(mock_debian)) == str(mock_debian.join("share/gdal/{}".format(str(gdal_version))))
+
+
+def test_search_wheel_proj_data_failure(tmpdir):
+    """Fail to find GDAL data in a non-wheel"""
+    finder = PROJDataFinder()
+    assert not finder.search_wheel(str(tmpdir))
+
+
+def test_search_wheel_proj_data(mock_wheel):
+    """Find GDAL data in a wheel"""
+    finder = PROJDataFinder()
+    assert finder.search_wheel(str(mock_wheel.join("_env.py"))) == str(mock_wheel.join("proj_data"))
+
+
+def test_search_prefix_proj_data_failure(tmpdir):
+    """Fail to find GDAL data in a bogus prefix"""
+    finder = PROJDataFinder()
+    assert not finder.search_prefix(str(tmpdir))
+
+
+def test_search_prefix_proj_data(mock_fhs):
+    """Find GDAL data under prefix"""
+    finder = PROJDataFinder()
+    assert finder.search_prefix(str(mock_fhs)) == str(mock_fhs.join("share/proj"))
+
+
+def test_search_proj_data_wheel(mock_wheel):
+    finder = PROJDataFinder()
+    assert finder.search(str(mock_wheel.join("_env.py"))) == str(mock_wheel.join("proj_data"))
+
+
+def test_search_proj_data_fhs(mock_fhs):
+    finder = PROJDataFinder()
+    assert finder.search(str(mock_fhs)) == str(mock_fhs.join("share/proj"))
diff --git a/tests/test_crs.py b/tests/test_crs.py
--- a/tests/test_crs.py
+++ b/tests/test_crs.py
@@ -24,6 +24,22 @@ def test_read_epsg(tmpdir):
         assert src.crs.to_dict() == {'init': 'epsg:32618'}
 
 
+def test_read_esri_wkt(tmpdir):
+    with rasterio.open('tests/data/test_esri_wkt.tif') as src:
+        assert src.crs.to_dict() == {
+            'datum': 'NAD83',
+            'lat_0': 23,
+            'lat_1': 29.5,
+            'lat_2': 45.5,
+            'lon_0': -96,
+            'no_defs': True,
+            'proj': 'aea',
+            'units': 'm',
+            'x_0': 0,
+            'y_0': 0,
+        }
+
+
 def test_read_epsg3857(tmpdir):
     tiffname = str(tmpdir.join('lol.tif'))
     subprocess.call([
@@ -301,3 +317,13 @@ def test_from_esri_wkt():
     assert proj_crs_str.to_string() == \
         ("+datum=NAD83 +lat_0=23 +lat_1=29.5 +lat_2=45.5 "
          "+lon_0=-96 +no_defs +proj=aea +units=m +x_0=0 +y_0=0")
+
+
+def test_compound_crs():
+    wkt = """COMPD_CS["unknown",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],VERT_CS["unknown",VERT_DATUM["unknown",2005],UNIT["metre",1.0,AUTHORITY["EPSG","9001"]],AXIS["Up",UP]]]"""
+    assert CRS.from_wkt(wkt).wkt.startswith('GEOGCS["WGS 84"')
+
+
+def test_dataset_compound_crs():
+    with rasterio.open("tests/data/compdcs.vrt") as dataset:
+        assert dataset.crs.wkt.startswith('GEOGCS["WGS 84"')
diff --git a/tests/test_env.py b/tests/test_env.py
--- a/tests/test_env.py
+++ b/tests/test_env.py
@@ -2,12 +2,14 @@
 # Collected here to make them easier to skip/xfail.
 
 import logging
+import os
 import sys
 
 import boto3
 import pytest
 
 import rasterio
+from rasterio import _env
 from rasterio._env import del_gdal_config, get_gdal_config, set_gdal_config
 from rasterio.env import Env, defenv, delenv, getenv, setenv, ensure_env, ensure_env_credentialled
 from rasterio.env import GDALVersion, require_gdal_version
@@ -112,6 +114,42 @@ def f():
     assert f() is True
 
 
+def test_ensure_env_decorator_sets_gdal_data(gdalenv, monkeypatch):
+    """ensure_env finds GDAL from environment"""
+    @ensure_env
+    def f():
+        return getenv()['GDAL_DATA']
+
+    monkeypatch.setenv('GDAL_DATA', '/lol/wut')
+    assert f() == '/lol/wut'
+
+
+def test_ensure_env_decorator_sets_gdal_data_prefix(gdalenv, monkeypatch, tmpdir):
+    """ensure_env finds GDAL data under a prefix"""
+    @ensure_env
+    def f():
+        return getenv()['GDAL_DATA']
+
+    tmpdir.ensure("share/gdal/pcs.csv")
+    monkeypatch.delenv('GDAL_DATA', raising=False)
+    monkeypatch.setattr(sys, 'prefix', str(tmpdir))
+
+    assert f() == str(tmpdir.join("share/gdal"))
+
+
+def test_ensure_env_decorator_sets_gdal_data_wheel(gdalenv, monkeypatch, tmpdir):
+    """ensure_env finds GDAL data in a wheel"""
+    @ensure_env
+    def f():
+        return getenv()['GDAL_DATA']
+
+    tmpdir.ensure("gdal_data/pcs.csv")
+    monkeypatch.delenv('GDAL_DATA', raising=False)
+    monkeypatch.setattr(_env, '__file__', str(tmpdir.join(os.path.basename(_env.__file__))))
+
+    assert f() == str(tmpdir.join("gdal_data"))
+
+
 def test_ensure_env_credentialled_decorator(monkeypatch, gdalenv):
     """Credentialization is ensured by wrapper"""
     monkeypatch.setenv('AWS_ACCESS_KEY_ID', 'id')
diff --git a/tests/test_rio_warp.py b/tests/test_rio_warp.py
--- a/tests/test_rio_warp.py
+++ b/tests/test_rio_warp.py
@@ -46,7 +46,7 @@ def test_dst_crs_error_epsg(runner, tmpdir):
     result = runner.invoke(main_group, [
         'warp', srcname, outputname, '--dst-crs', 'EPSG:'])
     assert result.exit_code == 2
-    assert 'for dst_crs: invalid literal for int()' in result.output
+    assert "for dst_crs: Invalid CRS:" in result.output
 
 
 def test_dst_crs_error_epsg_2(runner, tmpdir):
diff --git a/tests/test_warp.py b/tests/test_warp.py
--- a/tests/test_warp.py
+++ b/tests/test_warp.py
@@ -179,6 +179,43 @@ def test_transform_bounds():
         )
 
 
+def test_transform_bounds__esri_wkt():
+    left, bottom, right, top = \
+        (-78.95864996545055, 23.564991210854686,
+         -76.57492370013823, 25.550873767433984)
+    dst_projection_string = (
+        'PROJCS["USA_Contiguous_Albers_Equal_Area_Conic_USGS_version",'
+        'GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",'
+        'SPHEROID["GRS_1980",6378137.0,298.257222101]],'
+        'PRIMEM["Greenwich",0.0],'
+        'UNIT["Degree",0.0174532925199433]],'
+        'PROJECTION["Albers"],'
+        'PARAMETER["false_easting",0.0],'
+        'PARAMETER["false_northing",0.0],'
+        'PARAMETER["central_meridian",-96.0],'
+        'PARAMETER["standard_parallel_1",29.5],'
+        'PARAMETER["standard_parallel_2",45.5],'
+        'PARAMETER["latitude_of_origin",23.0],'
+        'UNIT["Meter",1.0],'
+        'VERTCS["NAVD_1988",'
+        'VDATUM["North_American_Vertical_Datum_1988"],'
+        'PARAMETER["Vertical_Shift",0.0],'
+        'PARAMETER["Direction",1.0],UNIT["Centimeter",0.01]]]')
+    assert np.allclose(
+        transform_bounds({"init": "EPSG:4326"},
+                         dst_projection_string,
+                         left,
+                         bottom,
+                         right,
+                         top),
+        (
+            1721263.7931814701,
+            219684.49332178483,
+            2002926.56696663,
+            479360.16562217404),
+    )
+
+
 def test_transform_bounds_densify():
     # This transform is non-linear along the edges, so densification produces
     # a different result than otherwise