Repository: IAMconsortium/pyam
Pull request #243 (2019-07-08 13:01:08)

Conn auth
# Please confirm that this PR has done the following:

- [x] Tests Added
- [x] Documentation Added
- [x] Description in RELEASE_NOTES.md Added

This PR adds the ability to connect to non-public databases by providing a username and password.

TODO: 

- [x] this currently also requires knowledge of an "application" name. For example, for the SR1.5 explorer, the "short name" is `iamc15` and the "application name" is `IXSE_SR15`.  We should decide which to go with in the end. To note, utilizing the "short name" will require an additional mapping in the python code (to the best of my knowledge), which was seen as desirable to avoid. In any case, I think we should limit the user to needing to know **only** one name.
- [x] decide on how to pass over credentials. Options: 1) leave as is and it is up to the user to store them properly. 2) support reading/writing from a config file. 3) support passing over env variables. 4) both 2 & 3
- [x] add functionality to check success of requests (likely as a context manager)


coveralls: 
[![Coverage Status](https://coveralls.io/builds/24425204/badge)](https://coveralls.io/builds/24425204)

Coverage increased (+0.06%) to 92.077% when pulling **10b6d0fe5a75617a0456b4343d351c7039e0cee2 on conn-auth** into **75e9d03bfdddf7afd641782486ea72aebccd4799 on master**.

gidden: @byers if you want to get started working on AR6, you can use this branch to do so. You'll need to coordinate the naming conventions with @zikolach and @fonfon 
fonfon: For the application names I'd prefer to fetch them from the auth-server and see how that goes. If we hard-code a lookup dict here it will become tedious (or forgotten) to keep it up to date, even if the application names do not change very often.
Users would then have to provide the application name though, like listed on https://data.ene.iiasa.ac.at/auth/#/apps
gidden: Ok @fonfon, happy to make this more amenable to the existing REST API features. Let's discuss when we're both in the office/on slack.
gidden: Here is the example how users can currently interact with the `pyam.iiasa.Connection` class: https://pyam-iamc.readthedocs.io/en/stable/tutorials/iiasa_dbs.html
byersiiasa: > @byers if you want to get started working on AR6, you can use this branch to do so. You'll need to coordinate the naming conventions with @zikolach and @fonfon

thanks I'll try this today - for some reason I didn't get a notification
gidden: Note that this needs iiasa/auth_server#53 to continue further
gidden: Hi folks, from my perspective this PR is now complete and ready for final review. @fonfon @byersiiasa and/or @danielhuppmann would you mind taking a final look?
gidden: @danielhuppmann @zikolach @fonfon (and @khaeru if you're interested)

I have listed a set of TODOs in the PR summary. Could you please provide input/opinions on items 1 and 2?

Namely, for credentials, I see a lot of potential problems here. E.g., if we want to support environment variables explicitly, once they are set, a user can no longer get explicitly to public dbs as "anonymous", because any sort of default/fall-back will always see them. Config files are less problematic, but is it our job in this library to support this? A user can always write their own file and read it in however they wish...

On the naming, I have no super strong opinions and would be happy if y'all could come to a consensus that I can implement.
gidden: Hi all, in bea472e I have added the ability to pass a credentials file with a warning if a non-file based option is used. I think that should cover the issues here.
gidden: Ok all, to my eye this PR is now complete. Please let me know the resolution of point 1 between yourselves.
byersiiasa: Hi
I have just tested (limited) and seems to work well. In particular - tested it that when credentials are supplied, then more instances are available... nice!

```
# Connection with instance / creds
Connection(creds=('xxxx','XXXXX')).valid_connections
```
returns
`['IXSE_AR6', 'IXSE_SETNAV', 'IXSE_SR15', 'IXSE_SR15_QA', 'IXSE_TEST_PUBLIC']`

```
# Connection without instance / creds
 Connection()
```
returns
`['IXSE_SETNAV', 'IXSE_SR15', 'IXSE_SR15_QA', 'IXSE_TEST_PUBLIC']`


Thanks all!
gidden: Hi @danielhuppmann @zikolach and @fonfon would be great to try to come to some resolution on the point above so this can be merged. Thanks!

Patch:
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index f20332a..3cf842e 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -1,6 +1,7 @@
 
 # Next Release
 
+- [#243](https://github.com/IAMconsortium/pyam/pull/243) Update `pyam.iiasa.Connection` to support all public and private database connections. DEPRECATED: the argument 'iamc15' has been deprecated in favor of names as queryable directly from the REST API.
 - [#241](https://github.com/IAMconsortium/pyam/pull/241) Add `set_meta_from_data` feature
 - [#236](https://github.com/IAMconsortium/pyam/pull/236) Add `swap_time_for_year` method and confirm datetime column is compatible with pyam features
 
diff --git a/doc/source/tutorials/iiasa_dbs.ipynb b/doc/source/tutorials/iiasa_dbs.ipynb
index 1a70db0..c9ff449 100644
--- a/doc/source/tutorials/iiasa_dbs.ipynb
+++ b/doc/source/tutorials/iiasa_dbs.ipynb
@@ -28,15 +28,14 @@
     }
    ],
    "source": [
-    "import pyam\n",
-    "from pyam.iiasa import valid_connection_names"
+    "import pyam"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "There are currently not many available data sources, but more will be added with time"
+    "Accessing an explorer is done via a `Connection` object. By default, all public explorers can be connected to. "
    ]
   },
   {
@@ -47,7 +46,7 @@
     {
      "data": {
       "text/plain": [
-       "['iamc15']"
+       "['IXSE_SETNAV', 'IXSE_SR15', 'IXSE_SR15_QA', 'IXSE_TEST_PUBLIC']"
       ]
      },
      "execution_count": 2,
@@ -56,21 +55,45 @@
     }
    ],
    "source": [
-    "valid_connection_names()"
+    "conn = pyam.iiasa.Connection()\n",
+    "conn.valid_connections"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this example, we will be pulling data from the Special Report on 1.5C explorer. This can be done in a number of ways, for example\n",
+    "If you have additional credentials, you can supply them as well via the `creds` key-word argument:\n",
     "\n",
     "```\n",
-    "pyam.read_iiasa('iamc15')\n",
-    "pyam.read_iiasa_iamc15()\n",
+    "pyam.iiasa.Connection(creds=(<username>, <password>))\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, we will be pulling data from the Special Report on 1.5C explorer. This can be done either via the constructor:\n",
+    "\n",
+    "```\n",
+    "pyam.iiasa.Connection('IXSE_SR15')\n",
     "```\n",
     "\n",
-    "However, this would pull all available data. It is also possible to query specific subsets of data in a manner similar to `pyam.IamDataFrame.filter()`."
+    "or, if you want to query multiple databases, via the explicit `connect()` method:\n",
+    "\n",
+    "```\n",
+    "conn = pyam.iiasa.Connection()\n",
+    "conn.connect('IXSE_SR15')\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We also provide some convenience functions to shorten the amount of code you have to write. Under the hood, `read_iiasa()` is just opening an connection to a database and making a query on that data.\n",
+    "In this tutorial, we will query specific subsets of data in a manner similar to `pyam.IamDataFrame.filter()`."
    ]
   },
   {
@@ -82,14 +105,13 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "INFO:root:You are connected to the iamc15 scenario explorer. Please cite as:\n",
-      "\n",
-      "D. Huppmann, E. Kriegler, V. Krey, K. Riahi, J. Rogelj, S.K. Rose, J. Weyant, et al., IAMC 1.5C Scenario Explorer and Data hosted by IIASA. IIASA & IAMC, 2018. doi: 10.22022/SR15/08-2018.15429, url: data.ene.iiasa.ac.at/iamc-1.5c-explorer\n"
+      "INFO:root:You are connected to the IXSE_SR15 scenario explorer hosted by IIASA. If you use this data in any published format, please cite the data as provided in the explorer guidelines: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/about.\n"
      ]
     }
    ],
    "source": [
-    "df = pyam.read_iiasa_iamc15(\n",
+    "df = pyam.read_iiasa(\n",
+    "    'IXSE_SR15',\n",
     "    model='MESSAGEix*', \n",
     "    variable=['Emissions|CO2', 'Primary Energy|Coal'], \n",
     "    region='World',\n",
@@ -183,14 +205,12 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "INFO:root:You are connected to the iamc15 scenario explorer. Please cite as:\n",
-      "\n",
-      "D. Huppmann, E. Kriegler, V. Krey, K. Riahi, J. Rogelj, S.K. Rose, J. Weyant, et al., IAMC 1.5C Scenario Explorer and Data hosted by IIASA. IIASA & IAMC, 2018. doi: 10.22022/SR15/08-2018.15429, url: data.ene.iiasa.ac.at/iamc-1.5c-explorer\n"
+      "INFO:root:You are connected to the IXSE_SR15 scenario explorer hosted by IIASA. If you use this data in any published format, please cite the data as provided in the explorer guidelines: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/about.\n"
      ]
     }
    ],
    "source": [
-    "conn = pyam.iiasa.Connection('iamc15')"
+    "conn = pyam.iiasa.Connection('IXSE_SR15')"
    ]
   },
   {
@@ -283,11 +303,11 @@
     {
      "data": {
       "text/plain": [
-       "0          World\n",
-       "1         R5ASIA\n",
-       "2          R5LAM\n",
-       "3          R5MAF\n",
-       "4    R5OECD90+EU\n",
+       "0     World\n",
+       "1    R5ROWO\n",
+       "2    R5ASIA\n",
+       "3     R5LAM\n",
+       "4     R5MAF\n",
        "Name: region, dtype: object"
       ]
      },
@@ -509,7 +529,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.2"
+   "version": "3.7.3"
   }
  },
  "nbformat": 4,
diff --git a/pyam/__init__.py b/pyam/__init__.py
index 9251bb4..5300ac7 100644
--- a/pyam/__init__.py
+++ b/pyam/__init__.py
@@ -5,7 +5,7 @@ from pyam.timeseries import *
 from pyam.read_ixmp import *
 from pyam.logger import *
 from pyam.run_control import *
-from pyam.iiasa import read_iiasa, read_iiasa_iamc15  # noqa: F401
+from pyam.iiasa import read_iiasa  # noqa: F401
 
 # in Jupyter notebooks: disable autoscroll, activate warnings
 try:
diff --git a/pyam/iiasa.py b/pyam/iiasa.py
index 8807bce..3184027 100644
--- a/pyam/iiasa.py
+++ b/pyam/iiasa.py
@@ -1,6 +1,10 @@
+import collections
 import json
 import logging
+import os
 import requests
+import warnings
+import yaml
 
 import numpy as np
 import pandas as pd
@@ -17,51 +21,129 @@ from pyam.utils import META_IDX, islistable, isstr, pattern_match
 # quiet this fool
 logging.getLogger('requests').setLevel(logging.WARNING)
 
-_URL_TEMPLATE = 'https://db1.ene.iiasa.ac.at/{}-api/rest/v2.1/'
-_AUTH_URL = 'https://db1.ene.iiasa.ac.at/EneAuth/config/v1/anonym'
+_BASE_URL = 'https://db1.ene.iiasa.ac.at/EneAuth/config/v1'
+_CITE_MSG = """
+You are connected to the {} scenario explorer hosted by IIASA.
+ If you use this data in any published format, please cite the
+ data as provided in the explorer guidelines: {}.
+""".replace('\n', '')
 
-_CITATIONS = {
-    'iamc15': 'D. Huppmann, E. Kriegler, V. Krey, K. Riahi, '
-    'J. Rogelj, S.K. Rose, J. Weyant, et al., '
-    'IAMC 1.5C Scenario Explorer and Data hosted by IIASA. '
-    'IIASA & IAMC, 2018. '
-    'doi: 10.22022/SR15/08-2018.15429, '
-    'url: data.ene.iiasa.ac.at/iamc-1.5c-explorer'
-}
 
+def _check_response(r, msg='Trouble with request', error=RuntimeError):
+    if not r.ok:
+        raise error('{}: {}'.format(msg, str(r)))
 
-def valid_connection_names():
-    return list(_CITATIONS.keys())
+
+def _get_token(creds):
+    if creds is None:  # get anonymous auth
+        url = '/'.join([_BASE_URL, 'anonym'])
+        r = requests.get(url)
+        _check_response(r, 'Could not get anonymous token')
+        return r.json()
+
+    # otherwise read creds and try to login
+    filecreds = False
+    try:
+        if isinstance(creds, collections.Mapping):
+            user, pw = creds['username'], creds['password']
+        elif os.path.exists(str(creds)):
+            with open(str(creds), 'r') as stream:
+                creds = yaml.safe_load(stream)
+            user, pw = creds['username'], creds['password']
+            filecreds = True
+        else:
+            user, pw = creds
+    except Exception as e:
+        msg = 'Could not read credentials: {}\n{}'.format(
+            creds, str(e))
+        raise type(e)(msg)
+    if not filecreds:
+        warnings.warn('You provided credentials in plain text. DO NOT save ' +
+                      'these in a repository or otherwise post them online')
+
+    headers = {'Accept': 'application/json',
+               'Content-Type': 'application/json'}
+    data = {'username': user, 'password': pw}
+    url = '/'.join([_BASE_URL, 'login'])
+    r = requests.post(url, headers=headers, data=json.dumps(data))
+    _check_response(r, 'Login failed for user: {}'.format(user))
+    return r.json()
 
 
 class Connection(object):
     """A class to facilitate querying an IIASA scenario explorer database"""
 
-    def __init__(self, name):
+    def __init__(self, name=None, creds=None):
         """
         Parameters
         ----------
-        name : str
+        name : str, optional
             A valid database name. For available options, see
             valid_connection_names().
+        creds : str, list-like, or dict, optional
+            Either:
+              - a yaml filename/path with entries of  'username' and 'password'
+                (preferred)
+              - an ordered container (tuple, list, etc.) with the same values
+              - a dictionary with the same keys
         """
-        valid = valid_connection_names()
+        self._token = _get_token(creds)
+
+        # connect if provided a name
+        self._connected = None
+        if name:
+            self.connect(name)
+
+    @property
+    @lru_cache()
+    def valid_connections(self):
+        url = '/'.join([_BASE_URL, 'applications'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
+        r = requests.get(url, headers=headers)
+        _check_response(r, 'Could not get valid connection list')
+        valid = [x['name'] for x in r.json()]
+        return valid
+
+    def connect(self, name):
+        # TODO: deprecate in next release
+        if name == 'iamc15':
+            warnings.warn(
+                'The name `iamc15` is deprecated and will be removed in the ' +
+                'next release. Please use `IXSE_SR15`.'
+            )
+            name = 'IXSE_SR15'
+
+        valid = self.valid_connections
+        if len(valid) == 0:
+            raise RuntimeError(
+                'No valid connections found for the provided credentials.'
+            )
+
         if name not in valid:
-            raise ValueError('{} is not a valid name. Choose one of {}'.format(
-                name, valid))
+            msg = """
+            {} not recognized as a valid connection name.
+            Choose from one of the supported connections for your user: {}.
+            """
+            raise ValueError(msg.format(name, valid))
+
+        url = '/'.join([_BASE_URL, 'applications', name, 'config'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
+        r = requests.get(url, headers=headers)
+        _check_response(r, 'Could not get application information')
+        response = r.json()
+        idxs = {x['path']: i for i, x in enumerate(response)}
 
-        logger().info(
-            'You are connected to the {} {}. Please cite as:\n\n{}'
-            .format(name, 'scenario explorer', _CITATIONS[name])
-        )
+        self._base_url = response[idxs['baseUrl']]['value']
+        # TODO: request the full citation to be added to this metadata intead
+        # of linking to the about page
+        about = '/'.join([response[idxs['uiUrl']]['value'], '#', 'about'])
+        logger().info(_CITE_MSG.format(name, about))
 
-        self.base_url = _URL_TEMPLATE.format(name)
+        self._connected = name
 
-    @lru_cache()
-    def auth(self):
-        """Anonymous user authentication token"""
-        r = requests.get(_AUTH_URL)
-        return r.json()
+    @property
+    def current_connection(self):
+        return self._connected
 
     @lru_cache()
     def scenario_list(self, default=True):
@@ -78,9 +160,10 @@ class Connection(object):
         """
         default = 'true' if default else 'false'
         add_url = 'runs?getOnlyDefaultRuns={}'
-        url = self.base_url + add_url.format(default)
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, add_url.format(default)])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r, 'Could not get scenario list')
         return pd.read_json(r.content, orient='records')
 
     @lru_cache()
@@ -89,9 +172,10 @@ class Connection(object):
         List all scenario metadata indicators available in the connected
         data source
         """
-        url = self.base_url + 'metadata/types'
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, 'metadata/types'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         return pd.read_json(r.content, orient='records')['name']
 
     @lru_cache()
@@ -110,9 +194,10 @@ class Connection(object):
         # up in the future to try to query a subset
         default = 'true' if default else 'false'
         add_url = 'runs?getOnlyDefaultRuns={}&includeMetadata=true'
-        url = self.base_url + add_url.format(default)
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, add_url.format(default)])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         df = pd.read_json(r.content, orient='records')
 
         def extract(row):
@@ -140,18 +225,20 @@ class Connection(object):
     @lru_cache()
     def variables(self):
         """All variables in the connected data source"""
-        url = self.base_url + 'ts'
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, 'ts'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         df = pd.read_json(r.content, orient='records')
         return pd.Series(df['variable'].unique(), name='variable')
 
     @lru_cache()
     def regions(self):
         """All regions in the connected data source"""
-        url = self.base_url + 'nodes?hierarchy=%2A'
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, 'nodes?hierarchy=%2A'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         df = pd.read_json(r.content, orient='records')
         return pd.Series(df['name'].unique(), name='region')
 
@@ -225,12 +312,13 @@ class Connection(object):
         ```
         """
         headers = {
-            'Authorization': 'Bearer {}'.format(self.auth()),
+            'Authorization': 'Bearer {}'.format(self._token),
             'Content-Type': 'application/json',
         }
         data = json.dumps(self._query_post_data(**kwargs))
-        url = self.base_url + 'runs/bulk/ts'
+        url = '/'.join([self._base_url, 'runs/bulk/ts'])
         r = requests.post(url, headers=headers, data=data)
+        _check_response(r)
         # refactor returned json object to be castable to an IamDataFrame
         df = (
             pd.read_json(r.content, orient='records')
@@ -284,11 +372,3 @@ def read_iiasa(name, meta=False, **kwargs):
         for col in mdf:
             df.set_meta(mdf[col])
     return df
-
-
-def read_iiasa_iamc15(**kwargs):
-    """
-    Query the IAMC 1.5C Scenario Explorer.
-    See Connection.query() for more documentation
-    """
-    return read_iiasa('iamc15', **kwargs)


Test patch:
diff --git a/tests/test_iiasa.py b/tests/test_iiasa.py
index 8501405..d148d13 100644
--- a/tests/test_iiasa.py
+++ b/tests/test_iiasa.py
@@ -1,41 +1,90 @@
 import copy
 import pytest
+import os
+import yaml
 
 import numpy.testing as npt
 
 from pyam import iiasa
 
+# check to see if we can do online testing of db authentication
+TEST_ENV_USER = 'IIASA_CONN_TEST_USER'
+TEST_ENV_PW = 'IIASA_CONN_TEST_PW'
+CONN_ENV_AVAILABLE = TEST_ENV_USER in os.environ and TEST_ENV_PW in os.environ
+CONN_ENV_REASON = 'Requires env variables defined: {} and {}'.format(
+    TEST_ENV_USER, TEST_ENV_PW
+)
 
-def test_auth():
+
+def test_anon_conn():
+    conn = iiasa.Connection('IXSE_SR15')
+    assert conn.current_connection == 'IXSE_SR15'
+
+
+def test_anon_conn_warning():
     conn = iiasa.Connection('iamc15')
-    assert conn.base_url == 'https://db1.ene.iiasa.ac.at/iamc15-api/rest/v2.1/'
-    conn.auth()
+    assert conn.current_connection == 'IXSE_SR15'
+
+
+@pytest.mark.skipif(not CONN_ENV_AVAILABLE, reason=CONN_ENV_REASON)
+def test_conn_creds_file(tmp_path):
+    user, pw = os.environ[TEST_ENV_USER], os.environ[TEST_ENV_PW]
+    path = tmp_path / 'config.yaml'
+    with open(path, 'w') as f:
+        yaml.dump({'username': user, 'password': pw}, f)
+    conn = iiasa.Connection('IXSE_SR15', creds=path)
+    assert conn.current_connection == 'IXSE_SR15'
+
 
+@pytest.mark.skipif(not CONN_ENV_AVAILABLE, reason=CONN_ENV_REASON)
+def test_conn_creds_tuple():
+    user, pw = os.environ[TEST_ENV_USER], os.environ[TEST_ENV_PW]
+    conn = iiasa.Connection('IXSE_SR15', creds=(user, pw))
+    assert conn.current_connection == 'IXSE_SR15'
 
-def test_connection_raises():
+
+def test_conn_bad_creds():
+    pytest.raises(RuntimeError, iiasa.Connection,
+                  'IXSE_SR15', creds=('_foo', '_bar'))
+
+
+def test_anon_conn_tuple_raises():
     pytest.raises(ValueError, iiasa.Connection, 'foo')
 
 
+@pytest.mark.skipif(not CONN_ENV_AVAILABLE, reason=CONN_ENV_REASON)
+def test_conn_creds_dict():
+    user, pw = os.environ[TEST_ENV_USER], os.environ[TEST_ENV_PW]
+    conn = iiasa.Connection(
+        'IXSE_SR15', creds={'username': user, 'password': pw})
+    assert conn.current_connection == 'IXSE_SR15'
+
+
+def test_conn_creds_dict_raises():
+    pytest.raises(KeyError, iiasa.Connection,
+                  'IXSE_SR15', creds={'username': 'foo'})
+
+
 def test_variables():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.variables().values
     assert 'Emissions|CO2' in obs
 
 
 def test_regions():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.regions().values
     assert 'World' in obs
 
 
 def test_metadata():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.scenario_list()['model'].values
     assert 'MESSAGEix-GLOBIOM 1.0' in obs
 
 
 def test_available_indicators():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.available_metadata()
     assert 'carbon price|2050' in list(obs)
 
@@ -49,11 +98,13 @@ QUERY_DATA_EXP = {
         "units": [],
         "times": []
     }
+
+
 }
 
 
 def test_query_data_model_scen():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn._query_post_data(model='AIM*', scenario='ADVANCE_2020_Med2C')
     exp = copy.deepcopy(QUERY_DATA_EXP)
     exp['filters']['runs'] = [2]
@@ -61,7 +112,7 @@ def test_query_data_model_scen():
 
 
 def test_query_data_region():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn._query_post_data(model='AIM*', scenario='ADVANCE_2020_Med2C',
                                 region='*World*')
     exp = copy.deepcopy(QUERY_DATA_EXP)
@@ -71,7 +122,7 @@ def test_query_data_region():
 
 
 def test_query_data_variables():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn._query_post_data(model='AIM*', scenario='ADVANCE_2020_Med2C',
                                 variable='Emissions|CO2*')
     exp = copy.deepcopy(QUERY_DATA_EXP)
@@ -97,19 +148,23 @@ def test_query_data_variables():
         npt.assert_array_equal(obs['filters'][k], exp['filters'][k])
 
 
-def test_query_iamc15():
-    df = iiasa.read_iiasa_iamc15(model='AIM*', scenario='ADVANCE_2020_Med2C',
-                                 variable='Emissions|CO2', region='World')
+def test_query_IXSE_SR15():
+    df = iiasa.read_iiasa('IXSE_SR15',
+                          model='AIM*',
+                          scenario='ADVANCE_2020_Med2C',
+                          variable='Emissions|CO2',
+                          region='World',
+                          )
     assert len(df) == 20
 
 
-def test_query_iamc15_with_metadata():
-    df = iiasa.read_iiasa_iamc15(
-        model='MESSAGEix*',
-        variable=['Emissions|CO2', 'Primary Energy|Coal'],
-        region='World',
-        meta=['carbon price|2100 (NPV)', 'category'],
-    )
+def test_query_IXSE_SR15_with_metadata():
+    df = iiasa.read_iiasa('IXSE_SR15',
+                          model='MESSAGEix*',
+                          variable=['Emissions|CO2', 'Primary Energy|Coal'],
+                          region='World',
+                          meta=['carbon price|2100 (NPV)', 'category'],
+                          )
     assert len(df) == 168
     assert len(df.data) == 168
     assert len(df.meta) == 7
diff --git a/tests/test_plotting.py b/tests/test_plotting.py
index 7d14421..83a382f 100644
--- a/tests/test_plotting.py
+++ b/tests/test_plotting.py
@@ -22,6 +22,13 @@ try:
 except ImportError:
     has_cartopy = False
 
+try:
+    import geopandas  # noqa: F401
+    has_geopandas = True
+except ImportError:
+    has_geopandas = False
+
+
 import matplotlib.pyplot as plt
 
 from contextlib import contextmanager
@@ -224,6 +231,7 @@ def test_line_plot_2_vars(plot_df):
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_iso_data.csv'))
@@ -239,6 +247,7 @@ def test_region():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_cbar():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_iso_data.csv'))
@@ -254,6 +263,7 @@ def test_region_cbar():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_cbar_args():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_iso_data.csv'))
@@ -269,6 +279,7 @@ def test_region_cbar_args():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_vmin_vmax():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_iso_data.csv'))
@@ -286,6 +297,7 @@ def test_region_vmin_vmax():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_cmap():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_iso_data.csv'))
@@ -302,6 +314,7 @@ def test_region_cmap():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_crs():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_iso_data.csv'))
@@ -318,6 +331,7 @@ def test_region_crs():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_map_regions():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_region_data.csv'))
@@ -334,6 +348,7 @@ def test_region_map_regions():
 
 
 @pytest.mark.skipif(not has_cartopy, reason="requires cartopy")
+@pytest.mark.skipif(not has_geopandas, reason="requires geopandas")
 @pytest.mark.mpl_image_compare(**MPL_KWARGS)
 def test_region_map_regions_legend():
     df = IamDataFrame(os.path.join(TEST_DATA_DIR, 'plot_region_data.csv'))