Repository: IAMconsortium/pyam
Pull request #247 (2019-07-22 09:22:20)

Fix required version as tests are failing with pandas v0.23.4 installed

coveralls: 
[![Coverage Status](https://coveralls.io/builds/24712800/badge)](https://coveralls.io/builds/24712800)

Coverage remained the same at 90.846% when pulling **af284a91a80ea6eb4a54d809c4073f8ecab851d3 on zikolach:hotfix/pandas-23-incompatibility** into **a134b46f7dd03b2e95e09c7770a388dd7ac29c37 on IAMconsortium:master**.

gidden: hi @zikolach out of curiosity, what is the issue with 0.23.4? Also, if we want to exclude a specific version, better to do so explicitly than require a new one, e.g.,: `pandas!=0.23.4`
zikolach: @gidden probably that's just thing you addressed in one of the latest hotfixes, just not tested against older versions of pandas.

```
________________________________________________________________________________________ test_set_meta_from_data_method_other_column[meta_df0] ________________________________________________________________________________________

meta_df = <pyam.core.IamDataFrame object at 0x113f017b8>

    def test_set_meta_from_data_method_other_column(meta_df):
        if 'year' in meta_df.data.columns:
            col, value = 'year', 2010
        else:
            col, value = 'time', pd.to_datetime('2010-07-21T00:00:00.0')
        meta_df.set_meta_from_data('pe_max_yr', variable='Primary Energy',
                                   method=np.max, column=col)
        exp = pd.Series(data=[value] * 2, index=EXP_IDX, name='pe_max_yr')
>       pd.testing.assert_series_equal(meta_df['pe_max_yr'], exp)

tests/test_feature_set_meta.py:137: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv/lib/python3.7/site-packages/pandas/util/testing.py:1210: in assert_series_equal
    assert_attr_equal('dtype', left, right)
venv/lib/python3.7/site-packages/pandas/util/testing.py:941: in assert_attr_equal
    raise_assert_detail(obj, msg, left_attr, right_attr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

obj = 'Attributes', message = 'Attribute "dtype" are different', left = dtype('float64'), right = dtype('int64'), diff = None

    def raise_assert_detail(obj, message, left, right, diff=None):
        if isinstance(left, np.ndarray):
            left = pprint_thing(left)
        elif is_categorical_dtype(left):
            left = repr(left)
    
        if PY2 and isinstance(left, string_types):
            # left needs to be printable in native text type in python2
            left = left.encode('utf-8')
    
        if isinstance(right, np.ndarray):
            right = pprint_thing(right)
        elif is_categorical_dtype(right):
            right = repr(right)
    
        if PY2 and isinstance(right, string_types):
            # right needs to be printable in native text type in python2
            right = right.encode('utf-8')
    
        msg = """{obj} are different
    
    {message}
    [left]:  {left}
    [right]: {right}""".format(obj=obj, message=message, left=left, right=right)
    
        if diff is not None:
            msg += "\n[diff]: {diff}".format(diff=diff)
    
>       raise AssertionError(msg)
E       AssertionError: Attributes are different
E       
E       Attribute "dtype" are different
E       [left]:  float64
E       [right]: int64

venv/lib/python3.7/site-packages/pandas/util/testing.py:1035: AssertionError

```
zikolach: @gidden thanks for suggestion. Just set `pandas==0.23.4`, waiting for tests results
zikolach: @gidden now builds failed, see e.g. [similar issue I mentioned above](https://ci.appveyor.com/project/gidden/pyam/builds/26306314/job/lhy0p0hol4nnkt92#L1585). Should we try another `0.23.*` versions?
gidden: Hey @zikolach, I think this is up to your personal preference. You're welcome to walk the version list to try to find all incompatible versions. Another option is just to say that we require `pandas>0.23` which I think will address your issue while also minimizing our dependency restrictions.
gidden: Apologies, this line should instead be `pandas>0.24.0`
zikolach: > Apologies, this line should instead be `pandas>0.24.0`

I wanted to check compatibility with exact version `0.24.1` (missed second `=`)
zikolach: @gidden could you please check it again. @zikolach thanks for suggestion with versions and PEP-440 reference.

Patch:
diff --git a/.gitignore b/.gitignore
index c14f6d6..fb3ca36 100644
--- a/.gitignore
+++ b/.gitignore
@@ -114,3 +114,7 @@ ENV/
 
 # pytest
 .pytest_cache/
+
+# idea
+.idea/
+
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index f20332a..3cf842e 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -1,6 +1,7 @@
 
 # Next Release
 
+- [#243](https://github.com/IAMconsortium/pyam/pull/243) Update `pyam.iiasa.Connection` to support all public and private database connections. DEPRECATED: the argument 'iamc15' has been deprecated in favor of names as queryable directly from the REST API.
 - [#241](https://github.com/IAMconsortium/pyam/pull/241) Add `set_meta_from_data` feature
 - [#236](https://github.com/IAMconsortium/pyam/pull/236) Add `swap_time_for_year` method and confirm datetime column is compatible with pyam features
 
diff --git a/doc/source/tutorials/iiasa_dbs.ipynb b/doc/source/tutorials/iiasa_dbs.ipynb
index 1a70db0..c9ff449 100644
--- a/doc/source/tutorials/iiasa_dbs.ipynb
+++ b/doc/source/tutorials/iiasa_dbs.ipynb
@@ -28,15 +28,14 @@
     }
    ],
    "source": [
-    "import pyam\n",
-    "from pyam.iiasa import valid_connection_names"
+    "import pyam"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "There are currently not many available data sources, but more will be added with time"
+    "Accessing an explorer is done via a `Connection` object. By default, all public explorers can be connected to. "
    ]
   },
   {
@@ -47,7 +46,7 @@
     {
      "data": {
       "text/plain": [
-       "['iamc15']"
+       "['IXSE_SETNAV', 'IXSE_SR15', 'IXSE_SR15_QA', 'IXSE_TEST_PUBLIC']"
       ]
      },
      "execution_count": 2,
@@ -56,21 +55,45 @@
     }
    ],
    "source": [
-    "valid_connection_names()"
+    "conn = pyam.iiasa.Connection()\n",
+    "conn.valid_connections"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this example, we will be pulling data from the Special Report on 1.5C explorer. This can be done in a number of ways, for example\n",
+    "If you have additional credentials, you can supply them as well via the `creds` key-word argument:\n",
     "\n",
     "```\n",
-    "pyam.read_iiasa('iamc15')\n",
-    "pyam.read_iiasa_iamc15()\n",
+    "pyam.iiasa.Connection(creds=(<username>, <password>))\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, we will be pulling data from the Special Report on 1.5C explorer. This can be done either via the constructor:\n",
+    "\n",
+    "```\n",
+    "pyam.iiasa.Connection('IXSE_SR15')\n",
     "```\n",
     "\n",
-    "However, this would pull all available data. It is also possible to query specific subsets of data in a manner similar to `pyam.IamDataFrame.filter()`."
+    "or, if you want to query multiple databases, via the explicit `connect()` method:\n",
+    "\n",
+    "```\n",
+    "conn = pyam.iiasa.Connection()\n",
+    "conn.connect('IXSE_SR15')\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We also provide some convenience functions to shorten the amount of code you have to write. Under the hood, `read_iiasa()` is just opening an connection to a database and making a query on that data.\n",
+    "In this tutorial, we will query specific subsets of data in a manner similar to `pyam.IamDataFrame.filter()`."
    ]
   },
   {
@@ -82,14 +105,13 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "INFO:root:You are connected to the iamc15 scenario explorer. Please cite as:\n",
-      "\n",
-      "D. Huppmann, E. Kriegler, V. Krey, K. Riahi, J. Rogelj, S.K. Rose, J. Weyant, et al., IAMC 1.5C Scenario Explorer and Data hosted by IIASA. IIASA & IAMC, 2018. doi: 10.22022/SR15/08-2018.15429, url: data.ene.iiasa.ac.at/iamc-1.5c-explorer\n"
+      "INFO:root:You are connected to the IXSE_SR15 scenario explorer hosted by IIASA. If you use this data in any published format, please cite the data as provided in the explorer guidelines: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/about.\n"
      ]
     }
    ],
    "source": [
-    "df = pyam.read_iiasa_iamc15(\n",
+    "df = pyam.read_iiasa(\n",
+    "    'IXSE_SR15',\n",
     "    model='MESSAGEix*', \n",
     "    variable=['Emissions|CO2', 'Primary Energy|Coal'], \n",
     "    region='World',\n",
@@ -183,14 +205,12 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "INFO:root:You are connected to the iamc15 scenario explorer. Please cite as:\n",
-      "\n",
-      "D. Huppmann, E. Kriegler, V. Krey, K. Riahi, J. Rogelj, S.K. Rose, J. Weyant, et al., IAMC 1.5C Scenario Explorer and Data hosted by IIASA. IIASA & IAMC, 2018. doi: 10.22022/SR15/08-2018.15429, url: data.ene.iiasa.ac.at/iamc-1.5c-explorer\n"
+      "INFO:root:You are connected to the IXSE_SR15 scenario explorer hosted by IIASA. If you use this data in any published format, please cite the data as provided in the explorer guidelines: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/about.\n"
      ]
     }
    ],
    "source": [
-    "conn = pyam.iiasa.Connection('iamc15')"
+    "conn = pyam.iiasa.Connection('IXSE_SR15')"
    ]
   },
   {
@@ -283,11 +303,11 @@
     {
      "data": {
       "text/plain": [
-       "0          World\n",
-       "1         R5ASIA\n",
-       "2          R5LAM\n",
-       "3          R5MAF\n",
-       "4    R5OECD90+EU\n",
+       "0     World\n",
+       "1    R5ROWO\n",
+       "2    R5ASIA\n",
+       "3     R5LAM\n",
+       "4     R5MAF\n",
        "Name: region, dtype: object"
       ]
      },
@@ -509,7 +529,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.2"
+   "version": "3.7.3"
   }
  },
  "nbformat": 4,
diff --git a/pyam/__init__.py b/pyam/__init__.py
index 9251bb4..5300ac7 100644
--- a/pyam/__init__.py
+++ b/pyam/__init__.py
@@ -5,7 +5,7 @@ from pyam.timeseries import *
 from pyam.read_ixmp import *
 from pyam.logger import *
 from pyam.run_control import *
-from pyam.iiasa import read_iiasa, read_iiasa_iamc15  # noqa: F401
+from pyam.iiasa import read_iiasa  # noqa: F401
 
 # in Jupyter notebooks: disable autoscroll, activate warnings
 try:
diff --git a/pyam/iiasa.py b/pyam/iiasa.py
index 8807bce..3184027 100644
--- a/pyam/iiasa.py
+++ b/pyam/iiasa.py
@@ -1,6 +1,10 @@
+import collections
 import json
 import logging
+import os
 import requests
+import warnings
+import yaml
 
 import numpy as np
 import pandas as pd
@@ -17,51 +21,129 @@ from pyam.utils import META_IDX, islistable, isstr, pattern_match
 # quiet this fool
 logging.getLogger('requests').setLevel(logging.WARNING)
 
-_URL_TEMPLATE = 'https://db1.ene.iiasa.ac.at/{}-api/rest/v2.1/'
-_AUTH_URL = 'https://db1.ene.iiasa.ac.at/EneAuth/config/v1/anonym'
+_BASE_URL = 'https://db1.ene.iiasa.ac.at/EneAuth/config/v1'
+_CITE_MSG = """
+You are connected to the {} scenario explorer hosted by IIASA.
+ If you use this data in any published format, please cite the
+ data as provided in the explorer guidelines: {}.
+""".replace('\n', '')
 
-_CITATIONS = {
-    'iamc15': 'D. Huppmann, E. Kriegler, V. Krey, K. Riahi, '
-    'J. Rogelj, S.K. Rose, J. Weyant, et al., '
-    'IAMC 1.5C Scenario Explorer and Data hosted by IIASA. '
-    'IIASA & IAMC, 2018. '
-    'doi: 10.22022/SR15/08-2018.15429, '
-    'url: data.ene.iiasa.ac.at/iamc-1.5c-explorer'
-}
 
+def _check_response(r, msg='Trouble with request', error=RuntimeError):
+    if not r.ok:
+        raise error('{}: {}'.format(msg, str(r)))
 
-def valid_connection_names():
-    return list(_CITATIONS.keys())
+
+def _get_token(creds):
+    if creds is None:  # get anonymous auth
+        url = '/'.join([_BASE_URL, 'anonym'])
+        r = requests.get(url)
+        _check_response(r, 'Could not get anonymous token')
+        return r.json()
+
+    # otherwise read creds and try to login
+    filecreds = False
+    try:
+        if isinstance(creds, collections.Mapping):
+            user, pw = creds['username'], creds['password']
+        elif os.path.exists(str(creds)):
+            with open(str(creds), 'r') as stream:
+                creds = yaml.safe_load(stream)
+            user, pw = creds['username'], creds['password']
+            filecreds = True
+        else:
+            user, pw = creds
+    except Exception as e:
+        msg = 'Could not read credentials: {}\n{}'.format(
+            creds, str(e))
+        raise type(e)(msg)
+    if not filecreds:
+        warnings.warn('You provided credentials in plain text. DO NOT save ' +
+                      'these in a repository or otherwise post them online')
+
+    headers = {'Accept': 'application/json',
+               'Content-Type': 'application/json'}
+    data = {'username': user, 'password': pw}
+    url = '/'.join([_BASE_URL, 'login'])
+    r = requests.post(url, headers=headers, data=json.dumps(data))
+    _check_response(r, 'Login failed for user: {}'.format(user))
+    return r.json()
 
 
 class Connection(object):
     """A class to facilitate querying an IIASA scenario explorer database"""
 
-    def __init__(self, name):
+    def __init__(self, name=None, creds=None):
         """
         Parameters
         ----------
-        name : str
+        name : str, optional
             A valid database name. For available options, see
             valid_connection_names().
+        creds : str, list-like, or dict, optional
+            Either:
+              - a yaml filename/path with entries of  'username' and 'password'
+                (preferred)
+              - an ordered container (tuple, list, etc.) with the same values
+              - a dictionary with the same keys
         """
-        valid = valid_connection_names()
+        self._token = _get_token(creds)
+
+        # connect if provided a name
+        self._connected = None
+        if name:
+            self.connect(name)
+
+    @property
+    @lru_cache()
+    def valid_connections(self):
+        url = '/'.join([_BASE_URL, 'applications'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
+        r = requests.get(url, headers=headers)
+        _check_response(r, 'Could not get valid connection list')
+        valid = [x['name'] for x in r.json()]
+        return valid
+
+    def connect(self, name):
+        # TODO: deprecate in next release
+        if name == 'iamc15':
+            warnings.warn(
+                'The name `iamc15` is deprecated and will be removed in the ' +
+                'next release. Please use `IXSE_SR15`.'
+            )
+            name = 'IXSE_SR15'
+
+        valid = self.valid_connections
+        if len(valid) == 0:
+            raise RuntimeError(
+                'No valid connections found for the provided credentials.'
+            )
+
         if name not in valid:
-            raise ValueError('{} is not a valid name. Choose one of {}'.format(
-                name, valid))
+            msg = """
+            {} not recognized as a valid connection name.
+            Choose from one of the supported connections for your user: {}.
+            """
+            raise ValueError(msg.format(name, valid))
+
+        url = '/'.join([_BASE_URL, 'applications', name, 'config'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
+        r = requests.get(url, headers=headers)
+        _check_response(r, 'Could not get application information')
+        response = r.json()
+        idxs = {x['path']: i for i, x in enumerate(response)}
 
-        logger().info(
-            'You are connected to the {} {}. Please cite as:\n\n{}'
-            .format(name, 'scenario explorer', _CITATIONS[name])
-        )
+        self._base_url = response[idxs['baseUrl']]['value']
+        # TODO: request the full citation to be added to this metadata intead
+        # of linking to the about page
+        about = '/'.join([response[idxs['uiUrl']]['value'], '#', 'about'])
+        logger().info(_CITE_MSG.format(name, about))
 
-        self.base_url = _URL_TEMPLATE.format(name)
+        self._connected = name
 
-    @lru_cache()
-    def auth(self):
-        """Anonymous user authentication token"""
-        r = requests.get(_AUTH_URL)
-        return r.json()
+    @property
+    def current_connection(self):
+        return self._connected
 
     @lru_cache()
     def scenario_list(self, default=True):
@@ -78,9 +160,10 @@ class Connection(object):
         """
         default = 'true' if default else 'false'
         add_url = 'runs?getOnlyDefaultRuns={}'
-        url = self.base_url + add_url.format(default)
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, add_url.format(default)])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r, 'Could not get scenario list')
         return pd.read_json(r.content, orient='records')
 
     @lru_cache()
@@ -89,9 +172,10 @@ class Connection(object):
         List all scenario metadata indicators available in the connected
         data source
         """
-        url = self.base_url + 'metadata/types'
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, 'metadata/types'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         return pd.read_json(r.content, orient='records')['name']
 
     @lru_cache()
@@ -110,9 +194,10 @@ class Connection(object):
         # up in the future to try to query a subset
         default = 'true' if default else 'false'
         add_url = 'runs?getOnlyDefaultRuns={}&includeMetadata=true'
-        url = self.base_url + add_url.format(default)
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, add_url.format(default)])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         df = pd.read_json(r.content, orient='records')
 
         def extract(row):
@@ -140,18 +225,20 @@ class Connection(object):
     @lru_cache()
     def variables(self):
         """All variables in the connected data source"""
-        url = self.base_url + 'ts'
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, 'ts'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         df = pd.read_json(r.content, orient='records')
         return pd.Series(df['variable'].unique(), name='variable')
 
     @lru_cache()
     def regions(self):
         """All regions in the connected data source"""
-        url = self.base_url + 'nodes?hierarchy=%2A'
-        headers = {'Authorization': 'Bearer {}'.format(self.auth())}
+        url = '/'.join([self._base_url, 'nodes?hierarchy=%2A'])
+        headers = {'Authorization': 'Bearer {}'.format(self._token)}
         r = requests.get(url, headers=headers)
+        _check_response(r)
         df = pd.read_json(r.content, orient='records')
         return pd.Series(df['name'].unique(), name='region')
 
@@ -225,12 +312,13 @@ class Connection(object):
         ```
         """
         headers = {
-            'Authorization': 'Bearer {}'.format(self.auth()),
+            'Authorization': 'Bearer {}'.format(self._token),
             'Content-Type': 'application/json',
         }
         data = json.dumps(self._query_post_data(**kwargs))
-        url = self.base_url + 'runs/bulk/ts'
+        url = '/'.join([self._base_url, 'runs/bulk/ts'])
         r = requests.post(url, headers=headers, data=data)
+        _check_response(r)
         # refactor returned json object to be castable to an IamDataFrame
         df = (
             pd.read_json(r.content, orient='records')
@@ -284,11 +372,3 @@ def read_iiasa(name, meta=False, **kwargs):
         for col in mdf:
             df.set_meta(mdf[col])
     return df
-
-
-def read_iiasa_iamc15(**kwargs):
-    """
-    Query the IAMC 1.5C Scenario Explorer.
-    See Connection.query() for more documentation
-    """
-    return read_iiasa('iamc15', **kwargs)
diff --git a/setup.py b/setup.py
index 24cd32c..c59c187 100755
--- a/setup.py
+++ b/setup.py
@@ -22,7 +22,7 @@ REQUIREMENTS = [
     'argparse',
     'numpy',
     'requests',
-    'pandas>=0.21.0,!=0.24.0',
+    'pandas>0.24',
     'PyYAML',
     'xlrd',
     'xlsxwriter',


Test patch:
diff --git a/tests/test_iiasa.py b/tests/test_iiasa.py
index 8501405..d148d13 100644
--- a/tests/test_iiasa.py
+++ b/tests/test_iiasa.py
@@ -1,41 +1,90 @@
 import copy
 import pytest
+import os
+import yaml
 
 import numpy.testing as npt
 
 from pyam import iiasa
 
+# check to see if we can do online testing of db authentication
+TEST_ENV_USER = 'IIASA_CONN_TEST_USER'
+TEST_ENV_PW = 'IIASA_CONN_TEST_PW'
+CONN_ENV_AVAILABLE = TEST_ENV_USER in os.environ and TEST_ENV_PW in os.environ
+CONN_ENV_REASON = 'Requires env variables defined: {} and {}'.format(
+    TEST_ENV_USER, TEST_ENV_PW
+)
 
-def test_auth():
+
+def test_anon_conn():
+    conn = iiasa.Connection('IXSE_SR15')
+    assert conn.current_connection == 'IXSE_SR15'
+
+
+def test_anon_conn_warning():
     conn = iiasa.Connection('iamc15')
-    assert conn.base_url == 'https://db1.ene.iiasa.ac.at/iamc15-api/rest/v2.1/'
-    conn.auth()
+    assert conn.current_connection == 'IXSE_SR15'
+
+
+@pytest.mark.skipif(not CONN_ENV_AVAILABLE, reason=CONN_ENV_REASON)
+def test_conn_creds_file(tmp_path):
+    user, pw = os.environ[TEST_ENV_USER], os.environ[TEST_ENV_PW]
+    path = tmp_path / 'config.yaml'
+    with open(path, 'w') as f:
+        yaml.dump({'username': user, 'password': pw}, f)
+    conn = iiasa.Connection('IXSE_SR15', creds=path)
+    assert conn.current_connection == 'IXSE_SR15'
+
 
+@pytest.mark.skipif(not CONN_ENV_AVAILABLE, reason=CONN_ENV_REASON)
+def test_conn_creds_tuple():
+    user, pw = os.environ[TEST_ENV_USER], os.environ[TEST_ENV_PW]
+    conn = iiasa.Connection('IXSE_SR15', creds=(user, pw))
+    assert conn.current_connection == 'IXSE_SR15'
 
-def test_connection_raises():
+
+def test_conn_bad_creds():
+    pytest.raises(RuntimeError, iiasa.Connection,
+                  'IXSE_SR15', creds=('_foo', '_bar'))
+
+
+def test_anon_conn_tuple_raises():
     pytest.raises(ValueError, iiasa.Connection, 'foo')
 
 
+@pytest.mark.skipif(not CONN_ENV_AVAILABLE, reason=CONN_ENV_REASON)
+def test_conn_creds_dict():
+    user, pw = os.environ[TEST_ENV_USER], os.environ[TEST_ENV_PW]
+    conn = iiasa.Connection(
+        'IXSE_SR15', creds={'username': user, 'password': pw})
+    assert conn.current_connection == 'IXSE_SR15'
+
+
+def test_conn_creds_dict_raises():
+    pytest.raises(KeyError, iiasa.Connection,
+                  'IXSE_SR15', creds={'username': 'foo'})
+
+
 def test_variables():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.variables().values
     assert 'Emissions|CO2' in obs
 
 
 def test_regions():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.regions().values
     assert 'World' in obs
 
 
 def test_metadata():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.scenario_list()['model'].values
     assert 'MESSAGEix-GLOBIOM 1.0' in obs
 
 
 def test_available_indicators():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn.available_metadata()
     assert 'carbon price|2050' in list(obs)
 
@@ -49,11 +98,13 @@ QUERY_DATA_EXP = {
         "units": [],
         "times": []
     }
+
+
 }
 
 
 def test_query_data_model_scen():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn._query_post_data(model='AIM*', scenario='ADVANCE_2020_Med2C')
     exp = copy.deepcopy(QUERY_DATA_EXP)
     exp['filters']['runs'] = [2]
@@ -61,7 +112,7 @@ def test_query_data_model_scen():
 
 
 def test_query_data_region():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn._query_post_data(model='AIM*', scenario='ADVANCE_2020_Med2C',
                                 region='*World*')
     exp = copy.deepcopy(QUERY_DATA_EXP)
@@ -71,7 +122,7 @@ def test_query_data_region():
 
 
 def test_query_data_variables():
-    conn = iiasa.Connection('iamc15')
+    conn = iiasa.Connection('IXSE_SR15')
     obs = conn._query_post_data(model='AIM*', scenario='ADVANCE_2020_Med2C',
                                 variable='Emissions|CO2*')
     exp = copy.deepcopy(QUERY_DATA_EXP)
@@ -97,19 +148,23 @@ def test_query_data_variables():
         npt.assert_array_equal(obs['filters'][k], exp['filters'][k])
 
 
-def test_query_iamc15():
-    df = iiasa.read_iiasa_iamc15(model='AIM*', scenario='ADVANCE_2020_Med2C',
-                                 variable='Emissions|CO2', region='World')
+def test_query_IXSE_SR15():
+    df = iiasa.read_iiasa('IXSE_SR15',
+                          model='AIM*',
+                          scenario='ADVANCE_2020_Med2C',
+                          variable='Emissions|CO2',
+                          region='World',
+                          )
     assert len(df) == 20
 
 
-def test_query_iamc15_with_metadata():
-    df = iiasa.read_iiasa_iamc15(
-        model='MESSAGEix*',
-        variable=['Emissions|CO2', 'Primary Energy|Coal'],
-        region='World',
-        meta=['carbon price|2100 (NPV)', 'category'],
-    )
+def test_query_IXSE_SR15_with_metadata():
+    df = iiasa.read_iiasa('IXSE_SR15',
+                          model='MESSAGEix*',
+                          variable=['Emissions|CO2', 'Primary Energy|Coal'],
+                          region='World',
+                          meta=['carbon price|2100 (NPV)', 'category'],
+                          )
     assert len(df) == 168
     assert len(df.data) == 168
     assert len(df.meta) == 7