Repository: automl/SMAC3

Issue:
Add seed to output directory name
In order to have an overview of what has been run one would sometimes like to know the seed of previous runs. One solution would be to store it in the output directory name, but this isn't as easy as it sounds as the scenario object does not know the seed for the SMAC object.

Discussion:
codecov-io: # [Codecov](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=h1) Report
> Merging [#418](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=desc) into [development](https://codecov.io/gh/automl/SMAC3/commit/2c259a0f3f67e43ee1674efb8c85d5ac8cec6e71?src=pr&el=desc) will **increase** coverage by `0.3%`.
> The diff coverage is `95.65%`.

[![Impacted file tree graph](https://codecov.io/gh/automl/SMAC3/pull/418/graphs/tree.svg?width=650&height=150&src=pr&token=0Z9cm7CRSY)](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=tree)

```diff
@@              Coverage Diff               @@
##           development     #418     +/-   ##
==============================================
+ Coverage        88.12%   88.43%   +0.3%     
==============================================
  Files               48       48             
  Lines             3150     3251    +101     
==============================================
+ Hits              2776     2875     +99     
- Misses             374      376      +2
```


| [Impacted Files](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [smac/facade/smac\_facade.py](https://codecov.io/gh/automl/SMAC3/pull/418/diff?src=pr&el=tree#diff-c21hYy9mYWNhZGUvc21hY19mYWNhZGUucHk=) | `93.25% <95.65%> (-0.24%)` | :arrow_down: |
| [smac/intensification/intensification.py](https://codecov.io/gh/automl/SMAC3/pull/418/diff?src=pr&el=tree#diff-c21hYy9pbnRlbnNpZmljYXRpb24vaW50ZW5zaWZpY2F0aW9uLnB5) | `94.02% <0%> (+0.23%)` | :arrow_up: |
| [smac/scenario/scenario.py](https://codecov.io/gh/automl/SMAC3/pull/418/diff?src=pr&el=tree#diff-c21hYy9zY2VuYXJpby9zY2VuYXJpby5weQ==) | `94.23% <0%> (+1.79%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=footer). Last update [2c259a0...dd76fd5](https://codecov.io/gh/automl/SMAC3/pull/418?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/README.md b/README.md
index 59ed9407c..797a06a81 100644
--- a/README.md
+++ b/README.md
@@ -1,9 +1,12 @@
 # SMAC v3 Project
 
-Copyright (C) 2017  [ML4AAD Group](http://www.ml4aad.org/)
+Copyright (C) 2016-2018  [ML4AAD Group](http://www.ml4aad.org/)
 
-__Attention__: This package is under heavy development and subject to change. 
-A stable release of SMAC (v2) in Java can be found [here](http://www.cs.ubc.ca/labs/beta/Projects/SMAC/).
+__Attention__: This package is a re-implementation of the original SMAC tool
+(see reference below).
+However, the reimplementation slightly differs from the original SMAC.
+For comparisons against the original SMAC, we refer to a stable release of SMAC (v2) in Java
+which can be found [here](http://www.cs.ubc.ca/labs/beta/Projects/SMAC/).
 
 The documentation can be found [here](https://automl.github.io/SMAC3/).
 
@@ -24,7 +27,7 @@ Status for development branch
 SMAC is a tool for algorithm configuration to optimize the parameters of
 arbitrary algorithms across a set of instances. This also includes
 hyperparameter optimization of ML algorithms. The main core consists of
-Bayesian Optimization in combination with a simple racing mechanism to
+Bayesian Optimization in combination with a aggressive racing mechanism to
 efficiently decide which of two configuration performs better.
 
 For a detailed description of its main idea,
@@ -35,26 +38,41 @@ we refer to
     In: Proceedings of the conference on Learning and Intelligent OptimizatioN (LION 5)
 
 
-SMAC v3 is written in python3 and continuously tested with python3.5 and
+SMAC v3 is written in Python3 and continuously tested with python3.5 and
 python3.6. Its [Random Forest](https://github.com/automl/random_forest_run)
 is written in C++.
 
 # Installation
 
+## Requirements
+
 Besides the listed requirements (see `requirements.txt`), the random forest
 used in SMAC3 requires SWIG (>= 3.0).
 
-	apt-get install swig 
+```apt-get install swig```
+
+
+## Installation via pip
+
+SMAC3 is available on pipy.
+
+```pip install smac```
+
+## Manual Installation
+
+```
+git clone https://github.com/automl/SMAC3.git && cd SMAC3
+cat requirements.txt | xargs -n 1 -L 1 pip install
+python setup.py install
+```
+
+## Installation in Anaconda
 
-    cat requirements.txt | xargs -n 1 -L 1 pip install
-    
-    python setup.py install
-    
 If you use Anaconda as your Python environment, you have to install three
-packages before you can install SMAC:
+packages **before** you can install SMAC:
+
+```conda install gxx_linux-64 gcc_linux-64 swig```
 
-	conda install gxx_linux-64 gcc_linux-64 swig
-    
 # License
 
 This program is free software: you can redistribute it and/or modify
@@ -64,28 +82,27 @@ This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
-You should have received a copy of the 3-clause BSD license 
-along with this program (see LICENSE file). 
+You should have received a copy of the 3-clause BSD license
+along with this program (see LICENSE file).
 If not, see <https://opensource.org/licenses/BSD-3-Clause>.
 
 # USAGE
 
 The usage of SMAC v3 is mainly the same as provided with [SMAC v2.08](http://www.cs.ubc.ca/labs/beta/Projects/SMAC/v2.08.00/manual.pdf).
-It supports the same parameter configuration space syntax and interface to
-target algorithms. Please note that we do not support the extended parameter
-configuration syntax introduced in SMACv2.10.
+It supports the same parameter configuration space syntax
+(except for extended forbidden constraints) and interface to
+target algorithms.
 
 # Examples
 
 See examples/
 
   * examples/rosenbrock.py - example on how to optimize a Python function
-    (REQUIRES [PYNISHER](https://github.com/sfalkner/pynisher) )
   * examples/spear_qcp/run.sh - example on how to optimize the SAT solver Spear
     on a set of SAT formulas
- 
+
 # Contact
- 
-SMAC v3 is developed by the [ML4AAD Group of the University of Freiburg](http://www.ml4aad.org/).
+
+SMAC3 is developed by the [ML4AAD Group of the University of Freiburg](http://www.ml4aad.org/).
 
 If you found a bug, please report to https://github.com/automl/SMAC3
diff --git a/doc/options.rst b/doc/options.rst
index 88272026c..2c0383116 100644
--- a/doc/options.rst
+++ b/doc/options.rst
@@ -47,6 +47,8 @@ The Parameter Configuration Space (PCS) defines the legal ranges of the
 parameters to be optimized and their default values. In the examples-folder you
 can find several examples for PCS-files. Generally, the format is:
 
+To define parameters and their ranges, the following format is supported:
+
 .. code-block:: bash
 
         parameter_name categorical {value_1, ..., value_N} [default value]
@@ -56,14 +58,29 @@ can find several examples for PCS-files. Generally, the format is:
         parameter_name real [min_value, max_value] [default value]
         parameter_name real [min_value, max_value] [default value] log
 
+The trailing "log" indicates that SMAC should sample from the defined ranges
+on a log scale.
+
+Furthermore, conditional dependencies can be expressed. That is useful if
+a parameter activates sub-parameters. For example, only if a certain heuristic
+is used, the heuristic's parameter are active and otherwise SMAC can ignore these.
+
+.. code-block:: bash
+
         # Conditionals:
         child_name | condition [&&,||] condition ...
 
-        # Condition Operators: 
+        # Condition Operators:
         # parent_x [<, >] parent_x_value (if parameter type is ordinal, integer or real)
         # parent_x [==,!=] parent_x_value (if parameter type is categorical, ordinal or integer)
         # parent_x in {parent_x_value1, parent_x_value2,...}
 
+Forbidden constraints allow for specifications of forbidden combinations of
+parameter values. Please note that SMAC uses a simple rejection sampling
+strategy. Therefore, SMAC cannot handle efficiently highly constrained spaces.
+
+.. code-block:: bash
+
         # Forbiddens:
         {parameter_name_1=value_1, ..., parameter_name_N=value_N}
 
diff --git a/examples/rosenbrock.py b/examples/rosenbrock.py
index bf45ed486..5a023806e 100644
--- a/examples/rosenbrock.py
+++ b/examples/rosenbrock.py
@@ -27,6 +27,6 @@ x, cost, _ = fmin_smac(func=rosenbrock_2d,
                        x0=[-3, -4],
                        bounds=[(-5, 5), (-5, 5)],
                        maxfun=325,
-                       rng=3)
+                       rng=3)  # Passing a seed makes fmin_smac determistic
 
 print("Best x: %s; with cost: %f"% (str(x), cost))
diff --git a/smac/facade/smac_facade.py b/smac/facade/smac_facade.py
index 48f86fb98..518d4de5c 100644
--- a/smac/facade/smac_facade.py
+++ b/smac/facade/smac_facade.py
@@ -1,6 +1,5 @@
 import logging
 import os
-import shutil
 import typing
 
 import numpy as np
@@ -27,7 +26,7 @@ from smac.optimizer.acquisition import EI, LogEI, AbstractAcquisitionFunction
 from smac.optimizer.ei_optimization import InterleavedLocalAndRandomSearch, \
     AcquisitionFunctionMaximizer
 from smac.optimizer.random_configuration_chooser import ChooserNoCoolDown, \
-    ChooserLinearCoolDown
+    ChooserLinearCoolDown, RandomConfigurationChooser
 from smac.epm.rf_with_instances import RandomForestWithInstances
 from smac.epm.rfr_imputator import RFRImputator
 from smac.epm.base_epm import AbstractEPM
@@ -59,21 +58,21 @@ class SMAC(object):
 
     def __init__(self,
                  scenario: Scenario,
-                 tae_runner: typing.Union[ExecuteTARun, typing.Callable]=None,
-                 runhistory: RunHistory=None,
-                 intensifier: Intensifier=None,
-                 acquisition_function: AbstractAcquisitionFunction=None,
-                 acquisition_function_optimizer: AcquisitionFunctionMaximizer=None,
-                 model: AbstractEPM=None,
-                 runhistory2epm: AbstractRunHistory2EPM=None,
-                 initial_design: InitialDesign=None,
-                 initial_configurations: typing.List[Configuration]=None,
-                 stats: Stats=None,
-                 restore_incumbent: Configuration=None,
-                 rng: typing.Union[np.random.RandomState, int]=None,
-                 smbo_class: SMBO=None,
-                 run_id: int=1,
-                 random_configuration_chooser=None):
+                 tae_runner: typing.Optional[typing.Union[ExecuteTARun, typing.Callable]]=None,
+                 runhistory: typing.Optional[RunHistory]=None,
+                 intensifier: typing.Optional[Intensifier]=None,
+                 acquisition_function: typing.Optional[AbstractAcquisitionFunction]=None,
+                 acquisition_function_optimizer: typing.Optional[AcquisitionFunctionMaximizer]=None,
+                 model: typing.Optional[AbstractEPM]=None,
+                 runhistory2epm: typing.Optional[AbstractRunHistory2EPM]=None,
+                 initial_design: typing.Optional[InitialDesign]=None,
+                 initial_configurations: typing.Optional[typing.List[Configuration]]=None,
+                 stats: typing.Optional[Stats]=None,
+                 restore_incumbent: typing.Optional[Configuration]=None,
+                 rng: typing.Optional[typing.Union[np.random.RandomState, int]]=None,
+                 smbo_class: typing.Optional[SMBO]=None,
+                 run_id: typing.Optional[int]=None,
+                 random_configuration_chooser: typing.Optional[RandomConfigurationChooser]=None):
         """Constructor
 
         Parameters
@@ -121,11 +120,11 @@ class SMAC(object):
         smbo_class : ~smac.optimizer.smbo.SMBO
             Class implementing the SMBO interface which will be used to
             instantiate the optimizer class.
-        run_id: int, (default: 1)
-            Run ID will be used as subfolder for output_dir.
-        random_configuration_chooser
-            when to choose a random configuration -- one of
-            ChooserNoCoolDown, ChooserLinearCoolDown
+        run_id : int (optional)
+            Run ID will be used as subfolder for output_dir. If no ``run_id`` is given, a random ``run_id`` will be
+            chosen.
+        random_configuration_chooser : ~smac.optimizer.random_configuration_chooser.RandomConfigurationChooser
+            How often to choose a random configuration during the intensification procedure.
         """
 
         self.logger = logging.getLogger(
@@ -136,16 +135,21 @@ class SMAC(object):
         self.scenario = scenario
         self.output_dir = ""
         if not restore_incumbent:
+            # restore_incumbent is used by the CLI interface which provides a method for restoring a SMAC run given an
+            # output directory. This is the default path.
+            # initial random number generator
+            run_id, rng = self._get_rng(rng=rng, run_id=run_id)
             self.output_dir = create_output_directory(scenario, run_id)
         elif scenario.output_dir is not None:
+            run_id, rng = self._get_rng(rng=rng, run_id=run_id)
             # output-directory is created in CLI when restoring from a
             # folder. calling the function again in the facade results in two
             # folders being created: run_X and run_X.OLD. if we are
             # restoring, the output-folder exists already and we omit creating it,
             # but set the self-output_dir to the dir.
             # necessary because we want to write traj to new output-dir in CLI.
-            self.output_dir = os.path.join(scenario.output_dir,
-                                           "run_%d" % (run_id))
+            self.output_dir = scenario.output_dir_for_this_run
+
         if (
             scenario.deterministic is True
             and getattr(scenario, 'tuner_timeout', None) is None
@@ -170,9 +174,6 @@ class SMAC(object):
         if runhistory.aggregate_func is None:
             runhistory.aggregate_func = aggregate_func
 
-        # initial random number generator
-        num_run, rng = self._get_rng(rng=rng)
-
         random_configuration_chooser = SMAC._get_random_configuration_chooser(
             random_configuration_chooser=random_configuration_chooser)
 
@@ -186,7 +187,7 @@ class SMAC(object):
         # initial EPM
         types, bounds = get_types(scenario.cs, scenario.feature_array)
         if model is None:
-            model = RandomForestWithInstances(types=types, 
+            model = RandomForestWithInstances(types=types,
                                               bounds=bounds,
                                               instance_features=scenario.feature_array,
                                               seed=rng.randint(MAXINT),
@@ -354,7 +355,7 @@ class SMAC(object):
             elif scenario.run_obj == 'quality':
                 runhistory2epm = RunHistory2EPM4Cost(scenario=scenario, num_params=num_params,
                                                      success_states=[
-                                                         StatusType.SUCCESS, 
+                                                         StatusType.SUCCESS,
                                                          StatusType.CRASHED],
                                                      impute_censored_data=False, impute_state=None)
 
@@ -374,7 +375,7 @@ class SMAC(object):
             'runhistory2epm': runhistory2epm,
             'intensifier': intensifier,
             'aggregate_func': aggregate_func,
-            'num_run': num_run,
+            'num_run': run_id,
             'model': model,
             'acq_optimizer': acquisition_function_optimizer,
             'acquisition_func': acquisition_function,
@@ -387,36 +388,61 @@ class SMAC(object):
         else:
             self.solver = smbo_class(**smbo_args)
 
-    def _get_rng(self, rng):
-        """Initialize random number generator
+    def _get_rng(
+            self,
+            rng: typing.Optional[typing.Union[int, np.random.RandomState]]=None,
+            run_id: typing.Optional[int]=None,
+    ) -> typing.Tuple[int, np.random.RandomState]:
+        """Initialize random number generator and set run_id
 
-        If rng is None, initialize a new generator
-        If rng is Int, create RandomState from that
-        If rng is RandomState, return it
+        * If rng and run_id are None, initialize a new generator and sample a run_id
+        * If rng is None and a run_id is given, use the run_id to initialize the rng
+        * If rng is an int, a RandomState object is created from that.
+        * If rng is RandomState, return it
+        * If only run_id is None, a run_id is sampled from the random state.
 
         Parameters
         ----------
-        rng: np.random.RandomState|int|None
+        rng : np.random.RandomState|int|None
+
+        run_id : int, optional
 
         Returns
         -------
-        int, np.random.RandomState
+        int
+        np.random.RandomState
         """
         # initialize random number generator
-        if rng is None:
-            self.logger.debug('no rng given: using default seed of 1')
-            num_run = 1
-            rng = np.random.RandomState(seed=num_run)
+        if rng is not None and not isinstance(rng, (int, np.random.RandomState)):
+            raise TypeError('Argument rng accepts only arguments of type None, int or np.random.RandomState, '
+                            'you provided %s.' % str(type(rng)))
+        if run_id is not None and not isinstance(run_id, int):
+            raise TypeError('Argument run_id accepts only arguments of type None, int or np.random.RandomState, '
+                            'you provided %s.' % str(type(run_id)))
+
+        if rng is None and run_id is None:
+            # Case that both are None
+            self.logger.debug('No rng and no run_id given: using a random value to initialize run_id.')
+            rng = np.random.RandomState()
+            run_id = rng.randint(MAXINT)
+        elif rng is None and isinstance(run_id, int):
+            self.logger.debug('No rng and no run_id given: using run_id %d as seed.', run_id)
+            rng = np.random.RandomState(seed=run_id)
         elif isinstance(rng, int):
-            num_run = rng
+            if run_id is None:
+                run_id = rng
+            else:
+                pass
             rng = np.random.RandomState(seed=rng)
         elif isinstance(rng, np.random.RandomState):
-            num_run = rng.randint(MAXINT)
-            rng = rng
+            if run_id is None:
+                run_id = rng.randint(MAXINT)
+            else:
+                pass
         else:
-            raise TypeError('Unknown type %s for argument rng. Only accepts '
-                            'None, int or np.random.RandomState' % str(type(rng)))
-        return num_run, rng
+            raise ValueError('This should not happen! Please contact the developers! Arguments: rng=%s of type %s and '
+                             'run_id=% of type %s' % (rng, type(rng), run_id, type(run_id)))
+        return run_id, rng
 
     @staticmethod
     def _get_random_configuration_chooser(random_configuration_chooser):
@@ -449,7 +475,7 @@ class SMAC(object):
         finally:
             self.solver.stats.save()
             self.solver.stats.print_stats()
-            self.logger.info("Final Incumbent: %s" % (self.solver.incumbent))
+            self.logger.info("Final Incumbent: %s", (self.solver.incumbent))
             self.runhistory = self.solver.runhistory
             self.trajectory = self.solver.intensifier.traj_logger.trajectory
 


Test patch:
diff --git a/test/test_facade/test_smac_facade.py b/test/test_facade/test_smac_facade.py
index 81b7e0f55..bc66013e5 100644
--- a/test/test_facade/test_smac_facade.py
+++ b/test/test_facade/test_smac_facade.py
@@ -2,6 +2,7 @@ from contextlib import suppress
 import os
 import shutil
 import unittest
+import unittest.mock
 
 import numpy as np
 from ConfigSpace.hyperparameters import UniformFloatHyperparameter
@@ -57,59 +58,91 @@ class TestSMACFacade(unittest.TestCase):
         self.assertIs(smac.solver.intensifier.tae_runner.ta, target_algorithm)
 
     def test_pass_invalid_tae_runner(self):
-        self.assertRaisesRegexp(TypeError, "Argument 'tae_runner' is <class "
-                                           "'int'>, but must be either a "
-                                           "callable or an instance of "
-                                           "ExecuteTaRun.",
-                                SMAC, tae_runner=1, scenario=self.scenario)
+        self.assertRaisesRegex(
+            TypeError,
+            "Argument 'tae_runner' is <class 'int'>, but must be either a callable or an instance of ExecuteTaRun.",
+            SMAC,
+            tae_runner=1,
+            scenario=self.scenario,
+        )
 
     def test_pass_tae_runner_objective(self):
-        tae = ExecuteTAFuncDict(lambda: 1,
-                                run_obj='runtime')
-        self.assertRaisesRegexp(ValueError, "Objective for the target algorithm"
-                                            " runner and the scenario must be "
-                                            "the same, but are 'runtime' and "
-                                            "'quality'",
-                                SMAC, tae_runner=tae, scenario=self.scenario)
-
-    def test_check_random_states(self):
-        ta = ExecuteTAFuncDict(lambda x: x**2)
-
-        # Get state immediately or it will change with the next calltest_check_random_states
-
+        tae = ExecuteTAFuncDict(lambda: 1, run_obj='runtime')
+        self.assertRaisesRegex(
+            ValueError,
+            "Objective for the target algorithm runner and the scenario must be the same, but are 'runtime' and "
+            "'quality'",
+            SMAC,
+            tae_runner=tae,
+            scenario=self.scenario,
+        )
+
+    @unittest.mock.patch.object(SMAC, '__init__')
+    def test_check_random_states(self, patch):
+        patch.return_value = None
+        smac = SMAC()
+        smac.logger = unittest.mock.MagicMock()
+
+        # Check some properties
         # Check whether different seeds give different random states
-        S1 = SMAC(tae_runner=ta, scenario=self.scenario, rng=1)
-        S1 = S1.solver.scenario.cs.random
-
-        S2 = SMAC(tae_runner=ta, scenario=self.scenario, rng=2)
-        S2 = S2.solver.scenario.cs.random
-        self.assertNotEqual(sum(S1.get_state()[1] - S2.get_state()[1]), 0)
-
-        # Check whether no seeds give the same random states (use default seed)
-        S1 = SMAC(tae_runner=ta, scenario=self.scenario)
-        S1 = S1.solver.scenario.cs.random
-
-        S2 = SMAC(tae_runner=ta, scenario=self.scenario)
-        S2 = S2.solver.scenario.cs.random
-        self.assertEqual(sum(S1.get_state()[1] - S2.get_state()[1]), 0)
-
-        # Check whether the same seeds give the same random states
-        S1 = SMAC(tae_runner=ta, scenario=self.scenario, rng=1)
-        S1 = S1.solver.scenario.cs.random
-
-        S2 = SMAC(tae_runner=ta, scenario=self.scenario, rng=1)
-        S2 = S2.solver.scenario.cs.random
-        self.assertEqual(sum(S1.get_state()[1] - S2.get_state()[1]), 0)
-
-        # Check whether the same RandomStates give the same random states
-        S1 = SMAC(tae_runner=ta, scenario=self.scenario,
-                  rng=np.random.RandomState(1))
-        S1 = S1.solver.scenario.cs.random
-
-        S2 = SMAC(tae_runner=ta, scenario=self.scenario,
-                  rng=np.random.RandomState(1))
-        S2 = S2.solver.scenario.cs.random
-        self.assertEqual(sum(S1.get_state()[1] - S2.get_state()[1]), 0)
+        _, rng_1 = smac._get_rng(1)
+        _, rng_2 = smac._get_rng(2)
+        self.assertNotEqual(sum(rng_1.get_state()[1] - rng_2.get_state()[1]), 0)
+
+        # Check whether no seeds gives different random states
+        _, rng_1 = smac._get_rng()
+        self.assertEqual(smac.logger.debug.call_count, 1)
+        _, rng_2 = smac._get_rng()
+        self.assertEqual(smac.logger.debug.call_count, 2)
+
+        self.assertNotEqual(sum(rng_1.get_state()[1] - rng_2.get_state()[1]), 0)
+
+        # Check whether the same int seeds give the same random states
+        _, rng_1 = smac._get_rng(1)
+        _, rng_2 = smac._get_rng(1)
+        self.assertEqual(sum(rng_1.get_state()[1] - rng_2.get_state()[1]), 0)
+
+        # Check all execution paths
+        self.assertRaisesRegex(
+            TypeError,
+            "Argument rng accepts only arguments of type None, int or np.random.RandomState, "
+            "you provided <class 'str'>.",
+            smac._get_rng,
+            rng='ABC',
+        )
+        self.assertRaisesRegex(
+            TypeError,
+            "Argument run_id accepts only arguments of type None, int or np.random.RandomState, "
+            "you provided <class 'str'>.",
+            smac._get_rng,
+            run_id='ABC'
+        )
+
+        run_id, rng_1 = smac._get_rng(rng=None, run_id=None)
+        self.assertIsInstance(run_id, int)
+        self.assertIsInstance(rng_1, np.random.RandomState)
+        self.assertEqual(smac.logger.debug.call_count, 3)
+
+        run_id, rng_1 = smac._get_rng(rng=None, run_id=1)
+        self.assertEqual(run_id, 1)
+        self.assertIsInstance(rng_1, np.random.RandomState)
+
+        run_id, rng_1 = smac._get_rng(rng=1, run_id=None)
+        self.assertEqual(run_id, 1)
+        self.assertIsInstance(rng_1, np.random.RandomState)
+
+        run_id, rng_1 = smac._get_rng(rng=1, run_id=1337)
+        self.assertEqual(run_id, 1337)
+        self.assertIsInstance(rng_1, np.random.RandomState)
+
+        rs = np.random.RandomState(1)
+        run_id, rng_1 = smac._get_rng(rng=rs, run_id=None)
+        self.assertIsInstance(run_id, int)
+        self.assertIs(rng_1, rs)
+
+        run_id, rng_1 = smac._get_rng(rng=rs, run_id=2505)
+        self.assertEqual(run_id, 2505)
+        self.assertIs(rng_1, rs)
 
     def test_check_deterministic_rosenbrock(self):
         def rosenbrock_2d(x):
diff --git a/test/test_smbo/test_smbo.py b/test/test_smbo/test_smbo.py
index cdb03c72d..d38bc36b5 100644
--- a/test/test_smbo/test_smbo.py
+++ b/test/test_smbo/test_smbo.py
@@ -93,11 +93,14 @@ class TestSMBO(unittest.TestCase):
         self.assertIsInstance(smbo.num_run, int)
         self.assertIs(smbo.rng, rng)
         # ML: I don't understand the following line and it throws an error
-        self.assertRaisesRegexp(TypeError,
-                                "Unknown type <(class|type) 'str'> for argument "
-                                'rng. Only accepts None, int or '
-                                'np.random.RandomState',
-                                SMAC, self.scenario, rng='BLA')
+        self.assertRaisesRegex(
+            TypeError,
+            "Argument rng accepts only arguments of type None, int or np.random.RandomState, you provided "
+            "<class 'str'>.",
+            SMAC,
+            self.scenario,
+            rng='BLA',
+        )
 
     def test_choose_next(self):
         seed = 42
@@ -110,7 +113,7 @@ class TestSMBO(unittest.TestCase):
         Y = self.branin(X)
         x = next(smbo.choose_next(X, Y)).get_array()
         assert x.shape == (2,)
-        
+
     def test_choose_next_w_empty_rh(self):
         seed = 42
         smbo = SMAC(self.scenario, rng=seed).solver
@@ -126,9 +129,9 @@ class TestSMBO(unittest.TestCase):
             **{"X":X, "Y":Y}
         )
 
-        x = next(smbo.choose_next(X, Y, incumbent_value=0.0)).get_array()        
+        x = next(smbo.choose_next(X, Y, incumbent_value=0.0)).get_array()
         assert x.shape == (2,)
-        
+
     def test_choose_next_empty_X(self):
         smbo = SMAC(self.scenario, rng=1).solver
         smbo.acquisition_func._compute = mock.Mock(
@@ -146,7 +149,7 @@ class TestSMBO(unittest.TestCase):
         self.assertEqual(x, [0, 1, 2])
         self.assertEqual(smbo._random_search.maximize.call_count, 1)
         self.assertEqual(smbo.acquisition_func._compute.call_count, 0)
-        
+
     def test_choose_next_empty_X_2(self):
         smbo = SMAC(self.scenario, rng=1).solver