Repository: HypothesisWorks/hypothesis

Issue:
Allow direct use of keyword arguments in `settings.register_profile`
Currently, registering a profile requires repetition of `settings`:

    settings.register_profile("ci", settings(max_examples=1000))
    ^^^^^^^^                        ^^^^^^^^
      once                           twice

This PR proposes to allow using the keyword arguments *directly*:

    settings.register_profile("ci", max_examples=1000)

Reusing stored instances of `settings` as a starting point, with the keyword arguments overriding individual settings, could be permitted like this:

    settings.register_profile("ci", stored_settings_instance, max_examples=1000)

I know how to do this conceptually, but in practice there are two steps I am unsure about

1. How to copy an instance of `settings`
2. How to override settings within an existing instance of `settings`

... or some more direct combination of the two.

`settings` does many things that have side-effects, and it has a non-standard metaclass, so I am not at all confident about getting it right.


Discussion:
I think I have found a simple solution, by using the `parent` construction parameter.

Now I have to start getting familiar with hypothesis' own test suite, and how to use it ...
In trying to follow the contribution guidelines, I've found myself writing `RELEASE.rst`. I'm wondering what value to choose for `RELEASE_TYPE`. Strictly speaking it's a new feature, so I should go with `minor`. But it's a trivial feature, and the previous entry used `patch` for something that seems to have added more functionality than this work. Is `minor` the correct choice here?

[Edit: `patch` -> `minor` and `bugfix` -> `patch`]
`documentation.rst` states that in the changelog you should

> include the current date and proposed version number (also updated in src/hypothesis/version.py)

Is this still current? It seems to me that Travis CI user does it automatically nowadays.
@Zac-HD, @DRMacIver I think I have, at least to a first-order approximation, complied with all the requirements, and thus have a complete, if small, PR.

I didn't manage to run the tests locally. The fundamental problem seems to be:

    pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.

I tried installing all sorts of SSL packages both in my conda environment and in Debian-apt, but didn't manage to fix it. As the instructions state that I don't *need* to run the tests locally, I'll leave it at that for now.

Could you please have a look and tell me if I have missed anything?

Off to a great start!  First, responses to questions:

- I agree that this is a `patch` release - while not strictly a bugfix, this is improving an interface without enabling any new behavior. 
- Unfortunately you based this on master between the last pull being merged and the commit deleting the release file - you'll need to rebase on (current) master and fix this for CI to work properly sorry 😢 
- Getting all the test environments working correctly can be a real pain.  FWIW, most of us only ever run the full test suite via CI!  If you're using conda, the way I do it is to create a conda environment with `conda create -n hydev python=3 numpy pandas`, then activate it and `pip install -e .[all]`, `pip install -r requirements/tools.txt`, `pip install -r requirements/test.txt`.  That's enough to use pytest on eg tests/cover and be confident I'm not wasting CI resources when we're busy.

Code-related comments:

- Any change to the signature *has* to be backwards compatible, which means no changing the argument names.  We tend to use `kwargs`, so that makes it `def register_profile(name, parent=None, **kwargs)` - and make sure to handle checking for `'settings' in kwargs`!
- Actually *implementing* this will be tricky - I think you'll need an alias for `settings` outside the method.  If you want to hand it over I'll spend some more time on it, but unless or until you decide that I'll try to help but not do it for you 😄 
> I think you'll need an alias for `settings` outside the method.

Or, rather than polluting the global namespace, how about importing it in the method itself? For example:

    def register_profile(name, settings=settings(), **kwargs):
        ...
        from hypothesis import settings as global_settings
        settings._profiles[name] = global_settings(parent=settings, **kwargs)

In the signature you proposed, you have `settings=None` instead of my `settings=settings()`. In which case it would look more like

    def register_profile(name, settings=None, **kwargs):
        ...
        from hypothesis import settings as global_settings
        if settings is None:
            new_profile = global_settings(**kwargs)
        else:
            new_profile = global_settings(parent=settings, **kwargs)
        settings._profiles[name] = new_profile

Is there some advantage in writing the latter over the former?

Finally, a Git style question: should I squash such modifications into the original commit, or add them in a separate commit? My tendency would be to squash it in, but I'm happy to follow whatever practices you have.
> I agree that this is a patch release - while not strictly a bugfix, this is improving an interface without enabling any new behavior.

I need to look at this in more detail to comment on the rest (I'll try to do so tomorrow), but no, sorry, I totally disagree with this and want to register that disagreement now before anything else happens.

If it's changing a public API in any way then it is a minor release, not a patch release. Patch releases are for things that fix bugs or in other way make zero changes to the public interface.
Oh, the import is much more elegant - well spotted.

The problem with using `settings()` as a default argument is what happens when the default settings are changed - the argument value isn't updated.  Using `None` avoids this, and is *also* a valid (in fact, default) argument for `parent`; in which case `global_settings` uses the current default settings.  So you don't need the branch!

For commits, it's up to you - I habitually squash and rebase my commits ~~to hide my typos~~ for a logical history, but I'm in a minority.  It goes onto master with a merge commit in any case.
> `pip install -e .[all]`

I'm struggling to parse this. I'm not a `pip` expert and I can't find the `-e` option documented anywhere.
The `-e` [means 'editable'](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs), similar to `python setup.py develop`.  Which means your environment will have it installed as soon as you save a file, without having to reinstall every time 😄 

Resolving patch:
diff --git a/src/hypothesis/_settings.py b/src/hypothesis/_settings.py
--- a/src/hypothesis/_settings.py
+++ b/src/hypothesis/_settings.py
@@ -33,6 +33,7 @@
 
 from hypothesis.errors import InvalidArgument, HypothesisDeprecationWarning
 from hypothesis.configuration import hypothesis_home_dir
+from hypothesis.internal.compat import text_type
 from hypothesis.utils.conventions import UniqueIdentifier, not_set
 from hypothesis.internal.validation import try_convert
 from hypothesis.utils.dynamicvariables import DynamicVariable
@@ -284,41 +285,53 @@ def __exit__(self, *args, **kwargs):
         return default_context_manager.__exit__(*args, **kwargs)
 
     @staticmethod
-    def register_profile(name, settings):
+    def register_profile(name, parent=None, **kwargs):
         """Registers a collection of values to be used as a settings profile.
 
-        These settings can be loaded in by name. Enable different
-        defaults for different settings.  ``settings`` must be a
-        settings object.
+        Settings profiles can be loaded by name - for example, you might
+        create a 'fast' profile which runs fewer examples, keep the 'default'
+        profile, and create a 'ci' profile that increases the number of
+        examples and uses a different database to store failures.
+
+        The arguments to this method are exactly as for
+        :class:`~hypothesis.settings`: optional ``parent`` settings, and
+        keyword arguments for each setting that will be set differently to
+        parent (or settings.default, if parent is None).
         """
-        settings._profiles[name] = settings
+        if not isinstance(name, (str, text_type)):
+            note_deprecation('name=%r must be a string' % (name,))
+        if 'settings' in kwargs:
+            if parent is None:
+                parent = kwargs.pop('settings')
+                note_deprecation('The `settings` argument is deprecated - '
+                                 'use `parent` instead.')
+            else:
+                raise InvalidArgument(
+                    'The `settings` argument is deprecated, and has been '
+                    'replaced by the `parent` argument.  Use `parent` only.'
+                )
+        settings._profiles[name] = settings(parent=parent, **kwargs)
 
     @staticmethod
     def get_profile(name):
-        """Return the profile with the given name.
-
-        - name is a string representing the name of the profile
-         to load
-        A InvalidArgument exception will be thrown if the
-         profile does not exist
-        """
+        """Return the profile with the given name."""
+        if not isinstance(name, (str, text_type)):
+            note_deprecation('name=%r must be a string' % (name,))
         try:
             return settings._profiles[name]
         except KeyError:
-            raise InvalidArgument(
-                "Profile '{0}' has not been registered".format(
-                    name
-                )
-            )
+            raise InvalidArgument('Profile %r is not registered' % (name,))
 
     @staticmethod
     def load_profile(name):
-        """Loads in the settings defined in the profile provided If the profile
-        does not exist an InvalidArgument will be thrown.
+        """Loads in the settings defined in the profile provided.
 
+        If the profile does not exist, InvalidArgument will be raised.
         Any setting not defined in the profile will be the library
-        defined default for that setting
+        defined default for that setting.
         """
+        if not isinstance(name, (str, text_type)):
+            note_deprecation('name=%r must be a string' % (name,))
         settings._current_profile = name
         settings._assign_default_internal(settings.get_profile(name))
 
@@ -716,10 +729,6 @@ class PrintSettings(Enum):
 
 settings.lock_further_definitions()
 
-settings.register_profile('default', settings())
-settings.load_profile('default')
-assert settings.default is not None
-
 
 def note_deprecation(message, s=None):
     if s is None:
@@ -729,3 +738,8 @@ def note_deprecation(message, s=None):
     warning = HypothesisDeprecationWarning(message)
     if verbosity > Verbosity.quiet:
         warnings.warn(warning, stacklevel=3)
+
+
+settings.register_profile('default', settings())
+settings.load_profile('default')
+assert settings.default is not None


Test patch:
diff --git a/tests/cover/test_settings.py b/tests/cover/test_settings.py
--- a/tests/cover/test_settings.py
+++ b/tests/cover/test_settings.py
@@ -95,6 +95,20 @@ def test_cannot_create_settings_with_invalid_options():
         settings(a_setting_with_limited_options=u'spoon')
 
 
+def test_cannot_register_with_parent_and_settings_args():
+    with pytest.raises(InvalidArgument):
+        settings.register_profile(
+            'conflicted', settings.default, settings=settings.default)
+    assert 'conflicted' not in settings._profiles
+
+
+@checks_deprecated_behaviour
+def test_register_profile_kwarg_settings_is_deprecated():
+    settings.register_profile('test', settings=settings(max_examples=10))
+    settings.load_profile('test')
+    assert settings.default.max_examples == 10
+
+
 def test_can_set_verbosity():
     settings(verbosity=Verbosity.quiet)
     settings(verbosity=Verbosity.normal)
@@ -134,14 +148,7 @@ def test_load_profile():
     assert settings.default.max_shrinks == 500
     assert settings.default.min_satisfying_examples == 5
 
-    settings.register_profile(
-        'test',
-        settings(
-            max_examples=10,
-            max_shrinks=5
-        )
-    )
-
+    settings.register_profile('test', settings(max_examples=10), max_shrinks=5)
     settings.load_profile('test')
 
     assert settings.default.max_examples == 10
@@ -155,6 +162,13 @@ def test_load_profile():
     assert settings.default.min_satisfying_examples == 5
 
 
+@checks_deprecated_behaviour
+def test_nonstring_profile_names_deprecated():
+    settings.register_profile(5, max_shrinks=5)
+    settings.load_profile(5)
+    assert settings.default.max_shrinks == 5
+
+
 def test_loading_profile_keeps_expected_behaviour():
     settings.register_profile('ci', settings(max_examples=10000))
     settings.load_profile('ci')