Repository: spyder-ide/spyder

Issue:
Use a separate configuration directory for beta and dev releases to avoid contaminating user settings
Can the *.beta releases be updated to use a different assortment of dot files, so `.spyder` could become `.spyder-beta`, this would preserve the beta alongside the current `3.3.1` release so that a production and testing environment can co-exist happily. I also wonder whether the same could be possible for the `.ipython` and `.matplotlib` subdirs?
Use a separate configuration directory for beta and dev releases to avoid contaminating user settings
Can the *.beta releases be updated to use a different assortment of dot files, so `.spyder` could become `.spyder-beta`, this would preserve the beta alongside the current `3.3.1` release so that a production and testing environment can co-exist happily. I also wonder whether the same could be possible for the `.ipython` and `.matplotlib` subdirs?


Discussion:
This is a good suggestion. @CAM-Gerlach, what do you think about it?

> I also wonder whether the same could be possible for the .ipython and .matplotlib subdirs?

This is not something Spyder should handle because it could cause more headaches than benefits. So I don't think we'd implement it.
@ccordoba12 glad you think it's worth considering. If they are pointing at different dirs, at least for spyder itself, then a beta user could relatively easily update to successive betas, providing feedback, but also enabling the use of the standard release as a fall back
@ccordoba12 Definitely agreed; this would be very useful to enable messing around without launching with ``--safe-mode`` all the time. Shouldn't changing the ``.spyder`` and ``.spyder-py3`` directory names to something like ``.spyder-beta`` and ``.spyder-py3-beta`` (for of Spyder 5, we can unify them, since Py2 will be dropped completely, but that's a different issue) be sufficient for now? Then for Beta 3 or whenever the project changes are implemented, we'd presumably need to change the ``.spyderproject`` name as well (or just rename it entirely).
> Shouldn't changing the .spyder and .spyder-py3 directory names to something like .spyder-beta and .spyder-py3-beta be sufficient for now?

Yes, it should. Would you like to implement it?

> Then for Beta 3 or whenever the project changes are implemented, we'd presumably need to change the .spyderproject name as well (or just rename it entirely).

This shouldn't be necessary. Our future changes shouldn't affect the current project structure.
> Yes, it should. Would you like to implement it?

Sure, I guess. Where do I make the change; just in config/base.py or are there other places too? Do you want iit be hardcoded on ``master`` until the final release? A command line flag enabled by default for now on ``master``? Or something that automatically parses the version string to determine if the current version qualifies as a "beta"? I can also also try to add a unit test if practicable.
> Sure, I guess. Where do I make the change; just in config/base.py or are there other places too?

I think there are no other places needed.

> Or something that automatically parses the version string to determine if the current version qualifies as a "beta"?

Exactly. That's what I'd do too.

> I can also also try to add a unit test if practicable.

Sure, it should be pretty simple: a test for `get_conf_path` should be enough.
> That's what I'd do too.

Okay, cool. That should be simple enough.
@ccordoba12 I've been finally doing the big legal housecleaning I've been planning on for a while, along with spinning up with the first day of my new research position yesterday as well as the start of classes before I do this, but with regard to the version string, should ``dev`` builds (checkouts from ``master``) also count as ``beta`` for settings isolation, or only the four released snapshots? I'd think the *former would make more sense, but its up to you what you think's best.
> but with regard to the version string, should dev builds (checkouts from master) also count as beta for settings isolation

That would be really helpful for the dev team and I haven't thought about that! So please also make `dev` as well as `beta` to count for settings isolation.
Great, that's what I thought too (I mistyped "latter" instead of "former" above). I'll probably get to it this weekend.
Hi folks, I'm just pinging myself on this as I'd forgotten where this issue was living.. What are the chances this could sneak into v4.0beta2?
I can take a look at it later this evening since it seems pretty simple to do. Sorry for leaving this for so long as I was working on other projects.
@CAM-Gerlach no problem! I've been a reluctant tester of v4.0beta1 as it's going to mess up my envs, so was hoping this could land so I can jump into testing alongside

Heh, its already wiped all my custom keyboard shortcuts and color schemes which is what I deserve for not implementing this sooner, I guess.
@CAM-Gerlach, exactly, you’ve hit one of the very reasons having a different dotfile subdir is a good idea for the betas
This is a good suggestion. @CAM-Gerlach, what do you think about it?

> I also wonder whether the same could be possible for the .ipython and .matplotlib subdirs?

This is not something Spyder should handle because it could cause more headaches than benefits. So I don't think we'd implement it.
@ccordoba12 glad you think it's worth considering. If they are pointing at different dirs, at least for spyder itself, then a beta user could relatively easily update to successive betas, providing feedback, but also enabling the use of the standard release as a fall back
@ccordoba12 Definitely agreed; this would be very useful to enable messing around without launching with ``--safe-mode`` all the time. Shouldn't changing the ``.spyder`` and ``.spyder-py3`` directory names to something like ``.spyder-beta`` and ``.spyder-py3-beta`` (for of Spyder 5, we can unify them, since Py2 will be dropped completely, but that's a different issue) be sufficient for now? Then for Beta 3 or whenever the project changes are implemented, we'd presumably need to change the ``.spyderproject`` name as well (or just rename it entirely).
> Shouldn't changing the .spyder and .spyder-py3 directory names to something like .spyder-beta and .spyder-py3-beta be sufficient for now?

Yes, it should. Would you like to implement it?

> Then for Beta 3 or whenever the project changes are implemented, we'd presumably need to change the .spyderproject name as well (or just rename it entirely).

This shouldn't be necessary. Our future changes shouldn't affect the current project structure.
> Yes, it should. Would you like to implement it?

Sure, I guess. Where do I make the change; just in config/base.py or are there other places too? Do you want iit be hardcoded on ``master`` until the final release? A command line flag enabled by default for now on ``master``? Or something that automatically parses the version string to determine if the current version qualifies as a "beta"? I can also also try to add a unit test if practicable.
> Sure, I guess. Where do I make the change; just in config/base.py or are there other places too?

I think there are no other places needed.

> Or something that automatically parses the version string to determine if the current version qualifies as a "beta"?

Exactly. That's what I'd do too.

> I can also also try to add a unit test if practicable.

Sure, it should be pretty simple: a test for `get_conf_path` should be enough.
> That's what I'd do too.

Okay, cool. That should be simple enough.
@ccordoba12 I've been finally doing the big legal housecleaning I've been planning on for a while, along with spinning up with the first day of my new research position yesterday as well as the start of classes before I do this, but with regard to the version string, should ``dev`` builds (checkouts from ``master``) also count as ``beta`` for settings isolation, or only the four released snapshots? I'd think the *former would make more sense, but its up to you what you think's best.
> but with regard to the version string, should dev builds (checkouts from master) also count as beta for settings isolation

That would be really helpful for the dev team and I haven't thought about that! So please also make `dev` as well as `beta` to count for settings isolation.
Great, that's what I thought too (I mistyped "latter" instead of "former" above). I'll probably get to it this weekend.
Hi folks, I'm just pinging myself on this as I'd forgotten where this issue was living.. What are the chances this could sneak into v4.0beta2?
I can take a look at it later this evening since it seems pretty simple to do. Sorry for leaving this for so long as I was working on other projects.
@CAM-Gerlach no problem! I've been a reluctant tester of v4.0beta1 as it's going to mess up my envs, so was hoping this could land so I can jump into testing alongside

Heh, its already wiped all my custom keyboard shortcuts and color schemes which is what I deserve for not implementing this sooner, I guess.
@CAM-Gerlach, exactly, you’ve hit one of the very reasons having a different dotfile subdir is a good idea for the betas

Resolving patch:
diff --git a/spyder/config/base.py b/spyder/config/base.py
--- a/spyder/config/base.py
+++ b/spyder/config/base.py
@@ -15,16 +15,18 @@
 from __future__ import print_function
 
 import codecs
+import getpass
 import locale
-import os.path as osp
 import os
+import os.path as osp
+import re
 import shutil
 import sys
-import warnings
-import getpass
 import tempfile
+import warnings
 
 # Local imports
+from spyder import __version__
 from spyder.utils import encoding
 from spyder.py3compat import (is_unicode, TEXT_TYPES, INT_TYPES, PY3,
                               to_text_string, is_text_string)
@@ -37,6 +39,9 @@
 # SPYDER_DEV is (and *only* has to be) set in bootstrap.py
 DEV = os.environ.get('SPYDER_DEV')
 
+# Manually override whether the dev configuration directory is used.
+USE_DEV_CONFIG_DIR = os.environ.get('SPYDER_USE_DEV_CONFIG_DIR')
+
 # Make Spyder use a temp clean configuration directory for testing purposes
 # SPYDER_SAFE_MODE can be set using the --safe-mode option of bootstrap.py
 SAFE_MODE = os.environ.get('SPYDER_SAFE_MODE')
@@ -52,6 +57,33 @@ def running_under_pytest():
     return bool(os.environ.get('SPYDER_PYTEST'))
 
 
+def is_stable_version(version):
+    """
+    Return true if version is stable, i.e. with letters in the final component.
+
+    Stable version examples: ``1.2``, ``1.3.4``, ``1.0.5``.
+    Non-stable version examples: ``1.3.4beta``, ``0.1.0rc1``, ``3.0.0dev0``.
+    """
+    if not isinstance(version, tuple):
+        version = version.split('.')
+    last_part = version[-1]
+
+    if not re.search(r'[a-zA-Z]', last_part):
+        return True
+    else:
+        return False
+
+
+def use_dev_config_dir(use_dev_config_dir=USE_DEV_CONFIG_DIR):
+    """Return whether the dev configuration directory should used."""
+    if use_dev_config_dir is not None:
+        if use_dev_config_dir.lower() in {'false', '0'}:
+            use_dev_config_dir = False
+    else:
+        use_dev_config_dir = DEV or not is_stable_version(__version__)
+    return use_dev_config_dir
+
+
 #==============================================================================
 # Debug helpers
 #==============================================================================
@@ -104,6 +136,12 @@ def debug_print(*message):
     SUBFOLDER = SUBFOLDER + '-py3'
 
 
+# If running a development/beta version, save config in a seperate directory
+# to avoid wiping or contaiminating the user's saved stable configuration.
+if use_dev_config_dir():
+    SUBFOLDER = SUBFOLDER + '-dev'
+
+
 def get_home_dir():
     """
     Return user home directory
diff --git a/spyder/utils/programs.py b/spyder/utils/programs.py
--- a/spyder/utils/programs.py
+++ b/spyder/utils/programs.py
@@ -20,6 +20,7 @@
 import tempfile
 
 # Local imports
+from spyder.config.base import is_stable_version
 from spyder.config.utils import is_anaconda
 from spyder.utils import encoding
 from spyder.utils.misc import get_python_executable
@@ -338,23 +339,6 @@ def run_python_script_in_terminal(fname, wdir, args, interact,
         raise NotImplementedError
 
 
-def is_stable_version(version):
-    """
-    A stable version has no letters in the final component, but only numbers.
-
-    Stable version example: 1.2, 1.3.4, 1.0.5
-    Not stable version: 1.2alpha, 1.3.4beta, 0.1.0rc1, 3.0.0dev
-    """
-    if not isinstance(version, tuple):
-        version = version.split('.')
-    last_part = version[-1]
-
-    if not re.search(r'[a-zA-Z]', last_part):
-        return True
-    else:
-        return False
-
-
 def check_version(actver, version, cmp_op):
     """
     Check version string of an active module against a required version.
diff --git a/spyder/workers/updates.py b/spyder/workers/updates.py
--- a/spyder/workers/updates.py
+++ b/spyder/workers/updates.py
@@ -16,10 +16,10 @@
 
 # Local imports
 from spyder import __version__
-from spyder.config.base import _
+from spyder.config.base import _, is_stable_version
 from spyder.py3compat import PY3, is_text_string
 from spyder.config.utils import is_anaconda
-from spyder.utils.programs import check_version, is_stable_version
+from spyder.utils.programs import check_version
 
 
 if PY3:


Test patch:
diff --git a/spyder/config/tests/test_base.py b/spyder/config/tests/test_base.py
new file mode 100644
--- /dev/null
+++ b/spyder/config/tests/test_base.py
@@ -0,0 +1,54 @@
+# -*- coding: utf-8 -*-
+# -----------------------------------------------------------------------------
+# Copyright (c) 2009- Spyder Project Contributors
+#
+# Distributed under the terms of the MIT License
+# (see spyder/__init__.py for details)
+# -----------------------------------------------------------------------------
+
+"""
+Tests for the spyder.config.base module.
+"""
+
+# Standard library imports
+import os.path as osp
+try:
+    from importlib import reload
+except ImportError:  # A builtin on Python 2
+    pass
+
+# Third party imports
+import pytest
+
+# Local imports
+import spyder.config.base
+
+
+# ============================================================================
+# ---- Tests
+# ============================================================================
+def test_is_stable_version():
+    """Test that stable and non-stable versions are recognized correctly."""
+    for stable_version in ['3.3.0', '2', ('0', '5')]:
+        assert spyder.config.base.is_stable_version(stable_version)
+    for not_stable_version in ['4.0.0b1', '3.3.2.dev0',
+                               'beta', ('2', '0', 'alpha')]:
+        assert not spyder.config.base.is_stable_version(not_stable_version)
+
+
+@pytest.mark.parametrize('use_dev_config_dir', [True, False])
+def test_get_conf_path(monkeypatch, use_dev_config_dir):
+    """Test that the config dir path is set under dev and release builds."""
+    monkeypatch.setenv('SPYDER_USE_DEV_CONFIG_DIR', str(use_dev_config_dir))
+    reload(spyder.config.base)
+    conf_path = spyder.config.base.get_conf_path()
+    assert conf_path
+    assert ((osp.basename(conf_path).split('-')[-1] == 'dev')
+            == use_dev_config_dir)
+    assert osp.isdir(conf_path)
+    monkeypatch.undo()
+    reload(spyder.config.base)
+
+
+if __name__ == '__main__':
+    pytest.main()
diff --git a/spyder/utils/tests/test_programs.py b/spyder/utils/tests/test_programs.py
--- a/spyder/utils/tests/test_programs.py
+++ b/spyder/utils/tests/test_programs.py
@@ -15,8 +15,7 @@
                                    is_python_interpreter,
                                    is_python_interpreter_valid_name,
                                    find_program, shell_split, check_version,
-                                   is_module_installed, get_temp_dir,
-                                   is_stable_version)
+                                   is_module_installed, get_temp_dir)
 
 
 if os.name == 'nt':
@@ -139,12 +138,5 @@ def test_get_temp_dir_ensure_dir_exists():
     assert another_call == temp_dir
 
 
-def test_is_stable_version():
-    """Test for is_stable_version."""
-    assert is_stable_version('3.3.0')
-    assert not is_stable_version('4.0.0b1')
-    assert not is_stable_version('3.3.2.dev0')
-
-
 if __name__ == '__main__':
     pytest.main()