Repository: spyder-ide/spyder

Issue:
Several tests fail locally under user environment with customized keyboard shortcuts due to relying on hardcoded keybindings
## Description of your problem

As discussed in PR #6044 , in a number of tests of the Spyder test suite, including the very first non-skipped test on Windows, keybindings (using `Qt.Key_XXX`) are used to activate certain routines (pylint, run file, etc) which are hardcoded to the defaults (affects static_analysis, dedicated_console, and several skipped checks). This prevents me or anyone with custom shortcuts from running the tests at all past the first one, as that one (dedicated_console) fals due to this issue (due to the -x option in runtests.py), at least without one of a few tedious and time consuming workarounds every time. Personally, I'm not a huge fan of many of the default shortcuts and use a heavily customized scheme harmonized between several different IDEs, editors and machines, so just using the stock ones all the time isn't really an efficient option.

Possible solutions would be to either call the routines directly without going through the keybindings, read the actual keybindings from the config file and use those, or (probably best for the general case, as it could solve/prevent other possible test failures due to a non-default config) have the tests run themselves with a clean config file, e.g. temporarily write and restore it, or otherwise telling the tested copy of Spyder to use a clean config. Longer term, probably project-based or at least overridable config files are the answer, but as of now this is preventing anyone with non default shortcuts from running the tests without backing up, wiping and restoring their config files every time, so a straightforward and effective fix is ideal. 

Therefore, as I am newer to this whole thing, I was hoping to get the rest of the team's feedback on what's the simplest and cleanest overall method to address this (either those mentioned above or something better I hadn't immediately thought of), and how to approach/structure it. Thanks!

**What steps will reproduce the problem?**

1. Set a relevant shortcut (run file, run pylint, etc) to something other than the default
2. Run the test suite (e.g. with `python runtests.py`)
3. Observe failure

**What is the expected output? What do you see instead?**
Expected: Tests do not depend on user shortcut configuration. Actual: Tests do, and fail if changed.



Discussion:

Resolving patch:
diff --git a/runtests.py b/runtests.py
--- a/runtests.py
+++ b/runtests.py
@@ -10,6 +10,8 @@
 
 # Standard library imports
 import os
+import os.path as osp
+import shutil
 
 # Third party imports
 import qtpy  # to ensure that Qt4 uses API v2
@@ -19,6 +21,8 @@
 # To activate/deactivate certain things for pytest's only
 os.environ['SPYDER_PYTEST'] = 'True'
 
+# Tests expect English as the interface language
+os.environ['LANG'] = 'en'
 
 # To run our slow tests only in our CIs
 run_slow = False
@@ -30,6 +34,12 @@ def main():
     """
     Run pytest tests.
     """
+    # Remove temp conf_dir before starting the tests
+    from spyder.config.base import get_conf_path
+    conf_dir = get_conf_path()
+    if osp.isdir(conf_dir):
+        shutil.rmtree(conf_dir)
+
     pytest_args = ['spyder',
                    'spyder_profiler',
                    '-x',
diff --git a/spyder/config/base.py b/spyder/config/base.py
--- a/spyder/config/base.py
+++ b/spyder/config/base.py
@@ -118,9 +118,16 @@ def get_home_dir():
 
 def get_conf_path(filename=None):
     """Return absolute path for configuration file with specified filename"""
-    # This makes us follow the XDG standard to save our settings
-    # on Linux, as it was requested on Issue 2629
-    if sys.platform.startswith('linux'):
+    # Define conf_dir
+    if PYTEST:
+        import py
+        from _pytest.tmpdir import get_user
+        conf_dir = osp.join(str(py.path.local.get_temproot()),
+                            'pytest-of-{}'.format(get_user()),
+                            SUBFOLDER)
+    elif sys.platform.startswith('linux'):
+        # This makes us follow the XDG standard to save our settings
+        # on Linux, as it was requested on Issue 2629
         xdg_config_home = os.environ.get('XDG_CONFIG_HOME', '')
         if not xdg_config_home:
             xdg_config_home = osp.join(get_home_dir(), '.config')
@@ -129,8 +136,13 @@ def get_conf_path(filename=None):
         conf_dir = osp.join(xdg_config_home, SUBFOLDER)
     else:
         conf_dir = osp.join(get_home_dir(), SUBFOLDER)
+
+    # Create conf_dir
     if not osp.isdir(conf_dir):
-        os.mkdir(conf_dir)
+        if PYTEST:
+            os.makedirs(conf_dir)
+        else:
+            os.mkdir(conf_dir)
     if filename is None:
         return conf_dir
     else:


Test patch:
diff --git a/spyder/app/tests/test_mainwindow.py b/spyder/app/tests/test_mainwindow.py
--- a/spyder/app/tests/test_mainwindow.py
+++ b/spyder/app/tests/test_mainwindow.py
@@ -10,7 +10,6 @@
 
 import os
 import os.path as osp
-from sys import version_info
 import shutil
 import tempfile
 try:
@@ -84,12 +83,14 @@ def open_file_in_editor(main_window, fname, directory=None):
             input_field.setText(fname)
             QTest.keyClick(w, Qt.Key_Enter)
 
+
 def get_thirdparty_plugin(main_window, plugin_title):
     """Get a reference to the thirdparty plugin with the title given."""
     for plugin in main_window.thirdparty_plugins:
         if plugin.get_plugin_title() == plugin_title:
             return plugin
 
+
 def reset_run_code(qtbot, shell, code_editor, nsb):
     """Reset state after a run code test"""
     with qtbot.waitSignal(shell.executed):
@@ -142,9 +143,12 @@ def main_window(request):
     app = initialize()
     options, args = get_options()
     window = run_spyder(app, options, args)
+
+    # Teardown
     def close_window():
         window.close()
     request.addfinalizer(close_window)
+
     return window