Repository: spyder-ide/spyder
Pull request: PR: Respect case sensitivity of working directory when running code (IPython console)
## Description of Changes

- We were setting the current working directory to be in lowercase on Windows after running/debugging a file, which didn't allow to make proper path comparisons in code.
- Depends on https://github.com/spyder-ide/spyder-kernels/pull/536

### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes #23835


### Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the [Developer Certificate of Origin](https://developercertificate.org)
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

<!--- TYPE YOUR USER/NAME AFTER THE FOLLOWING: --->
I certify the above statement is true and correct:
@jsbautista 
<!--- Thanks for your help making Spyder better for everyone! --->


Problem statement:
Initial runfile changes cwd to lowercase version which fails all code needing case sensitive paths
I am using the latest version of Spyder through Anaconda, but this problem started with the first version of spyder 6+.
On launch, my path is correctly reflected in the working directory field
if I import os in the console, and use os.getcwd() it outputs a correctly formatted uppercase cwd (C://Users/Myname).
If I then run an empty script, I see the following command in the console
```
%runfile 'C:/Users/MyName/PathToPlace/EmptyScript.py' --wdir
```
My console working directory has now been swapped to:
```
'c:\\users\\myname\\pathtoplace"
```
which means that any commands that need winbows style path formatting fails. Is there a secret option I missed that stops this from happening? Right now, all of my code that uses os.listdir() and os.getcwd() to get filelists for plotting data can't be used because cwd is reflected incorrectly.
I thought it might be a package problem, but like I said, if I  run a script that contains literally nothing, it changes my cwd.

Temporary solution currently is:
If you try to right click your tab and choose "set console working directory" it swaps to the uppercase path, then immediately back to a lowercase one... unless you swap to another file tab, choose "set console working directory", then swap back to your target file and do the same thing.
Now it works, but why?

Permanent solution currently is:
Downgrade to 5.5.1, which works flawlessly in comparison. 

output of pip list in case it's relevant

<details>
Package                         Version
------------------------------- -------------------
aext-assistant                  4.1.0
aext-assistant-server           4.1.0
aext-core                       4.1.0
aext-core-server                4.1.0
aext-panels                     4.1.0
aext-panels-server              4.1.0
aext-project-filebrowser-server 4.1.0
aext-share-notebook             4.1.0
aext-share-notebook-server      4.1.0
aext-shared                     4.1.0
aext-toolbox                    4.1.0
aiofiles                        22.1.0
aiohappyeyeballs                2.4.4
aiohttp                         3.11.10
aiosignal                       1.2.0
aiosqlite                       0.18.0
alabaster                       0.7.16
alembic                         1.13.3
anaconda-anon-usage             0.5.0
anaconda-cli-base               0.4.2
anaconda-client                 1.13.0
anaconda-cloud-auth             0.7.2
anaconda-navigator              2.6.4
anaconda-project                0.11.1
annotated-types                 0.6.0
anyio                           4.6.2
archspec                        0.2.3
argon2-cffi                     21.3.0
argon2-cffi-bindings            21.2.0
arrow                           1.3.0
astroid                         3.3.8
astropy                         6.0.0
astropy-iers-data               0.2024.3.11.18.33.3
asttokens                       2.0.5
async-lru                       2.0.4
async-timeout                   5.0.1
asyncssh                        2.17.0
atomicwrites                    1.4.0
attrs                           24.3.0
autopep8                        2.0.4
babel                           2.16.0
backports.functools-lru-cache   1.6.4
backports.tempfile              1.0
backports.weakref               1.0.post1
beautifulsoup4                  4.12.3
binaryornot                     0.4.4
black                           24.10.0
bleach                          6.2.0
bokeh                           3.6.2
boltons                         24.1.0
Bottleneck                      1.4.2
Brotli                          1.0.9
cairocffi                       1.7.1
certifi                         2025.1.31
cffi                            1.17.1
chardet                         4.0.0
charset-normalizer              3.3.2
click                           8.1.7
cloudpickle                     3.0.0
colorama                        0.4.6
comm                            0.2.1
conda                           25.1.1
conda-build                     25.1.2
conda-content-trust             0.2.0
conda_index                     0.5.0
conda-libmamba-solver           25.1.1
conda-pack                      0.7.1
conda-package-handling          2.4.0
conda_package_streaming         0.11.0
conda-repo-cli                  1.0.137
conda-token                     0.5.0+1.g2209e04
conda-verify                    3.4.2
configparser                    7.0.0
contourpy                       1.2.0
cookiecutter                    2.6.0
cryptography                    44.0.1
cssselect2                      0.7.0
cycler                          0.12.1
datefinder                      0.7.3
dateparser                      1.2.0
debugpy                         1.8.11
decorator                       5.1.1
defusedxml                      0.7.1
Deprecated                      1.2.13
diff-match-patch                20200713
dill                            0.3.8
distro                          1.9.0
DMU                             0.3.8
docstring-to-markdown           0.11
docutils                        0.18.1
entrypoints                     0.4
et-xmlfile                      1.1.0
exceptiongroup                  1.2.0
executing                       0.8.3
fastjsonschema                  2.20.0
filelock                        3.13.1
flake8                          7.1.1
fonttools                       4.49.0
frozendict                      2.4.2
frozenlist                      1.5.0
future                          1.0.0
greenlet                        3.1.1
h11                             0.14.0
h5py                            3.10.0
hdf5storage                     0.1.19
httpcore                        1.0.2
httpx                           0.27.0
idna                            3.7
imagej                          0.3.2
imagesize                       1.4.1
imglyb                          2.1.0
importlib_metadata              8.5.0
inflection                      0.5.1
intervaltree                    3.1.0
ipykernel                       6.29.5
ipython                         8.30.0
ipython-genutils                0.2.0
isort                           5.13.2
jaraco.classes                  3.2.1
jedi                            0.19.2
jellyfish                       1.1.3
jgo                             1.0.6
Jinja2                          3.1.5
JPype1                          1.5.0
json5                           0.9.25
jsonpatch                       1.33
jsonpointer                     2.1
jsonschema                      4.23.0
jsonschema-specifications       2023.7.1
jupyter_client                  8.6.3
jupyter_core                    5.7.2
jupyter-events                  0.10.0
jupyter-lsp                     2.2.0
jupyter_server                  2.14.1
jupyter_server_fileid           0.9.0
jupyter_server_terminals        0.4.4
jupyter_server_ydoc             0.8.0
jupyter-ydoc                    0.2.5
jupyterlab                      4.3.4
jupyterlab-pygments             0.1.2
jupyterlab_server               2.27.3
keyring                         24.3.1
kiwisolver                      1.4.5
labeling                        0.1.14
latex2excel                     0.1.dev0
latexcodec                      2.0.1
libarchive-c                    5.1
libmambapy                      2.0.5
linkify-it-py                   2.0.0
lxml                            5.3.0
Mako                            1.2.3
Markdown                        3.4.1
markdown-it-py                  2.2.0
MarkupSafe                      3.0.2
mat73                           0.63
matplotlib                      3.8.3
matplotlib-inline               0.1.6
mccabe                          0.7.0
mdit-py-plugins                 0.3.0
mdurl                           0.1.0
menuinst                        2.2.0
mistune                         2.0.4
mkl_fft                         1.3.11
mkl_random                      1.2.8
mkl-service                     2.4.0
more-itertools                  10.3.0
mpl-axes-aligner                1.3
multidict                       6.1.0
mypy-extensions                 1.0.0
natsort                         8.3.1
navigator-updater               0.5.1
nb_conda_kernels                2.5.2
nbclassic                       1.1.0
nbclient                        0.8.0
nbconvert                       7.16.4
nbformat                        5.10.4
nest-asyncio                    1.6.0
notebook                        7.3.2
notebook_shim                   0.2.3
numexpr                         2.10.1
numpy                           1.26.4
numpydoc                        1.7.0
opencv-python                   4.9.0.80
openpyxl                        3.1.2
overrides                       7.4.0
packaging                       24.2
pandas                          2.2.2
pandocfilters                   1.5.0
panel                           1.6.0
param                           2.2.0
parso                           0.8.4
pathspec                        0.10.3
pexpect                         4.8.0
pickleshare                     0.7.5
pillow                          11.1.0
pip                             25.0.1
pkce                            1.0.3
pkginfo                         1.11.2
plasTeX                         3.0
platformdirs                    3.10.0
pluggy                          1.5.0
ply                             3.11
prometheus_client               0.21.0
prompt-toolkit                  3.0.43
propcache                       0.2.0
psutil                          5.9.0
ptyprocess                      0.7.0
pure-eval                       0.2.2
py-slvs                         1.0.6
pycodestyle                     2.12.1
pycosat                         0.6.6
pycparser                       2.21
pydantic                        2.10.3
pydantic_core                   2.27.1
pydantic-settings               2.6.1
pydocstyle                      6.3.0
pyerfa                          2.0.1.1
pyflakes                        3.2.0
PyGithub                        2.4.0
Pygments                        2.15.1
pyimagej                        1.5.0
PyJWT                           2.10.1
pylatexenc                      2.10
pylint                          3.3.3
pylint-venv                     3.0.3
pyls-spyder                     0.4.0
PyMySQL                         1.1.1
PyNaCl                          1.5.0
pyparsing                       3.1.2
PyQt5                           5.15.10
PyQt5_sip                       12.13.0
PyQtWebEngine                   5.15.6
PySocks                         1.7.1
python-dateutil                 2.9.0.post0
python-dotenv                   0.21.0
python-json-logger              3.2.1
python-lsp-black                2.0.0
python-lsp-jsonrpc              1.1.2
python-lsp-server               1.12.0
python-slugify                  5.0.2
pytoolconfig                    1.2.6
pytz                            2024.1
pyuca                           1.2
pyvips                          2.2.3
pyviz_comms                     3.0.2
pywin32                         308
pywin32-ctypes                  0.2.2
pywinpty                        2.0.14
PyYAML                          6.0.2
pyzmq                           26.2.0
QDarkStyle                      3.2.3
qstylizer                       0.2.2
QtAwesome                       1.3.1
qtconsole                       5.6.1
QtPy                            2.4.1
readchar                        4.0.5
referencing                     0.30.2
regex                           2024.5.15
reportlab                       4.2.2
requests                        2.32.3
requests-toolbelt               1.0.0
rfc3339-validator               0.1.4
rfc3986-validator               0.1.1
rich                            13.9.4
rope                            1.12.0
rpds-py                         0.22.3
Rtree                           1.0.1
ruamel.yaml                     0.18.6
ruamel.yaml.clib                0.2.8
ruamel-yaml-conda               0.17.21
scipy                           1.12.0
scyjava                         1.10.0
semver                          3.0.2
Send2Trash                      1.8.2
sentry-sdk                      2.18.0
setuptools                      69.0.3
shellingham                     1.5.0
sip                             6.7.12
six                             1.16.0
sniffio                         1.3.0
snowballstemmer                 2.2.0
sortedcontainers                2.4.0
soupsieve                       2.5
Sphinx                          7.3.7
sphinxcontrib-applehelp         1.0.2
sphinxcontrib-devhelp           1.0.2
sphinxcontrib-htmlhelp          2.0.0
sphinxcontrib-jsmath            1.0.1
sphinxcontrib-qthelp            1.0.3
sphinxcontrib-serializinghtml   1.1.10
spyder                          6.0.3
spyder-kernels                  3.0.2
SQLAlchemy                      2.0.37
stack-data                      0.2.0
superqt                         0.6.7
svglib                          1.5.1
svgutils                        0.3.4
svgwrite                        1.4.3
tabulate                        0.9.0
terminado                       0.17.1
text-unidecode                  1.3
textdistance                    4.6.3
three-merge                     0.1.1
tifffile                        2024.2.12
tinycss2                        1.4.0
toml                            0.10.2
tomli                           2.0.1
tomlkit                         0.13.2
tornado                         6.4.2
tqdm                            4.67.1
traitlets                       5.14.3
truststore                      0.10.0
typer                           0.9.0
typing_extensions               4.12.2
tzdata                          2023.3
tzlocal                         5.2
uc-micro-py                     1.0.1
ujson                           5.10.0
Unidecode                       1.3.8
urllib3                         2.3.0
watchdog                        4.0.2
wcwidth                         0.2.5
webencodings                    0.5.1
websocket-client                1.8.0
whatthepatch                    1.0.2
wheel                           0.45.1
win-inet-pton                   1.1.0
wrapt                           1.17.0
xarray                          2024.9.0
xlrd                            2.0.1
xyzservices                     2022.9.0
y-py                            0.6.2
yapf                            0.40.2
yarl                            1.18.0
ypy-websocket                   0.8.4
zipp                            3.21.0
zstandard                       0.23.0

</details>

Hints from issue discussion:
Hey @DeltaMod, thanks for reporting. We were not aware of this problem, so thanks for bringing it up to our attention.

We'll try to fix it in 6.0.6, to be released in a couple of months.
@dalthviz, are you able to reproduce this one?
I think I was able to reproduce this:

![Image](https://github.com/user-attachments/assets/c0986d92-3d56-4cd6-97ba-213fa0472aee)

I guess is caused due to the path after running a file being set as the working directory with a normalized version of the path (maybe at some point `os.path.normcase` is being used 🤔)
> Hey [@DeltaMod](https://github.com/DeltaMod), thanks for reporting. We were not aware of this problem, so thanks for bringing it up to our attention.
> 
> We'll try to fix it in 6.0.6, to be released in a couple of months.

Thank you! For now, simply reverting to 5.5.1 solves any and all issues, so I can keep using Spyder. I'll pay attention to new release cycles. I just find it rather confusing that it only remains a bug if you stay in the first tab-document after running it once. If you swap to another document, it accepts the new change of cwd with correct formatting such that if you now return to your initial document, running it will now correctly re-assign cwd.

I can imagine that most people just neglect to report, and instead silently revert versions to when it worked last., so I'm just happy to help 
> I just find it rather confusing that it only remains a bug if you stay in the first tab-document after running it once.

Ok, that's interesting too.

> I can imagine that most people just neglect to report, and instead silently revert versions to when it worked last., so I'm just happy to help

Thanks! We rely on users for this kind of problems because they're difficult to notice by us since we don't use Spyder in the way you do.

Commit message: add test

Patch to review:
diff --git a/external-deps/spyder-kernels/.gitrepo b/external-deps/spyder-kernels/.gitrepo
index ddc441685ab..dcce1ff92e1 100644
--- a/external-deps/spyder-kernels/.gitrepo
+++ b/external-deps/spyder-kernels/.gitrepo
@@ -4,9 +4,9 @@
 ; git-subrepo command. See https://github.com/ingydotnet/git-subrepo#readme
 ;
 [subrepo]
-	remote = https://github.com/spyder-ide/spyder-kernels.git
-	branch = master
-	commit = 1c2113ab9e6ecd722882d45efc27a393cafe366b
-	parent = 75901418214287ede9722b71c7ab735f0a258b82
+	remote = https://github.com/jsbautista/spyder-kernels.git
+	branch = cwdLowercase
+	commit = 9598bfd10be5aa393d07f7a487cbb5ac81d82ddf
+	parent = 5b30b090f05e36c8c4bc4ca9d2f831250126d263
 	method = merge
-	cmdver = 0.4.3
+	cmdver = 0.4.9
diff --git a/external-deps/spyder-kernels/spyder_kernels/customize/code_runner.py b/external-deps/spyder-kernels/spyder_kernels/customize/code_runner.py
index 8f3ac12cf41..41aff7a1b25 100644
--- a/external-deps/spyder-kernels/spyder_kernels/customize/code_runner.py
+++ b/external-deps/spyder-kernels/spyder_kernels/customize/code_runner.py
@@ -286,6 +286,8 @@ def _exec_file(
             self.shell.showtraceback(exception_only=True)
             return
 
+        original_filename = filename
+
         # Here the remote filename has been used. It must now be valid locally.
         filename = canonic_filename
 
@@ -313,7 +315,7 @@ def _exec_file(
             if wdir is not None:
                 if wdir is True:
                     # True means use file dir
-                    wdir = os.path.dirname(filename)
+                    wdir = os.path.dirname(original_filename)
                 if os.path.isdir(wdir):
                     os.chdir(wdir)
 
diff --git a/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py b/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
index cde6338e10e..43313e319e2 100644
--- a/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
+++ b/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
@@ -2516,5 +2516,37 @@ def test_filter_frames_in_tracebacks(ipyconsole, qtbot, tmp_path):
     assert "BdbQuit" not in control.toPlainText()
 
 
+def test_lowercase_wdir(ipyconsole, qtbot, tmp_path):
+
+    shell = ipyconsole.get_current_shellwidget()
+    control = shell._control
+    qtbot.waitUntil(
+        lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT
+    )
+    dir_a = tmp_path / 'SensitiveCaseA'
+    dir_a.mkdir()
+    filename_a = dir_a / 'a.py'
+    filename_a.write_text('a = 1')
+
+    # Filename in the format used when running magics from the main toolbar
+    fname = str(filename_a).replace('\\', '/')
+    fdir = str(dir_a).replace('\\', '/')
+
+    # Run file
+    with qtbot.waitSignal(shell.executed):
+        shell.execute(f"%runfile {fname} --wdir {fdir}")
+
+    with qtbot.waitSignal(shell.executed):
+        shell.execute("%clear")
+
+    with qtbot.waitSignal(shell.executed):
+        shell.execute("%pwd")
+
+
+    assert "SensitiveCaseA" in control.toPlainText()
+    assert "sensitivecasea" not in control.toPlainText()
+
+
+
 if __name__ == "__main__":
     pytest.main()


Code review comments:
diff_hunk: @@ -2516,5 +2516,37 @@ def test_filter_frames_in_tracebacks(ipyconsole, qtbot, tmp_path):
     assert "BdbQuit" not in control.toPlainText()
 
 
+def test_lowercase_wdir(ipyconsole, qtbot, tmp_path):
+
+    shell = ipyconsole.get_current_shellwidget()
+    control = shell._control
+    qtbot.waitUntil(
+        lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT
+    )
+    dir_a = tmp_path / 'SensitiveCaseA'
+    dir_a.mkdir()
+    filename_a = dir_a / 'a.py'
+    filename_a.write_text('a = 1')
+
+    # Filename in the format used when running magics from the main toolbar
+    fname = str(filename_a).replace('\\', '/')
+    fdir = str(dir_a).replace('\\', '/')
+
+    # Run file
+    with qtbot.waitSignal(shell.executed):
+        shell.execute(f"%runfile {fname} --wdir {fdir}")
+
+    with qtbot.waitSignal(shell.executed):
+        shell.execute("%clear")
+
+    with qtbot.waitSignal(shell.executed):
+        shell.execute("%pwd")
+
+
path: spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
text: @user1:
```suggestion
```

Remove unnecessary line.

diff_hunk: @@ -2516,5 +2516,37 @@ def test_filter_frames_in_tracebacks(ipyconsole, qtbot, tmp_path):
     assert "BdbQuit" not in control.toPlainText()
 
 
+def test_lowercase_wdir(ipyconsole, qtbot, tmp_path):
+
+    shell = ipyconsole.get_current_shellwidget()
+    control = shell._control
+    qtbot.waitUntil(
+        lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT
+    )
+    dir_a = tmp_path / 'SensitiveCaseA'
+    dir_a.mkdir()
+    filename_a = dir_a / 'a.py'
+    filename_a.write_text('a = 1')
+
+    # Filename in the format used when running magics from the main toolbar
+    fname = str(filename_a).replace('\\', '/')
+    fdir = str(dir_a).replace('\\', '/')
+
+    # Run file
+    with qtbot.waitSignal(shell.executed):
+        shell.execute(f"%runfile {fname} --wdir {fdir}")
+
+    with qtbot.waitSignal(shell.executed):
+        shell.execute("%clear")
+
+    with qtbot.waitSignal(shell.executed):
+        shell.execute("%pwd")
+
+
+    assert "SensitiveCaseA" in control.toPlainText()
+    assert "sensitivecasea" not in control.toPlainText()
+
path: spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
text: @user1:
```suggestion
```

Another unnecessary line.

diff_hunk: @@ -2516,5 +2516,37 @@ def test_filter_frames_in_tracebacks(ipyconsole, qtbot, tmp_path):
     assert "BdbQuit" not in control.toPlainText()
 
 
+def test_lowercase_wdir(ipyconsole, qtbot, tmp_path):
path: spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py
text: @user1:
```suggestion
def test_case__sensitive_wdir(ipyconsole, qtbot, tmp_path):
    """Test that we preserve case sensitive working directories."""
```

Improve test name and docstring to it.