Repository: beeware/briefcase

Issue:
Accommodate AppData Redirection for Windows Store Python
**Describe the bug**
Windows will redirect interaction with `%LOCALAPPDATA%` when Python is installed from the Windows Store. This means that the `briefcase` data directory will be created at somewhere like `%LOCALAPPDATA%\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\Local\BeeWare` instead of `%LOCALAPPDATA%\BeeWare`.



**To Reproduce**
Steps to reproduce the behavior:
1. Ensure `%LOCALAPPDATA%\BeeWare` directory does not exist.
2. Run `briefcase build android` using Python from the Windows Store
3. See error on trying to verify licenses

**Expected behavior**
Briefcase should prevent this redirection or incorporate it in to sourcing the `briefcase` data directory.

Of note, if the `%LOCALAPPDATA%\BeeWare` exists, this redirection will not happen.

**Environment:**
 - Operating System: Windows 10 1903 and later
 - Python version: 3.10
 - Software versions:
   - Briefcase: 0.3.11

**Additional context**
https://github.com/python/cpython/issues/84557
https://docs.microsoft.com/en-us/windows/msix/desktop/desktop-to-uwp-behind-the-scenes#file-system


## Workarounds
- Use the public [Python installer](https://www.python.org/downloads/)
- Move the `%LOCALAPPDATA%\Packages\<Python>\LocalCache\Local\BeeWare` directory to `%LOCALAPPDATA%\BeeWare`

Discussion:

Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0b932557..f4d824b5 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -7,7 +7,7 @@ on:
   workflow_call:
 
 env:
-  python_version: '3.9'
+  python_version: "3.9"
 
 defaults:
   run:
@@ -75,25 +75,34 @@ jobs:
   unit-tests:
     name: Unit tests
     needs: [pre-commit, towncrier, package]
+    runs-on: ${{ matrix.platform }}-latest
+    continue-on-error: ${{ matrix.experimental }}
     strategy:
       matrix:
         platform: ["macos", "ubuntu", "windows"]
         python-version: ["3.8", "3.9", "3.10", "3.11", "3.12-dev"]
         include:
-          - experimental: false
-
-          - python-version: "3.12-dev"
-            experimental: true
-    runs-on: ${{ matrix.platform }}-latest
-    continue-on-error: ${{ matrix.experimental }}
+        - experimental: false
+        - python-version: "3.12-dev"
+          experimental: true
+        # Run tests against the latest Windows Store Python
+        - platform: "windows"
+          python-version: "winstore"
+          experimental: false
     steps:
     - uses: actions/checkout@v3.3.0
       with:
         fetch-depth: 0
     - name: Set up Python
+      if: matrix.python-version != 'winstore'
       uses: actions/setup-python@v4.5.0
       with:
         python-version: ${{ matrix.python-version }}
+    - name: Install Windows Store Python
+      if: matrix.python-version == 'winstore'
+      uses: beeware/.github/.github/actions/install-win-store-python@main
+      with:
+        python-version: 3.11
     - name: Get packages
       uses: actions/download-artifact@v3
       with:
@@ -123,8 +132,7 @@ jobs:
   coverage:
     name: Combine & check coverage.
     runs-on: ubuntu-latest
-    needs:
-    - unit-tests
+    needs: [unit-tests]
     steps:
     - uses: actions/checkout@v3.3.0
       with:
@@ -132,7 +140,7 @@ jobs:
     - uses: actions/setup-python@v4.5.0
       with:
         # Use latest, so it understands all syntax.
-        python-version: "3.10"
+        python-version: '3.x'
     - name: Install dev dependencies
       run: |
         python -m pip install --upgrade pip
@@ -160,7 +168,8 @@ jobs:
 
   verify-apps:
     name: Build apps
-    needs: unit-tests
+    needs: [unit-tests]
+    runs-on: ${{ matrix.platform }}
     strategy:
       matrix:
         os_name: ['macOS', 'windows', 'linux']
@@ -183,7 +192,6 @@ jobs:
           pip-cache-dir: ~/.cache/pip
           # cache action cannot cache docker images (actions/cache#31)
           # docker-cache-dir: /var/lib/docker
-    runs-on: ${{ matrix.platform }}
     steps:
     - name: Cache Briefcase tools
       uses: actions/cache@v3.2.3
diff --git a/changes/922.bugfix.rst b/changes/922.bugfix.rst
new file mode 100644
index 00000000..456d072b
--- /dev/null
+++ b/changes/922.bugfix.rst
@@ -0,0 +1,1 @@
+When using the Windows Store version of Python, Briefcase now ensures the cache directory is created in %LOCALAPPDATA% instead of the sandboxed location enforced for Windows Store apps.
diff --git a/src/briefcase/commands/base.py b/src/briefcase/commands/base.py
index a3c9eb47..09c1c3cc 100644
--- a/src/briefcase/commands/base.py
+++ b/src/briefcase/commands/base.py
@@ -197,7 +197,9 @@ class BaseCommand(ABC):
                     appauthor="BeeWare",
                 ).user_cache_path
 
-        if " " in os.fsdecode(data_path):
+        data_path = os.fsdecode(data_path)
+
+        if " " in data_path:
             raise BriefcaseCommandError(
                 f"""
 The location Briefcase will use to store tools and support files:
@@ -213,6 +215,37 @@ a custom location for Briefcase's tools.
 """
             )
 
+        if not os.path.exists(data_path):
+            try:
+                # The Windows Store version of Python can redirect filesystem
+                # interactions within %LOCALAPPDATA% to a sandboxed location.
+                # To bypass this, the Briefcase cache directory creation is
+                # performed via ``cmd.exe`` in a different process. Once this
+                # directory exists in the "real" %LOCALAPPDATA%, Windows will
+                # allow normal interactions without attempting to sandbox them.
+                if platform.system() == "Windows":
+                    subprocess.run(
+                        ["mkdir", data_path],
+                        shell=True,
+                        check=True,
+                        stdout=subprocess.DEVNULL,
+                        stderr=subprocess.DEVNULL,
+                    )
+                else:
+                    os.makedirs(data_path, exist_ok=True)
+            except (subprocess.CalledProcessError, OSError):
+                raise BriefcaseCommandError(
+                    f"""
+Failed to create the Briefcase directory to store tools and support files:
+
+    {data_path}
+
+You can set the environment variable BRIEFCASE_HOME to specify
+a custom location for Briefcase's tools.
+
+"""
+                )
+
         return Path(data_path)
 
     def _command_factory(self, command_name: str):
diff --git a/tox.ini b/tox.ini
index 87eb94ab..de4e80c7 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,8 @@
 # The leading comma generates the "py" environment.
 [testenv:py{,38,39,310,311,312}]
+passenv =
+    # LOCALAPPDATA is needed to test data directory creation on Windows.
+    LOCALAPPDATA
 extras =
     dev
 commands =


Test patch:
diff --git a/tests/commands/base/test_paths.py b/tests/commands/base/test_paths.py
index b1eeec92..cfc34b5c 100644
--- a/tests/commands/base/test_paths.py
+++ b/tests/commands/base/test_paths.py
@@ -1,5 +1,7 @@
 import os
 import platform
+import subprocess
+from contextlib import suppress
 from pathlib import Path
 
 import pytest
@@ -9,6 +11,44 @@ from briefcase.exceptions import BriefcaseCommandError
 from .conftest import DummyCommand
 
 
+@pytest.mark.skipif(platform.system() != "Windows", reason="Windows specific tests")
+def test_path_is_realpath(tmp_path):
+    """Briefcase's data path matches its realpath.
+
+    For the Windows Store Python, filesystem interaction with
+    `%LOCALAPPDATA%` can be redirected to a sandboxed location. However,
+    `os.path.realpath()` will reveal such a redirection. This test
+    ensures that Briefcase bypasses the sandboxing.
+    """
+    data_path = Path(os.environ["LOCALAPPDATA"]) / "realpathbase"
+    try:
+        command = DummyCommand(data_path=data_path)
+        assert command.data_path == Path(os.path.realpath(data_path))
+    finally:
+        with suppress(FileNotFoundError):
+            os.rmdir(data_path)
+
+
+def test_data_path_creation_failure(tmp_path, monkeypatch):
+    """An error is raised if the data path cannot be created."""
+
+    def raise_calledprpcesserror(*a, **kw):
+        raise subprocess.CalledProcessError(returncode=1, cmd=["mkdir", str(data_path)])
+
+    # On Linux and macOS, use /etc/ to raise an OSError
+    data_path = Path("/etc/mydatadir")
+
+    # Patch run() since it's apparently quite difficult to find a
+    # location in Windows that is guaranteed to throw an error...
+    monkeypatch.setattr(subprocess, "run", raise_calledprpcesserror)
+
+    with pytest.raises(
+        BriefcaseCommandError,
+        match=r"Failed to create the Briefcase directory to store tools and support files:",
+    ):
+        DummyCommand(data_path=data_path)
+
+
 def test_space_in_path(tmp_path):
     """The briefcase data path cannot contain spaces."""
     with pytest.raises(
@@ -59,7 +99,9 @@ def templated_path_test(
         base_path=tmp_path / "base",
         data_path=data_path.format(tmp_path=tmp_path) if data_path else None,
     )
+
     assert command.base_path == tmp_path / "base"
+    assert command.data_path.exists()
     assert command.data_path == Path(
         os.path.expanduser(expected_data_path.format(tmp_path=tmp_path))
     )
@@ -80,14 +122,14 @@ def templated_path_test(
             "{tmp_path}custom/briefcase/path",
         ),
         (  # Explicit paths
-            "/path/to/data",
+            "{tmp_path}path/to/data",
             None,
-            "/path/to/data",
+            "{tmp_path}path/to/data",
         ),
         (  # Explicit paths and an environment variable present
-            "/path/to/data",
+            "{tmp_path}path/to/data",
             "{tmp_path}custom/briefcase/path",
-            "/path/to/data",
+            "{tmp_path}path/to/data",
         ),
     ],
 )
@@ -122,14 +164,14 @@ def test_macOS_paths(
             "{tmp_path}custom\\briefcase\\path",
         ),
         (  # Explicit paths
-            "Z:\\path\\to\\data",
+            "{tmp_path}path\\to\\data",
             None,
-            "Z:\\path\\to\\data",
+            "{tmp_path}path\\to\\data",
         ),
         (  # Explicit paths and an environment variable present
-            "Z:\\path\\to\\data",
+            "{tmp_path}path\\to\\data",
             "{tmp_path}\\briefcase\\path",
-            "Z:\\path\\to\\data",
+            "{tmp_path}path\\to\\data",
         ),
     ],
 )
@@ -164,14 +206,14 @@ def test_windows_paths(
             "{tmp_path}custom/briefcase/path",
         ),
         (  # Explicit paths
-            "/path/to/data",
+            "{tmp_path}path/to/data",
             None,
-            "/path/to/data",
+            "{tmp_path}path/to/data",
         ),
         (  # Explicit paths and an environment variable present
-            "/path/to/data",
+            "{tmp_path}path/to/data",
             "{tmp_path}custom/briefcase/path",
-            "/path/to/data",
+            "{tmp_path}path/to/data",
         ),
     ],
 )