PyPyInstaller Powershell Module Initial Code Push
Tags: powershell, python, opensource, windows
Published: 2023-09-25 12:58:23

{%- # TOC start (generated with https://github.com/derlin/bitdowntoc) -%}

- [Motivation](#motivation)
- [Why Powershell](#why-powershell)
- [Code Layout](#code-layout)
- [Test Test Test...](#test-test-test)
- [Interesting Powershell](#interesting-powershell)
- [What's Next](#whats-next)

{%- # TOC end -%}

I've already written about this in a few places but I've finally done an initial release of [PyPyInstaller](https://github.com/cwgem/pypy-powershell-install). This is one of the projects I mentioned in my last [distraction post](https://dev.to/cwprogram/dev-stuff-distracting-me-from-article-writing-4a4f). 

## Motivation

One of the articles I wanted to write was for managing multiple installations of Python. Unfortunately, I noticed that PyPy seemed to be left out in terms of Windows support. There wasn't a solid option to both add it to PATH and add the appropriate [PEP 514](https://peps.python.org/pep-0514/) entries to be listed in the [Python Launcher](https://docs.python.org/3/using/windows.html#python-launcher-for-windows). I decided this would be an interesting project to work on as a hobby type deal.

## Why Powershell

There are many Windows solutions that run off a command line or GUI executable. The problem is you can't really see what's going on. Even if the source code is available you don't really know it's the code in that executable unless you build it yourself. Making this a Powershell module means everything is a script and the user can easily see every command that's being run.

Another reason is that using commandlets allows for flexibility in how installations are done. For example, there's a command for finding the latest installation which can be piped into the command to install a PyPy version:

```powershell
> Find-PyPyLatest -PythonSeries "3.9", "3.10" | Install-PyPy
```

But what if you really need that specific version? You can just run `Install-PyPy` by itself with the appropriate option:

```
> Install-PyPy -PythonVersions "3.8.16"
```

I think it helps strike a balance between just get it done and still allow some more controlled uses.

## Code Layout

The layout of the code looks like this:

```
├─.github
│  └─workflows
├─.vscode
├─src
│  └─PyPyInstaller
│      ├─Functions
│      │  ├─Private
│      │  └─Public
│      └─Wrappers
└─test
    └─fixtures
```

`src` contains the source of the module itself. Under this is a `Functions` directory containing the actual `ps1` files. This directory is also where code coverage looks to ensure tests are built against everything. Functions are then broken up into `Public` and `Private`. `Private` functions are meant to support the `Public` versions. It's then the `Public` versions that are exposed to the user. `test` contain the Pester tests used to ensure the code works.

## Test Test Test...

The is one of the first personal projects where I really went all out on testing. Part of it was because PyPyInstaller touches PATH and registry keys. At the moment the code coverage is at 99% (I still have a big question mark on how to deal with the last %):

```
Tests completed in 3.35s
Tests Passed: 37, Failed: 0, Skipped: 0 NotRun: 0
Processing code coverage result.
Covered 99.46% / 90%. 185 analyzed Commands in 8 Files.
```

[Pester](https://pester.dev/) is what's used for most Powershell testing. While it has some pretty amazing features, it does have issues with .NET class method mocking and sometimes scope can be weird. Here's a simple example:

```Powershell
BeforeAll {
    $PackageRoot = "$PSScriptRoot\..\src\PyPyInstaller\"
    Remove-Module PyPyInstaller
    Import-Module $PackageRoot
    . "$PackageRoot\Functions\Private\Utility.ps1"
}
InModuleScope PyPyInstaller {
    Describe "Update-PyPyMirror" {
        BeforeAll {
            $TestRootPath = "$env:temp"
            Mock -CommandName Read-PyPyInstallerConfig -MockWith { return @{ RootPath = $TestRootPath } }
            Mock -CommandName Out-File -ParameterFilter { $FilePath -eq "$TestRootPath\versions.json" } -MockWith { return $null }
            Mock -CommandName Invoke-WebRequest -ParameterFilter { $Uri -eq "https://buildbot.pypy.org/mirror/versions.json" } -MockWith { return @{ Content = Get-Content "$PSScriptRoot\fixtures\test_versions.json" } }
        }
        It "Passes Downloads Mirror JSON" {
            Update-PyPyMirror
            Assert-MockCalled Invoke-WebRequest -Exactly -Times 1 -ParameterFilter { $Uri -eq "https://buildbot.pypy.org/mirror/versions.json" } -Scope It
            Assert-MockCalled Out-File -ParameterFilter { $FilePath -eq "$TestRootPath\versions.json" }
        }
    }
}

```

First `BeforeAll` imports the code that will be tested. Then `InModuleScope` is used to deal with the scope oddities mentioned. `Describe` has the function being mentioned, with an additional `BeforeAll` declaration. This is where the mocking happens in the this particular test case. As an example for one of the mocks:

```powershell
Mock -CommandName Invoke-WebRequest -ParameterFilter { $Uri -eq "https://buildbot.pypy.org/mirror/versions.json" } -MockWith { return @{ Content = Get-Content "$PSScriptRoot\fixtures\test_versions.json" } }
```

This will mock the `Invoke-Request` command when the `-Uri` argument is the PyPy mirror `versions.json` file. Instead of returning that it will return a test version of the file with a specific layout for my tests. When the code is run, it asserts that my mocked version of `Invoke-WebRequest` was called instead of the actual command:

```powershell
Assert-MockCalled Invoke-WebRequest -Exactly -Times 1
``` 

This makes sure I'm not on the bad side of PyPy mirror infra by hitting their versions file every time I run the test cases! It can also test the registry:

```powershell
    Context "Registry Keys Don't Exist" {
        BeforeAll {
            New-Item -Path "TestRegistry:\Software\Python" -Force
            Mock -CommandName Test-Path -MockWith { return $false }
        }
        It "Passes Adds Entries To Registry" {
            Set-PyPyLauncherEntry -PythonVersion "3.10.12" -PyPyVersion "7.3.12" -InstallPath "C:\PyPy" -RegistryDrive "TestRegistry"

            $CoreInfo = Get-ItemProperty "TestRegistry:\Software\Python\PyPyInstaller\3.10"
            $CoreInfo.DisplayName | Should -Be "PyPy 3.10.12"
            $CoreInfo.SupportUrl | Should -Be "https://github.com/cwgem/pypy-powershell-install"
            $CoreInfo.Version | Should -Be "7.3.12"
            $CoreInfo.SysVersion | Should -Be "3.10.12"
            $CoreInfo.SysArchitecture | Should -Be "64bit"

            $InstallPathInfo = Get-ItemProperty "TestRegistry:\Software\Python\PyPyInstaller\3.10\InstallPath"
            $InstallPathInfo.'(default)' | Should -Be "C:\PyPy"
            $InstallPathInfo.ExecutablePath | Should -Be "C:\PyPy\pypy.exe"
            $InstallPathInfo.WindowedExecutablePath | Should -Be "C:\PyPy\pypyw.exe"
        }
    }
```

It does so by creating a `TestRegistry:` entry that can be used in place of the actual registry. This is something I then pass in to the command that deals with the registry as a path, and then assert against the various registry keys.

## Interesting Powershell

Of particular interest was how Powershell was easily able to handle pipeline input:

```powershell
function Find-PyPyLatest {
    [CmdletBinding()]
    param (
        [Parameter(ValueFromPipeline)]
        [string[]]
        $PythonSeries
    )
```

Here's an example which accepts the value of PythonSeries as a pipeline argument (I can also just pass it in as the `-PythonSeries` argument):

```powershell
> "3.9", "3.10" | Find-PyPyLatest

pypy_version   : 7.3.12
python_version : 3.9.17
stable         : True
latest_pypy    : True
date           : 2023-06-16
files          : {@{filename=pypy3.9-v7.3.12-aarch64.tar.bz2; arch=aarch64; platform=linux; download_url=https://downlo
                 ads.python.org/pypy/pypy3.9-v7.3.12-aarch64.tar.bz2}, @{filename=pypy3.9-v7.3.12-linux32.tar.bz2; arch
                 =i686; platform=linux; download_url=https://downloads.python.org/pypy/pypy3.9-v7.3.12-linux32.tar.bz2}
                 , @{filename=pypy3.9-v7.3.12-linux64.tar.bz2; arch=x64; platform=linux; download_url=https://downloads
                 .python.org/pypy/pypy3.9-v7.3.12-linux64.tar.bz2}, @{filename=pypy3.9-v7.3.12-macos_x86_64.tar.bz2; ar
                 ch=x64; platform=darwin; download_url=https://downloads.python.org/pypy/pypy3.9-v7.3.12-macos_x86_64.t
                 ar.bz2}…}

pypy_version   : 7.3.12
python_version : 3.10.12
stable         : True
latest_pypy    : True
date           : 2023-06-16
files          : {@{filename=pypy3.10-v7.3.12-aarch64.tar.bz2; arch=aarch64; platform=linux; download_url=https://downl
                 oads.python.org/pypy/pypy3.10-v7.3.12-aarch64.tar.bz2}, @{filename=pypy3.10-v7.3.12-linux32.tar.bz2; a
                 rch=i686; platform=linux; download_url=https://downloads.python.org/pypy/pypy3.10-v7.3.12-linux32.tar.
                 bz2}, @{filename=pypy3.10-v7.3.12-linux64.tar.bz2; arch=x64; platform=linux; download_url=https://down
                 loads.python.org/pypy/pypy3.10-v7.3.12-linux64.tar.bz2}, @{filename=pypy3.10-v7.3.12-macos_x86_64.tar.
                 bz2; arch=x64; platform=darwin; download_url=https://downloads.python.org/pypy/pypy3.10-v7.3.12-macos_
                 x86_64.tar.bz2}…}
```

Objects can also be accepted via pipeline:

```powershell
function Install-PyPy {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)]
        [string[]]
        [Alias('python_version')]
        $PythonVersions,
```

This will bind the `python_version` property of any passed in objects to the `PythonVersions` argument. It's is quite a blessing from someone who has had to deal with the *NIX version of pipe handling... Another oddity I found was adding to PATH:

```powershell
function Get-PyPyPathEnvironmentVariable {
    return [System.Environment]::GetEnvironmentVariable('Path', 'User')
}

function Set-PyPyPathEnvironmentVariable {
    param (
        [Parameter()]
        [String]
        $NewPath
    )
    [System.Environment]::SetEnvironmentVariable('Path', $NewPath, 'User')
}
```

These are wrappers around .NET class methods to make Pester play nice as it can't mock those very well (instead it mocks the commands that wrap them). The issue I found was that normal PATH registration in Powershell doesn't fire off the [WM_SETTINGCHANGE](https://learn.microsoft.com/en-us/windows/win32/winmsg/wm-settingchange) event. So instead the .NET method is used which does fire off the event. This allows for a simple restart of a terminal session to pick up the changes.

## What's Next

This is far from complete and there's still a lot that can be done:

- Code layout consolidation
- Test case consolidation
- List/Remove/Update PyPyInstaller Installations
- Developer documentation
- GitHub workflows for testing and maybe a coverage badge

Can't wait to see where this project leads. Even if it doesn't go places I learned quite a lot about Powershell!