Repository: Hubs-Foundation/reticulum
Pull request #727: Updates Elixir, Erlang OTP, Alpine & GitOps
Author: login: DougReeder
Created: 2025-04-16T22:31:51Z | Merged: 2025-04-25T21:32:09Z

## What?
Updates Elixir and Erlang/OTP to latest that pass tests


## Why?
The proximal cause was GitOps lint-and-test being blocked for building using an old version of Ubuntu.
As we need to test for regressions, it made sense to update what else we can.

## How to test
I. GitOps lint-and-test Action should succed for Elixir 1.16 and Erlang/OTP 25.3 (the versions in turkeyDockerfile)
II. GitOps turkeyGitops Action should succeed (when running in the HubsFoundation repo)
III. hubs-compose with this should build and not have regressions
    1. run the `down` script
    2. cd to reticulum in hubs-compose
    3. pull this branch
    4. run `mix deps.get`
    5. run the `up` script
    6. if the reticulum container builds successfully, the Elixir automated tests have passed
    7. when all containers are running, smoke test Hubs

(This is currently working without any apparent errors in https://hubs.hominidsoftware.com/

## Documentation of functionality
No changes to functionality are expected.

## Limitations
The tests are run under Ubuntu, which has no equivalent to the Alpine used to build the image

## Alternatives considered
If there are regressions, we can switch to older version of Elixir and Erlang, that is still newer than what we're currently using.



Diff:
diff --git a/.github/workflows/lint-and-test.yml b/.github/workflows/lint-and-test.yml
index 55ba09afe..f08ea51da 100644
--- a/.github/workflows/lint-and-test.yml
+++ b/.github/workflows/lint-and-test.yml
@@ -3,8 +3,107 @@ name: lint-and-test
 on: [push, workflow_dispatch]
 
 jobs:
-  lint-and-test:
-    runs-on: ubuntu-22.04
+  test-current:
+    strategy:
+      fail-fast: false
+      matrix:
+        os:
+          - 'ubuntu-22.04'
+          - 'ubuntu-24.04'
+
+    runs-on: ${{ matrix.os }}
+
+    services:
+      postgres:
+        image: postgres:10
+        env:
+          POSTGRES_USER: admin
+          POSTGRES_PASSWORD: admin
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        ports:
+          - 5432:5432
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: erlef/setup-beam@v1
+        with:
+          elixir-version: '1.16'
+          otp-version: '25.3'
+
+      - uses: actions/cache@v4
+        with:
+          path: |
+            deps
+            _build
+          key: mix-${{ hashFiles('mix.lock') }}
+
+      - run: mix local.hex --force
+      - run: mix local.rebar --force
+      - run: mix deps.get
+      - run: mix compile   # after fixing warnings, add --warnings-as-errors
+      - run: mix test
+
+
+  test-matrix:
+    # if a matrix-job with newer versions passes, advance the current versions!
+    continue-on-error: true
+
+    strategy:
+      fail-fast: false
+      matrix:
+        versions:
+#          - elixir: '1.14'
+#            otp: '24.2'
+#          - elixir: '1.14'
+#            otp: '24.3'
+#          - elixir: '1.14'
+#            otp: '25.3'
+#
+#          - elixir: '1.15'
+#            otp: '24.3'
+#          - elixir: '1.15'
+#            otp: '25.3'
+#          - elixir: '1.15'
+#            otp: '26.2'
+
+          - elixir: '1.16'
+            otp: '24.3'
+          - elixir: '1.16'
+            otp: '25.3'
+          - elixir: '1.16'
+            otp: '26.2'
+
+          - elixir: '1.17'
+            otp: '25.3'
+          - elixir: '1.17'
+            otp: '26.2'
+          - elixir: '1.17'
+            otp: '27.3'
+
+        os:
+          - 'ubuntu-22.04'
+          - 'ubuntu-24.04'
+
+        exclude:
+          - versions:   # current
+              elixir: '1.16'
+              otp: '25.3'
+
+#          - versions:   # not supported
+#              elixir: '1.14'
+#              otp: '24.2'
+#            os: 'ubuntu-24.04'
+          - versions:   # not supported
+              elixir: '1.16'
+              otp: '24.3'
+            os: 'ubuntu-22.04'
+
+    runs-on: ${{ matrix.os }}
 
     services:
       postgres:
@@ -25,8 +124,8 @@ jobs:
 
       - uses: erlef/setup-beam@v1
         with:
-          otp-version: '24.2'
-          elixir-version: '1.14'
+          elixir-version: ${{ matrix.versions.elixir }}
+          otp-version: ${{ matrix.versions.otp }}
 
       - uses: actions/cache@v4
         with:
@@ -38,6 +137,43 @@ jobs:
       - run: mix local.hex --force
       - run: mix local.rebar --force
       - run: mix deps.get
-      - run: mix compile --warnings-as-errors
+      - run: mix compile   # after fixing warnings, add --warnings-as-errors
       - run: mix test
+
+
+  pseudo-lint:
+    continue-on-error: true   # TODO: use `mix format` & check in files, then set this false
+    strategy:
+      fail-fast: false
+      matrix:
+        versions:
+          - elixir: '1.14'
+            otp: '24.3'
+
+          - elixir: '1.15'
+            otp: '24.3'
+
+          - elixir: '1.16'
+            otp: '24.3'
+
+          - elixir: '1.17'
+            otp: '25.3'
+
+    runs-on: 'ubuntu-24.04'
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: erlef/setup-beam@v1
+        with:
+          elixir-version: ${{ matrix.versions.elixir }}
+          otp-version: ${{ matrix.versions.otp }}
+
+      - uses: actions/cache@v4
+        with:
+          path: |
+            deps
+            _build
+          key: mix-${{ hashFiles('mix.lock') }}
+
       - run: mix format --check-formatted '{lib,priv,test,config}/**/*.{ex,exs}'
diff --git a/TurkeyDockerfile b/TurkeyDockerfile
index 7d88c83bd..9629412ef 100644
--- a/TurkeyDockerfile
+++ b/TurkeyDockerfile
@@ -1,7 +1,7 @@
 # syntax=docker/dockerfile:1
-ARG ALPINE_VERSION=3.16.2
-ARG ELIXIR_VERSION=1.14.3
-ARG ERLANG_VERSION=23.3.4.18
+ARG ALPINE_VERSION=3.21.3
+ARG ELIXIR_VERSION=1.16.3
+ARG ERLANG_VERSION=25.3.2.20
 
 FROM hexpm/elixir:${ELIXIR_VERSION}-erlang-${ERLANG_VERSION}-alpine-${ALPINE_VERSION} AS base
 RUN mix do local.hex --force, local.rebar --force
diff --git a/mix.lock b/mix.lock
index c8ecd056a..45c0ff287 100644
--- a/mix.lock
+++ b/mix.lock
@@ -79,7 +79,7 @@
   "sleeplocks": {:hex, :sleeplocks, "1.1.1", "3d462a0639a6ef36cc75d6038b7393ae537ab394641beb59830a1b8271faeed3", [:rebar3], [], "hexpm", "84ee37aeff4d0d92b290fff986d6a95ac5eedf9b383fadfd1d88e9b84a1c02e1"},
   "slugger": {:hex, :slugger, "0.3.0", "efc667ab99eee19a48913ccf3d038b1fb9f165fa4fbf093be898b8099e61b6ed", [:mix], [], "hexpm", "20d0ded0e712605d1eae6c5b4889581c3460d92623a930ddda91e0e609b5afba"},
   "socket": {:hex, :socket, "0.3.13", "98a2ab20ce17f95fb512c5cadddba32b57273e0d2dba2d2e5f976c5969d0c632", [:mix], [], "hexpm", "f82ea9833ef49dde272e6568ab8aac657a636acb4cf44a7de8a935acb8957c2e"},
-  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.6", "cf344f5692c82d2cd7554f5ec8fd961548d4fd09e7d22f5b62482e5aeaebd4b0", [:make, :mix, :rebar3], [], "hexpm", "bdb0d2471f453c88ff3908e7686f86f9be327d065cc1ec16fa4540197ea04680"},
+  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.7", "354c321cf377240c7b8716899e182ce4890c5938111a1296add3ec74cf1715df", [:make, :mix, :rebar3], [], "hexpm", "fe4c190e8f37401d30167c8c405eda19469f34577987c76dde613e838bbc67f8"},
   "statix": {:hex, :statix, "1.4.0", "c822abd1e60e62828e8460e932515d0717aa3c089b44cc3f795d43b94570b3a8", [:mix], [], "hexpm", "507373cc80925a9b6856cb14ba17f6125552434314f6613c907d295a09d1a375"},
   "stream_data": {:git, "https://github.com/whatyouhide/stream_data.git", "c7ef8ef9c1fe78e6f404272c7129c4eac83db53c", [ref: "c7ef8ef"]},
   "telemetry": {:hex, :telemetry, "0.4.3", "a06428a514bdbc63293cd9a6263aad00ddeb66f608163bdec7c8995784080818", [:rebar3], [], "hexpm", "eb72b8365ffda5bed68a620d1da88525e326cb82a75ee61354fc24b844768041"},


Reviews:
user: login: Exairnous
body: Nice.  Thank you.  This sounds great.

Is the matrix of different version tests planned to be left in or is it just for confirmation that the proposed version we're moving to works?

The lint part fails for me when I run it through my fork, but I noticed that it doesn't include a version for Elixir 1.16 and Erlang 25.3 (it does 1.16 and 24.3 or 1.17 and 25.3).

The ret-turkey.yml changes have been moved to #731, so they can be removed from this PR.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-25T01:55:21Z
user: login: Exairnous
body: 
> My thinking is that the "linting" only needs to be done once per version of Elixir. I use the earliest matching version of Erlang/OTP on the assumption that if the syntax is good for the first version, it's good for the others.

Ah, okay.  That makes sense.  Thanks.

I found a typo in the `test-current` section, but otherwise the tests looks good.

About the linting, I forgot that we had files that weren't formatted as expected, so they should fail the workflow, my bad.  Sorry! :flushed:

Would it be possible to add in a `current` property to the `lint` matrix and set `continue-on-error` to not `current`?  This would seem to fit with the behaviour of the tests (i.e. fails on non-current versions are fine, but a fail on the current version fails the whole workflow).
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-25T05:43:01Z
user: login: DougReeder
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T07:52:02Z
user: login: Exairnous
body: LGTM.  Thanks.  Merging.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-25T21:31:54Z

Review comments:
diff_hunk: @@ -3,8 +3,105 @@ name: lint-and-test
 on: [push, workflow_dispatch]
 
 jobs:
-  lint-and-test:
-    runs-on: ubuntu-22.04
+  test-current:
+    strategy:
+      fail-fast: false
+      matrix:
+        os:
+          - 'ubuntu-22.04'
+          - 'ubuntu-24.04'
+
+    env:
+      elixir_version: '1.16'
+      otp_version: '25.3'
+
+    runs-on: ${{ matrix.os }}
+
+    services:
+      postgres:
+        image: postgres:10
+        env:
+          POSTGRES_USER: admin
+          POSTGRES_PASSWORD: admin
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        ports:
+          - 5432:5432
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: erlef/setup-beam@v1
+        with:
+          otp-version: $otp_version
+          elixir-version: $elixir_version
path: .github/workflows/lint-and-test.yml
user: login: Exairnous
body: Typos:
```suggestion
          otp-version: ${{ env.otp_version }}
          elixir-version: ${{ env.elixir_version }}
```
created_at: 2025-04-25T05:18:57Z
author_association: MEMBER
diff_hunk: @@ -3,8 +3,105 @@ name: lint-and-test
 on: [push, workflow_dispatch]
 
 jobs:
-  lint-and-test:
-    runs-on: ubuntu-22.04
+  test-current:
+    strategy:
+      fail-fast: false
+      matrix:
+        os:
+          - 'ubuntu-22.04'
+          - 'ubuntu-24.04'
+
+    env:
+      elixir_version: '1.16'
+      otp_version: '25.3'
+
+    runs-on: ${{ matrix.os }}
+
+    services:
+      postgres:
+        image: postgres:10
+        env:
+          POSTGRES_USER: admin
+          POSTGRES_PASSWORD: admin
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        ports:
+          - 5432:5432
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: erlef/setup-beam@v1
+        with:
+          otp-version: $otp_version
+          elixir-version: $elixir_version
path: .github/workflows/lint-and-test.yml
user: login: DougReeder
body: It appears the syntax I was using is allowed in steps, but not here. :-(  Thanks!
created_at: 2025-04-25T07:52:02Z
author_association: MEMBER

PR comments:
user: login: DougReeder
created_at: 2025-04-20T09:49:08Z
body: It appears that this will fix #711 and support https://discord.com/channels/498741086295031808/1158476691384062012/1361647882305798335
author_association: MEMBER
user: login: DougReeder
created_at: 2025-04-25T02:43:34Z
body: My thinking is that the "linting" only needs to be done once per version of Elixir.  I use the earliest matching version of Erlang/OTP on the assumption that if the syntax is good for the first version, it's good for the others.
author_association: MEMBER
user: login: Exairnous
created_at: 2025-04-25T13:07:11Z
body: The workflow fails because it can't access env variables in the matrix specification.  This should work to get around that.
```yaml
env:
  elixir_version_current: '1.16'
  otp_version_current: '25.3'

jobs:
  variables:
    runs-on: ubuntu-latest
    outputs:
      elixir_version_current: ${{ env.elixir_version_current }}
      otp_version_current: ${{ env.otp_version_current }}
    steps:
      - name: Job to satisfy syntax requirements
        run: |
          :
      
  <snip>
  
  test-matrix:
    needs: [variables]
    continue-on-error: true

    strategy:
      fail-fast: false
      matrix:
      <snip>
        exclude:
          - versions:
              elixir: ${{ needs.variables.outputs.elixir_version_current }}
              otp: ${{ needs.variables.outputs.otp_version_current }}
```





References:
https://stackoverflow.com/questions/74072206/github-actions-use-variables-in-matrix-definition
https://stackoverflow.com/questions/73305126/passing-environment-variables-as-inputs-to-a-reusable-workflow/77512339#77512339
author_association: MEMBER
user: login: DougReeder
created_at: 2025-04-25T15:46:08Z
body: Your approach would work fine, but since the Elixir and Erlang/OTP versions need to be updated  in TurkeyDockerfile as well, I think hard-coding is more consistent. (You **have** to update multiple files, so you're more-or-less doing a search and replace anyway.)  This should be documented, but where?
author_association: MEMBER
user: login: rawnsley
created_at: 2025-05-20T08:31:39Z
body: @DougReeder I think there might be a problem with version upgrades: https://github.com/Hubs-Foundation/reticulum/issues/735
author_association: NONE