Repository: thesofproject/sof
Pull request #9870: [SKIP SOFCI-TEST] topology: build using a local ALSA install
Author: login: lgirdwood
Created: 2025-03-03T16:43:10Z | Merged: 2025-05-08T14:49:39Z

Build topologies using a locally installed ALSA lib and ALSA utils without having to use Docker or upgrade the system ALSA installation.


Diff:
diff --git a/.github/workflows/installer.yml b/.github/workflows/installer.yml
deleted file mode 100644
index 012b1bddd42e..000000000000
--- a/.github/workflows/installer.yml
+++ /dev/null
@@ -1,63 +0,0 @@
----
-# Tools that can save round-trips to github and a lot of time:
-#
-# yamllint -f parsable this.yml
-# pip3 install ruamel.yaml.cmd
-# yaml merge-expand this.yml exp.yml && diff -w -u this.yml exp.yml
-#
-# github.com also has a powerful web editor that can be used without
-# committing.
-
-name: installer
-
-# 'workflow_dispatch' allows running this workflow manually from the
-# 'Actions' tab
-# yamllint disable-line rule:truthy
-on: [push, pull_request, workflow_dispatch, workflow_call]
-
-jobs:
-  checktree:
-    runs-on: ubuntu-22.04
-
-    strategy:
-      fail-fast: false
-      matrix:  # just a vector in this case
-        make_env: [
-          IPC_VERSION= UNSIGNED_list='imx8m' SIGNED_list=,  # default version
-        ]
-
-    steps:
-      - uses: actions/checkout@v4
-        with: {fetch-depth: 0, submodules: recursive, filter: 'tree:0'}
-
-      - name: docker
-        run: docker pull thesofproject/sof && docker tag thesofproject/sof sof
-
-      # -j3 tests the ability to build multiple platforms
-      # concurrently. It makes the build log unreadable, so retry with
-      # a single thread in case of failure
-      - name: build all and stage
-        env:
-          gh_make_env: ${{ matrix.make_env }}
-        # Use 'eval' to have two different levels of whitespace:
-        # 1. between vars, and 2. inside _lists. Quoting is hard, even
-        # harder through 'docker-run.sh'.
-        run: eval env_array=($gh_make_env) &&
-             ./scripts/docker-run.sh
-                make -j3 -C installer/ tarball "${env_array[@]}" ||
-            VERBOSE=1 NO_PROCESSORS=1 USE_XARGS=no
-             ./scripts/docker-run.sh
-                make -j1 -C installer/ tarball "${env_array[@]}"
-
-      - name: check staging tree
-        env:
-          gh_make_env: ${{ matrix.make_env }}
-        run: eval env_array=($gh_make_env) &&
-             make -C installer/ checktree "${env_array[@]}"
-
-      - name: test make cleanall
-        run: |
-           make -C installer cleanall
-           # Make sure there's nothing left
-           rm -rf scripts/kconfig/__pycache__/
-           ! git status --porcelain --ignored | grep .
diff --git a/.github/workflows/testbench.yml b/.github/workflows/testbench.yml
index 9c1dfdfb22ec..7a7b32a99ac0 100644
--- a/.github/workflows/testbench.yml
+++ b/.github/workflows/testbench.yml
@@ -29,33 +29,63 @@ on:
   workflow_call:
 
 jobs:
-
-  build-run:
-    runs-on: ubuntu-22.04
+  build-and-test:
+    runs-on: ubuntu-24.10
 
     steps:
-      - uses: actions/checkout@v4
-        with: {fetch-depth: 0, filter: 'tree:0'}
-
-      - name: docker
-        run: docker pull thesofproject/sof && docker tag thesofproject/sof sof
+      - name: Checkout SOF repository (PR source)
+        uses: actions/checkout@v4
+        with:
+          path: sof
 
       - name: apt get
         run: sudo apt-get update &&
-             sudo apt-get -y install valgrind alsa-utils libasound2-dev ninja-build
-             octave octave-signal
+               sudo apt-get -y install valgrind ninja-build
+                  octave octave-signal automake autoconf libtool
+                  gettext linux-headers-6.11.0-8
+
+      - name: Build Alsa-lib
+        run: |
+          cd ${GITHUB_WORKSPACE}
+          git clone https://github.com/thesofproject/alsa-lib.git
+          cd alsa-lib
+          git checkout df8f1cc1ec9d9ee15be5e2c23ad25b9389fd8766 -b build
+          ./gitcompile --prefix=${GITHUB_WORKSPACE}/tools
+          make install
+
+      - name: Build Alsa-utils
+        run: |
+          cd ${GITHUB_WORKSPACE}
+          git clone https://github.com/thesofproject/alsa-utils.git
+          cd alsa-utils
+          git checkout 0ffa105942a06cdfa98e5918b8dc82e3cac12792 -b build
+          ./gitcompile --prefix=${GITHUB_WORKSPACE}/tools \
+             --with-alsa-prefix=${GITHUB_WORKSPACE}/tools \
+             --with-alsa-inc-prefix=${GITHUB_WORKSPACE}/tools/include \
+             --with-sysroot=${GITHUB_WORKSPACE}/tools \
+             --with-udev-rules-dir=${GITHUB_WORKSPACE}/tools \
+             PKG_CONFIG_PATH=${GITHUB_WORKSPACE}/tools \
+             LDFLAGS=-L${GITHUB_WORKSPACE}/tools/lib \
+             --disable-old-symbols \
+             --enable-alsatopology \
+             --with-asound-state-dir=${GITHUB_WORKSPACE}/tools/var/lib/alsa \
+             --with-systemdsystemunitdir=${GITHUB_WORKSPACE}/tools/lib/systemd/system
+          make install
 
       # testbench needs some topologies.
       - name: build test topologies
-        run: CMAKE_BUILD_TYPE=Release ./scripts/docker-run.sh
-          ./scripts/build-tools.sh -Y ||
+        run: |
+          CMAKE_BUILD_TYPE=Release ./sof/scripts/build-tools.sh -Y ||
              VERBOSE=1 NO_PROCESSORS=1 USE_XARGS=no
-             CMAKE_BUILD_TYPE=Release ./scripts/docker-run.sh
-          ./scripts/build-tools.sh -Y
+          CMAKE_BUILD_TYPE=Release ./sof/scripts/build-tools.sh -Y
 
+      # build testbench
       - name: build testbench
-        run: ./scripts/rebuild-testbench.sh ||
-             ./scripts/rebuild-testbench.sh -j 1
+        run: |
+          ./sof/scripts/rebuild-testbench.sh ||
+          ./sof/scripts/rebuild-testbench.sh -j 1
 
+      # run testbench
       - name: run testbench
-        run: ./scripts/host-testbench.sh
+        run: |
+          ./sof/scripts/host-testbench.sh
diff --git a/scripts/build-alsa-tools.sh b/scripts/build-alsa-tools.sh
new file mode 100755
index 000000000000..5e1e7f75d39d
--- /dev/null
+++ b/scripts/build-alsa-tools.sh
@@ -0,0 +1,127 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation. All rights reserved.
+
+# fail immediately on any errors
+set -e
+
+# Array of ALSA Git repository URLs.  Add or remove repositories as needed.
+declare -a REPOS=(
+    "https://github.com/thesofproject/alsa-lib.git"
+    "https://github.com/thesofproject/alsa-utils.git"
+    # Add more repositories here...
+)
+
+# Commit ID to check for (optional). If specified, the script will update
+# the repository if this commit ID is not found.  Leave empty to skip.
+# This array order must align with REPO array above.
+declare -a COMMIT_ID=(
+    "df8f1cc1ec9d9ee15be5e2c23ad25b9389fd8766"
+    "09550cd393b1a7d307ee6f26637b1ed7bd275e38"
+    # Add more IDs here...
+)
+
+# Directory where repositories will be cloned/updated.
+if [[ -z "$SOF_WORKSPACE" ]]; then
+  # Environment variable is empty or unset so use default
+  BASE_DIR="$HOME/work/sof"
+else
+  # Environment variable exists and has a value
+  BASE_DIR="$SOF_WORKSPACE"
+fi
+
+# Arguments to pass to ./configure for each repository.  Add or remove
+declare -a CONFIGURE_ARGS=(
+    "--prefix=${BASE_DIR}/tools"
+    "--prefix=${BASE_DIR}/tools \
+        --with-alsa-prefix=${BASE_DIR}/tools \
+        --with-alsa-inc-prefix=${BASE_DIR}/tools/include \
+        --with-sysroot=${BASE_DIR}/tools \
+        --with-udev-rules-dir=${BASE_DIR}/tools \
+        PKG_CONFIG_PATH=${BASE_DIR}/tools \
+        LDFLAGS=-L${BASE_DIR}/tools/lib \
+        --with-asound-state-dir=${BASE_DIR}/tools/var/lib/alsa \
+        --with-systemdsystemunitdir=${BASE_DIR}/tools/lib/systemd/system"
+)
+
+# Arguments to pass to make for each repository.  Add or remove arguments as needed.
+declare -a TARGET_ARGS=(
+    "--disable-old-symbols"
+    "--enable-alsatopology"
+)
+
+# Function to check if a commit ID exists in a repository
+check_commit() {
+    local repo_dir="$1"
+    local commit_id="$2"
+
+    if [ -z "$commit_id" ]; then
+        return 0  # Skip check if no commit ID is provided
+    fi
+
+    if ! git -C "$repo_dir" rev-parse --quiet --verify "$commit_id" >/dev/null 2>&1; then
+        return 1  # Commit ID not found
+    else
+        return 0  # Commit ID found
+    fi
+}
+
+
+# Function to update the repository
+update_repo() {
+    local repo_dir="$1"
+    echo "Updating repository: $repo_dir"
+    git -C "$repo_dir" fetch --all
+    git -C "$repo_dir" pull
+}
+
+# Function to build and install the repository
+build_and_install() {
+    local repo_dir="$1"
+    local configure_args="$2"
+    local target_args="$3"
+
+    echo "Building and installing: $repo_dir $configure_args $target_args"
+
+    if [ ! -f "$repo_dir/gitcompile" ]; then
+          echo "Error: gitcompile not found in $repo_dir" >&2
+          exit 1
+    fi
+
+    # if Makefile exists then we can just run make
+    if [ ! -f "$repo_dir/Makefile" ]; then
+        (cd "$repo_dir" && ./gitcompile $configure_args $target_args) || \
+            { echo "configure failed in $repo_dir"; exit 1; }
+    else
+        (cd "$repo_dir" && make -j) || { echo "make failed in $repo_dir"; exit 1; }
+    fi
+
+    (cd "$repo_dir" && make install) || { echo "make install failed in $repo_dir"; exit 1; }
+
+    echo "Build and installation complete for $repo_dir"
+}
+
+# Main script logic
+mkdir -p "$BASE_DIR"
+
+for ((i = 0; i < ${#REPOS[@]}; i++)); do
+    echo "Counter: $i, Value: ${REPOS[i]}"
+    repo_url=${REPOS[i]}
+
+    repo_name=$(basename "$repo_url" .git)  # Extract repo name
+    repo_dir="$BASE_DIR/$repo_name"
+
+    if [ ! -d "$repo_dir" ]; then
+        echo "Cloning repository: $repo_url"
+        git clone "$repo_url" "$repo_dir" || { echo "git clone failed for $repo_url"; exit 1; }
+    elif ! check_commit "$repo_dir" "${COMMIT_ID[i]}"; then
+        update_repo "$repo_dir"
+    else
+        echo "Repository $repo_name is up to date."
+    fi
+
+    build_and_install "$repo_dir" "${CONFIGURE_ARGS[i]}"
+
+done
+
+echo "All repositories processed."
diff --git a/scripts/build-tools.sh b/scripts/build-tools.sh
index de41f652debf..6304265820cd 100755
--- a/scripts/build-tools.sh
+++ b/scripts/build-tools.sh
@@ -16,7 +16,7 @@ Attention: the list of selected shortcuts below is _not_ exhaustive. To
 build _everything_ don't select any particular target; this will build
 CMake's default target "ALL".
 
-usage: $0 [-c|-f|-h|-l|-p|-t|-T|-X|-Y]
+usage: $0 [-c|-f|-h|-l|-p|-t|-T|-X|-Y|-A]
        -h Display help
 
        -c Rebuild ctl/
@@ -26,6 +26,7 @@ usage: $0 [-c|-f|-h|-l|-p|-t|-T|-X|-Y]
        -X Rebuild topology1 only
        -Y Rebuild topology2 only
        -t Rebuild test/topology/ (or tools/test/topology/tplg-build.sh directly)
+       -A Clone and rebuild local ALSA lib and utils.
 
        -C No build, only CMake re-configuration. Shows CMake targets.
 EOFUSAGE
@@ -106,6 +107,7 @@ main()
         fi
 
         DO_BUILD_ctl=false
+        DO_BUILD_alsa=false
         DO_BUILD_logger=false
         DO_BUILD_probes=false
         DO_BUILD_tests=false
@@ -118,7 +120,7 @@ main()
 
         # eval is a sometimes necessary evil
         # shellcheck disable=SC2034
-        while getopts "cfhlptTCXY" OPTION; do
+        while getopts "cfhlptTCXYA" OPTION; do
                 case "$OPTION" in
                 c) DO_BUILD_ctl=true ;;
                 l) DO_BUILD_logger=true ;;
@@ -128,12 +130,17 @@ main()
                 X) DO_BUILD_topologies1=true ;;
                 Y) DO_BUILD_topologies2=true ;;
                 C) CMAKE_ONLY=true ;;
+                A) DO_BUILD_alsa=true ;;
                 h) print_usage; exit 1;;
                 *) print_usage; exit 1;;
                 esac
         done
         shift "$((OPTIND - 1))"
 
+        if "$DO_BUILD_alsa"; then
+                $SOF_TOP/scripts/build-alsa-tools.sh
+        fi
+
         if "$CMAKE_ONLY"; then
                 reconfigure_build
                 print_build_info
diff --git a/tools/ctl/CMakeLists.txt b/tools/ctl/CMakeLists.txt
index 70735f88da65..5c4c2dd315b5 100644
--- a/tools/ctl/CMakeLists.txt
+++ b/tools/ctl/CMakeLists.txt
@@ -4,6 +4,9 @@ add_executable(sof-ctl
 	ctl.c
 )
 
+target_link_directories(sof-ctl BEFORE
+	PRIVATE "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/lib")
+
 target_link_libraries(sof-ctl PRIVATE
 	"-lasound"
 )
@@ -13,6 +16,7 @@ target_compile_options(sof-ctl PRIVATE
 )
 
 target_include_directories(sof-ctl PRIVATE
+	"${SOF_ROOT_SOURCE_DIRECTORY}/../tools/include"
 	"${SOF_ROOT_SOURCE_DIRECTORY}/src/include"
 	"${SOF_ROOT_SOURCE_DIRECTORY}"
 )
diff --git a/tools/topology/CMakeLists.txt b/tools/topology/CMakeLists.txt
index 33722120e41b..92a7acf6e5a6 100644
--- a/tools/topology/CMakeLists.txt
+++ b/tools/topology/CMakeLists.txt
@@ -1,18 +1,19 @@
 set(SOF_TOPOLOGY_BINARY_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
-
+set(SOF_ALSA_TOOLS_DIR "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/bin")
+set(ALSATPLG_CMD "${SOF_ALSA_TOOLS_DIR}/alsatplg")
 
 function(alsatplg_version  OUT_STATUS  OUT_VERSION)
-	execute_process(COMMAND alsatplg --version
+	execute_process(COMMAND ${ALSATPLG_CMD} --version
 	  RESULT_VARIABLE status
 	  OUTPUT_VARIABLE stdout
 	  OUTPUT_STRIP_TRAILING_WHITESPACE)
-	message(DEBUG "alsatplg --version: status=${status}, output=${stdout}")
+	message(DEBUG "${ALSATPLG_CMD} --version: status=${status}, output=${stdout}")
 
 	set(${OUT_STATUS} "${status}" PARENT_SCOPE)
 
         # Some error messages have already been printed on stderr
 	if(NOT status EQUAL 0)
-	  message(WARNING "alsatplg --version returned status: ${status},
+	  message(WARNING "${ALSATPLG_CMD} --version returned status: ${status},
 ${stdout}")
 	  return()
 	endif()
@@ -32,7 +33,7 @@ endfunction()
 # Longer story in #5192.
 alsatplg_version(STATUS ALSATPLG_VERSION_NUMBER)
 if(NOT STATUS EQUAL 0)
-	message(WARNING "alsatplg failed: ${STATUS}; all topologies skipped")
+	message(WARNING "${ALSATPLG_CMD} failed: ${STATUS}; all topologies skipped")
 	return()
 else()
 	if(${ALSATPLG_VERSION_NUMBER} VERSION_LESS "1.2.5")
@@ -67,7 +68,7 @@ macro(add_alsatplg_command)
 		# permissions are hardcoded and only the user can read
 		# the -o(utput) file.
 		# See bug https://github.com/alsa-project/alsa-utils/issues/126
-		COMMAND alsatplg \$\${VERBOSE:+-v 1} -c ${ARGV0} -o ${ARGV1}
+		COMMAND ${ALSATPLG_CMD} \$\${VERBOSE:+-v 1} -c ${ARGV0} -o ${ARGV1}
 		USES_TERMINAL
 	)
 endmacro()
@@ -94,7 +95,7 @@ macro(add_alsatplg2_command conf_header conf_target input_name output_name inclu
 		COMMAND cat ${conf_header} ${input_name}.conf > ${output_name}.conf
 
 		# -p to pre-process Topology2.0 conf file
-		COMMAND ALSA_CONFIG_DIR=${CMAKE_SOURCE_DIR}/topology/topology2 alsatplg \$\${VERBOSE:+-v 1}
+		COMMAND ALSA_CONFIG_DIR=${CMAKE_SOURCE_DIR}/topology/topology2 ${ALSATPLG_CMD} \$\${VERBOSE:+-v 1}
 		        -I ${include_path} -D "'${defines}'" -p -c ${output_name}.conf -o ${output_name}.tplg
 		USES_TERMINAL
 		)


Reviews:
user: login: kv2019i
body: Looks good, Windows CI jobs looking green, so seems topology builds are attempted at limited places. We do need to update the CI flows were topologies are built before merge.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-03T17:19:02Z
user: login: lgirdwood
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T17:51:43Z
user: login: lgirdwood
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T13:35:27Z
user: login: lgirdwood
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T13:35:59Z
user: login: lgirdwood
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T16:09:15Z
user: login: kv2019i
body: Looks good. We now only need the SOF CI to adapt so this can be merged...
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-02T07:30:55Z
user: login: singalsu
body: It seemed to work for me. I wonder if the alsa-utils build question "Press Return to acknowledge the previous three paragraphs." would cause issues in CI batch env? 
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-08T14:44:55Z
user: login: jsarha
body: Works for me too, and did not see anything wrong with the commits either.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T20:12:25Z
user: login: kv2019i
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-24T14:20:48Z
user: login: lgirdwood
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T10:39:14Z
user: login: kv2019i
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-02T07:24:57Z
user: login: kv2019i
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-02T07:54:01Z
user: login: lgirdwood
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-02T09:46:30Z

Review comments:
diff_hunk: @@ -0,0 +1,117 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation. All rights reserved.
path: scripts/build-alsa-tools.sh
user: login: kv2019i
body: 2025?
created_at: 2025-03-03T17:05:41Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,117 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation. All rights reserved.
+
+# fail immediately on any errors
+set -e
+
+# Array of ALSA Git repository URLs.  Add or remove repositories as needed.
+declare -a REPOS=(
+    "https://github.com/alsa-project/alsa-lib.git"
+    "https://github.com/alsa-project/alsa-utils.git"
path: scripts/build-alsa-tools.sh
user: login: kv2019i
body: Btw, the docker script for alsa-utils has built ALSA from thesofproject repos of alsa-lib and alsa-utils. I don't see an issue using upstream repos though, just as long we capture the commit-id to build in SOF side to ensure only versions we have validated are used.
created_at: 2025-03-03T17:07:17Z
author_association: COLLABORATOR
diff_hunk: @@ -1,18 +1,19 @@
 set(SOF_TOPOLOGY_BINARY_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
-
+set(SOF_ALSA_TOOLS_DIR "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/bin")
+set(ALSATPLG_CMD "${SOF_ALSA_TOOLS_DIR}/alsatplg")
path: tools/topology/CMakeLists.txt
user: login: kv2019i
body: It seems this is passing Windows CI runs, so apparently there's no attempt to run this (which is good).

We do need to add a call to new build script somewhere in our build flows, and somewhere that is specific to Linux. E.g. our SOF jenkins build now failing:
https://[sof-ci.01.org/sofpr/PR9870/build11207/build/topology/tplg.log](https://sof-ci.01.org/sofpr/PR9870/build11207/build/topology/tplg.log)

CMake Warning at topology/CMakeLists.txt:16 (message):
  /home/sof/work/sof.git/tools/../../tools/bin/alsatplg --version returned
  status: No such file or directory,


created_at: 2025-03-03T17:13:24Z
author_association: COLLABORATOR
diff_hunk: @@ -1,18 +1,19 @@
 set(SOF_TOPOLOGY_BINARY_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
-
+set(SOF_ALSA_TOOLS_DIR "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/bin")
+set(ALSATPLG_CMD "${SOF_ALSA_TOOLS_DIR}/alsatplg")
path: tools/topology/CMakeLists.txt
user: login: lgirdwood
body: This is due to the missing alsa-utils patch
created_at: 2025-03-03T17:51:43Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation. All rights reserved.
path: scripts/build-alsa-tools.sh
user: login: lgirdwood
body: fixed
created_at: 2025-03-05T13:35:27Z
author_association: MEMBER
diff_hunk: @@ -1,18 +1,19 @@
 set(SOF_TOPOLOGY_BINARY_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
-
+set(SOF_ALSA_TOOLS_DIR "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/bin")
+set(ALSATPLG_CMD "${SOF_ALSA_TOOLS_DIR}/alsatplg")
path: tools/topology/CMakeLists.txt
user: login: lgirdwood
body: fixed that part for github workflows that doen't use Docker
created_at: 2025-03-05T13:35:59Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,117 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation. All rights reserved.
+
+# fail immediately on any errors
+set -e
+
+# Array of ALSA Git repository URLs.  Add or remove repositories as needed.
+declare -a REPOS=(
+    "https://github.com/alsa-project/alsa-lib.git"
+    "https://github.com/alsa-project/alsa-utils.git"
path: scripts/build-alsa-tools.sh
user: login: lgirdwood
body: fixed
created_at: 2025-03-05T16:09:14Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,125 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation. All rights reserved.
+
+# fail immediately on any errors
+set -e
+
+# Array of ALSA Git repository URLs.  Add or remove repositories as needed.
+declare -a REPOS=(
+    "https://github.com/thesofproject/alsa-lib.git"
+    "https://github.com/thesofproject/alsa-utils.git"
+    # Add more repositories here...
+)
+
+# Commit ID to check for (optional). If specified, the script will update
+# the repository if this commit ID is not found.  Leave empty to skip.
+declare -a COMMIT_ID=(
+    "df8f1cc1ec9d9ee15be5e2c23ad25b9389fd8766"
+    "09550cd393b1a7d307ee6f26637b1ed7bd275e38"
+)
+
+# Directory where repositories will be cloned/updated.
+if [[ -z "$SOF_WORKSPACE" ]]; then
+  # Environment variable is empty or unset so use default
+  BASE_DIR="$HOME/work/sof"
path: scripts/build-alsa-tools.sh
user: login: kv2019i
body: I think this should be

```
BASE_DIR="$HOME/work"
```

For the default case, the BASE_DIR should point to parent directory of sof as this is the directory assumed in tools/topology/CMakeLists.txt rules:

```
set(SOF_ALSA_TOOLS_DIR "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/bin") 
```

So BASE_DIR should match  "${SOF_ROOT_SOURCE_DIRECTORY}".

This is causing failuress e.g. when we try to use "build-tools.sh -A" in docker in github (and might be a reason why @lgirdwood you "open-coded" the build recipe for github testbench workflow in this PR). 
created_at: 2025-04-24T14:20:48Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,125 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation. All rights reserved.
+
+# fail immediately on any errors
+set -e
+
+# Array of ALSA Git repository URLs.  Add or remove repositories as needed.
+declare -a REPOS=(
+    "https://github.com/thesofproject/alsa-lib.git"
+    "https://github.com/thesofproject/alsa-utils.git"
+    # Add more repositories here...
+)
+
+# Commit ID to check for (optional). If specified, the script will update
+# the repository if this commit ID is not found.  Leave empty to skip.
+declare -a COMMIT_ID=(
+    "df8f1cc1ec9d9ee15be5e2c23ad25b9389fd8766"
+    "09550cd393b1a7d307ee6f26637b1ed7bd275e38"
+)
+
+# Directory where repositories will be cloned/updated.
+if [[ -z "$SOF_WORKSPACE" ]]; then
+  # Environment variable is empty or unset so use default
+  BASE_DIR="$HOME/work/sof"
path: scripts/build-alsa-tools.sh
user: login: lgirdwood
body: This is correct. We are cloning into the ```~/work/sof``` workspace directory with contains the ```~/work/sof/sof``` FW directory. e.g. my SOF workspace directory looks like 
```
~/work/sof$ ls
alsa-lib      build-sof-staging  LICENSE.workspace  ruy                 sof-docs      zephyr
alsa-utils    flatbuffers        modules            sof                 sof-test
build-mtl     gemmlowp           nnlib-hifi4        sof-bin             tflite-micro
build-rimage  keys               README.md          sof.code-workspace  tools
```
The workspace should include all ingredients used to build FW, topologies, tooling i.e. everything a developer needs to create a new module and deploy it.  I'll fix the other makefiles.
created_at: 2025-05-01T10:39:14Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,125 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation. All rights reserved.
+
+# fail immediately on any errors
+set -e
+
+# Array of ALSA Git repository URLs.  Add or remove repositories as needed.
+declare -a REPOS=(
+    "https://github.com/thesofproject/alsa-lib.git"
+    "https://github.com/thesofproject/alsa-utils.git"
+    # Add more repositories here...
+)
+
+# Commit ID to check for (optional). If specified, the script will update
+# the repository if this commit ID is not found.  Leave empty to skip.
+declare -a COMMIT_ID=(
+    "df8f1cc1ec9d9ee15be5e2c23ad25b9389fd8766"
+    "09550cd393b1a7d307ee6f26637b1ed7bd275e38"
+)
+
+# Directory where repositories will be cloned/updated.
+if [[ -z "$SOF_WORKSPACE" ]]; then
+  # Environment variable is empty or unset so use default
+  BASE_DIR="$HOME/work/sof"
path: scripts/build-alsa-tools.sh
user: login: kv2019i
body: @lgirdwood Right so "$ZEPHYR_WORKSPACE=work/sof" is terms of 
https://thesofproject.github.io/latest/getting_started/build-guide/build-with-zephyr.html

Ok I didn't realize that, I never use "sof" as the workspace directory name, but indeed then the problem is elsewhere setting the workspace path.

UPDATE2: ahaa, the build-from SOF source instructions do have "work/sof/sof". https://thesofproject.github.io/latest/getting_started/build-guide/build-from-scratch.html#step-1-set-up-the-workspace-directory . so then this is indeed aligned!
UPDATE: our docker images have used workspace path "/home/sof/work" with sof git checked out in "/home/sof/work/sof". This just means the workspace has to be manually be set with "--env SOF_WORKSPACE=/home/sof/work/". 
created_at: 2025-05-02T07:23:54Z
author_association: COLLABORATOR
diff_hunk: @@ -13,6 +13,7 @@ target_compile_options(sof-ctl PRIVATE
 )
 
 target_include_directories(sof-ctl PRIVATE
+	"${SOF_ROOT_SOURCE_DIRECTORY}/../tools/include"
path: tools/ctl/CMakeLists.txt
user: login: kv2019i
body: I tested this and works in my local environment, but with @cgturner1 's test docker image (thesofproject/sof:20250410_no-alsa) with no system ALSA installed, the build fails as linkage still uses system libasound:

```

: && /usr/bin/cc -g  ctl/CMakeFiles/sof-ctl.dir/ctl.c.o -o ctl/sof-ctl  -lasound && :
/usr/bin/ld: cannot find -lasound: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
```

created_at: 2025-05-02T07:54:00Z
author_association: COLLABORATOR
diff_hunk: @@ -13,6 +13,7 @@ target_compile_options(sof-ctl PRIVATE
 )
 
 target_include_directories(sof-ctl PRIVATE
+	"${SOF_ROOT_SOURCE_DIRECTORY}/../tools/include"
path: tools/ctl/CMakeLists.txt
user: login: lgirdwood
body: @kv2019i now fixed
created_at: 2025-05-02T09:46:30Z
author_association: MEMBER

PR comments:
user: login: lgirdwood
created_at: 2025-03-03T16:48:23Z
body: This wont pass CI yet, pending a alsa-utils fix to upstream 1st before merge.
author_association: MEMBER
user: login: lgirdwood
created_at: 2025-04-08T18:00:06Z
body: > It seemed to work for me. I wonder if the alsa-utils build question "Press Return to acknowledge the previous three paragraphs." would cause issues in CI batch env?

Tested that - it passes straight thru as no console stdin is available.
author_association: MEMBER
user: login: cgturner1
created_at: 2025-04-28T22:36:59Z
body: @lrgirdwo In addition to the changes for the BASE_DIR i was testing this in the internal CI build and all the one specific flag does not work after installing the alsa tools and that is for the sof-ctl portion

so for example '.scripts/build-tools.sh -A -c' the tools install great and I'm able to build topologies but the sof-ctl returns:
```
tools/ctl/ctl.c:13:10: fatal error: alsa/asoundlib.h: No such file or directory
   13 | #include <alsa/asoundlib.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
```

Is there a separate cmake file for sof-ctl that needs to be updated like the one for the topologies to work with the local ALSA install?

author_association: CONTRIBUTOR
user: login: lgirdwood
created_at: 2025-05-01T13:12:48Z
body: > Is there a separate cmake file for sof-ctl that needs to be updated like the one for the topologies to work with the local ALSA install?

Thanks - now fixed, was actually using system ALSA path
author_association: MEMBER
user: login: lgirdwood
created_at: 2025-05-02T10:25:04Z
body: @cgturner1 thinking aloud - is there a hard need for a docker container now after this update is working/merged ? This has been quite painful to make changes here - it would be great if CI could build topologies and tools without Docker in the future.
author_association: MEMBER
user: login: cgturner1
created_at: 2025-05-05T16:22:43Z
body: > @cgturner1 thinking aloud - is there a hard need for a docker container now after this update is working/merged ? This has been quite painful to make changes here - it would be great if CI could build topologies and tools without Docker in the future.

After this is merged then there wont really be a need to continually update the docker but a lot of the current CI scripts are written around docker and the docker workspaces that removing that is a much bigger change than adding the command to build things using the -A script and build it locally inside the docker (which currently is working after the latest updates)

So I think for step 1 is to just get these local builds working in a docker that doesn't have Alsa-tools since it will require the least amount of changes to the jenkins and then longer term can remove the dependency all together but that touches way more CI scripts and dependencies.

One last open/question @lrgirdwo was doing final tests of this and was wondering if it can be possible to set the `DO_BUILD_alsa=true` when running ./scripts/build-tools.sh with no flags set? I've run into instances where we have it set to run ./scripts/build-tools.sh without any flags after but i don't know if the intended behavior is for alsa-build to be optional or included in the build-all scenario
author_association: CONTRIBUTOR
user: login: marc-hb
created_at: 2025-05-05T23:24:17Z
body: I think this topic was already discussed there:
- https://github.com/thesofproject/linux/issues/2211

... and in internal issue sofcitest/issues/692

I don't have time to look at this PR but the sheer length of it gives me the impression that a Docker container was a good idea. Sorry...

EDIT: and also
- https://github.com/thesofproject/sof-test/pull/363

\+ many others.
author_association: COLLABORATOR
user: login: lgirdwood
created_at: 2025-05-07T13:42:09Z
body: > One last open/question @lrgirdwo was doing final tests of this and was wondering if it can be possible to set the `DO_BUILD_alsa=true` when running ./scripts/build-tools.sh with no flags set? I've run into instances where we have it set to run ./scripts/build-tools.sh without any flags after but i don't know if the intended behavior is for alsa-build to be optional or included in the build-all scenario

@cgturner1 Ok, so by default do you want the flags inverted i.e. always build ALSA unless a flag says dont build ALSA ? I can do that if preferred by you. 
author_association: MEMBER
user: login: cgturner1
created_at: 2025-05-07T17:49:45Z
body: > > One last open/question @lrgirdwo was doing final tests of this and was wondering if it can be possible to set the `DO_BUILD_alsa=true` when running ./scripts/build-tools.sh with no flags set? I've run into instances where we have it set to run ./scripts/build-tools.sh without any flags after but i don't know if the intended behavior is for alsa-build to be optional or included in the build-all scenario
> 
> @cgturner1 Ok, so by default do you want the flags inverted i.e. always build ALSA unless a flag says dont build ALSA ? I can do that if preferred by you.

ah didn't think about it that way i think the current optional way makes more sense to me where the flag includes it (to match the behavior of the other flags)

for now i have it that when we want to build all tools + install rather than just running

`.scripts/buils-tools.sh`

i have:

`.scripts/build-tools.sh -A -c -l -p -t -T ` 

I think for now we can leave as is and we can always revisit... I am trying to get time carved out tomorrow to have Kai merge as is and try to see if the internal CI changes work out worst case we back out on Friday and I figure out what changes were missing on the Jenkins side while not impacting production CI runs any more than a day
author_association: CONTRIBUTOR
user: login: kv2019i
created_at: 2025-05-08T14:50:13Z
body: @cgturner1 Merged now. We need to do this in steps and now proceed to update the jenkins task.
author_association: COLLABORATOR
user: login: marc-hb
created_at: 2025-07-30T22:50:17Z
body: Funny enough this broke the daily tests and no one noticed the front page :-)
Should be a one-line fix

https://github.com/thesofproject/sof
<img width="408" height="261" alt="Screenshot 2025-07-30 at 15 49 50" src="https://github.com/user-attachments/assets/162139cc-b5e0-4d26-a664-695776cb0af7" />

```
error parsing called workflow ".github/workflows/daily-tests.yml" -> "./.github/workflows/installer.yml"
```
author_association: COLLABORATOR