Repository: chainguard-dev/melange

Issue:
Private libs end up in public provides
```yaml
gcc-12-test-sca.yaml
package:
  name: gcc-12-test-sca
  version: 12.4.0
  epoch: 0
  description: "the GNU compiler collection - version 12"
  copyright:
    - license: GPL-3.0-or-later

environment:
  contents:
    packages:
      - busybox
      - gcc-12

pipeline:
  - runs: |
      mkdir -p ${{targets.destdir}}/usr/lib
      cp -r /usr/lib/gcc ${{targets.destdir}}/usr/lib
```

build output:

```console
024/07/11 18:58:44 INFO generating package gcc-12-test-sca-12.4.0-r0
2024/07/11 18:58:44 INFO scanning for shared object dependencies...
2024/07/11 18:58:44 INFO   found lib libc.so.6 for usr/lib/gcc/x86_64-pc-linux-gnu/12.4.0/libgomp.so.1.0.0
2024/07/11 18:58:44 INFO   found lib libc.so.6 for usr/lib/gcc/x86_64-pc-linux-gnu/12.4.0/libitm.so.1.0.0
2024/07/11 18:58:44 INFO   found lib libm.so.6 for usr/lib/gcc/x86_64-pc-linux-gnu/12.4.0/libquadmath.so.0.0.0
2024/07/11 18:58:44 INFO   found lib libc.so.6 for usr/lib/gcc/x86_64-pc-linux-gnu/12.4.0/libquadmath.so.0.0.0
2024/07/11 18:58:44 INFO   found lib libc.so.6 for usr/lib/gcc/x86_64-pc-linux-gnu/12.4.0/libssp.so.0.0.0
2024/07/11 18:58:44 INFO scanning for commands...
2024/07/11 18:58:44 INFO scanning for pkg-config data...
2024/07/11 18:58:44 INFO scanning for python modules...
2024/07/11 18:58:44 INFO scanning for shbang deps...
2024/07/11 18:58:44 INFO   runtime:
2024/07/11 18:58:44 INFO     so:libc.so.6
2024/07/11 18:58:44 INFO     so:libm.so.6
2024/07/11 18:58:44 INFO   provides:
2024/07/11 18:58:44 INFO     so:libgomp.so.1=1
2024/07/11 18:58:44 INFO     so:libitm.so.1=1
2024/07/11 18:58:44 INFO     so:libquadmath.so.0=0
2024/07/11 18:58:44 INFO     so:libssp.so.0=0
2024/07/11 18:58:44 INFO   installed-size: 74429639
2024/07/11 18:58:44 INFO   data.tar.gz digest: f4370156bd67b791b8c0cb8eb0856a4a25a6a0301cad1f2ee85cfc26aa308e93
2024/07/11 18:58:44 INFO wrote packages/x86_64/gcc-12-test-sca-12.4.0-r0.apk
```

Expecting this to not happen:

```
2024/07/11 18:58:44 INFO   provides:
2024/07/11 18:58:44 INFO     so:libgomp.so.1=1
2024/07/11 18:58:44 INFO     so:libitm.so.1=1
2024/07/11 18:58:44 INFO     so:libquadmath.so.0=0
2024/07/11 18:58:44 INFO     so:libssp.so.0=0
```

As these libraries are in the private `/usr/lib/gcc/` location, these are not public libraries and should not conflict with `gcc` which ships them publically. As intention is for `gcc-12` to be co-installable with `gcc`.

Seems like we check for `usr/lib/` prefix, but then we have unlimited depth, and subdirs shouldn't typically be included in automatic SCA provides as typically they are outside of the ldconfig.conf paths.

(cc @smoser @imjasonh @jonjohnsonjr )

Hints from issue comments:
@xnox: This bug is the reason why `gcc-12` package uses `no-provides` which intern is buggy w.r.t. turning of depends, see https://github.com/chainguard-dev/melange/issues/1366@imjasonh: Do you have an idea how many packages list a `provides:` `so:` that's "private" today?

(Is "private" here being defined as being under a subdirectory in `/usr/lib`?)@smoser: Maybe this is helpful. Maybe not. I put together this crappy shell script https://gist.github.com/smoser/0a11e2643b884960c1e5349d4dc0b8c7 .  

I'm going to attach the output in case you find it useful.
[out.d.tar.gz](https://github.com/user-attachments/files/16183178/out.d.tar.gz)

more crappy shell scripts with that data can answer your question.@imjasonh: Just checking my understanding: another example of this is `py3-seaborn` which has

```
-rwxr-xr-x root/root    249696 2024-01-26 10:15 usr/lib/python3.12/site-packages/numpy.libs/libquadmath-96973f99.so.0.0.0
```

and so has it in its provides:

```
provides = so:libquadmath-96973f99.so.0.0.0=0.0.0
```

https://apk.dag.dev/https/packages.wolfi.dev/os/x86_64/py3-seaborn-0.13.2-r0.apk@sha1:6d5ff93ebf4f34a98d67ca94f2c073f98832f57e/.PKGINFO

It sounds like that should be detected as a "vendored" .so, and not as a provided one. Is that right?@smoser: > It sounds like that should be detected as a "vendored" .so, and not as a provided one. Is that right?

I agree with your 'vendored' statement. 

It feels to me like we should only generate provides: for .so files in "normal" locations... those that we would put in /etc/ld.so.conf or /etc/so.conf.d/ .  If something had a dependency on libquadmath above, then it is extremely unlikely that py-seaborn would successfully provide it.

I feel like there are probably some cases where that is not true, but i would expect those to be outliers.  I can imagine that
 * within the same origin package , things might "know" to look in additional paths
 * for some well defined/well known plugin architectures there might be paths that are not expected to be added to system search path.@xnox: py3-seaborn => shouldn't really vendor in numpy, and should really depend on matching py3-numpy.
Also our libquadmath currently is not correctly shipped by gcc as a separate package, as that really out to be a separate package for things to depend on at runtime (as in py3-numpy likely should have dependency on so:libquadmath.so which is provided by libquadmath package which is currently empty, because bugs).@murraybd: I'm closing this with the impression that https://github.com/chainguard-dev/melange/pull/1372 fixed it. If I'm wrong please reopen it and let me know!

Solution patch:
diff --git a/.github/workflows/melange-test-pipelines.yaml b/.github/workflows/melange-test-pipelines.yaml
index b5d1e875..3e940697 100644
--- a/.github/workflows/melange-test-pipelines.yaml
+++ b/.github/workflows/melange-test-pipelines.yaml
@@ -72,10 +72,6 @@ jobs:
           go-version-file: './go.mod'
           check-latest: true
 
-      - name: Go generate and diff
-        run: |
-          go generate ./...
-          git diff --exit-code
       - name: Run e2e-tests
         run: |
           make test-e2e
diff --git a/.github/workflows/verify.yaml b/.github/workflows/verify.yaml
index 90f01796..e28d11e5 100644
--- a/.github/workflows/verify.yaml
+++ b/.github/workflows/verify.yaml
@@ -37,6 +37,5 @@ jobs:
           git diff --exit-code
 
       - run: |
-          make generate
           go mod tidy
           git diff --exit-code
diff --git a/.gitignore b/.gitignore
index 9e8ef30e..58420647 100644
--- a/.gitignore
+++ b/.gitignore
@@ -21,7 +21,7 @@ melange.rsa.pub
 packages/
 .idea/
 bin/
-/generated/
+generated/
 melange.images
 /.DS_Store
 dist/
diff --git a/Makefile b/Makefile
index 4bd81c48..7da0a340 100644
--- a/Makefile
+++ b/Makefile
@@ -88,7 +88,7 @@ ko-apply:  ## Build the image and apply the manifests
 ##########
 
 .PHONY: generate
-generate: ## Generates jsonschema for melange types.
+generate:
 	go generate ./...
 
 ##########
@@ -146,13 +146,12 @@ lint: checkfmt setup-golangci-lint ## Run linters and checks like golangci-lint
 	$(GOLANGCI_LINT_BIN) run --verbose --concurrency 4 --skip-dirs .modcache ./...
 
 .PHONY: test
-test: melange ## Run go test
-	# build test package
-	 ./melange build --generate-index=false pkg/sca/testdata/go-fips-bin/go-fips-bin.yaml --arch=`uname -m` --source-dir=pkg/sca/testdata/go-fips-bin/ --out-dir pkg/sca/testdata/go-fips-bin/packages/ --log-level error
-	go test -tags e2e ./... -race
+test:
+	go test ./... -race
 
 .PHONY: test-e2e
-test-e2e:
+test-e2e: test generate
+	go test -tags e2e ./... -race
 	cd e2e-tests && ./run-tests
 
 .PHONY: clean
diff --git a/docs/md/melange_build.md b/docs/md/melange_build.md
index 92e8b5e9..34bd2275 100644
--- a/docs/md/melange_build.md
+++ b/docs/md/melange_build.md
@@ -57,7 +57,7 @@ melange build [flags]
       --pipeline-dir string                                     directory used to extend defined built-in pipelines
   -r, --repository-append strings                               path to extra repositories to include in the build environment
       --rm                                                      clean up intermediate artifacts (e.g. container images)
-      --runner string                                           which runner to use to enable running commands, default is based on your platform. Options are ["bubblewrap" "docker" "kubernetes"]
+      --runner string                                           which runner to use to enable running commands, default is based on your platform. Options are ["bubblewrap" "docker"]
       --signing-key string                                      key to use for signing
       --source-dir string                                       directory used for included sources
       --strip-origin-name                                       whether origin names should be stripped (for bootstrap)
diff --git a/docs/md/melange_compile.md b/docs/md/melange_compile.md
index ec6b8209..be6727ea 100644
--- a/docs/md/melange_compile.md
+++ b/docs/md/melange_compile.md
@@ -56,7 +56,7 @@ melange compile [flags]
       --pipeline-dir string         directory used to extend defined built-in pipelines
   -r, --repository-append strings   path to extra repositories to include in the build environment
       --rm                          clean up intermediate artifacts (e.g. container images)
-      --runner string               which runner to use to enable running commands, default is based on your platform. Options are ["bubblewrap" "docker" "kubernetes"]
+      --runner string               which runner to use to enable running commands, default is based on your platform. Options are ["bubblewrap" "docker"]
       --signing-key string          key to use for signing
       --source-dir string           directory used for included sources
       --strip-origin-name           whether origin names should be stripped (for bootstrap)
diff --git a/docs/md/melange_test.md b/docs/md/melange_test.md
index 297c5ff9..6b6cb10f 100644
--- a/docs/md/melange_test.md
+++ b/docs/md/melange_test.md
@@ -42,7 +42,7 @@ melange test [flags]
       --overlay-binsh string          use specified file as /bin/sh overlay in build environment
       --pipeline-dirs strings         directories used to extend defined built-in pipelines
   -r, --repository-append strings     path to extra repositories to include in the build environment
-      --runner string                 which runner to use to enable running commands, default is based on your platform. Options are ["bubblewrap" "docker" "kubernetes"]
+      --runner string                 which runner to use to enable running commands, default is based on your platform. Options are ["bubblewrap" "docker"]
       --source-dir string             directory used for included sources
       --test-option strings           build options to enable
       --test-package-append strings   extra packages to install for each of the test environments
diff --git a/pkg/build/pipelines/cargo/build.yaml b/pkg/build/pipelines/cargo/build.yaml
index 51a96f12..17fbe473 100644
--- a/pkg/build/pipelines/cargo/build.yaml
+++ b/pkg/build/pipelines/cargo/build.yaml
@@ -40,7 +40,7 @@ pipeline:
 
       # Build and install package(s)
       cargo auditable build "${{inputs.opts}}"
-      if [[ -z "${{inputs.output}}" ]]; then
+      if [[ ! -z "${{inputs.output}}" ]]; then
         install -Dm755 "${OUTPUT_PATH}/${{inputs.output}}" "${INSTALL_PATH}/${{inputs.output}}"
       else
         install -Dm755 "${OUTPUT_PATH}"/* -t "${INSTALL_PATH}"
diff --git a/pkg/build/pipelines/npm/install.yaml b/pkg/build/pipelines/npm/install.yaml
index 024e2123..dd2639ac 100644
--- a/pkg/build/pipelines/npm/install.yaml
+++ b/pkg/build/pipelines/npm/install.yaml
@@ -19,7 +19,7 @@ inputs:
   prefix:
     description: |
       The -prefix argument to pass to npm install; where /bin and /lib will be copied to.
-    default: "${{targets.destdir}}/usr/"
+    default: "${{targets.contextdir}}/usr/"
 
   overrides:
     description: |
diff --git a/pkg/build/pipelines/pecl/install.yaml b/pkg/build/pipelines/pecl/install.yaml
index 3d5d2b24..c2c09dbc 100644
--- a/pkg/build/pipelines/pecl/install.yaml
+++ b/pkg/build/pipelines/pecl/install.yaml
@@ -12,6 +12,6 @@ inputs:
 
 pipeline:
   - runs: |
-      make INSTALL_ROOT="${{targets.destdir}}" install
-      install -d ${{targets.destdir}}/etc/php/conf.d
-      echo "extension=${{inputs.extension}}.so" > ${{targets.destdir}}/etc/php/conf.d/${{inputs.extension}}.ini
+      make INSTALL_ROOT="${{targets.contextdir}}" install
+      install -d ${{targets.contextdir}}/etc/php/conf.d
+      echo "extension=${{inputs.extension}}.so" > ${{targets.contextdir}}/etc/php/conf.d/${{inputs.extension}}.ini
diff --git a/pkg/build/pipelines/perl/cleanup.yaml b/pkg/build/pipelines/perl/cleanup.yaml
index 7590cb4c..abead60c 100644
--- a/pkg/build/pipelines/perl/cleanup.yaml
+++ b/pkg/build/pipelines/perl/cleanup.yaml
@@ -6,4 +6,4 @@ needs:
 
 pipeline:
   - runs: |
-      find "${{targets.destdir}}" \( -name perllocal.pod -o -name .packlist \) -delete
+      find "${{targets.contextdir}}" \( -name perllocal.pod -o -name .packlist \) -delete
diff --git a/pkg/build/pipelines/ruby/clean.yaml b/pkg/build/pipelines/ruby/clean.yaml
index 3206b6e7..7b7f5e6c 100644
--- a/pkg/build/pipelines/ruby/clean.yaml
+++ b/pkg/build/pipelines/ruby/clean.yaml
@@ -12,6 +12,6 @@ pipeline:
         exit 1
       fi
   - runs: |
-      INSTALL_DIR=${{targets.destdir}}/$(ruby -e 'puts Gem.default_dir')
+      INSTALL_DIR=${{targets.contextdir}}/$(ruby -e 'puts Gem.default_dir')
       rm -rf ${INSTALL_DIR}/build_info \
              ${INSTALL_DIR}/cache
diff --git a/pkg/build/runner.go b/pkg/build/runner.go
index 0282d34e..0397f1ee 100644
--- a/pkg/build/runner.go
+++ b/pkg/build/runner.go
@@ -5,8 +5,6 @@ type Runner string
 const (
 	runnerBubblewrap Runner = "bubblewrap"
 	runnerDocker     Runner = "docker"
-	runnerKubernetes Runner = "kubernetes"
-	// more to come
 )
 
 // GetAllRunners returns a list of all valid runners.
@@ -14,6 +12,5 @@ func GetAllRunners() []Runner {
 	return []Runner{
 		runnerBubblewrap,
 		runnerDocker,
-		runnerKubernetes,
 	}
 }
diff --git a/pkg/sca/sca.go b/pkg/sca/sca.go
index ff16ec72..a61b8b64 100644
--- a/pkg/sca/sca.go
+++ b/pkg/sca/sca.go
@@ -77,16 +77,6 @@ type SCAHandle interface {
 // findings based on analysis.
 type DependencyGenerator func(context.Context, SCAHandle, *config.Dependencies) error
 
-func allowedPrefix(path string, prefixes []string) bool {
-	for _, pfx := range prefixes {
-		if strings.HasPrefix(path, pfx) {
-			return true
-		}
-	}
-
-	return false
-}
-
 func isInDir(path string, dirs []string) bool {
 	mydir := filepath.Dir(path)
 	for _, d := range dirs {
@@ -97,8 +87,6 @@ func isInDir(path string, dirs []string) bool {
 	return false
 }
 
-var pathBinDirs = []string{"bin/", "sbin/", "usr/bin/", "usr/sbin/"}
-
 func generateCmdProviders(ctx context.Context, hdl SCAHandle, generated *config.Dependencies) error {
 	log := clog.FromContext(ctx)
 	if hdl.Options().NoCommands {
@@ -127,7 +115,7 @@ func generateCmdProviders(ctx context.Context, hdl SCAHandle, generated *config.
 		}
 
 		if mode.Perm()&0555 == 0555 {
-			if isInDir(path, pathBinDirs) {
+			if isInDir(path, []string{"bin/", "sbin/", "usr/bin/", "usr/sbin/"}) {
 				basename := filepath.Base(path)
 				log.Infof("  found command %s", path)
 				if !hdl.Options().NoProvides {
@@ -360,7 +348,7 @@ func generateSharedObjectNameDeps(ctx context.Context, hdl SCAHandle, generated
 			for _, soname := range sonames {
 				libver := sonameLibver(soname)
 
-				if allowedPrefix(path, libDirs) {
+				if isInDir(path, libDirs) {
 					if !hdl.Options().NoProvides {
 						generated.Provides = append(generated.Provides, fmt.Sprintf("so:%s=%s", soname, libver))
 					}
@@ -404,8 +392,6 @@ var pkgConfigVersionRegexp = regexp.MustCompile("-(alpha|beta|rc|pre)")
 // TODO(kaniini): Turn this feature on once enough of Wolfi is built with provider data.
 var generateRuntimePkgConfigDeps = false
 
-var pcDirs = []string{"lib/pkgconfig/", "usr/lib/pkgconfig/", "lib64/pkgconfig/", "usr/lib64/pkgconfig/"}
-
 // generatePkgConfigDeps generates a list of provided pkg-config package names and versions,
 // as well as dependency relationships.
 func generatePkgConfigDeps(ctx context.Context, hdl SCAHandle, generated *config.Dependencies) error {
@@ -468,7 +454,7 @@ func generatePkgConfigDeps(ctx context.Context, hdl SCAHandle, generated *config
 
 		apkVersion := pkgConfigVersionRegexp.ReplaceAllString(pkg.Version, "_$1")
 		if !hdl.Options().NoProvides {
-			if allowedPrefix(path, pcDirs) {
+			if isInDir(path, []string{"lib/pkgconfig/", "usr/lib/pkgconfig/", "lib64/pkgconfig/", "usr/lib64/pkgconfig/"}) {
 				log.Infof("  found pkg-config %s for %s", pcName, path)
 				generated.Provides = append(generated.Provides, fmt.Sprintf("pc:%s=%s", pcName, sigh(apkVersion)))
 			} else {



Test patch:
diff --git a/pkg/sca/e2e_test.go b/pkg/sca/e2e_test.go
index 95b36001..3ccdf67b 100644
--- a/pkg/sca/e2e_test.go
+++ b/pkg/sca/e2e_test.go
@@ -15,13 +15,9 @@
 //go:build e2e
 // +build e2e
 
-//go:generate go run ./../../ build --out-dir=./generated ./testdata/shbang-test.yaml --arch=x86_64
-
 package sca
 
 import (
-	"fmt"
-	"runtime"
 	"testing"
 
 	"chainguard.dev/melange/pkg/config"
@@ -30,67 +26,187 @@ import (
 	"github.com/google/go-cmp/cmp"
 )
 
-// test a fips like go binary package for SCA depends
-// Chainguard go-fips toolchain generates binaries like these
-// which at runtime require openssl and fips provider
 func TestGoFipsBinDeps(t *testing.T) {
 	ctx := slogtest.TestContextWithLogger(t)
-
-	var ldso, archdir string
-	switch runtime.GOARCH {
-	case "arm64":
-		ldso = "so:ld-linux-aarch64.so.1"
-		archdir = "aarch64"
-	case "amd64":
-		ldso = "so:ld-linux-x86-64.so.2"
-		archdir = "x86_64"
-	}
-
-	th := handleFromApk(ctx, t, fmt.Sprintf("go-fips-bin/packages/%s/go-fips-bin-v0.0.1-r0.apk", archdir), "go-fips-bin/go-fips-bin.yaml")
+	th := handleFromApk(ctx, t, "generated/x86_64/go-fips-bin-0.0.1-r0.apk", "go-fips-bin/go-fips-bin.yaml")
 	defer th.exp.Close()
 
 	got := config.Dependencies{}
 	if err := Analyze(ctx, th, &got); err != nil {
 		t.Fatal(err)
 	}
-
 	want := config.Dependencies{
 		Runtime: util.Dedup([]string{
 			"openssl-config-fipshardened",
-			ldso,
+			"so:ld-linux-x86-64.so.2",
 			"so:libc.so.6",
 			"so:libcrypto.so.3",
 			"so:libssl.so.3",
 		}),
-		Provides: []string{"cmd:go-fips-bin=v0.0.1-r0"},
+		Provides: []string{"cmd:go-fips-bin=0.0.1-r0"},
 	}
 	if diff := cmp.Diff(want, got); diff != "" {
 		t.Errorf("Analyze(): (-want, +got):\n%s", diff)
 	}
 }
 
-func TestShbangDeps(t *testing.T) {
-	ctx := slogtest.TestContextWithLogger(t)
-	// Generated with `go generate ./...`
-	th := handleFromApk(ctx, t, "generated/x86_64/shbang-test-1-r1.apk", "shbang-test.yaml")
-	defer th.exp.Close()
+func TestAnalyze(t *testing.T) {
+	for _, c := range []struct {
+		apk     string
+		cfgpath string
+		want    config.Dependencies
+	}{{
+		apk:     "py3-seaborn-0.13.2-r0.apk",
+		cfgpath: "py3-seaborn.yaml",
+		want: config.Dependencies{
+			Runtime: []string{
+				"so:ld-linux-x86-64.so.2",
+				"so:libXau-154567c4.so.6.0.0",
+				"so:libbrotlicommon-3ecfe81c.so.1",
+				"so:libbrotlidec-ba690955.so.1",
+				"so:libc.so.6",
+				"so:libdl.so.2",
+				"so:libfreetype-f154df84.so.6.20.1",
+				"so:libgcc_s.so.1",
+				"so:libgfortran-040039e1.so.5.0.0",
+				"so:libharfbuzz-2093a78b.so.0.60830.0",
+				"so:libjpeg-e44fd0cd.so.62.4.0",
+				"so:liblcms2-e69eef39.so.2.0.16",
+				"so:liblzma-13fa198c.so.5.4.5",
+				"so:libm.so.6",
+				"so:libopenjp2-eca49203.so.2.5.0",
+				"so:libpng16-78d422d5.so.16.40.0",
+				"so:libpthread.so.0",
+				"so:libquadmath-96973f99.so.0.0.0",
+				"so:libsharpyuv-20f78091.so.0.0.1",
+				"so:libstdc++.so.6",
+				"so:libtiff-91af027d.so.6.0.2",
+				"so:libwebp-850e2bec.so.7.1.8",
+				"so:libwebpdemux-df9b36c7.so.2.0.14",
+				"so:libwebpmux-9fe05867.so.3.0.13",
+				"so:libxcb-f0538cc0.so.1.1.0",
+				"so:libz.so.1",
+			},
+			Provides: []string{
+				"cmd:f2py=0.13.2-r0",
+				"cmd:fonttools=0.13.2-r0",
+				"cmd:pyftmerge=0.13.2-r0",
+				"cmd:pyftsubset=0.13.2-r0",
+				"cmd:ttx=0.13.2-r0",
+			},
+			Vendored: []string{
+				"so:libXau-154567c4.so.6.0.0=6.0.0",
+				"so:libbrotlicommon-3ecfe81c.so.1=1",
+				"so:libbrotlidec-ba690955.so.1=1",
+				"so:libfreetype-f154df84.so.6.20.1=6.20.1",
+				"so:libgfortran-040039e1.so.5.0.0=5.0.0",
+				"so:libharfbuzz-2093a78b.so.0.60830.0=0.60830.0",
+				"so:libjpeg-e44fd0cd.so.62.4.0=62.4.0",
+				"so:liblcms2-e69eef39.so.2.0.16=2.0.16",
+				"so:liblzma-13fa198c.so.5.4.5=5.4.5",
+				"so:libopenblas64_p-r0-0cf96a72.3.23.dev.so=0",
+				"so:libopenjp2-eca49203.so.2.5.0=2.5.0",
+				"so:libpng16-78d422d5.so.16.40.0=16.40.0",
+				"so:libquadmath-96973f99.so.0.0.0=0.0.0",
+				"so:libsharpyuv-20f78091.so.0.0.1=0.0.1",
+				"so:libtiff-91af027d.so.6.0.2=6.0.2",
+				"so:libwebp-850e2bec.so.7.1.8=7.1.8",
+				"so:libwebpdemux-df9b36c7.so.2.0.14=2.0.14",
+				"so:libwebpmux-9fe05867.so.3.0.13=3.0.13",
+				"so:libxcb-f0538cc0.so.1.1.0=1.1.0",
+			},
+		},
+	}, {
+		apk:     "systemd-256.2-r1.apk",
+		cfgpath: "systemd.yaml",
+		want: config.Dependencies{
+			Runtime: []string{
+				"so:ld-linux-x86-64.so.2",
+				"so:libblkid.so.1",
+				"so:libc.so.6",
+				"so:libcap.so.2",
+				"so:libcrypt.so.1",
+				"so:libcrypto.so.3",
+				"so:libfdisk.so.1",
+				"so:libm.so.6",
+				"so:libmount.so.1",
+				"so:libssl.so.3",
+				"so:libudev.so.1",
+			},
+			Provides: []string{
+				"cmd:bootctl=256.2-r1",
+				"cmd:busctl=256.2-r1",
+				"cmd:coredumpctl=256.2-r1",
+				"cmd:hostnamectl=256.2-r1",
+				"cmd:journalctl=256.2-r1",
+				"cmd:kernel-install=256.2-r1",
+				"cmd:localectl=256.2-r1",
+				"cmd:loginctl=256.2-r1",
+				"cmd:machinectl=256.2-r1",
+				"cmd:networkctl=256.2-r1",
+				"cmd:oomctl=256.2-r1",
+				"cmd:portablectl=256.2-r1",
+				"cmd:resolvectl=256.2-r1",
+				"cmd:systemctl=256.2-r1",
+				"cmd:systemd-ac-power=256.2-r1",
+				"cmd:systemd-analyze=256.2-r1",
+				"cmd:systemd-ask-password=256.2-r1",
+				"cmd:systemd-cat=256.2-r1",
+				"cmd:systemd-cgls=256.2-r1",
+				"cmd:systemd-cgtop=256.2-r1",
+				"cmd:systemd-creds=256.2-r1",
+				"cmd:systemd-delta=256.2-r1",
+				"cmd:systemd-detect-virt=256.2-r1",
+				"cmd:systemd-dissect=256.2-r1",
+				"cmd:systemd-escape=256.2-r1",
+				"cmd:systemd-firstboot=256.2-r1",
+				"cmd:systemd-hwdb=256.2-r1",
+				"cmd:systemd-id128=256.2-r1",
+				"cmd:systemd-inhibit=256.2-r1",
+				"cmd:systemd-machine-id-setup=256.2-r1",
+				"cmd:systemd-mount=256.2-r1",
+				"cmd:systemd-notify=256.2-r1",
+				"cmd:systemd-nspawn=256.2-r1",
+				"cmd:systemd-path=256.2-r1",
+				"cmd:systemd-repart=256.2-r1",
+				"cmd:systemd-run=256.2-r1",
+				"cmd:systemd-socket-activate=256.2-r1",
+				"cmd:systemd-stdio-bridge=256.2-r1",
+				"cmd:systemd-sysext=256.2-r1",
+				"cmd:systemd-sysusers=256.2-r1",
+				"cmd:systemd-tmpfiles=256.2-r1",
+				"cmd:systemd-tty-ask-password-agent=256.2-r1",
+				"cmd:systemd-vmspawn=256.2-r1",
+				"cmd:systemd-vpick=256.2-r1",
+				"cmd:timedatectl=256.2-r1",
+				"cmd:udevadm=256.2-r1",
+				"cmd:userdbctl=256.2-r1",
+				"cmd:varlinkctl=256.2-r1",
+				"so:libnss_myhostname.so.2=2",
+				"so:libnss_mymachines.so.2=2",
+				"so:libnss_resolve.so.2=2",
+				"so:libnss_systemd.so.2=2",
+				"so:libudev.so.1=1",
+			},
+			Vendored: []string{
+				"so:libsystemd-core-256.so=0",
+				"so:libsystemd-shared-256.so=0",
+			},
+		},
+	}} {
+		t.Run(c.apk, func(t *testing.T) {
+			ctx := slogtest.TestContextWithLogger(t)
+			url := "https://packages.wolfi.dev/os/x86_64/" + c.apk
+			th := handleFromApk(ctx, t, url, c.cfgpath)
+			defer th.exp.Close()
 
-	want := config.Dependencies{
-		Runtime: util.Dedup([]string{
-			"cmd:bash",
-			"cmd:envDashSCmd",
-			"cmd:python3.12",
-			"so:ld-linux-x86-64.so.2",
-			"so:libc.so.6",
-		}),
-		Provides: nil,
-	}
-
-	got := config.Dependencies{}
-	if err := Analyze(ctx, th, &got); err != nil {
-		t.Fatal(err)
-	}
-	if diff := cmp.Diff(want, got); diff != "" {
-		t.Errorf("Analyze(): (-want, +got):\n%s", diff)
+			got := config.Dependencies{}
+			if err := Analyze(ctx, th, &got); err != nil {
+				t.Fatal(err)
+			}
+			if diff := cmp.Diff(c.want, got); diff != "" {
+				t.Errorf("Analyze(): (-want, +got):\n%s", diff)
+			}
+		})
 	}
 }
diff --git a/pkg/sca/sca_test.go b/pkg/sca/sca_test.go
index 724b410e..a5bda94b 100644
--- a/pkg/sca/sca_test.go
+++ b/pkg/sca/sca_test.go
@@ -12,13 +12,21 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
+//go:generate go run ./../../ build --generate-index=false --out-dir=./testdata/generated ./testdata/shbang-test.yaml --arch=x86_64
+//go:generate go run ./../../ build --generate-index=false --source-dir=./testdata/go-fips-bin/ --out-dir=./testdata/generated ./testdata/go-fips-bin/go-fips-bin.yaml --arch=x86_64
+//go:generate curl -s -o ./testdata/py3-seaborn.yaml https://raw.githubusercontent.com/wolfi-dev/os/7a39ac1d0603a3561790ea2201dd8ad7c2b7e51e/py3-seaborn.yaml
+//go:generate curl -s -o ./testdata/systemd.yaml https://raw.githubusercontent.com/wolfi-dev/os/7a39ac1d0603a3561790ea2201dd8ad7c2b7e51e/systemd.yaml
+
 package sca
 
 import (
 	"context"
 	"fmt"
+	"io"
+	"net/http"
 	"os"
 	"path/filepath"
+	"strings"
 	"testing"
 	"time"
 
@@ -76,9 +84,20 @@ func (th *testHandle) BaseDependencies() config.Dependencies {
 // TODO: Loose coupling.
 func handleFromApk(ctx context.Context, t *testing.T, apkfile, melangefile string) *testHandle {
 	t.Helper()
-	file, err := os.Open(filepath.Join("testdata", apkfile))
-	if err != nil {
-		t.Fatal(err)
+	var file io.Reader
+	if strings.HasPrefix(apkfile, "https://") {
+		resp, err := http.Get(apkfile)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer resp.Body.Close()
+		file = resp.Body
+	} else {
+		var err error
+		file, err = os.Open(filepath.Join("testdata", apkfile))
+		if err != nil {
+			t.Fatal(err)
+		}
 	}
 
 	exp, err := expandapk.ExpandApk(ctx, file, "")
@@ -215,3 +234,29 @@ func TestUnstableSonames(t *testing.T) {
 		t.Errorf("Analyze(): (-want, +got):\n%s", diff)
 	}
 }
+
+func TestShbangDeps(t *testing.T) {
+	ctx := slogtest.TestContextWithLogger(t)
+	// Generated with `go generate ./...`
+	th := handleFromApk(ctx, t, "generated/x86_64/shbang-test-1-r1.apk", "shbang-test.yaml")
+	defer th.exp.Close()
+
+	want := config.Dependencies{
+		Runtime: util.Dedup([]string{
+			"cmd:bash",
+			"cmd:envDashSCmd",
+			"cmd:python3.12",
+			"so:ld-linux-x86-64.so.2",
+			"so:libc.so.6",
+		}),
+		Provides: nil,
+	}
+
+	got := config.Dependencies{}
+	if err := Analyze(ctx, th, &got); err != nil {
+		t.Fatal(err)
+	}
+	if diff := cmp.Diff(want, got); diff != "" {
+		t.Errorf("Analyze(): (-want, +got):\n%s", diff)
+	}
+}
diff --git a/pkg/sca/testdata/generated/x86_64/APKINDEX.json b/pkg/sca/testdata/generated/x86_64/APKINDEX.json
deleted file mode 100644
index 8847413e..00000000
--- a/pkg/sca/testdata/generated/x86_64/APKINDEX.json
+++ /dev/null
@@ -1,34 +0,0 @@
-{
-  "Signature": null,
-  "Description": "",
-  "Packages": [
-    {
-      "Name": "shbang-test",
-      "Version": "1-r1",
-      "Arch": "x86_64",
-      "Description": "shbang test",
-      "License": "MIT",
-      "Origin": "shbang-test",
-      "Maintainer": "",
-      "URL": "",
-      "Checksum": "lh8N8wmjML0PzAr+ocS3r0+SeOQ=",
-      "Dependencies": [
-        "cmd:bash",
-        "cmd:envDashSCmd",
-        "cmd:python3.12",
-        "so:ld-linux-x86-64.so.2",
-        "so:libc.so.6"
-      ],
-      "Provides": null,
-      "InstallIf": null,
-      "Size": 4519,
-      "InstalledSize": 17302,
-      "ProviderPriority": 0,
-      "BuildTime": "1970-01-01T00:00:00Z",
-      "BuildDate": 0,
-      "RepoCommit": "",
-      "Replaces": null,
-      "DataHash": "bd2e7d74e652d107cd67ba2f31c02f8bb489779c881e300c2dddacbbdbace252"
-    }
-  ]
-}
\ No newline at end of file
diff --git a/pkg/sca/testdata/generated/x86_64/APKINDEX.tar.gz b/pkg/sca/testdata/generated/x86_64/APKINDEX.tar.gz
deleted file mode 100644
index 024c9a92..00000000
Binary files a/pkg/sca/testdata/generated/x86_64/APKINDEX.tar.gz and /dev/null differ
diff --git a/pkg/sca/testdata/go-fips-bin/go-fips-bin.yaml b/pkg/sca/testdata/go-fips-bin/go-fips-bin.yaml
index a104a6ac..fd3cd83b 100644
--- a/pkg/sca/testdata/go-fips-bin/go-fips-bin.yaml
+++ b/pkg/sca/testdata/go-fips-bin/go-fips-bin.yaml
@@ -11,7 +11,7 @@
 # please see go-install.yaml in this directory.
 package:
   name: go-fips-bin
-  version: v0.0.1
+  version: 0.0.1
   epoch: 0
   description: "A tiny fips-like go binary"
 
diff --git a/pkg/sca/testdata/py3-seaborn.yaml b/pkg/sca/testdata/py3-seaborn.yaml
new file mode 100644
index 00000000..ece1f85a
--- /dev/null
+++ b/pkg/sca/testdata/py3-seaborn.yaml
@@ -0,0 +1,43 @@
+# Generated from https://pypi.org/project/seaborn/
+package:
+  name: py3-seaborn
+  version: 0.13.2
+  epoch: 0
+  description: Statistical data visualization
+  copyright:
+    - license: BSD-3-Clause
+  dependencies:
+    runtime:
+      - numpy
+      - py3-pandas
+      - py3-matplotlib
+      - python3
+
+environment:
+  contents:
+    packages:
+      - build-base
+      - ca-certificates-bundle
+      - py3-pip
+      - python3
+      - wolfi-base
+
+pipeline:
+  - uses: git-checkout
+    with:
+      repository: https://github.com/mwaskom/seaborn
+      expected-commit: 9521ea1f29b5ce1df1aa2ed6f65f3bd1c63884bb
+      tag: v${{package.version}}
+
+  - name: Python build
+    runs: |
+      pip install . --prefix=/usr --root=${{targets.destdir}}
+
+  - uses: strip
+
+update:
+  enabled: true
+  github:
+    identifier: mwaskom/seaborn
+    tag-filter: v
+    strip-prefix: v
diff --git a/pkg/sca/testdata/systemd.yaml b/pkg/sca/testdata/systemd.yaml
new file mode 100644
index 00000000..d25bb47f
--- /dev/null
+++ b/pkg/sca/testdata/systemd.yaml
@@ -0,0 +1,77 @@
+package:
+  name: systemd
+  version: "256.2"
+  epoch: 1
+  description: The systemd System and Service Manager
+  copyright:
+    - license: LGPL-2.1-or-later AND GPL-2.0-or-later
+
+environment:
+  contents:
+    packages:
+      - build-base
+      - busybox-full
+      - ca-certificates-bundle
+      - clang-16
+      - cmake
+      - coreutils
+      - gperf
+      - kmod-dev
+      - libbpf
+      - libcap-dev
+      - libmount
+      - libuuid
+      - llvm16
+      - meson
+      - ninja
+      - posix-libc-utils
+      - py3-jinja2
+      - py3-pyelftools
+      - python3
+      - util-linux-dev
+
+pipeline:
+  - uses: git-checkout
+    with:
+      repository: https://github.com/systemd/systemd
+      tag: v${{package.version}}
+      expected-commit: c7e144eb4a2ed3838e2d361262d3eee8830d9bf5
+
+  - uses: meson/configure
+
+  - uses: meson/compile
+
+  - uses: meson/install
+
+  - runs: |
+      mkdir -p ${{targets.destdir}}/lib
+      mv ${{targets.destdir}}/usr/lib/libsystemd.so* ${{targets.destdir}}/lib/
+
+subpackages:
+  - name: "systemd-dev"
+    description: "headers for systemd"
+    pipeline:
+      - uses: split/dev
+
+  - name: "libsystemd"
+    description: "systemd library"
+    pipeline:
+      - runs: |
+          mkdir -p ${{targets.subpkgdir}}/lib
+          mv ${{targets.destdir}}/lib/libsystemd.so.* ${{targets.subpkgdir}}/lib
+
+  - name: "systemd-init"
+    description: "Configure systemd for use as an init system"
+    dependencies:
+      runtime:
+        - ${{package.name}}
+    pipeline:
+      - runs: |
+          mkdir -p ${{targets.subpkgdir}}/
+          ln -s /usr/lib/systemd/systemd ${{targets.subpkgdir}}/init
+
+update:
+  enabled: true
+  github:
+    identifier: systemd/systemd
+    strip-prefix: v