Change:
While working on for , I found that our kubernetes1.24 package stopped to produce bit-identical results on 2 fresh builds. The first chunk is likely just the hash of the remainder, so can be ignored. The other chunks indicate that a random build temporary path is used for Sources should be able to reproduce bit-identical binaries e.g. for SLSA level4 On Debian or openSUSE do No response <detailsWARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version. Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-05-05T00:00:00Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v4.5.4 console $ cat /etc/os-release $ uname -a Linux bmwiedemann-factory 5.18.6-1-default SMP PREEMPTDYNAMIC Thu Jun 23 05:46:18 UTC 2022 () x8664 x8664 x86_64 GNU/Linux </details<detailsgo1.18 </details<details</details<details</details
/sig release /area build-release
can you please post a link to the full logs from the build?
any news on this? Haven't heard from (context: we worked to fix this a couple of times in )
Unfortunately not, is this something we can fix on the OBS side? If the go build timestamp () is the main issue here then we can approach a fix in our build pipeline.
It would be good if would not be embedded in the first place. It might be possible to replace it somehow in our builds, but all other people wanting reproducible builds, would need to replicate that.
full logs from the build: from kubernetes-1.24.3 built 2022-08-18
can you please try setting GOCACHE=off when you run the build? I am suspecting that the temp directory is being generated here: (or add an export in for that env var)
I to but still got these diffs.
So i built golang from source, peppered it with changes to track down where this is getting created and found that it is coming from: I don't know yet how to fix it. will dig more when i can (travelling for the next few days)
Would it work to override the , I see that they use the : Ref: https://reproducible- Edit: Probably not, we may make it configurable in golang :thinking:
this is so tricky :( will dig more when i get back from my event this week
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue with Mark this issue as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close not-planned [1]:
Closing this issue, marking it as "Not Planned". <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue with Mark this issue as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This is still relevant even with 1.27. It might be worth revisiting this issue again. /reopen /triage accepted
Reopened this issue. <detailsIn response to : This is still relevant even with 1.27. It might be worth revisiting this issue again. /reopen /triage accepted Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-lifecycle rotten
latest run -
I found that our package has the same issue, so it probably it is a toolchain issue in golang - and much faster to debug outside of the slow kubernetes build. =filed
AIUI, the conclusion by the golang people seems to be that commit is not the right approach and one should use with the go call instead. Not sure how that interacts with other parts of your build.
cc (please see referenced commit above!)
I see. I inherited the gcflags before so I plead innocence. It seems plausible to change it from to - I don't know when I will have time to look at that, though.
My patch doesn't seem to fix the whole problem: thockin-glaptop4 kubernetes buildtrimpath /$ make kubelet +++ [1114 19:10:12] Building go targets for linux/amd64 (non-static) thockin-glaptop4 kubernetes buildtrimpath /$ md5sum output/bin/kubelet output/bin/kubelet thockin-glaptop4 kubernetes buildtrimpath /$ make kubelet +++ [1114 19:12:40] Building go targets for linux/amd64 (non-static) thockin-glaptop4 kubernetes buildtrimpath /$ md5sum output/bin/kubelet output/bin/kubelet thockin-glaptop4 kubernetes build_trimpath /$
diff --- a 2023-11-14 19:13:45. -0800 +++ b 2023-11-14 19:13:56. -0800 -1,4 +1,4 -38rZHdQckYF9EEzmndR/BAXOwzCcq0Mk1UHT62ha/QLiRgXP4oojWh3oWP9/TWG7KW90IL8vKt1UOEPR +LeLArvl7sgWAd55ROKN2/BAXOwzCcq0Mk1UHT62ha/QLiRgXP4oojWh3oWP9/cYb5ZGQwa6js077YiAvK /lib64/ld-linux-x86- AUATUSH []AA] -,8 +,8 go1.21.3 runtime/cgo: call frame too large -2023-11-15T03:12:40Z -2023-11-15T03:12:40Z +2023-11-15T03:13:47Z +2023-11-15T03:13:47Z pthreadcreate failed: %s bad stack bounds: lo=%p hi=%p expand 32-byte kexpand 32-byte k
That seems unrelated. I tested with and it made our openSUSE package fully reproducible. Your diff contains a timestamp - maybe would get rid of that. The top part of the diff is just the go build-id hash of the remainder and can be ignored.
Thanks for confirming
Will there be backports to 1.24++ ? I found it applies cleanly to 1.26++ and works with some small tweaks in 1.24 and 1.25. If not, I'd submit the patch for our openSUSE packages.
1.24 and 1.25 reached EOL and we don't support these branches any longer. We can cherry-pick this to 1.26+ if folks ( agree
Cherry-picks: 1.29 1.28 1.27 1.26

Discussion:
done V=2 kube::log::info "Env for ${platform}: GOOS=${GOOS-} GOARCH=${GOARCH-} GOROOT=${GOROOT-} CGO_ENABLED=${CGO_ENABLED-} CC=${CC-}" <del> V=3 kube::log::info "Building binaries with GCFLAGS=${gogcflags} ASMFLAGS=${goasmflags} LDFLAGS=${goldflags}" </del> <ins> V=3 kube::log::info "Building binaries with GCFLAGS=${gogcflags} LDFLAGS=${goldflags}" </ins> local -a build_args if [[ "${#statics[@]}" != 0 ]]; then

-installsuffix=static ${goflags:+"${goflags[@]}"} -gcflags="${gogcflags}" <del> -asmflags="${goasmflags}" </del> -ldflags="${goldflags}" -tags="${gotags:-}" )

build_args=( ${goflags:+"${goflags[@]}"} -gcflags="${gogcflags}" <del> -asmflags="${goasmflags}" </del> -ldflags="${goldflags}" -tags="${gotags:-}" )

go test -c  ${goflags:+"${goflags[@]}"}  -gcflags="${gogcflags}"  <del> -asmflags="${goasmflags}"  </del> -ldflags="${goldflags}"  -tags="${gotags:-}"  -o "${outfile}" 

# These are "local" but are visible to and relied on by functions this # function calls.  They are effectively part of the calling API to # build_binaries_for_platform. <del> local goflags goldflags goasmflags gogcflags gotags </del> <ins> local goflags goldflags gogcflags gotags </ins> <del> # This is $(pwd) because we use run-in-gopath to build.  Once that is # excised, this can become ${KUBE_ROOT}. local trimroot # two lines to appease shellcheck SC2155 trimroot=$(pwd) goasmflags="all=-trimpath=${trimroot}" </del> <ins> goflags=() gogcflags="${GOGCFLAGS:-}" goldflags="all=$(kube::version::ldflags) ${GOLDFLAGS:-}" </ins> <del> gogcflags="all=-trimpath=${trimroot} ${GOGCFLAGS:-}" </del> if [[ "${DBG:-}" == 1 ]]; then # Debugging - disable optimizations and inlining and trimPath <del> gogcflags="${GOGCFLAGS:-} all=-N -l" goasmflags="" fi goldflags="all=$(kube::version::ldflags) ${GOLDFLAGS:-}" if [[ "${DBG:-}" != 1 ]]; then # Not debugging - disable symbols and DWARF. </del> <ins> gogcflags="${gogcflags} all=-N -l" else # Not debugging - disable symbols and DWARF, trim embedded paths </ins> goldflags="${goldflags} -s -w" <ins> goflags+=("-trimpath") </ins> fi # Extract tags if any specified in GOFLAGS

cd "${KUBE_ROOT}" # build ginkgo and e2e.test <del> hack/make-rules/build.sh github.com/onsi/ginkgo/v2/ginkgo test/e2e/e2e.test </del> <ins> # # Set DBG=1 to build with embedded filenames as filenames rather than # module-relative names (e.g. /src/kube/foo/bar.go vs. # k8s.io/kubernetes/foo/bar.go).  These names are used by gingko in # `--spec-dump` which is consumed later in conformance verification. DBG=1 hack/make-rules/build.sh github.com/onsi/ginkgo/v2/ginkgo test/e2e/e2e.test </ins> # dump spec ./_output/bin/ginkgo --dry-run=true --focus='[Conformance]' ./_output/bin/e2e.test -- --spec-dump "${KUBE_ROOT}/_output/specsummaries.json" > /dev/null