Repository: prometheus/prometheus (language: go)

Issue:
Unable to update to version 2.40.0 Docker images
### What did you do?

Upgrade Docker image to 2.40.0

### What did you expect to see?

_No response_

### What did you see instead? Under which circumstances?

I was unable to update to version 2.40.0 from this docker 

`prom/prometheus `

linux/arm tag image.

https://hub.docker.com/r/prom/prometheus/tags

I got the following error and then downgraded to version 2.37.2 linux/arm tag

```
ts=2022-11-07T20:16:17.105Z caller=main.go:564 level=info vm_limits="(soft=unlimited, hard=unlimited)"
ts=2022-11-07T20:16:17.114Z caller=web.go:559 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090
ts=2022-11-07T20:16:17.116Z caller=main.go:993 level=info msg="Starting TSDB ..."
ts=2022-11-07T20:16:17.117Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666553016057 maxt=1666562400000 ulid=01GG5RM85XTZPDKNFVAH5YBJZG
ts=2022-11-07T20:16:17.117Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666611455225 maxt=1666648800000 ulid=01GG7TH469Y1X60RYCZ8MKYRYC
ts=2022-11-07T20:16:17.118Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666693846133 maxt=1666720800000 ulid=01GGAW9PF4Q53DEMQ69XN9G77N
ts=2022-11-07T20:16:17.118Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666720801137 maxt=1666785600000 ulid=01GGCPS355YQWVHV4VX9QGCPP5
ts=2022-11-07T20:16:17.118Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666812486364 maxt=1666821600000 ulid=01GGEYKY0FT8M87V99NZFZYXH4
ts=2022-11-07T20:16:17.119Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666873792368 maxt=1666908000000 ulid=01GGFAVGPGBX5Q7PZ2SJZHY0WH
ts=2022-11-07T20:16:17.119Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666945576133 maxt=1666972800000 ulid=01GGT5K9J4TX86JH8RAMAHE663
ts=2022-11-07T20:16:17.119Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667034043617 maxt=1667044800000 ulid=01GGVJQQRFQJFJ5F3YJFXRMF3M
ts=2022-11-07T20:16:17.120Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667325548814 maxt=1667332800000 ulid=01GGYD53WBMEP7Z2YBDEF89GSS
ts=2022-11-07T20:16:17.120Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667372858273 maxt=1667383200000 ulid=01GGZPP0F7YES5VTCT1R8Y09QG
ts=2022-11-07T20:16:17.120Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667464135019 maxt=1667484000000 ulid=01GH4S7BEEVK05XAK2FSR6K9PW
ts=2022-11-07T20:16:17.120Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667681678720 maxt=1667685600000 ulid=01GH5S078BZPSG7AV4D9VA5HKH
ts=2022-11-07T20:16:17.121Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667714992879 maxt=1667721600000 ulid=01GH798S30PB93QYSQYBW53C0F
ts=2022-11-07T20:16:17.121Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667511243017 maxt=1667563200000 ulid=01GH798XV036M9GTJE57RFTFPW
ts=2022-11-07T20:16:17.121Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667765610463 maxt=1667772000000 ulid=01GH8MH1GF0F28GETBYPBS7KQ0
ts=2022-11-07T20:16:17.121Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667810974955 maxt=1667815200000 ulid=01GH8YSFQCSTC95CDK9K2H0CBW
ts=2022-11-07T20:16:17.122Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667815200000 maxt=1667822400000 ulid=01GH9STVC38EEZPRQWJ87C8683
ts=2022-11-07T20:16:17.122Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667822400398 maxt=1667829600000 ulid=01GH9STWV3S5T1081ANBE4AHGR
ts=2022-11-07T20:16:17.123Z caller=dir_locker.go:77 level=warn component=tsdb msg="A lockfile from a previous execution already existed. It was replaced" file=/data/prometheus/lock
ts=2022-11-07T20:16:17.123Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9090
ts=2022-11-07T20:16:17.124Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9090
panic: unaligned 64-bit atomic operation
goroutine 106 [running]:
runtime/internal/atomic.panicUnaligned()
	/usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x24
runtime/internal/atomic.Store64(0x68bb024, 0x20)
	/usr/local/go/src/runtime/internal/atomic/atomic_arm.s:291 +0x14
go.uber.org/atomic.(*Int64).Store(...)
	/go/pkg/mod/go.uber.org/atomic@v1.10.0/int64.go:82
github.com/prometheus/prometheus/tsdb.DefaultHeadOptions()
	/app/tsdb/head.go:174 +0xec
github.com/prometheus/prometheus/tsdb.open({0xbec9af7a, 0x10}, {0x33aa4e4, 0x6d36240}, {0x33b6d00, 0x6966940}, 0x6a0e690, {0x6d54000, 0xa, 0xa}, ...)
	/app/tsdb/db.go:771 +0xb88
github.com/prometheus/prometheus/tsdb.Open({0xbec9af7a, 0x10}, {0x33aa4e4, 0x6d36240}, {0x33b6d00, 0x6966940}, 0x6a0e690, 0x6dfd3c0)
	/app/tsdb/db.go:617 +0x8c
main.openDBWithMetrics({0xbec9af7a, 0x10}, {0x33aa548, 0x6d32be0}, {0x33b6d00, 0x6966940}, 0x6a0e690, 0x6dfd3c0)
	/app/cmd/prometheus/main.go:1139 +0xa4
main.main.func23()
	/app/cmd/prometheus/main.go:1005 +0x2bc
github.com/oklog/run.(*Group).Run.func1({0x689c570, 0x6c00000})
	/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:38 +0x24
created by github.com/oklog/run.(*Group).Run
	/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:37 +0x70
ts=2022-11-07T20:16:21.720Z caller=main.go:512 level=info msg="No time or size retention was set so using the default time retention" duration=15d
ts=2022-11-07T20:16:21.721Z caller=main.go:556 level=info msg="Starting Prometheus Server" mode=server version="(version=2.40.0, branch=HEAD, revision=c08d76b363f0d8052f3d2298df06b616af30900e)"

ts=2022-11-07T20:16:21.721Z caller=main.go:558 level=warn msg="This Prometheus binary has not been compiled for a 64-bit architecture. Due to virtual memory constraints of 32-bit systems, it is highly recommended to switch to a 64-bit binary of Prometheus." GOARCH=arm
ts=2022-11-07T20:16:21.721Z caller=main.go:561 level=info build_context="(go=go1.19.3, user=root@f1cf1846754b, date=20221107-18:17:50)"
ts=2022-11-07T20:16:21.721Z caller=main.go:562 level=info host_details="(Linux 5.15.61-v7l+ #1579 SMP Fri Aug 26 11:13:03 BST 2022 armv7l prometheus (none))"
ts=2022-11-07T20:16:21.721Z caller=main.go:563 level=info fd_limits="(soft=1048576, hard=1048576)"
ts=2022-11-07T20:16:21.721Z caller=main.go:564 level=info vm_limits="(soft=unlimited, hard=unlimited)"
ts=2022-11-07T20:16:21.733Z caller=web.go:559 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090
ts=2022-11-07T20:16:21.741Z caller=main.go:993 level=info msg="Starting TSDB ..."
ts=2022-11-07T20:16:21.742Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666553016057 maxt=1666562400000 ulid=01GG5RM85XTZPDKNFVAH5YBJZG
ts=2022-11-07T20:16:21.743Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666611455225 maxt=1666648800000 ulid=01GG7TH469Y1X60RYCZ8MKYRYC
ts=2022-11-07T20:16:21.743Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666693846133 maxt=1666720800000 ulid=01GGAW9PF4Q53DEMQ69XN9G77N
ts=2022-11-07T20:16:21.744Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666720801137 maxt=1666785600000 ulid=01GGCPS355YQWVHV4VX9QGCPP5
ts=2022-11-07T20:16:21.744Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666812486364 maxt=1666821600000 ulid=01GGEYKY0FT8M87V99NZFZYXH4
ts=2022-11-07T20:16:21.745Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666873792368 maxt=1666908000000 ulid=01GGFAVGPGBX5Q7PZ2SJZHY0WH
ts=2022-11-07T20:16:21.745Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1666945576133 maxt=1666972800000 ulid=01GGT5K9J4TX86JH8RAMAHE663
ts=2022-11-07T20:16:21.745Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667034043617 maxt=1667044800000 ulid=01GGVJQQRFQJFJ5F3YJFXRMF3M
ts=2022-11-07T20:16:21.746Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667325548814 maxt=1667332800000 ulid=01GGYD53WBMEP7Z2YBDEF89GSS
ts=2022-11-07T20:16:21.746Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667372858273 maxt=1667383200000 ulid=01GGZPP0F7YES5VTCT1R8Y09QG
ts=2022-11-07T20:16:21.747Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667464135019 maxt=1667484000000 ulid=01GH4S7BEEVK05XAK2FSR6K9PW
ts=2022-11-07T20:16:21.747Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667681678720 maxt=1667685600000 ulid=01GH5S078BZPSG7AV4D9VA5HKH
ts=2022-11-07T20:16:21.747Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667714992879 maxt=1667721600000 ulid=01GH798S30PB93QYSQYBW53C0F
ts=2022-11-07T20:16:21.747Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667511243017 maxt=1667563200000 ulid=01GH798XV036M9GTJE57RFTFPW
ts=2022-11-07T20:16:21.748Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667765610463 maxt=1667772000000 ulid=01GH8MH1GF0F28GETBYPBS7KQ0
ts=2022-11-07T20:16:21.748Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667810974955 maxt=1667815200000 ulid=01GH8YSFQCSTC95CDK9K2H0CBW
ts=2022-11-07T20:16:21.748Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667815200000 maxt=1667822400000 ulid=01GH9STVC38EEZPRQWJ87C8683
ts=2022-11-07T20:16:21.749Z caller=repair.go:56 level=info component=tsdb msg="Found healthy block" mint=1667822400398 maxt=1667829600000 ulid=01GH9STWV3S5T1081ANBE4AHGR
ts=2022-11-07T20:16:21.752Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9090
ts=2022-11-07T20:16:21.753Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9090
ts=2022-11-07T20:16:21.753Z caller=dir_locker.go:77 level=warn component=tsdb msg="A lockfile from a previous execution already existed. It was replaced" file=/data/prometheus/lock
panic: unaligned 64-bit atomic operation
goroutine 244 [running]:
runtime/internal/atomic.panicUnaligned()
	/usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x24
runtime/internal/atomic.Store64(0x68c2124, 0x20)
	/usr/local/go/src/runtime/internal/atomic/atomic_arm.s:291 +0x14
go.uber.org/atomic.(*Int64).Store(...)
	/go/pkg/mod/go.uber.org/atomic@v1.10.0/int64.go:82
github.com/prometheus/prometheus/tsdb.DefaultHeadOptions()
	/app/tsdb/head.go:174 +0xec
github.com/prometheus/prometheus/tsdb.open({0xbea27f7a, 0x10}, {0x33aa4e4, 0x689de60}, {0x33b6d00, 0x69ee900}, 0x6d49b90, {0x68434a0, 0xa, 0xa}, ...)
	/app/tsdb/db.go:771 +0xb88
github.com/prometheus/prometheus/tsdb.Open({0xbea27f7a, 0x10}, {0x33aa4e4, 0x689de60}, {0x33b6d00, 0x69ee900}, 0x6d49b90, 0x680efd8)
	/app/tsdb/db.go:617 +0x8c
main.openDBWithMetrics({0xbea27f7a, 0x10}, {0x33aa548, 0x6d43aa0}, {0x33b6d00, 0x69ee900}, 0x6d49b90, 0x680efd8)
	/app/cmd/prometheus/main.go:1139 +0xa4
main.main.func23()
	/app/cmd/prometheus/main.go:1005 +0x2bc
github.com/oklog/run.(*Group).Run.func1({0x6ab3dd0, 0x6fea3d8})
	/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:38 +0x24
created by github.com/oklog/run.(*Group).Run
	/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:37 +0x70
```

### System information

Raspberry Pi4 with RaspianOS bullseye 32bit

### Prometheus version

2.40.0

### Prometheus configuration file

_No response_

### Alertmanager version

_No response_

### Alertmanager configuration file

_No response_

### Logs

_No response_


Discussion / hints:
Besides that, is it intended to have a `v2.40.0` (without the rc suffix) while `v2.40.0` is still a pre-release? Wouldn't it make more sense to tag e.g. `v2.40.0` only after it becomes a stable release?

Fix patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index f635db74697..c49b0c6014d 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -16,6 +16,7 @@ jobs:
       - uses: ./.github/actions/setup_environment
       - run: make GO_ONLY=1 SKIP_GOLANGCI_LINT=1
       - run: go test ./tsdb/ -test.tsdb-isolation=false
+      - run: GOARCH=386 go test ./cmd/prometheus
       - run: make -C documentation/examples/remote_storage
       - run: make -C documentation/examples
       - uses: ./.github/actions/check_proto
diff --git a/tsdb/head.go b/tsdb/head.go
index 85baad680d5..be4b3b6a95f 100644
--- a/tsdb/head.go
+++ b/tsdb/head.go
@@ -132,6 +132,9 @@ type HeadOptions struct {
 	// https://pkg.go.dev/sync/atomic#pkg-note-BUG
 	MaxExemplars atomic.Int64
 
+	OutOfOrderTimeWindow atomic.Int64
+	OutOfOrderCapMax     atomic.Int64
+
 	// EnableNativeHistograms enables the ingestion of native histograms.
 	EnableNativeHistograms atomic.Bool
 
@@ -141,8 +144,6 @@ type HeadOptions struct {
 	ChunkPool            chunkenc.Pool
 	ChunkWriteBufferSize int
 	ChunkWriteQueueSize  int
-	OutOfOrderTimeWindow atomic.Int64
-	OutOfOrderCapMax     atomic.Int64
 
 	// StripeSize sets the number of entries in the hash map, it must be a power of 2.
 	// A larger StripeSize will allocate more memory up-front, but will increase performance when handling a large number of series.


Test patch: