Repository: armadaproject/armada
Pull request #2539 (2023-06-05 13:28:31)

Scheduling key optimisations
The current scheduling key implementation is relatively slow and requires many allocs, causing unnecessary GC pressure. Here, I've switched to a new implementation that is much faster and doesn't require allocs. Further, affinities that are equivalent from a scheduling point of view but differ in some other way will now map to the same hash. I've added tests to check that the affinity docs don't change unexpectedly to avoid the affinity definition changing causing us issues.

The new implementation is about 5x faster for jobs without affinities and about 100x faster for jobs with affinities.

The speedup is in large part due to using a 256-bit highway hash instead of sha1. While sha1 is designed to make it hard to find X and Y such that hash(X) = hash(Y), highway hash is designed to make it hard to find Y such that hash(Y) = hash(X, k), where k is an unknown key and X and hash(X) are known. As a result, highway hash can be made more efficient and is a better fit for this application than sha1. For affinities, the speedup is largely due to hashing the relevant parts of the affinities directly instead of serialising to json and hashing the resulting byte array. 

Current implementation:
```
go.exe test -benchmem -run=^$ -bench ^BenchmarkSchedulingKey$ github.com/armadaproject/armada/internal/scheduler/schedulerobjects
goos: windows
goarch: amd64
pkg: github.com/armadaproject/armada/internal/scheduler/schedulerobjects
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkSchedulingKey-20         646980              1796 ns/op             640 B/op         39 allocs/op
PASS
ok      github.com/armadaproject/armada/internal/scheduler/schedulerobjects     1.218s


go.exe test -benchmem -run=^$ -bench ^BenchmarkSchedulingKey_Affinity$ github.com/armadaproject/armada/internal/scheduler/schedulerobjects
goos: windows
goarch: amd64
pkg: github.com/armadaproject/armada/internal/scheduler/schedulerobjects
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkSchedulingKey_Affinity-20         31515             38902 ns/op           17025 B/op        504 allocs/op
PASS
ok      github.com/armadaproject/armada/internal/scheduler/schedulerobjects     1.651s
```

New implementation:
```
go.exe test -v -benchmem -run=^$ -bench ^BenchmarkSchedulingKey$ github.com/armadaproject/armada/internal/scheduler/schedulerobjects
goos: windows
goarch: amd64
pkg: github.com/armadaproject/armada/internal/scheduler/schedulerobjects
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkSchedulingKey
BenchmarkSchedulingKey-20        3922611               304.3 ns/op             0 B/op          0 allocs/op
PASS
ok      github.com/armadaproject/armada/internal/scheduler/schedulerobjects     1.540s

go.exe test -v -benchmem -run=^$ -bench ^BenchmarkSchedulingKey_Affinity$ github.com/armadaproject/armada/internal/scheduler/schedulerobjects
goos: windows
goarch: amd64
pkg: github.com/armadaproject/armada/internal/scheduler/schedulerobjects
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkSchedulingKey_Affinity
BenchmarkSchedulingKey_Affinity-20       3030038               394.9 ns/op             0 B/op          0 allocs/op
PASS
ok      github.com/armadaproject/armada/internal/scheduler/schedulerobjects     1.636s
```

┆Issue is synchronized with this [Jira Task](https://gr-oss.atlassian.net/browse/BATCH-319) by [Unito](https://www.unito.io)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject) Report
Patch coverage: **`68.43`**% and project coverage change: **`+0.15`** :tada:
> Comparison is base [(`9188597`)](https://app.codecov.io/gh/armadaproject/armada/commit/9188597ceb3969442e0b83b1c9171dea671d8733?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject) 58.57% compared to head [(`052ae59`)](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject) 58.72%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #2539      +/-   ##
==========================================
+ Coverage   58.57%   58.72%   +0.15%     
==========================================
  Files         236      236              
  Lines       29889    30207     +318     
==========================================
+ Hits        17506    17739     +233     
- Misses      11041    11120      +79     
- Partials     1342     1348       +6     
```

| Flag | Coverage Δ | |
|---|---|---|
| armada-server | `58.72% <68.43%> (+0.15%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject) | Coverage Δ | |
|---|---|---|
| [internal/scheduleringester/instructions.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyaW5nZXN0ZXIvaW5zdHJ1Y3Rpb25zLmdv) | `74.13% <ø> (-0.18%)` | :arrow_down: |
| [internal/scheduler/context/context.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL2NvbnRleHQvY29udGV4dC5nbw==) | `41.07% <7.69%> (+1.56%)` | :arrow_up: |
| [internal/scheduler/jobdb/job.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL2pvYmRiL2pvYi5nbw==) | `81.33% <26.47%> (-8.88%)` | :arrow_down: |
| [internal/scheduler/schedulerobjects/podutils.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL3NjaGVkdWxlcm9iamVjdHMvcG9kdXRpbHMuZ28=) | `64.53% <71.68%> (+1.70%)` | :arrow_up: |
| [internal/scheduler/queue\_scheduler.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL3F1ZXVlX3NjaGVkdWxlci5nbw==) | `65.16% <76.92%> (ø)` | |
| [internal/scheduler/pool\_assigner.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL3Bvb2xfYXNzaWduZXIuZ28=) | `75.45% <92.59%> (+2.72%)` | :arrow_up: |
| [internal/scheduler/gang\_scheduler.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL2dhbmdfc2NoZWR1bGVyLmdv) | `78.12% <100.00%> (-2.07%)` | :arrow_down: |
| [internal/scheduler/submitcheck.go](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject#diff-aW50ZXJuYWwvc2NoZWR1bGVyL3N1Ym1pdGNoZWNrLmdv) | `74.73% <100.00%> (+0.98%)` | :arrow_up: |

... and [1 file with indirect coverage changes](https://app.codecov.io/gh/armadaproject/armada/pull/2539/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject)


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/armadaproject/armada/pull/2539?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=armadaproject).

severinson: For some reason the affinity tests aren't showing in the diff. They're here:
https://github.com/armadaproject/armada/blob/e5b8b70aaacac2d666e30f83bc7ab4d073f25286/internal/scheduler/schedulerobjects/podutils_test.go

Patch:
diff --git a/.golangci.yml b/.golangci.yml
index 8b9647cd..b7d8b3f0 100644
--- a/.golangci.yml
+++ b/.golangci.yml
@@ -1,6 +1,10 @@
 issues:
   max-issues-per-linter: 0
   max-same-issues: 0
+  exclude-rules:
+    - path: internal/scheduler/schedulerobjects/podutils_test.go
+      linters:
+        - lll
 
 output:
   print-issued-lines: true
diff --git a/go.mod b/go.mod
index 03cb8615..0f03be2a 100644
--- a/go.mod
+++ b/go.mod
@@ -88,6 +88,7 @@ require (
 	github.com/jackc/pgx/v5 v5.3.1
 	github.com/jessevdk/go-flags v1.5.0
 	github.com/magefile/mage v1.14.0
+	github.com/minio/highwayhash v1.0.2
 	github.com/openconfig/goyang v1.2.0
 	github.com/prometheus/common v0.37.0
 	github.com/sanity-io/litter v1.5.5
diff --git a/go.sum b/go.sum
index 5c336f81..3b3ef71a 100644
--- a/go.sum
+++ b/go.sum
@@ -655,6 +655,8 @@ github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5
 github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI=
 github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
 github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
+github.com/minio/highwayhash v1.0.2 h1:Aak5U0nElisjDCfPSG79Tgzkn2gl66NxOMspRrKnA/g=
+github.com/minio/highwayhash v1.0.2/go.mod h1:BQskDq+xkJ12lmlUUi7U0M5Swg3EWR+dLTk+kldvVxY=
 github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
 github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
 github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
@@ -1105,6 +1107,7 @@ golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5h
 golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190130150945-aca44879d564/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
diff --git a/internal/scheduler/context/context.go b/internal/scheduler/context/context.go
index d618d136..58b85fe6 100644
--- a/internal/scheduler/context/context.go
+++ b/internal/scheduler/context/context.go
@@ -56,6 +56,8 @@ type SchedulingContext struct {
 	NumEvictedGangs int
 	// Reason for why the scheduling round finished.
 	TerminationReason string
+	// Used to efficiently generate scheduling keys.
+	SchedulingKeyGenerator *schedulerobjects.SchedulingKeyGenerator
 	// Record of job scheduling requirements known to be unfeasible.
 	// Used to immediately reject new jobs with identical reqirements.
 	// Maps to the JobSchedulingContext of a previous job attempted to schedule with the same key.
@@ -82,10 +84,25 @@ func NewSchedulingContext(
 		ScheduledResources:           schedulerobjects.NewResourceListWithDefaultSize(),
 		ScheduledResourcesByPriority: make(schedulerobjects.QuantityByPriorityAndResourceType),
 		EvictedResourcesByPriority:   make(schedulerobjects.QuantityByPriorityAndResourceType),
+		SchedulingKeyGenerator:       schedulerobjects.NewSchedulingKeyGenerator(),
 		UnfeasibleSchedulingKeys:     make(map[schedulerobjects.SchedulingKey]*JobSchedulingContext),
 	}
 }
 
+func (sctx *SchedulingContext) SchedulingKeyFromLegacySchedulerJob(job interfaces.LegacySchedulerJob) schedulerobjects.SchedulingKey {
+	var priority int32
+	if priorityClass, ok := sctx.PriorityClasses[job.GetPriorityClassName()]; ok {
+		priority = priorityClass.Priority
+	}
+	return sctx.SchedulingKeyGenerator.Key(
+		job.GetNodeSelector(),
+		job.GetAffinity(),
+		job.GetTolerations(),
+		job.GetResourceRequirements().Requests,
+		priority,
+	)
+}
+
 func (sctx *SchedulingContext) ClearUnfeasibleSchedulingKeys() {
 	sctx.UnfeasibleSchedulingKeys = make(map[schedulerobjects.SchedulingKey]*JobSchedulingContext)
 }
diff --git a/internal/scheduler/gang_scheduler.go b/internal/scheduler/gang_scheduler.go
index e8a1ee72..1ebbcb18 100644
--- a/internal/scheduler/gang_scheduler.go
+++ b/internal/scheduler/gang_scheduler.go
@@ -6,7 +6,6 @@ import (
 
 	"github.com/pkg/errors"
 
-	"github.com/armadaproject/armada/internal/armada/configuration"
 	"github.com/armadaproject/armada/internal/common/util"
 	schedulerconstraints "github.com/armadaproject/armada/internal/scheduler/constraints"
 	schedulercontext "github.com/armadaproject/armada/internal/scheduler/context"
@@ -75,12 +74,12 @@ func (sch *GangScheduler) Schedule(ctx context.Context, gctx *schedulercontext.G
 			//
 			// Only record unfeasible scheduling keys for single-job gangs.
 			// Since a gang may be unschedulable even if all its members are individually schedulable.
-			if !sch.skipUnsuccessfulSchedulingKeyCheck {
-				if schedulingKey, jctx, ok := schedulingKeyIfSingleJobGang(gctx, sch.schedulingContext.PriorityClasses); ok {
-					if _, ok := sch.schedulingContext.UnfeasibleSchedulingKeys[schedulingKey]; !ok {
-						// Keep the first jctx for each unfeasible schedulingKey.
-						sch.schedulingContext.UnfeasibleSchedulingKeys[schedulingKey] = jctx
-					}
+			if !sch.skipUnsuccessfulSchedulingKeyCheck && len(gctx.JobSchedulingContexts) == 1 {
+				jctx := gctx.JobSchedulingContexts[0]
+				schedulingKey := sch.schedulingContext.SchedulingKeyFromLegacySchedulerJob(jctx.Job)
+				if _, ok := sch.schedulingContext.UnfeasibleSchedulingKeys[schedulingKey]; !ok {
+					// Keep the first jctx for each unfeasible schedulingKey.
+					sch.schedulingContext.UnfeasibleSchedulingKeys[schedulingKey] = jctx
 				}
 			}
 		}
@@ -151,21 +150,3 @@ func requestIsLargeEnough(totalResourceRequests, minRequest schedulerobjects.Res
 	}
 	return true, ""
 }
-
-func schedulingKeyIfSingleJobGang(
-	gctx *schedulercontext.GangSchedulingContext,
-	priorityClasses map[string]configuration.PriorityClass,
-) (schedulerobjects.SchedulingKey, *schedulercontext.JobSchedulingContext, bool) {
-	if len(gctx.JobSchedulingContexts) == 1 {
-		jctx := gctx.JobSchedulingContexts[0]
-		schedulingKey, ok := schedulingKeyFromLegacySchedulerJob(jctx.Job, priorityClasses)
-		return schedulingKey, jctx, ok
-	}
-	return schedulerobjects.SchedulingKey{}, nil, false
-}
-
-func schedulingKeyFromLegacySchedulerJob(job interfaces.LegacySchedulerJob, priorityClasses map[string]configuration.PriorityClass) (schedulerobjects.SchedulingKey, bool) {
-	jobSchedulingInfo := job.GetRequirements(priorityClasses)
-	schedulingKey, ok := jobSchedulingInfo.SchedulingKey()
-	return schedulingKey, ok
-}
diff --git a/internal/scheduler/interfaces/interfaces.go b/internal/scheduler/interfaces/interfaces.go
index 86e3d011..7786fb99 100644
--- a/internal/scheduler/interfaces/interfaces.go
+++ b/internal/scheduler/interfaces/interfaces.go
@@ -1,6 +1,8 @@
 package interfaces
 
 import (
+	v1 "k8s.io/api/core/v1"
+
 	"github.com/armadaproject/armada/internal/armada/configuration"
 	"github.com/armadaproject/armada/internal/scheduler/schedulerobjects"
 )
@@ -13,6 +15,10 @@ type LegacySchedulerJob interface {
 	GetAnnotations() map[string]string
 	GetRequirements(map[string]configuration.PriorityClass) *schedulerobjects.JobSchedulingInfo
 	GetPriorityClassName() string
+	GetNodeSelector() map[string]string
+	GetAffinity() *v1.Affinity
+	GetTolerations() []v1.Toleration
+	GetResourceRequirements() v1.ResourceRequirements
 }
 
 func PodRequirementFromLegacySchedulerJob(job LegacySchedulerJob, priorityClasses map[string]configuration.PriorityClass) *schedulerobjects.PodRequirements {
diff --git a/internal/scheduler/jobdb/job.go b/internal/scheduler/jobdb/job.go
index 22850f55..aae2e08b 100644
--- a/internal/scheduler/jobdb/job.go
+++ b/internal/scheduler/jobdb/job.go
@@ -5,6 +5,7 @@ import (
 
 	"github.com/google/uuid"
 	"golang.org/x/exp/maps"
+	v1 "k8s.io/api/core/v1"
 
 	"github.com/armadaproject/armada/internal/armada/configuration"
 	"github.com/armadaproject/armada/internal/scheduler/schedulerobjects"
@@ -149,18 +150,69 @@ func (job *Job) JobSchedulingInfo() *schedulerobjects.JobSchedulingInfo {
 	return job.jobSchedulingInfo
 }
 
-// GetRequirements  returns the scheduling requirements associated with the job.
-// this is needed for compatibility with interfaces.LegacySchedulerJob
+// GetAnnotations returns the annotations on the job.
+// This is needed for compatibility with interfaces.LegacySchedulerJob
+func (job *Job) GetAnnotations() map[string]string {
+	if req := job.getPodRequirements(); req != nil {
+		return req.Annotations
+	}
+	return nil
+}
+
+// GetRequirements returns the scheduling requirements associated with the job.
+// Needed for compatibility with interfaces.LegacySchedulerJob
 func (job *Job) GetRequirements(_ map[string]configuration.PriorityClass) *schedulerobjects.JobSchedulingInfo {
 	return job.JobSchedulingInfo()
 }
 
-// GetPriorityClassName returns the priorityClassName of the job.
 // Needed for compatibility with interfaces.LegacySchedulerJob
 func (job *Job) GetPriorityClassName() string {
 	return job.JobSchedulingInfo().PriorityClassName
 }
 
+// Needed for compatibility with interfaces.LegacySchedulerJob
+func (job *Job) GetNodeSelector() map[string]string {
+	if req := job.getPodRequirements(); req != nil {
+		return req.NodeSelector
+	}
+	return nil
+}
+
+// Needed for compatibility with interfaces.LegacySchedulerJob
+func (job *Job) GetAffinity() *v1.Affinity {
+	if req := job.getPodRequirements(); req != nil {
+		return req.Affinity
+	}
+	return nil
+}
+
+// Needed for compatibility with interfaces.LegacySchedulerJob
+func (job *Job) GetTolerations() []v1.Toleration {
+	if req := job.getPodRequirements(); req != nil {
+		return req.Tolerations
+	}
+	return nil
+}
+
+// Needed for compatibility with interfaces.LegacySchedulerJob
+func (job *Job) GetResourceRequirements() v1.ResourceRequirements {
+	if req := job.getPodRequirements(); req != nil {
+		return req.ResourceRequirements
+	}
+	return v1.ResourceRequirements{}
+}
+
+func (job *Job) getPodRequirements() *schedulerobjects.PodRequirements {
+	requirements := job.jobSchedulingInfo.GetObjectRequirements()
+	if len(requirements) == 0 {
+		return nil
+	}
+	if podReqs := requirements[0].GetPodRequirements(); podReqs != nil {
+		return podReqs
+	}
+	return nil
+}
+
 // Queued returns true if the job should be considered by the scheduler for assignment or false otherwise.
 func (job *Job) Queued() bool {
 	return job.queued
@@ -250,19 +302,6 @@ func (job *Job) Created() int64 {
 	return job.created
 }
 
-// GetAnnotations returns the annotations on the job.
-// This is needed for compatibility with interfaces.LegacySchedulerJob
-func (job *Job) GetAnnotations() map[string]string {
-	requirements := job.jobSchedulingInfo.GetObjectRequirements()
-	if len(requirements) == 0 {
-		return nil
-	}
-	if podReqs := requirements[0].GetPodRequirements(); podReqs != nil {
-		return podReqs.GetAnnotations()
-	}
-	return nil
-}
-
 // InTerminalState returns true if the job  is in a terminal state
 func (job *Job) InTerminalState() bool {
 	return job.succeeded || job.cancelled || job.failed
diff --git a/internal/scheduler/pool_assigner.go b/internal/scheduler/pool_assigner.go
index 4e0faa16..d399b434 100644
--- a/internal/scheduler/pool_assigner.go
+++ b/internal/scheduler/pool_assigner.go
@@ -29,17 +29,18 @@ type executor struct {
 }
 
 type DefaultPoolAssigner struct {
-	executorTimeout    time.Duration
-	priorityClasses    map[string]configuration.PriorityClass
-	priorities         []int32
-	indexedResources   []string
-	indexedTaints      []string
-	indexedNodeLabels  []string
-	poolByExecutorId   map[string]string
-	executorsByPool    map[string][]*executor
-	executorRepository database.ExecutorRepository
-	poolCache          *lru.Cache
-	clock              clock.Clock
+	executorTimeout        time.Duration
+	priorityClasses        map[string]configuration.PriorityClass
+	priorities             []int32
+	indexedResources       []string
+	indexedTaints          []string
+	indexedNodeLabels      []string
+	poolByExecutorId       map[string]string
+	executorsByPool        map[string][]*executor
+	executorRepository     database.ExecutorRepository
+	schedulingKeyGenerator *schedulerobjects.SchedulingKeyGenerator
+	poolCache              *lru.Cache
+	clock                  clock.Clock
 }
 
 func NewPoolAssigner(executorTimeout time.Duration,
@@ -51,17 +52,18 @@ func NewPoolAssigner(executorTimeout time.Duration,
 		return nil, errors.Wrap(err, "error  creating PoolAssigner pool cache")
 	}
 	return &DefaultPoolAssigner{
-		executorTimeout:    executorTimeout,
-		priorityClasses:    schedulingConfig.Preemption.PriorityClasses,
-		executorsByPool:    map[string][]*executor{},
-		poolByExecutorId:   map[string]string{},
-		priorities:         schedulingConfig.Preemption.AllowedPriorities(),
-		indexedResources:   schedulingConfig.IndexedResources,
-		indexedTaints:      schedulingConfig.IndexedTaints,
-		indexedNodeLabels:  schedulingConfig.IndexedNodeLabels,
-		executorRepository: executorRepository,
-		poolCache:          poolCache,
-		clock:              clock.RealClock{},
+		executorTimeout:        executorTimeout,
+		priorityClasses:        schedulingConfig.Preemption.PriorityClasses,
+		executorsByPool:        map[string][]*executor{},
+		poolByExecutorId:       map[string]string{},
+		priorities:             schedulingConfig.Preemption.AllowedPriorities(),
+		indexedResources:       schedulingConfig.IndexedResources,
+		indexedTaints:          schedulingConfig.IndexedTaints,
+		indexedNodeLabels:      schedulingConfig.IndexedNodeLabels,
+		executorRepository:     executorRepository,
+		schedulingKeyGenerator: schedulerobjects.NewSchedulingKeyGenerator(),
+		poolCache:              poolCache,
+		clock:                  clock.RealClock{},
 	}, nil
 }
 
@@ -88,6 +90,7 @@ func (p *DefaultPoolAssigner) Refresh(ctx context.Context) error {
 	}
 	p.executorsByPool = executorsByPool
 	p.poolByExecutorId = poolByExecutorId
+	p.schedulingKeyGenerator = schedulerobjects.NewSchedulingKeyGenerator()
 	p.poolCache.Purge()
 	return nil
 }
@@ -100,10 +103,19 @@ func (p *DefaultPoolAssigner) AssignPool(j *jobdb.Job) (string, error) {
 	}
 
 	// See if we have this set of reqs cached.
-	if schedulingKey, ok := j.JobSchedulingInfo().SchedulingKey(); ok {
-		if cachedPool, ok := p.poolCache.Get(schedulingKey); ok {
-			return cachedPool.(string), nil
-		}
+	var priority int32
+	if priorityClass, ok := p.priorityClasses[j.GetPriorityClassName()]; ok {
+		priority = priorityClass.Priority
+	}
+	schedulingKey := p.schedulingKeyGenerator.Key(
+		j.GetNodeSelector(),
+		j.GetAffinity(),
+		j.GetTolerations(),
+		j.GetResourceRequirements().Requests,
+		priority,
+	)
+	if cachedPool, ok := p.poolCache.Get(schedulingKey); ok {
+		return cachedPool.(string), nil
 	}
 
 	req := PodRequirementFromJobSchedulingInfo(j.JobSchedulingInfo())
@@ -124,9 +136,7 @@ func (p *DefaultPoolAssigner) AssignPool(j *jobdb.Job) (string, error) {
 					return "", errors.WithMessagef(err, "error selecting node for job %s", j.Id())
 				}
 				if report.Node != nil {
-					if schedulingKey, ok := j.JobSchedulingInfo().SchedulingKey(); ok {
-						p.poolCache.Add(schedulingKey, pool)
-					}
+					p.poolCache.Add(schedulingKey, pool)
 					return pool, nil
 				}
 			}
diff --git a/internal/scheduler/queue_scheduler.go b/internal/scheduler/queue_scheduler.go
index 50c3fb16..9408015e 100644
--- a/internal/scheduler/queue_scheduler.go
+++ b/internal/scheduler/queue_scheduler.go
@@ -212,21 +212,20 @@ func (it *QueuedGangIterator) Peek() (*schedulercontext.GangSchedulingContext, e
 
 		// Skip this job if it's known to be unschedulable.
 		if len(it.schedulingContext.UnfeasibleSchedulingKeys) > 0 {
-			if schedulingKey, ok := schedulingKeyFromLegacySchedulerJob(job, it.schedulingContext.PriorityClasses); ok {
-				if unsuccessfulJctx, ok := it.schedulingContext.UnfeasibleSchedulingKeys[schedulingKey]; ok {
-					jctx := &schedulercontext.JobSchedulingContext{
-						Created:              time.Now(),
-						ExecutorId:           it.schedulingContext.ExecutorId,
-						JobId:                job.GetId(),
-						Job:                  job,
-						UnschedulableReason:  unsuccessfulJctx.UnschedulableReason,
-						PodSchedulingContext: unsuccessfulJctx.PodSchedulingContext,
-					}
-					if _, err := it.schedulingContext.AddJobSchedulingContext(jctx); err != nil {
-						return nil, err
-					}
-					continue
+			schedulingKey := it.schedulingContext.SchedulingKeyFromLegacySchedulerJob(job)
+			if unsuccessfulJctx, ok := it.schedulingContext.UnfeasibleSchedulingKeys[schedulingKey]; ok {
+				jctx := &schedulercontext.JobSchedulingContext{
+					Created:              time.Now(),
+					ExecutorId:           it.schedulingContext.ExecutorId,
+					JobId:                job.GetId(),
+					Job:                  job,
+					UnschedulableReason:  unsuccessfulJctx.UnschedulableReason,
+					PodSchedulingContext: unsuccessfulJctx.PodSchedulingContext,
 				}
+				if _, err := it.schedulingContext.AddJobSchedulingContext(jctx); err != nil {
+					return nil, err
+				}
+				continue
 			}
 		}
 
diff --git a/internal/scheduler/schedulerobjects/podutils.go b/internal/scheduler/schedulerobjects/podutils.go
index 418eead4..287401d2 100644
--- a/internal/scheduler/schedulerobjects/podutils.go
+++ b/internal/scheduler/schedulerobjects/podutils.go
@@ -1,23 +1,15 @@
 package schedulerobjects
 
 import (
-	"crypto/sha1"
-	"io"
-	"strconv"
-	"strings"
+	"crypto/rand"
+	"encoding/binary"
 
-	"github.com/gogo/protobuf/jsonpb"
-	"golang.org/x/exp/maps"
+	"github.com/minio/highwayhash"
 	"golang.org/x/exp/slices"
 	v1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/resource"
-
-	armadaslices "github.com/armadaproject/armada/internal/common/slices"
 )
 
-const SchedulingKeySize = 20
-
-type SchedulingKey [SchedulingKeySize]byte
+type SchedulingKey [highwayhash.Size]byte
 
 func (req *PodRequirements) GetAffinityNodeSelector() *v1.NodeSelector {
 	affinity := req.Affinity
@@ -31,93 +23,193 @@ func (req *PodRequirements) GetAffinityNodeSelector() *v1.NodeSelector {
 	return nodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution
 }
 
-// SchedulingKey returns the canonical hash of the scheduling requirements of a job.
+// SchedulingKeyGenerator is used to generate scheduling keys efficiently.
+// A scheduling key is the canonical hash of the scheduling requirements of a job.
+// All memory is allocated up-front and re-used. Not thread-safe.
+type SchedulingKeyGenerator struct {
+	s      PodRequirementsSerialiser
+	key    []byte
+	buffer []byte
+}
+
+func NewSchedulingKeyGenerator() *SchedulingKeyGenerator {
+	key := make([]byte, 32)
+	if _, err := rand.Read(key); err != nil {
+		// This should never happen.
+		panic(err)
+	}
+	return &SchedulingKeyGenerator{
+		s:      *NewPodRequirementsSerialiser(),
+		key:    key,
+		buffer: make([]byte, 2048),
+	}
+}
+
+func (skg *SchedulingKeyGenerator) Key(
+	nodeSelector map[string]string,
+	affinity *v1.Affinity,
+	tolerations []v1.Toleration,
+	requests v1.ResourceList,
+	priority int32,
+) SchedulingKey {
+	skg.buffer = skg.buffer[0:0]
+	skg.buffer = skg.s.AppendRequirements(
+		skg.buffer,
+		nodeSelector,
+		affinity,
+		tolerations,
+		requests,
+		priority,
+	)
+	return highwayhash.Sum(skg.buffer, skg.key)
+}
+
+// PodRequirementsSerialiser produces the canonical byte representation of a set of pod scheduling requirements.
+// The resulting byte array can, e.g., be used to produce a hash guaranteed to be equal for equivalent requirements.
+// Not thread-safe.
 //
-// The hash is of size SchedulingKeySize and is guaranteed to always be the same for equivalent requirements,
-// unless Affinity is used, in which case they may differ as a result of unordered map keys.
-func (jobSchedulingInfo *JobSchedulingInfo) SchedulingKey() (SchedulingKey, bool) {
-	if jobSchedulingInfo == nil {
-		return [SchedulingKeySize]byte{}, false
+// Fields are separated by =, $, &, and =, since these characters are not allowed in taints and labels; see
+// https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
+// https://man.archlinux.org/man/community/kubectl/kubectl-taint.1.en
+type PodRequirementsSerialiser struct {
+	stringBuffer                  []string
+	byteBuffer                    []byte
+	nodeSelectorRequirementBuffer []v1.NodeSelectorRequirement
+	tolerationBuffer              []v1.Toleration
+	resourceNameBuffer            []v1.ResourceName
+}
+
+func NewPodRequirementsSerialiser() *PodRequirementsSerialiser {
+	return &PodRequirementsSerialiser{
+		stringBuffer:                  make([]string, 0, 16),
+		byteBuffer:                    make([]byte, 0, 1024),
+		nodeSelectorRequirementBuffer: make([]v1.NodeSelectorRequirement, 16),
+		tolerationBuffer:              make([]v1.Toleration, 0, 16),
+		resourceNameBuffer:            make([]v1.ResourceName, 0, 4),
 	}
-	for _, objReq := range jobSchedulingInfo.ObjectRequirements {
-		if req := objReq.GetPodRequirements(); req != nil {
-			return req.SchedulingKey(), true
+}
+
+func (skg *PodRequirementsSerialiser) AppendRequirements(
+	out []byte,
+	nodeSelector map[string]string,
+	affinity *v1.Affinity,
+	tolerations []v1.Toleration,
+	requests v1.ResourceList,
+	priority int32,
+) []byte {
+	out = skg.AppendNodeSelector(out, nodeSelector)
+	out = skg.AppendAffinity(out, affinity)
+	out = skg.AppendTolerations(out, tolerations)
+	out = skg.AppendResourceList(out, requests)
+	out = binary.LittleEndian.AppendUint32(out, uint32(priority))
+	return out
+}
+
+func (skg *PodRequirementsSerialiser) AppendNodeSelector(out []byte, nodeSelector map[string]string) []byte {
+	skg.stringBuffer = skg.stringBuffer[0:0]
+	for _, key := range nodeSelector {
+		skg.stringBuffer = append(skg.stringBuffer, key)
+	}
+	slices.Sort(skg.stringBuffer)
+	for _, key := range skg.stringBuffer {
+		value := nodeSelector[key]
+		out = append(out, []byte(key)...)
+		out = append(out, []byte("=")...)
+		out = append(out, []byte(value)...)
+		out = append(out, []byte("$")...)
+	}
+	out = append(out, []byte("&")...)
+	return out
+}
+
+// AppendAffinity writes a v1.Affinity into the hash.
+// Only NodeAffinity (i.e., not PodAffinity) and RequiredDuringSchedulingIgnoredDuringExecution fields are considered.
+func (skg *PodRequirementsSerialiser) AppendAffinity(out []byte, affinity *v1.Affinity) []byte {
+	if affinity == nil {
+		return out
+	}
+	if affinity.NodeAffinity == nil {
+		return out
+	}
+	return skg.AppendAffinityNodeSelector(out, affinity.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution)
+}
+
+func (skg *PodRequirementsSerialiser) AppendAffinityNodeSelector(out []byte, nodeSelector *v1.NodeSelector) []byte {
+	if nodeSelector == nil {
+		return out
+	}
+	// For simplicity, these are not sorted. Hence, the hash may depend on their order.
+	for _, nodeSelectorTerm := range nodeSelector.NodeSelectorTerms {
+		out = skg.AppendNodeSelectorRequirements(out, nodeSelectorTerm.MatchExpressions)
+		out = skg.AppendNodeSelectorRequirements(out, nodeSelectorTerm.MatchFields)
+	}
+	if len(nodeSelector.NodeSelectorTerms) > 0 {
+		out = append(out, []byte("&")...)
+	}
+	return out
+}
+
+func (skg *PodRequirementsSerialiser) AppendNodeSelectorRequirements(out []byte, nodeSelectorRequirements []v1.NodeSelectorRequirement) []byte {
+	skg.nodeSelectorRequirementBuffer = skg.nodeSelectorRequirementBuffer[0:0]
+	skg.nodeSelectorRequirementBuffer = append(skg.nodeSelectorRequirementBuffer, nodeSelectorRequirements...)
+	slices.SortFunc(skg.nodeSelectorRequirementBuffer, lessNodeSelectorRequirement)
+	for _, nodeSelectorRequirement := range skg.nodeSelectorRequirementBuffer {
+		out = append(out, []byte(nodeSelectorRequirement.Key)...)
+		out = append(out, []byte("=")...)
+		for _, value := range nodeSelectorRequirement.Values {
+			out = append(out, []byte(value)...)
+			out = append(out, []byte("$")...)
 		}
+		out = append(out, []byte(":")...)
+		out = append(out, []byte(nodeSelectorRequirement.Operator)...)
+		out = append(out, []byte("$")...)
+	}
+	if len(skg.nodeSelectorRequirementBuffer) > 0 {
+		out = append(out, []byte("&")...)
 	}
-	return SchedulingKey{}, false
+	return out
 }
 
-// SchedulingKey returns the canonical hash of the scheduling requirements of a pod.
-//
-// The hash is of size SchedulingKeySize and is guaranteed to always be the same for equivalent requirements,
-// unless Affinity is used, in which case they may differ as a result of unordered map keys.
-func (req *PodRequirements) SchedulingKey() SchedulingKey {
-	if req.CachedSchedulingKey == nil {
-		// Cache the key such that the next invocation returns a pre-computed key.
-		schedulingKey := schedulingKeyFromPodRequirements(req)
-		req.CachedSchedulingKey = schedulingKey[:]
-	}
-	return SchedulingKey(req.CachedSchedulingKey)
-}
-
-func schedulingKeyFromPodRequirements(req *PodRequirements) SchedulingKey {
-	// We separate taints/labels by $, labels and values by =, and and groups by &,
-	// since these characters are not allowed in taints and labels; see
-	// https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
-	// https://man.archlinux.org/man/community/kubectl/kubectl-taint.1.en
-	h := sha1.New()
-
-	nodeSelectorKeys := maps.Keys(req.NodeSelector)
-	slices.Sort(nodeSelectorKeys)
-	for _, key := range nodeSelectorKeys {
-		value := req.NodeSelector[key]
-		_, _ = io.WriteString(h, key)
-		_, _ = io.WriteString(h, "=")
-		_, _ = io.WriteString(h, value)
-		_, _ = io.WriteString(h, "$")
-	}
-	_, _ = io.WriteString(h, "&")
-
-	if req.Affinity != nil {
-		_ = (&jsonpb.Marshaler{EnumsAsInts: true, OrigName: true}).Marshal(h, req.Affinity)
-		_, _ = io.WriteString(h, "&")
-	}
-
-	tolerations := slices.Clone(req.Tolerations)
-	slices.SortFunc(tolerations, lessToleration)
-	for _, toleration := range tolerations {
-		_, _ = io.WriteString(h, toleration.Key)
-		_, _ = io.WriteString(h, "=")
-		_, _ = io.WriteString(h, toleration.Value)
-		_, _ = io.WriteString(h, ":")
-		_, _ = io.WriteString(h, string(toleration.Operator))
-		_, _ = io.WriteString(h, ":")
-		_, _ = io.WriteString(h, string(toleration.Effect))
-		_, _ = io.WriteString(h, "$")
-	}
-	_, _ = io.WriteString(h, "&")
-
-	_, _ = io.WriteString(h, strconv.Itoa(int(req.Priority)))
-	_, _ = io.WriteString(h, "&")
-
-	requestKeys := maps.Keys(req.ResourceRequirements.Requests)
-	requestKeys = armadaslices.Filter(
-		requestKeys,
-		func(key v1.ResourceName) bool {
-			q := req.ResourceRequirements.Requests[key]
-			return q.Cmp(resource.Quantity{}) != 0
-		},
-	)
-	slices.Sort(requestKeys)
-	for _, key := range requestKeys {
-		value := req.ResourceRequirements.Requests[key]
-		_, _ = io.WriteString(h, string(key))
-		_, _ = io.WriteString(h, "=")
-		_, _ = h.Write(EncodeQuantity(value))
-		_, _ = io.WriteString(h, "$")
+func (skg *PodRequirementsSerialiser) AppendTolerations(out []byte, tolerations []v1.Toleration) []byte {
+	skg.tolerationBuffer = skg.tolerationBuffer[0:0]
+	skg.tolerationBuffer = append(skg.tolerationBuffer, tolerations...)
+	slices.SortFunc(skg.tolerationBuffer, lessToleration)
+	for _, toleration := range skg.tolerationBuffer {
+		out = append(out, []byte(toleration.Key)...)
+		out = append(out, []byte("=")...)
+		out = append(out, []byte(toleration.Value)...)
+		out = append(out, []byte(":")...)
+		out = append(out, []byte(toleration.Operator)...)
+		out = append(out, []byte(":")...)
+		out = append(out, []byte((toleration.Effect))...)
+		out = append(out, []byte("$")...)
 	}
+	if len(tolerations) > 0 {
+		out = append(out, []byte("&")...)
+	}
+	return out
+}
 
-	return SchedulingKey(h.Sum(nil))
+func (skg *PodRequirementsSerialiser) AppendResourceList(out []byte, resourceList v1.ResourceList) []byte {
+	skg.resourceNameBuffer = skg.resourceNameBuffer[0:0]
+	for t, q := range resourceList {
+		if !q.IsZero() {
+			// Zero-valued requests don't affect scheduling.
+			skg.resourceNameBuffer = append(skg.resourceNameBuffer, t)
+		}
+	}
+	slices.Sort(skg.resourceNameBuffer)
+	for _, t := range skg.resourceNameBuffer {
+		q := resourceList[t]
+		out = append(out, []byte(t)...)
+		out = append(out, []byte("=")...)
+		out = binary.LittleEndian.AppendUint64(out, uint64(q.MilliValue()))
+		out = append(out, []byte("$")...)
+	}
+	if len(skg.resourceNameBuffer) > 0 {
+		out = append(out, "&"...)
+	}
+	return out
 }
 
 // ClearCachedSchedulingKey clears any cached scheduling keys.
@@ -140,24 +232,43 @@ func (req *PodRequirements) ClearCachedSchedulingKey() {
 }
 
 func lessToleration(a, b v1.Toleration) bool {
-	if cmp := strings.Compare(a.Key, b.Key); cmp == -1 {
+	if a.Key < b.Key {
+		return true
+	} else if a.Key > b.Key {
+		return false
+	}
+	if a.Value < b.Value {
+		return true
+	} else if a.Value > b.Value {
+		return false
+	}
+	if string(a.Operator) < string(b.Operator) {
 		return true
-	} else if cmp == 1 {
+	} else if string(a.Operator) > string(b.Operator) {
 		return false
 	}
-	if cmp := strings.Compare(a.Value, b.Value); cmp == -1 {
+	if string(a.Effect) < string(b.Effect) {
+		return true
+	} else if string(a.Effect) > string(b.Effect) {
+		return false
+	}
+	return true
+}
+
+func lessNodeSelectorRequirement(a, b v1.NodeSelectorRequirement) bool {
+	if a.Key < b.Key {
 		return true
-	} else if cmp == 1 {
+	} else if a.Key > b.Key {
 		return false
 	}
-	if cmp := strings.Compare(string(a.Operator), string(b.Operator)); cmp == -1 {
+	if string(a.Operator) < string(b.Operator) {
 		return true
-	} else if cmp == 1 {
+	} else if string(a.Operator) > string(b.Operator) {
 		return false
 	}
-	if cmp := strings.Compare(string(a.Effect), string(b.Effect)); cmp == -1 {
+	if len(a.Values) < len(b.Values) {
 		return true
-	} else if cmp == 1 {
+	} else if len(a.Values) > len(b.Values) {
 		return false
 	}
 	return true
diff --git a/internal/scheduler/submitcheck.go b/internal/scheduler/submitcheck.go
index 05257692..64d3924e 100644
--- a/internal/scheduler/submitcheck.go
+++ b/internal/scheduler/submitcheck.go
@@ -50,6 +50,7 @@ type SubmitChecker struct {
 	executorRepository        database.ExecutorRepository
 	clock                     clock.Clock
 	mu                        sync.Mutex
+	schedulingKeyGenerator    *schedulerobjects.SchedulingKeyGenerator
 	jobSchedulingResultsCache *lru.Cache
 }
 
@@ -73,6 +74,7 @@ func NewSubmitChecker(
 		indexedNodeLabels:         schedulingConfig.IndexedNodeLabels,
 		executorRepository:        executorRepository,
 		clock:                     clock.RealClock{},
+		schedulingKeyGenerator:    schedulerobjects.NewSchedulingKeyGenerator(),
 		jobSchedulingResultsCache: jobSchedulingResultsCache,
 	}
 }
@@ -122,15 +124,16 @@ func (srv *SubmitChecker) updateExecutors(ctx context.Context) {
 		} else {
 			log.WithError(err).Warnf("Error clearing nodedb for executor %s", executor.Id)
 		}
-
 	}
 
-	// Reset cache as the executors may have updated - changing what can be scheduled
+	// Reset cache as the executors may have updated, changing what can be scheduled.
+	// Create a new schedulingKeyGenerator to get a new initial state.
+	srv.schedulingKeyGenerator = schedulerobjects.NewSchedulingKeyGenerator()
 	srv.jobSchedulingResultsCache.Purge()
 }
 
-func (srv *SubmitChecker) CheckPodRequirements(podRequirement *schedulerobjects.PodRequirements) (bool, string) {
-	schedulingResult := srv.getSchedulingResult([]*schedulerobjects.PodRequirements{podRequirement})
+func (srv *SubmitChecker) CheckPodRequirements(req *schedulerobjects.PodRequirements) (bool, string) {
+	schedulingResult := srv.getSchedulingResult(req)
 	if !schedulingResult.isSchedulable {
 		return schedulingResult.isSchedulable, fmt.Sprintf("requirements unschedulable:\n%s", schedulingResult.reason)
 	}
@@ -140,8 +143,8 @@ func (srv *SubmitChecker) CheckPodRequirements(podRequirement *schedulerobjects.
 func (srv *SubmitChecker) CheckApiJobs(jobs []*api.Job) (bool, string) {
 	// First, check if all jobs can be scheduled individually.
 	for i, job := range jobs {
-		reqs := PodRequirementFromLegacySchedulerJob(job, srv.priorityClasses)
-		schedulingResult := srv.getSchedulingResult([]*schedulerobjects.PodRequirements{reqs})
+		req := PodRequirementFromLegacySchedulerJob(job, srv.priorityClasses)
+		schedulingResult := srv.getSchedulingResult(req)
 		if !schedulingResult.isSchedulable {
 			return schedulingResult.isSchedulable, fmt.Sprintf("%d-th job unschedulable:\n%s", i, schedulingResult.reason)
 		}
@@ -173,19 +176,25 @@ func GroupJobsByAnnotation(annotation string, jobs []*api.Job) map[string][]*api
 	return rv
 }
 
-func (srv *SubmitChecker) getSchedulingResult(reqs []*schedulerobjects.PodRequirements) schedulingResult {
-	for _, req := range reqs {
-		schedulingKey := req.SchedulingKey()
-		var result schedulingResult
-		if obj, ok := srv.jobSchedulingResultsCache.Get(schedulingKey); ok {
-			result = obj.(schedulingResult)
-		} else {
-			result = srv.check(reqs)
-			srv.jobSchedulingResultsCache.Add(schedulingKey, result)
-		}
-		if !result.isSchedulable {
-			return result
-		}
+func (srv *SubmitChecker) getSchedulingResult(req *schedulerobjects.PodRequirements) schedulingResult {
+	srv.mu.Lock()
+	schedulingKey := srv.schedulingKeyGenerator.Key(
+		req.NodeSelector,
+		req.Affinity,
+		req.Tolerations,
+		req.ResourceRequirements.Requests,
+		req.Priority,
+	)
+	srv.mu.Unlock()
+	var result schedulingResult
+	if obj, ok := srv.jobSchedulingResultsCache.Get(schedulingKey); ok {
+		result = obj.(schedulingResult)
+	} else {
+		result = srv.check([]*schedulerobjects.PodRequirements{req})
+		srv.jobSchedulingResultsCache.Add(schedulingKey, result)
+	}
+	if !result.isSchedulable {
+		return result
 	}
 	return schedulingResult{isSchedulable: true}
 }
diff --git a/internal/scheduleringester/instructions.go b/internal/scheduleringester/instructions.go
index 2166c842..3ff86553 100644
--- a/internal/scheduleringester/instructions.go
+++ b/internal/scheduleringester/instructions.go
@@ -381,9 +381,5 @@ func (c *InstructionConverter) schedulingInfoFromSubmitJob(submitJob *armadaeven
 	default:
 		return nil, errors.Errorf("unsupported object type %T", object)
 	}
-
-	// Call SchedulingKey() to trigger computing and caching the key.
-	_, _ = schedulingInfo.SchedulingKey()
-
 	return schedulingInfo, nil
 }
diff --git a/pkg/api/util.go b/pkg/api/util.go
index 98d37061..45e047bf 100644
--- a/pkg/api/util.go
+++ b/pkg/api/util.go
@@ -109,15 +109,6 @@ func NewNodeTypeFromNodeInfo(nodeInfo *NodeInfo, indexedTaints map[string]interf
 func (job *Job) GetRequirements(priorityClasses map[string]configuration.PriorityClass) *schedulerobjects.JobSchedulingInfo {
 	podSpec := job.GetMainPodSpec()
 
-	// Use pre-computed schedulingResourceRequirements if available.
-	// Otherwise compute it from the containers in podSpec.
-	var schedulingResourceRequirements v1.ResourceRequirements
-	if len(job.SchedulingResourceRequirements.Requests) > 0 || len(job.SchedulingResourceRequirements.Limits) > 0 {
-		schedulingResourceRequirements = job.SchedulingResourceRequirements
-	} else {
-		schedulingResourceRequirements = SchedulingResourceRequirementsFromPodSpec(podSpec)
-	}
-
 	priority, ok := PriorityFromPodSpec(podSpec, priorityClasses)
 	if priorityClasses != nil && !ok {
 		// Ignore this error if priorityByPriorityClassName is explicitly set to nil.
@@ -137,7 +128,7 @@ func (job *Job) GetRequirements(priorityClasses map[string]configuration.Priorit
 		Annotations:          maps.Clone(job.Annotations),
 		Priority:             priority,
 		PreemptionPolicy:     preemptionPolicy,
-		ResourceRequirements: schedulingResourceRequirements,
+		ResourceRequirements: job.GetResourceRequirements(),
 	}
 	return &schedulerobjects.JobSchedulingInfo{
 		PriorityClassName: podSpec.PriorityClassName,
@@ -228,6 +219,32 @@ func (job *Job) GetPriorityClassName() string {
 	return podSpec.PriorityClassName
 }
 
+func (job *Job) GetNodeSelector() map[string]string {
+	podSpec := job.GetMainPodSpec()
+	return podSpec.NodeSelector
+}
+
+func (job *Job) GetAffinity() *v1.Affinity {
+	podSpec := job.GetMainPodSpec()
+	return podSpec.Affinity
+}
+
+func (job *Job) GetTolerations() []v1.Toleration {
+	podSpec := job.GetMainPodSpec()
+	return podSpec.Tolerations
+}
+
+func (job *Job) GetResourceRequirements() v1.ResourceRequirements {
+	// Use pre-computed schedulingResourceRequirements if available.
+	// Otherwise compute it from the containers in podSpec.
+	podSpec := job.GetMainPodSpec()
+	if len(job.SchedulingResourceRequirements.Requests) > 0 || len(job.SchedulingResourceRequirements.Limits) > 0 {
+		return job.SchedulingResourceRequirements
+	} else {
+		return SchedulingResourceRequirementsFromPodSpec(podSpec)
+	}
+}
+
 func (job *Job) GetJobSet() string {
 	return job.JobSetId
 }


Test patch:
diff --git a/internal/scheduler/reports_test.go b/internal/scheduler/reports_test.go
index 083fdec5..6b2f2f4d 100644
--- a/internal/scheduler/reports_test.go
+++ b/internal/scheduler/reports_test.go
@@ -302,5 +302,6 @@ func testSchedulingContext(executorId string) *schedulercontext.SchedulingContex
 	)
 	sctx.Started = time.Time{}
 	sctx.Finished = time.Time{}
+	sctx.SchedulingKeyGenerator = nil
 	return sctx
 }
diff --git a/internal/scheduler/schedulerobjects/podutils_test.go b/internal/scheduler/schedulerobjects/podutils_test.go
index 043a84f9..39581a96 100644
--- a/internal/scheduler/schedulerobjects/podutils_test.go
+++ b/internal/scheduler/schedulerobjects/podutils_test.go
@@ -10,6 +10,128 @@ import (
 	"k8s.io/utils/pointer"
 )
 
+// TestPodRequirementsSerialiser_AffinityDocsUnchanged fails if the docs for the parts of v1.Affinity
+// we use for scheduling change, so that we are warned if its definition changes and need to update PodRequirementsSerialiser.
+func TestPodRequirementsSerialiser_AffinityDocsUnchanged(t *testing.T) {
+	affinity := &v1.Affinity{
+		NodeAffinity: &v1.NodeAffinity{
+			RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+				NodeSelectorTerms: []v1.NodeSelectorTerm{
+					{
+						MatchExpressions: []v1.NodeSelectorRequirement{
+							{
+								Key:      "k1",
+								Operator: "o1",
+								Values:   []string{"v1", "v2"},
+							},
+						},
+						MatchFields: []v1.NodeSelectorRequirement{
+							{
+								Key:      "k2",
+								Operator: "o2",
+								Values:   []string{"v10", "v20"},
+							},
+						},
+					},
+				},
+			},
+		},
+		PodAffinity: &v1.PodAffinity{
+			RequiredDuringSchedulingIgnoredDuringExecution: []v1.PodAffinityTerm{
+				{
+					LabelSelector: &metav1.LabelSelector{
+						MatchLabels: map[string]string{
+							"label1": "labelval1",
+							"label2": "labelval2",
+							"label3": "labelval3",
+						},
+						MatchExpressions: []metav1.LabelSelectorRequirement{
+							{
+								Key:      "k1",
+								Operator: "o1",
+								Values:   []string{"v1", "v2", "v3"},
+							},
+						},
+					},
+					Namespaces:  []string{"n1, n2, n3"},
+					TopologyKey: "topkey1",
+					NamespaceSelector: &metav1.LabelSelector{
+						MatchLabels: map[string]string{
+							"label10": "labelval1",
+							"label20": "labelval2",
+							"label30": "labelval3",
+						},
+						MatchExpressions: []metav1.LabelSelectorRequirement{
+							{
+								Key:      "k10",
+								Operator: "o10",
+								Values:   []string{"v10", "v20", "v30"},
+							},
+						},
+					},
+				},
+			},
+		},
+		PodAntiAffinity: nil,
+	}
+	assert.Equal(
+		t,
+		map[string]string{
+			"":                "Affinity is a group of affinity scheduling rules.",
+			"nodeAffinity":    "Describes node affinity scheduling rules for the pod.",
+			"podAffinity":     "Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).",
+			"podAntiAffinity": "Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).",
+		},
+		affinity.SwaggerDoc(),
+	)
+	assert.Equal(
+		t,
+		map[string]string{
+			"": "Node affinity is a group of node affinity scheduling rules.",
+			"requiredDuringSchedulingIgnoredDuringExecution":  "If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.",
+			"preferredDuringSchedulingIgnoredDuringExecution": "The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \"weight\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.",
+		},
+		affinity.NodeAffinity.SwaggerDoc(),
+	)
+	assert.Equal(
+		t,
+		map[string]string{
+			"":                  "A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.",
+			"nodeSelectorTerms": "Required. A list of node selector terms. The terms are ORed.",
+		},
+		affinity.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.SwaggerDoc(),
+	)
+	assert.Equal(
+		t,
+		map[string]string{
+			"":                 "A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.",
+			"matchExpressions": "A list of node selector requirements by node's labels.",
+			"matchFields":      "A list of node selector requirements by node's fields.",
+		},
+		affinity.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms[0].SwaggerDoc(),
+	)
+	assert.Equal(
+		t,
+		map[string]string{
+			"":         "A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.",
+			"key":      "The label key that the selector applies to.",
+			"operator": "Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.",
+			"values":   "An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.",
+		},
+		affinity.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms[0].MatchExpressions[0].SwaggerDoc(),
+	)
+	assert.Equal(
+		t,
+		map[string]string{
+			"":         "A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.",
+			"key":      "The label key that the selector applies to.",
+			"operator": "Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.",
+			"values":   "An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.",
+		},
+		affinity.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms[0].MatchFields[0].SwaggerDoc(),
+	)
+}
+
 func TestSchedulingKey(t *testing.T) {
 	defaultN := 10 // Run the check several times to check for consistency.
 	tests := map[string]struct {
@@ -642,27 +764,8 @@ func TestSchedulingKey(t *testing.T) {
 				},
 			},
 		},
-		"affinity MatchLabels": {
+		"affinity PodAffinity ignored": {
 			a: &PodRequirements{
-				NodeSelector: map[string]string{
-					"property1": "value1",
-					"property3": "value3",
-				},
-				Tolerations: []v1.Toleration{{
-					Key:               "a",
-					Operator:          "b",
-					Value:             "b",
-					Effect:            "d",
-					TolerationSeconds: pointer.Int64(1),
-				}},
-				Priority: 1,
-				ResourceRequirements: v1.ResourceRequirements{
-					Requests: map[v1.ResourceName]resource.Quantity{
-						"cpu":    resource.MustParse("4"),
-						"memory": resource.MustParse("5"),
-						"gpu":    resource.MustParse("6"),
-					},
-				},
 				Affinity: &v1.Affinity{
 					NodeAffinity: &v1.NodeAffinity{
 						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
@@ -726,25 +829,6 @@ func TestSchedulingKey(t *testing.T) {
 				},
 			},
 			b: &PodRequirements{
-				NodeSelector: map[string]string{
-					"property3": "value3",
-					"property1": "value1",
-				},
-				Tolerations: []v1.Toleration{{
-					Key:               "a",
-					Operator:          "b",
-					Value:             "b",
-					Effect:            "d",
-					TolerationSeconds: pointer.Int64(1),
-				}},
-				Priority: 1,
-				ResourceRequirements: v1.ResourceRequirements{
-					Requests: map[v1.ResourceName]resource.Quantity{
-						"cpu":    resource.MustParse("4"),
-						"memory": resource.MustParse("5"),
-						"gpu":    resource.MustParse("6"),
-					},
-				},
 				Affinity: &v1.Affinity{
 					NodeAffinity: &v1.NodeAffinity{
 						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
@@ -807,12 +891,197 @@ func TestSchedulingKey(t *testing.T) {
 					PodAntiAffinity: nil,
 				},
 			},
+			equal: true,
+		},
+		"affinity NodeAffinity MatchExpressions": {
+			a: &PodRequirements{
+				Affinity: &v1.Affinity{
+					NodeAffinity: &v1.NodeAffinity{
+						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+							NodeSelectorTerms: []v1.NodeSelectorTerm{
+								{
+									MatchExpressions: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k1",
+											Operator: "o1",
+											Values:   []string{"v1", "v2"},
+										},
+									},
+									MatchFields: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k2",
+											Operator: "o2",
+											Values:   []string{"v10", "v20"},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			b: &PodRequirements{
+				Affinity: &v1.Affinity{
+					NodeAffinity: &v1.NodeAffinity{
+						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+							NodeSelectorTerms: []v1.NodeSelectorTerm{
+								{
+									MatchExpressions: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k1",
+											Operator: "o1",
+											Values:   []string{"v1", "v3"},
+										},
+									},
+									MatchFields: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k2",
+											Operator: "o2",
+											Values:   []string{"v10", "v20"},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			equal: false,
+		},
+		"affinity NodeAffinity MatchFields": {
+			a: &PodRequirements{
+				Affinity: &v1.Affinity{
+					NodeAffinity: &v1.NodeAffinity{
+						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+							NodeSelectorTerms: []v1.NodeSelectorTerm{
+								{
+									MatchExpressions: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k1",
+											Operator: "o1",
+											Values:   []string{"v1", "v2"},
+										},
+									},
+									MatchFields: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k2",
+											Operator: "o2",
+											Values:   []string{"v10", "v20"},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			b: &PodRequirements{
+				Affinity: &v1.Affinity{
+					NodeAffinity: &v1.NodeAffinity{
+						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+							NodeSelectorTerms: []v1.NodeSelectorTerm{
+								{
+									MatchExpressions: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k1",
+											Operator: "o1",
+											Values:   []string{"v1", "v2"},
+										},
+									},
+									MatchFields: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k2",
+											Operator: "o2",
+											Values:   []string{"v10", "v21"},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			equal: false,
+		},
+		"affinity NodeAffinity multiple MatchFields": {
+			a: &PodRequirements{
+				Affinity: &v1.Affinity{
+					NodeAffinity: &v1.NodeAffinity{
+						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+							NodeSelectorTerms: []v1.NodeSelectorTerm{
+								{
+									MatchExpressions: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k1",
+											Operator: "o1",
+											Values:   []string{"v1", "v2"},
+										},
+									},
+									MatchFields: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k2",
+											Operator: "o2",
+											Values:   []string{"v10", "v20"},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			b: &PodRequirements{
+				Affinity: &v1.Affinity{
+					NodeAffinity: &v1.NodeAffinity{
+						RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{
+							NodeSelectorTerms: []v1.NodeSelectorTerm{
+								{
+									MatchExpressions: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k1",
+											Operator: "o1",
+											Values:   []string{"v1", "v2"},
+										},
+									},
+									MatchFields: []v1.NodeSelectorRequirement{
+										{
+											Key:      "k2",
+											Operator: "o2",
+											Values:   []string{"v10", "v20"},
+										},
+										{
+											Key:      "k3",
+											Operator: "o2",
+											Values:   []string{"v10", "v20"},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			equal: false,
 		},
 	}
 	for name, tc := range tests {
 		t.Run(name, func(t *testing.T) {
-			schedulingKeyA := tc.a.SchedulingKey()
-			schedulingKeyB := tc.b.SchedulingKey()
+			skg := NewSchedulingKeyGenerator()
+			schedulingKeyA := skg.Key(
+				tc.a.NodeSelector,
+				tc.a.Affinity,
+				tc.a.Tolerations,
+				tc.a.ResourceRequirements.Requests,
+				tc.a.Priority,
+			)
+			schedulingKeyB := skg.Key(
+				tc.b.NodeSelector,
+				tc.b.Affinity,
+				tc.b.Tolerations,
+				tc.b.ResourceRequirements.Requests,
+				tc.b.Priority,
+			)
+
 			var prevSchedulingKeyA SchedulingKey
 			var prevSchedulingKeyB SchedulingKey
 			n := defaultN
@@ -820,8 +1089,6 @@ func TestSchedulingKey(t *testing.T) {
 				n = tc.n
 			}
 			for i := 0; i < n; i++ {
-				tc.a.ClearCachedSchedulingKey()
-				tc.b.ClearCachedSchedulingKey()
 				if tc.equal {
 					assert.Equal(t, schedulingKeyA, schedulingKeyB)
 				} else {
@@ -838,8 +1105,57 @@ func TestSchedulingKey(t *testing.T) {
 	}
 }
 
+func benchmarPodRequirementsSerialiser(b *testing.B, jobSchedulingInfo *JobSchedulingInfo) {
+	skg := NewPodRequirementsSerialiser()
+	req := (jobSchedulingInfo.ObjectRequirements[0]).GetPodRequirements()
+	out := make([]byte, 0, 1024)
+	b.ResetTimer()
+	for n := 0; n < b.N; n++ {
+		out = out[0:0]
+		out = skg.AppendRequirements(
+			out,
+			req.NodeSelector,
+			req.Affinity,
+			req.Tolerations,
+			req.ResourceRequirements.Requests,
+			req.Priority,
+		)
+	}
+}
+
+func BenchmarkPodRequirementsSerialiser(b *testing.B) {
+	benchmarPodRequirementsSerialiser(b, getBenchmarkJobSchedulingSchedulingInfo())
+}
+
+func BenchmarkPodRequirementsSerialiser_Affinity(b *testing.B) {
+	benchmarPodRequirementsSerialiser(b, getBenchmarkJobSchedulingSchedulingInfoWithAffinity())
+}
+
+func benchmarkSchedulingKey(b *testing.B, jobSchedulingInfo *JobSchedulingInfo) {
+	skg := NewSchedulingKeyGenerator()
+	req := (jobSchedulingInfo.ObjectRequirements[0]).GetPodRequirements()
+	b.ResetTimer()
+	for n := 0; n < b.N; n++ {
+		skg.Key(
+			req.NodeSelector,
+			req.Affinity,
+			req.Tolerations,
+			req.ResourceRequirements.Requests,
+			req.Priority,
+		)
+	}
+}
+
 func BenchmarkSchedulingKey(b *testing.B) {
-	jobSchedulingInfo := &JobSchedulingInfo{
+	benchmarkSchedulingKey(b, getBenchmarkJobSchedulingSchedulingInfo())
+}
+
+func BenchmarkSchedulingKey_Affinity(b *testing.B) {
+	benchmarkSchedulingKey(b, getBenchmarkJobSchedulingSchedulingInfoWithAffinity())
+}
+
+func getBenchmarkJobSchedulingSchedulingInfo() *JobSchedulingInfo {
+	return &JobSchedulingInfo{
 		Lifetime:          1,
 		AtMostOnce:        true,
 		Preemptible:       true,
@@ -885,15 +1201,10 @@ func BenchmarkSchedulingKey(b *testing.B) {
 			},
 		},
 	}
-	b.ResetTimer()
-	for n := 0; n < b.N; n++ {
-		jobSchedulingInfo.ClearCachedSchedulingKey()
-		jobSchedulingInfo.SchedulingKey()
-	}
 }
 
-func BenchmarkSchedulingKey_Affinity(b *testing.B) {
-	jobSchedulingInfo := &JobSchedulingInfo{
+func getBenchmarkJobSchedulingSchedulingInfoWithAffinity() *JobSchedulingInfo {
+	return &JobSchedulingInfo{
 		Lifetime:          1,
 		AtMostOnce:        true,
 		Preemptible:       true,
@@ -1000,9 +1311,4 @@ func BenchmarkSchedulingKey_Affinity(b *testing.B) {
 			},
 		},
 	}
-	b.ResetTimer()
-	for n := 0; n < b.N; n++ {
-		jobSchedulingInfo.ClearCachedSchedulingKey()
-		jobSchedulingInfo.SchedulingKey()
-	}
 }
diff --git a/internal/scheduleringester/instructions_test.go b/internal/scheduleringester/instructions_test.go
index fc95ec09..45debe5c 100644
--- a/internal/scheduleringester/instructions_test.go
+++ b/internal/scheduleringester/instructions_test.go
@@ -300,6 +300,5 @@ func getExpectedSubmitMessageSchedulingInfo(t *testing.T) *schedulerobjects.JobS
 			},
 		},
 	}
-	_, _ = expectedSubmitSchedulingInfo.SchedulingKey()
 	return expectedSubmitSchedulingInfo
 }