Code change:
} // Write stores the given "val" in CycleState with the given "key". <del> // This function is not thread safe. In multi-threaded code, lock should be // acquired first. </del> <ins> // This function is thread safe by acquiring an internal lock first. </ins> func (c *CycleState) Write(key StateKey, val StateData) { <ins> c.mx.Lock() </ins> c.storage[key] = val <ins> c.mx.Unlock() </ins> } // Delete deletes data with the given key from CycleState. <del> // This function is not thread safe. In multi-threaded code, lock should be // acquired first. </del> <ins> // This function is thread safe by acquiring an internal lock first. </ins> func (c *CycleState) Delete(key StateKey) { <del> delete(c.storage, key) } // Lock acquires CycleState lock. func (c *CycleState) Lock() { </del> c.mx.Lock() <del> } // Unlock releases CycleState lock. func (c *CycleState) Unlock() { </del> <ins> delete(c.storage, key) </ins> c.mx.Unlock() } <del> // RLock acquires CycleState read lock. func (c *CycleState) RLock() { c.mx.RLock() } // RUnlock releases CycleState read lock. func (c *CycleState) RUnlock() { c.mx.RUnlock() } </del>

Related discussion:
text: What happened: kube-scheduler () crashes due to a panic while running : The race condition is triggered when an out-of-tree plugin, that writes to during its Filter phase, is registered through . Access to in general happens from a single goroutine and there are no practical concurrency issues, with the exception of the phase, that parallelizes calls for each node across plugins: A quick glance at the tip of the branch seems to indicate the bug is still present there. What you expected to happen: Ideally access to should be synchronized everywhere for correctness, including by all in-tree plugins, since there are no guarantees about how will call multiple plugins concurrently. It seems to me it is something that was considered at some point, given that has methods. Sadly, almost no in-tree plugins are synchronizing access. For now, I will try to work around the problem by doing a single write to during and storing a data bag structure where I will write state into during (with proper synchronization), instead of writing to directly. How to reproduce it (as minimally and precisely as possible): Install a scheduler plugin that writes to during its phase. In-tree (builtin) plugins are not synchronizing their access to . Environment: Kubernetes version (use ): v1.18.8 Cloud provider or hardware configuration: linux-amd64 (AWS) OS (e.g: ): Ubuntu 18.04.5 LTS (Bionic Beaver) Others: custom kube-scheduler binary built with an out-of-tree plugin using
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
AFAICT all in-tree plugins write to the cycle state in PreFilter and then only read from cycle state in Filter. So this shouldn't be a problem with in-tree plugins as concurrent read access should be fine during RunFilterPlugins. But since your out-of-tree plugin writes in Filter, I can see why this can be a problem.