Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: during phase from the plugin when the is updated: What you expected to happen: Not crash. How to reproduce it (as minimally and precisely as possible): I am suspecting that the issue is in the naive implementation of the 's method: Most likely the is cloned during filter phase, e.g. in the called during in the : ... and when is cloned during filtering, new mutex is created, but is not cloned properly, this mean that potentially there are two or more different mutexes protecting the same map from writes. Anything else we need to know?: I think potential solution is to have mutex in the or change the 's . Environment: Kubernetes version (use ): 1.19.0 Cloud provider or hardware configuration: not relevant OS (e.g: ): not relevant Kernel (e.g. ): not relevant Install tools: not relevant Network plugin and version (if this is a network-related bug): not relevant Others: not relevant /sig scheduling
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
It looks like relevant code is from and mutex per was considered intially. CC
IIUC, the may (edit: if the node has nominated pods) be duplicated in for every node before passing to extension point. So we cann't rely on 's lock to prevent the state from being modified concurrently. Am I right?
I may have given an incorrect analysis as VolumeBinding's plugin-specific state are always using the same pointer, sorry for the confusion. I removed the confusing comments. It's actually different from what was discussed in . Your and 's analysis make more sense to me. For nodes that have nominated nodes (cloned CycleState) or not (original CycleState), the original mutex may not be able to protect the CycleState: I will take another look tomorrow.
can you please post the actual core dump, the description in the original comment doesn't seem complete. If the "concurrent map" fatal error is indeed happening at , then it means we are operating on two different CycleStates instances at the same time, which may happen when adding nominated pods () Using the CycleState lock to synchronize access to the plugin's state in the map is not proper, the plugin state itself should have its private lock, this is certainly a bug.
/cc

Discussion:
"context" "errors" "fmt" <ins> "sync" </ins> "time" v1 "k8s.io/api/core/v1"

// phase for each node // it's initialized in the PreFilter phase podVolumesByNode map[string]*scheduling.PodVolumes <ins> sync.Mutex </ins> } func (d *stateData) Clone() framework.StateData {

return status } <del> cs.Lock() </del> <ins> // multiple goroutines call `Filter` on different nodes simultaneously and the `CycleState` may be duplicated, so we must use a local lock here state.Lock() </ins> state.podVolumesByNode[node.Name] = podVolumes <del> cs.Unlock() </del> <ins> state.Unlock() </ins> return nil }