Code change:
// from the registered device plugins. func (m *ManagerImpl) Allocate(node *schedulercache.NodeInfo, attrs *lifecycle.PodAdmitAttributes) error { pod := attrs.Pod <ins> devicesToReuse := make(map[string]sets.String) </ins> // TODO: Reuse devices between init containers and regular containers. for _, container := range pod.Spec.InitContainers { <del> if err := m.allocateContainerResources(pod, &container); err != nil { </del> <ins> if err := m.allocateContainerResources(pod, &container, devicesToReuse); err != nil { </ins> return err } <ins> m.podDevices.addContainerAllocatedResources(string(pod.UID), container.Name, devicesToReuse) </ins> } for _, container := range pod.Spec.Containers { <del> if err := m.allocateContainerResources(pod, &container); err != nil { </del> <ins> if err := m.allocateContainerResources(pod, &container, devicesToReuse); err != nil { </ins> return err } <ins> m.podDevices.removeContainerAllocatedResources(string(pod.UID), container.Name, devicesToReuse) </ins> } m.mutex.Lock()

Related discussion:
text: The reason is because in the current device plugin implementation, we don't have a good way to free devices used by init containers of an active pod. It is hard to do this kind of validation at API level or even scheduler because we can't know whether an extended resource is device resource or not. So for now, I think we should probably just return a clear error message during admission handler. I am not sure yet whether there will be strong use cases that need to share devices between init containers and normal containers. We can revisit this based on user feedback. If there is strong user demand to support this, something we could do is to differentiate init containers and normal containers in podDevices, and free up devices used by init containers when any normal container is about to run. That would request some code structure change, and we may wait to see whether there is strong demand to support this.
It is hard to do this at API level or even scheduler because we can't know whether an extended resource is device resource or not. Oh, you are right, I ignored the resource awareness issue. I also agree that it's hard to address the issue without big change to current implementation. But I still feel the inconsistency of behavior between API and kubelet might confusing users. However, its effect should be sustainable, since we haven't received feedback about init containers up to now.
Right. So for now, perhaps just return an error when this happens, to give clear error signals.
Oh, I'm afraid we miss something, the check may not work as expected. As I commented, because init-containers run sequentially, we did reuse between init containers in , that is, if we have two init containers, each request 1 device, then we expect 1 device in total in scheduler, but expect 2 in deviceplugin. The check could not cover the scenario.
Thinking about this problem a bit more, I wonder whether we may tackle this issue from the following approaches, ordered by their complexities: 1) Just return an error whenever an init container requests a device plugin resource, and document in the device plugin user guide that requiring device plugin resource in init containers are not supported. This may be the short-term change we can try to get into 1.9. 2) Maintain a list of devices allocated to init containers in ManagerImpl.Allocate().