Code change:
// from the registered device plugins. func (m *ManagerImpl) Allocate(node *schedulercache.NodeInfo, attrs *lifecycle.PodAdmitAttributes) error { pod := attrs.Pod <ins> devicesToReuse := make(map[string]sets.String) </ins> // TODO: Reuse devices between init containers and regular containers. for _, container := range pod.Spec.InitContainers { <del> if err := m.allocateContainerResources(pod, &container); err != nil { </del> <ins> if err := m.allocateContainerResources(pod, &container, devicesToReuse); err != nil { </ins> return err } <ins> m.podDevices.addContainerAllocatedResources(string(pod.UID), container.Name, devicesToReuse) </ins> } for _, container := range pod.Spec.Containers { <del> if err := m.allocateContainerResources(pod, &container); err != nil { </del> <ins> if err := m.allocateContainerResources(pod, &container, devicesToReuse); err != nil { </ins> return err } <ins> m.podDevices.removeContainerAllocatedResources(string(pod.UID), container.Name, devicesToReuse) </ins> } m.mutex.Lock()

Related discussion:
text: Try to allocate from this list first for later init containers and normal containers. The required no. of devices we pass to ManagerImpl.devicesToAllocate() should subtract the no. of devices we have allocated to init containers. This may require some code structure change around ManagerImpl.Allocate(), ManagerImpl.allocateContainerResources(), and ManagerImpl.devicesToAllocate(). However, Allocate gRPC call is still issued during Admission all together, and it is possible that we issue an Allocate gRPC for [dev1] (say dev1 allocated to initContianer1), followed by another Allocate gRPC call for [dev1, dev2] (say dev1 and dev2 allocated to initContainer2), with none of containers started or finished. This assumes containers running in the same pod share the same security context, and device plugin doesn't need to perform per-container allocation operations. We just use Allocate gRPC call to learn container runtime settings. 3) Take a further step from 2), during Admission, we just determine what devices to allocate to containers without issuing the actual Allocate gRPC calls. I.e., we only fill deviceIds field in podDevices.deviceAllocateInfo but leaves allocResp field empty. During ManagerImpl.GetDeviceRunContainerOptions(), we check for this case and issue the actual Allocate gRPC call for the container. This ensures we can issue a gRPC Allocate call per container. However, if a pod is restarted multiple times, we don't guarantee Allocate gRPC call will be issued during subsequent pod runs. 4) Take a further step from 3), we can check for overlapping device allocations in ManagerImpl.GetDeviceRunContainerOptions() and invalidate their allocResp so that an Allocate gRPC call will be issued next time for containers using re-allocated devices. E.g., for containerA, if we find it is allocated with [dev1, dev2] and dev1 is also allocated to containerB, we set containerB's allocResp to nil so that next time before containerB is running, we will issue an Allocate gRPC call for dev1. Note that this approach does have the drawback that device plugin failure would cause previously running pods not be able to restart, so we need to carefully evaluate our use cases to see whether we need to implement this. My suggestion is to implement 1) now and gather some user feedbacks to determine whether we need to implement the following approaches.