Ensure that the init layer is removed with the container

Fixes #2951