I was examining the wrong message
None of that helped. Even after attempting to stop the container separately and handling the errors there, I still get INTERNAL_SERVER_ERRORS when attempting to remove those.