One option is to build a "reference count" collection to hold the spaces.  This would allow the spaces to be managed as the actions are added/removed from the ExecutableList.  We could safely handle "removing" spaces when an action is removed because we would keep track of the overall number of referenced added and only physically remove them when the reference count got to zero.  I think in most cases collecting the spaces as actions are added, and "removing" then as an action is removed is (in most cases, I think) more performant.

I don't know of any available reference count collections, but we could encapsulate this behind ExecutableList using a Map<String,Integer>