> I'd probably add an goog.asserts.fail() as well though, as this really should not happen.

Thinking about it, are you sure that makes sense? It could be convenient that removeControl does nothing without complaining when it's passed a control that's not in the map, no?