It could be a convention to silently ignore it, but I think in most cases this would indicate a bug in the application.

`ol.control.Control` has a `getMap` method, so users can always do
```javascript
// remove control if has not already been removed
if (control.getMap()) {
  map.removeControl(control);
}
```
I'd prefer this to the option of being silent when a user mixes up which control is on which map.