No-op Control#remove if not on a map (fixes #2877)