Properly erase/remove an observer