Fix failure of peer removal after adding a peer set to another peer set. Fix a possible db use after close bug.