More cleanup in KV after remove