Allow removing the config

Signed-off-by: Bao Haojun <baohaojun@gmail.com>