Support deleting keys from web ui