Fix a bug of removing access