Don't update object when deleting access points

Fixes #5946