This won't work in the way that we're using the permissions/voters. We grant EDIT access to the admin for a user, so that he is allowed to edit. And in addition the permissions per object are checked with custom voters. So we use role based security with additional voters to deny edit access to some objects.

With your solution, the edit links will still be shown (because of the "global_permission") but the user may not be able to edit the object.