Only change category restriction for a function when it was deny