Make sure excluded fields are not a part of only

Fixes the scenario in which a field is a part of `only` and `readonly_fields`