Reverse partial changes to N-related matrices