This is an expected behavior and will neither cause any functional failure nor reliability concern to the device if the power-up sequence and power-down sequence are followed.
This behavior can only be observed if the regulators were designed to leave the power supplies to float before they ramp up and after they ramp down.