It is suggested to run the A/B test for your app for not more than one to two weeks. This is the best duration where you can track users’ activity on your main different and modified versions and figure out which one is meeting their needs in better ways.
You can further implement the same strategy for some other functions to check if these are working fine as expected.