Fix some logic surrounding when to move branches

Change-Id: I083f0156640a9de7df6252d4806ca914a14fdff9