Repository: git-town/git-town

Issue:
fix flaky sync behavior, demonstrated by flaky end-to-end tests
Now that production code is doing more conditional rebasing, it's behavior becomes non-deterministic due to Git's coarse timestamps (with 1 second resolution).

The problem occurs when Git Town syncs a branch stack whose oldest branch was shipped using the "rebase" strategy. This now re-creates commits over several stages, including additional rebasing to remove the shipped commits from the branches.

If all these rebases happen in the same second, the rewritten commits end up with the same SHA. If different rebases happen in different seconds, the rewritten commits end up with different SHAs. As a result, Git Town sometimes executes additional rebase operations (because it perceives differing commits in ancestor branches) and sometimes it doesn't.

This flakiness happens especially under load, i.e. on CI, but also locally.

This is also not a nice user experience, because Git Town sometimes rebases parent branches and sometimes it doesn't do that, which will trip up users who pay attention to these things.

Discussion:
@kevgo: Example of the problem: when running `features/sync/stack/rebase_sync_strategy/branches_gone/ancestor_shipped_and_main_updated.feature`, it sometimes executes these additional commands.

```diff
  | BRANCH   | COMMAND                                                    |
  | branch-4 | git fetch --prune --tags                                   |
  |          | git checkout main                                          |
  | main     | git -c rebase.updateRefs=false rebase origin/main          |
  |          | git push                                                   |
  |          | git checkout branch-2                                      |
  | branch-2 | git -c rebase.updateRefs=false rebase --onto main branch-1 |
  |          | git checkout branch-3                                      |
  | branch-3 | git pull                                                   |
  |          | git -c rebase.updateRefs=false rebase --onto main branch-2 |
  |          | git push --force-with-lease                                |
  |          | git checkout branch-4                                      |
  | branch-4 | git pull                                                   |
  |          | git -c rebase.updateRefs=false rebase --onto main branch-2 |
  |          | git push --force-with-lease                                |
+ |          | git -c rebase.updateRefs=false rebase branch-3             |
+ |          | git push --force-with-lease --force-if-includes            |
  |          | git branch -D branch-1                                     |
  |          | git branch -D branch-2                                     |
```
@kevgo: A possible solution would be to always execute these additional commands, whether they are needed or not. It would make sense: it's intuitive that `branch-4` should always pull in updates from its parent `branch-3`.@kevgo: Or at least always do that if there are updates to the perennial branch at the root of the stack.@kevgo: Or, `branch-4` should always perform `git -c rebase.updateRefs=false rebase --onto main branch-2` and `git -c rebase.updateRefs=false rebase branch-3` together. This also avoids the double-force-push. In other words, instead of this 

```cucumber
| branch-4 | git pull                                                   |
|          | git -c rebase.updateRefs=false rebase --onto main branch-2 |
|          | git push --force-with-lease                                |
|          | git -c rebase.updateRefs=false rebase branch-3             |
|          | git push --force-with-lease --force-if-includes            |
```

It should run this:

```cucumber
| branch-4 | git pull                                                   |
|          | git -c rebase.updateRefs=false rebase --onto main branch-2 |
|          | git -c rebase.updateRefs=false rebase branch-3             |
|          | git push --force-with-lease --force-if-includes            |
```
@kevgo: This is still happening on CI:

![Image](https://github.com/user-attachments/assets/6a97ec81-605e-48a9-a8a4-d03d17ce4532)@tranhl: @kevgo If coarse-grained commit dates are leading to unpredictable commit re-hashing, could setting the `GIT_AUTHOR_DATE` & `GIT_COMMITTER_DATE` environment variables help here? If we use the same date for both the author date & committer date across the entire Git Town command, theoretically the hash shouldn't change 🤔

Link to docs: https://git-scm.com/docs/git-commit#_commit_information@kevgo: Thanks for the pointer, I wasn't aware that this is possible!

Solution patch:
diff --git a/internal/subshell/doc.go b/internal/subshell/doc.go
index 110c44bad..9c90f8b62 100644
--- a/internal/subshell/doc.go
+++ b/internal/subshell/doc.go
@@ -8,3 +8,5 @@ const concurrentGitRetries = 5
 
 // the amount of time Git Town should wait between retries when there is another Git process running
 const concurrentGitRetryDelay = 1 * time.Second
+
+const TestToken = "GIT_TOWN_TEST"
diff --git a/internal/vm/opcodes/rebase_onto_keep_deleted.go b/internal/vm/opcodes/rebase_onto_keep_deleted.go
index 699a518b9..1c7737fee 100644
--- a/internal/vm/opcodes/rebase_onto_keep_deleted.go
+++ b/internal/vm/opcodes/rebase_onto_keep_deleted.go
@@ -2,8 +2,11 @@ package opcodes
 
 import (
 	"fmt"
+	"os"
+	"time"
 
 	"github.com/git-town/git-town/v20/internal/git/gitdomain"
+	"github.com/git-town/git-town/v20/internal/subshell"
 	"github.com/git-town/git-town/v20/internal/vm/shared"
 	. "github.com/git-town/git-town/v20/pkg/prelude"
 )
@@ -29,6 +32,12 @@ func (self *RebaseOntoKeepDeleted) ContinueProgram() []shared.Opcode {
 }
 
 func (self *RebaseOntoKeepDeleted) Run(args shared.RunArgs) error {
+	// Fix for https://github.com/git-town/git-town/issues/4942.
+	// Waiting here in end-to-end tests to ensure new timestamps for the rebased commits,
+	// which avoids flaky end-to-end tests.
+	if len(os.Getenv(subshell.TestToken)) > 0 {
+		time.Sleep(1 * time.Second)
+	}
 	err := args.Git.RebaseOnto(args.Frontend, self.BranchToRebaseOnto.Location(), self.CommitsToRemove, self.Upstream)
 	if err != nil {
 		conflictingFiles, err := args.Git.FileConflictQuickInfos(args.Backend)
diff --git a/internal/vm/opcodes/rebase_tracking_branch.go b/internal/vm/opcodes/rebase_tracking_branch.go
index 7e3613e8d..960923c98 100644
--- a/internal/vm/opcodes/rebase_tracking_branch.go
+++ b/internal/vm/opcodes/rebase_tracking_branch.go
@@ -14,8 +14,8 @@ type RebaseTrackingBranch struct {
 }
 
 func (self *RebaseTrackingBranch) Run(args shared.RunArgs) error {
-	// Try to force-push the local branch with lease and includes to the remote branch.
 	if self.PushBranches {
+		// force-push-with-lease-if-includes here first, this avoids phantom merge conflicts from amended local commits
 		err := args.Git.ForcePushBranchSafely(args.Frontend, args.Config.Value.NormalConfig.NoPushHook(), true)
 		if err == nil {
 			// The force-push succeeded --> the remote branch didn't contain new commits, we are done.



Test patch:
diff --git a/internal/test/subshell/test_runner.go b/internal/test/subshell/test_runner.go
index 28883d599..911b1a9e6 100644
--- a/internal/test/subshell/test_runner.go
+++ b/internal/test/subshell/test_runner.go
@@ -205,6 +205,8 @@ func (self *TestRunner) QueryWithCode(opts *Options, cmd string, args ...string)
 	if gitEditor, hasGitEditor := self.gitEditor.Get(); hasGitEditor {
 		opts.Env = envvars.Replace(opts.Env, "GIT_EDITOR", filepath.Join(self.BinDir, gitEditor))
 	}
+	// mark as test run
+	opts.Env = append(opts.Env, subshell.TestToken+"=1")
 	// set the working dir
 	opts.Dir = filepath.Join(self.WorkingDir, opts.Dir)
 	// run the command inside the custom environment
@@ -212,9 +214,7 @@ func (self *TestRunner) QueryWithCode(opts *Options, cmd string, args ...string)
 	if len(opts.Dir) > 0 {
 		subProcess.Dir = opts.Dir
 	}
-	if opts.Env != nil {
-		subProcess.Env = opts.Env
-	}
+	subProcess.Env = opts.Env
 	var outputBuf bytes.Buffer
 	subProcess.Stdout = &outputBuf
 	subProcess.Stderr = &outputBuf