Repository: runatlantis/atlantis (language: go)

Issue:
workspace does not exist
<!--- Please keep this note for the community --->

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
* If you are interested in working on this issue or have submitted a pull request, please leave a comment.

<!--- Thank you for keeping this note for the community --->

---

<!---
When filing a bug, please include the following headings if possible.
Any example text in this template can be deleted.
--->

### Overview of the Issue

My PRs get comments of

```
checking if workspace exists: stat /atlantis-data/repos/goldsky-io/goldsky-infra/1286/default: no such file or directory
```

My helm values are as follows

```
atlantisUrl: https://atlantis.goldsky.com
environment:
  ATLANTIS_AUTOPLAN_FILE_LIST: '**/*.tf,**/*.tfvars,**/*.tfvars.json,**/cdk.tf.json'
  ATLANTIS_GH_ORG: goldsky-io
  ATLANTIS_INCLUDE_GIT_UNTRACKED_FILES: true
  ATLANTIS_TFE_TOKEN: secret
githubApp:
  id: 963113
  installationId: "53537338"
  key: |
    key
  secret: secret
  slug: atlantis-for-atlantis-goldsky-com
image:
  pullPolicy: Always
  repository: public.ecr.aws/b0y6c8j0/atlantis
  tag: v0.28.5-alpine
orgAllowlist: github.com/goldsky-io/goldsky-infra
readinessProbe:
  enabled: true
  failureThreshold: 5
  initialDelaySeconds: 5
  periodSeconds: 5
  scheme: HTTP
  successThreshold: 1
  timeoutSeconds: 5
```

I'm using a custom docker image like so

```
FROM ghcr.io/runatlantis/atlantis:v0.28.5

USER root

# Install Doppler CLI and npm
RUN wget -q -t3 'https://packages.doppler.com/public/cli/rsa.8004D9FF50437357.key' -O /etc/apk/keys/cli@doppler-8004D9FF50437357.rsa.pub && \
    echo 'https://packages.doppler.com/public/cli/alpine/any-version/main' | tee -a /etc/apk/repositories && \
    apk add doppler npm

USER atlantis
```

I'm trying to follow the [docs for getting CDKTF working](https://www.runatlantis.io/docs/custom-workflows.html#cdktf).

I have the following `atlantis.yaml` in my repo

```
version: 3
projects:
  - name: default
    dir: .
    branch: dev
    pre_workflow_hooks:
      - run: npm i && npm run get && npm run synth:dev:atlantis
```


### Logs

```
{"level":"error","ts":"2024-08-06T20:37:29.512Z","caller":"events/instrumented_project_command_builder.go:75","msg":"Error building auto plan commands: checking if workspace exists: stat /atlantis-data/repos/goldsky-io/goldsky-infra/1286/default: no such file or directory","json":{},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).buildAndEmitStats\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:75\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildAutoplanCommands\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:26\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).runAutoplan\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:86\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:305\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunAutoplanCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:223"}
{"level":"info","ts":"2024-08-06T20:37:29.512Z","caller":"vcs/instrumented_client.go:218","msg":"updating vcs status","json":{"repo":"goldsky-io/goldsky-infra","pull":"1286"}}
{"level":"error","ts":"2024-08-06T20:37:30.127Z","caller":"events/pull_updater.go:18","msg":"checking if workspace exists: stat /atlantis-data/repos/goldsky-io/goldsky-infra/1286/default: no such file or directory","json":{"repo":"goldsky-io/goldsky-infra","pull":"1286"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*PullUpdater).updatePull\n\tgithub.com/runatlantis/atlantis/server/events/pull_updater.go:18\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).runAutoplan\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:91\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:305\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunAutoplanCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:223"}

```

### Additional Context

Using helm chart 5.4.4


Discussion / hints:
```
 ❮❮❮ kubectl exec --namespace default -it statefulset/atlantis -- /bin/bash
atlantis-0:/$ terraform workspace list
* default
```

It seems the default workspace does exist.

The path printed in the error does not exist though

```
atlantis-0:/$ cd /atlantis-data/repos/goldsky-io/goldsky-infra/
atlantis-0:/atlantis-data/repos/goldsky-io/goldsky-infra$ ls
atlantis-0:/atlantis-data/repos/goldsky-io/goldsky-infra$
```
ah, removing 

```
  ATLANTIS_INCLUDE_GIT_UNTRACKED_FILES: true
```

seems to fix the issue, now my logs say

```
atlantis-0 atlantis {"level":"info","ts":"2024-08-06T20:47:20.823Z","caller":"events/working_dir.go:235","msg":"creating dir '/atlantis-data/repos/goldsky-io/goldsky-infra/1286/default'","json":{"repo":"goldsky-io/goldsky-infra","pull":"1286"}}
```

but there are other issues for me to work though.

It seems that `ATLANTIS_INCLUDE_GIT_UNTRACKED_FILES` doesn't properly account for creating the missing directory (or there's other config I need which isn't well documented)
I just ran into this. Why is this no longer marked as a bug?

If `include-git-untracked-files` is set, Atlantis will try to check for untracked files before it checks out the PR:

```
2024-11-14T00:29:29.769Z events/project_command_builder.go:330 'include-git-untracked-files' option is set, getting
 untracked files
2024-11-14T00:29:29.770Z events/instrumented_project_command_builder.go:75 Error building plan commands: checking i
f workspace exists: stat /home/deploy/.atlantis/repos/foo/bar/123/default: no such file or directory
github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).buildAndEmitStats
	github.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:75
github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildPlanCommands
	github.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:35
github.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).run
	github.com/runatlantis/atlantis/server/events/plan_command_runner.go:195
github.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run
	github.com/runatlantis/atlantis/server/events/plan_command_runner.go:307
github.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand
	github.com/runatlantis/atlantis/server/events/command_runner.go:367
```

(This is with Atlantis 0.28.5. I could try 0.30.0, but I don't see any bug fixes that seem relevant in the changelog.)
Have also been running into this issue on Atlantis 0.29.0. I tried to work around it by setting `TF_WORKSPACE`, but with @quentinmit's [comment](https://github.com/runatlantis/atlantis/issues/4822#issuecomment-2475113036), it seems that `inlcude-git-untracked-files` kicks things off before Atlantis can read my repo config and know what it should be planning.

I _don't_ see this error on my projects that use a repo config generated with a pre-workflow hook. 

Fix patch:
diff --git a/server/events/project_command_builder.go b/server/events/project_command_builder.go
index 2e42cfc8a4..f626d4b605 100644
--- a/server/events/project_command_builder.go
+++ b/server/events/project_command_builder.go
@@ -455,23 +455,17 @@ func (p *DefaultProjectCommandBuilder) buildAllCommandsByCfg(ctx *command.Contex
 		return nil, err
 	}
 
-	if p.IncludeGitUntrackedFiles {
-		ctx.Log.Debug(("'include-git-untracked-files' option is set, getting untracked files"))
-		untrackedFiles, err := p.WorkingDir.GetGitUntrackedFiles(ctx.Log, ctx.HeadRepo, ctx.Pull, DefaultWorkspace)
+	ctx.Log.Debug("%d files were modified in this pull request. Modified files: %v", len(modifiedFiles), modifiedFiles)
+
+	// If we're not including git untracked files, we can skip the clone if there are no modified files.
+	if !p.IncludeGitUntrackedFiles {
+		shouldSkipClone, err := p.shouldSkipClone(ctx, modifiedFiles)
 		if err != nil {
 			return nil, err
 		}
-		modifiedFiles = append(modifiedFiles, untrackedFiles...)
-	}
-
-	ctx.Log.Debug("%d files were modified in this pull request. Modified files: %v", len(modifiedFiles), modifiedFiles)
-
-	shouldSkipClone, err := p.shouldSkipClone(ctx, modifiedFiles)
-	if err != nil {
-		return nil, err
-	}
-	if shouldSkipClone {
-		return []command.ProjectContext{}, nil
+		if shouldSkipClone {
+			return []command.ProjectContext{}, nil
+		}
 	}
 
 	// Need to lock the workspace we're about to clone to.
@@ -490,6 +484,15 @@ func (p *DefaultProjectCommandBuilder) buildAllCommandsByCfg(ctx *command.Contex
 		return nil, err
 	}
 
+	if p.IncludeGitUntrackedFiles {
+		ctx.Log.Debug(("'include-git-untracked-files' option is set, getting untracked files"))
+		untrackedFiles, err := p.WorkingDir.GetGitUntrackedFiles(ctx.Log, ctx.HeadRepo, ctx.Pull, DefaultWorkspace)
+		if err != nil {
+			return nil, err
+		}
+		modifiedFiles = append(modifiedFiles, untrackedFiles...)
+	}
+
 	// Parse config file if it exists.
 	repoCfgFile := p.GlobalCfg.RepoConfigFile(ctx.Pull.BaseRepo.ID())
 	hasRepoCfg, err := p.ParserValidator.HasRepoCfg(repoDir, repoCfgFile)


Test patch:
diff --git a/server/events/project_command_builder_test.go b/server/events/project_command_builder_test.go
index bb16148893..e74c563ed7 100644
--- a/server/events/project_command_builder_test.go
+++ b/server/events/project_command_builder_test.go
@@ -45,7 +45,7 @@ var defaultUserConfig = struct {
 	AutoplanFileList:         "**/*.tf,**/*.tfvars,**/*.tfvars.json,**/terragrunt.hcl,**/.terraform.lock.hcl",
 	RestrictFileList:         false,
 	SilenceNoProjects:        false,
-	IncludeGitUntrackedFiles: true,
+	IncludeGitUntrackedFiles: false,
 	AutoDiscoverMode:         "auto",
 }
 
@@ -1695,27 +1695,40 @@ projects:
 // Test that we don't clone the repo if there were no changes based on the atlantis.yaml file.
 func TestDefaultProjectCommandBuilder_SkipCloneNoChanges(t *testing.T) {
 	cases := []struct {
-		AtlantisYAML   string
-		ExpectedCtxs   int
-		ExpectedClones InvocationCountMatcher
-		ModifiedFiles  []string
+		AtlantisYAML             string
+		ExpectedCtxs             int
+		ExpectedClones           InvocationCountMatcher
+		ModifiedFiles            []string
+		IncludeGitUntrackedFiles bool
 	}{
 		{
 			AtlantisYAML: `
 version: 3
 projects:
 - dir: dir1`,
-			ExpectedCtxs:   0,
-			ExpectedClones: Never(),
-			ModifiedFiles:  []string{"dir2/main.tf"},
+			ExpectedCtxs:             0,
+			ExpectedClones:           Never(),
+			ModifiedFiles:            []string{"dir2/main.tf"},
+			IncludeGitUntrackedFiles: false,
+		},
+		{
+			AtlantisYAML: `
+version: 3
+projects:
+- dir: dir1`,
+			ExpectedCtxs:             0,
+			ExpectedClones:           Once(),
+			ModifiedFiles:            []string{"dir2/main.tf"},
+			IncludeGitUntrackedFiles: true,
 		},
 		{
 			AtlantisYAML: `
 version: 3
 parallel_plan: true`,
-			ExpectedCtxs:   0,
-			ExpectedClones: Once(),
-			ModifiedFiles:  []string{"README.md"},
+			ExpectedCtxs:             0,
+			ExpectedClones:           Once(),
+			ModifiedFiles:            []string{"README.md"},
+			IncludeGitUntrackedFiles: false,
 		},
 		{
 			AtlantisYAML: `
@@ -1724,9 +1737,10 @@ autodiscover:
   mode: enabled
 projects:
 - dir: dir1`,
-			ExpectedCtxs:   0,
-			ExpectedClones: Once(),
-			ModifiedFiles:  []string{"dir2/main.tf"},
+			ExpectedCtxs:             0,
+			ExpectedClones:           Once(),
+			ModifiedFiles:            []string{"dir2/main.tf"},
+			IncludeGitUntrackedFiles: false,
 		},
 	}
 
@@ -1770,7 +1784,7 @@ projects:
 			userConfig.AutoplanFileList,
 			userConfig.RestrictFileList,
 			userConfig.SilenceNoProjects,
-			userConfig.IncludeGitUntrackedFiles,
+			c.IncludeGitUntrackedFiles,
 			userConfig.AutoDiscoverMode,
 			scope,
 			terraformClient,