Repository: woodpecker-ci/woodpecker (Go)

Issue:
BitBucket Cloud - Pipeline definition not found (when using multiple workflows, manual run)
### Component

server, agent, web-ui

### Describe the bug

When using ".woodpecker/" directory with a workflow file (myworkflow.yml) inside, the error message: `[generic]pipeline definition not found in company/my-repo` appears (manual run).

When using the same file (myworkflow.yml) in the root directory of the project with name ".woodpecker.yml", the workflow is found and executed.

The documentation states that BitBucket Cloud supports multiple workflows. See [https://woodpecker-ci.org/docs/next/administration/forges/overview](https://woodpecker-ci.org/docs/next/administration/forges/overview)

### System Info

```shell
version	"next-d0e63375fa"
```


### Additional context

```
woodpecker-server-1  | {"level":"debug","time":"2024-04-09T07:06:55Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/session/repo.go:150","message":"myuser granted &{UserID:1 RepoID:1 Repo:<nil> Pull:true Push:true Admin:true Synced:1712646369 Created:1712589724 Updated:1712646369} permission to company/my-repo"}
woodpecker-server-1  | {"level":"error","error":"Resource not found","repo":"company/my-repo","user":"myuser","time":"2024-04-09T07:06:56Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:152","message":"could not get folder from forge"}
woodpecker-server-1  | {"level":"error","error":"Resource not found","repo":"company/my-repo","user":"myuser","time":"2024-04-09T07:06:57Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:152","message":"could not get folder from forge"}
woodpecker-server-1  | {"level":"error","error":"Resource not found","repo":"company/my-repo","user":"myuser","time":"2024-04-09T07:06:57Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:152","message":"could not get folder from forge"}
woodpecker-server-1  | {"level":"debug","repo":"company/my-repo","error":"configFetcher: fallback did not find config: Resource not found\nNo such file or directory: .woodpecker.yaml\nNo such file or directory: .woodpecker.yml","time":"2024-04-09T07:06:58Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/pipeline/create.go:82","message":"error while fetching config '' in 'ci/woodpecker' with user: 'myuser'"}
```

Woodpecker-UI: Pipeline Path -> empty (default)

### Validations

- [X] Read the [docs](https://woodpecker-ci.org/docs/intro).
- [X] Check that there isn't [already an issue](https://github.com/woodpecker-ci/woodpecker/issues) that reports the same bug to avoid creating a duplicate.
- [X] Checked that the bug isn't fixed in the `next` version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]


Discussion:
Can you check the logs?
And what's configured as config path in repo settings?
Hi. I added the logs in the original post. The pipeline path is left empty (default).
Thanks for the quick help @qwerty287 

I tested the change (using server version: `next-00f0fcd416`). The "Resource not found" error messages are gone. But the workflows are still not found. Same debug message:

```
woodpecker-server-1  | {"level":"debug","repo":"company/my-repo","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-15T09:43:29Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/pipeline/create.go:77","message":"cannot find config '' in 'ci/woodpecker' with user: 'myuser'"}
```

I have two workflow files in ./woodpecker:

.woodpecker/workflow1.yml
```
steps:
  test-step-1:
    image: alpine
    when:
      - event: [tag, manual]
    commands:
      - echo "Test step 1"

```

.woodpecker/workflow2.yml
```
steps:
  test-step-2:
    image: alpine
    when:
      - event: [tag, manual]
    commands:
      - echo "Test step 2"

```

The UI is showing this error in console when doing a manual run:
```
Error: Missing required param "pipelineId"
```

Do I need to specify the "pipelineId" somehow? Maybe it's just an error on my side.

Thanks.
Hi @emit-ah, could you check the logs with `trace` level for some more lines related to config files?
Hi, here are the trace logs:

```
Tag:

{"level":"trace","time":"2024-04-29T07:33:14Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/cron/cron.go:49","message":"cron: fetch next crons"}
{"level":"trace","time":"2024-04-29T07:33:22Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/router.go:43","message":"[POST] /api/hook?access_token=redacted"}
{"level":"trace","time":"2024-04-29T07:33:22Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:101","message":"configFetcher[mycompany/myrepo]: user did not define own config, following default procedure"}
{"level":"trace","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:33:23Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:64","message":"1. try failed"}
{"level":"trace","time":"2024-04-29T07:33:23Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:101","message":"configFetcher[mycompany/myrepo]: user did not define own config, following default procedure"}
{"level":"trace","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:33:23Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:64","message":"2. try failed"}
{"level":"trace","time":"2024-04-29T07:33:23Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:101","message":"configFetcher[mycompany/myrepo]: user did not define own config, following default procedure"}
{"level":"trace","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:33:24Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:64","message":"3. try failed"}
{"level":"debug","repo":"mycompany/myrepo","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:33:24Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/pipeline/create.go:83","message":"cannot find config '' in 'refs/tags/mytag' with user: 'myuser'"}
{"level":"debug","ip":"redacted","latency":1712.565727,"method":"POST","path":"/api/hook","status":204,"time":"2024-04-29T07:33:24Z","user-agent":"Bitbucket-Webhooks/2.0","time":"2024-04-29T07:33:24Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/logger.go:59"}
{"level":"trace","time":"2024-04-29T07:33:24Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/cron/cron.go:49","message":"cron: fetch next crons"}


Manual run:

{"level":"trace","time":"2024-04-29T07:36:34Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/cron/cron.go:49","message":"cron: fetch next crons"}
{"level":"trace","time":"2024-04-29T07:36:35Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/router.go:43","message":"[POST] /api/repos/1/pipelines"}
{"level":"debug","time":"2024-04-29T07:36:35Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/session/repo.go:157","message":"myuser granted &{UserID:1 RepoID:1 Repo:<nil> Pull:true Push:true Admin:true Synced:1714375214 Created:1714030001 Updated:1714375214} permission to mycompany/myrepo"}
{"level":"trace","time":"2024-04-29T07:36:36Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:101","message":"configFetcher[mycompany/myrepo]: user did not define own config, following default procedure"}
{"level":"trace","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:36:37Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:64","message":"1. try failed"}
{"level":"trace","time":"2024-04-29T07:36:37Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:101","message":"configFetcher[mycompany/myrepo]: user did not define own config, following default procedure"}
{"level":"trace","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:36:37Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:64","message":"2. try failed"}
{"level":"trace","time":"2024-04-29T07:36:37Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:101","message":"configFetcher[mycompany/myrepo]: user did not define own config, following default procedure"}
{"level":"trace","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:36:38Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/services/config/forge.go:64","message":"3. try failed"}
{"level":"debug","repo":"mycompany/myrepo","error":"configFetcher: fallback did not find config: configs not found: .woodpecker/, .woodpecker.yaml, .woodpecker.yml","time":"2024-04-29T07:36:38Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/pipeline/create.go:83","message":"cannot find config '' in 'mybranch' with user: 'myuser'"}
{"level":"debug","ip":"redacted","latency":2760.83674,"method":"POST","path":"/api/repos/1/pipelines","status":204,"time":"2024-04-29T07:36:38Z","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0","time":"2024-04-29T07:36:38Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/logger.go:59"}
{"level":"trace","time":"2024-04-29T07:36:44Z","caller":"/woodpecker/src/github.com/woodpecker-ci/woodpecker/server/cron/cron.go:49","message":"cron: fetch next crons"}
```




Patch:
diff --git a/server/forge/bitbucket/fixtures/handler.go b/server/forge/bitbucket/fixtures/handler.go
index ab2104f7b69..146dcfef854 100644
--- a/server/forge/bitbucket/fixtures/handler.go
+++ b/server/forge/bitbucket/fixtures/handler.go
@@ -114,7 +114,7 @@ func getRepoFile(c *gin.Context) {
 	switch c.Param("file") {
 	case "dir":
 		c.String(http.StatusOK, repoDirPayload)
-	case "dir_not_found/":
+	case "dir_not_found":
 		c.String(http.StatusNotFound, "")
 	case "file_not_found":
 		c.String(http.StatusNotFound, "")
diff --git a/server/forge/bitbucket/internal/client.go b/server/forge/bitbucket/internal/client.go
index 16e4380bc15..9bae76ae8c9 100644
--- a/server/forge/bitbucket/internal/client.go
+++ b/server/forge/bitbucket/internal/client.go
@@ -52,7 +52,7 @@ const (
 	pathOrgPerms      = "%s/2.0/workspaces/%s/permissions?%s"
 	pathPullRequests  = "%s/2.0/repositories/%s/%s/pullrequests?%s"
 	pathBranchCommits = "%s/2.0/repositories/%s/%s/commits/%s"
-	pathDir           = "%s/2.0/repositories/%s/%s/src/%s%s"
+	pathDir           = "%s/2.0/repositories/%s/%s/src/%s/%s"
 	pageSize          = 100
 )
 
diff --git a/server/forge/bitbucketdatacenter/bitbucketdatacenter.go b/server/forge/bitbucketdatacenter/bitbucketdatacenter.go
index b881a701db9..58b0c6d3726 100644
--- a/server/forge/bitbucketdatacenter/bitbucketdatacenter.go
+++ b/server/forge/bitbucketdatacenter/bitbucketdatacenter.go
@@ -262,8 +262,14 @@ func (c *client) File(ctx context.Context, u *model.User, r *model.Repo, p *mode
 		return nil, fmt.Errorf("unable to create bitbucket client: %w", err)
 	}
 
-	b, _, err := bc.Projects.GetTextFileContent(ctx, r.Owner, r.Name, f, p.Commit)
+	b, resp, err := bc.Projects.GetTextFileContent(ctx, r.Owner, r.Name, f, p.Commit)
 	if err != nil {
+		if resp.StatusCode == http.StatusNotFound {
+			// requested directory might not exist
+			return nil, &forge_types.ErrConfigNotFound{
+				Configs: []string{f},
+			}
+		}
 		return nil, err
 	}
 	return b, nil
@@ -281,7 +287,10 @@ func (c *client) Dir(ctx context.Context, u *model.User, r *model.Repo, p *model
 		list, resp, err := bc.Projects.ListFiles(ctx, r.Owner, r.Name, path, opts)
 		if err != nil {
 			if resp.StatusCode == http.StatusNotFound {
-				break // requested directory might not exist
+				// requested directory might not exist
+				return nil, &forge_types.ErrConfigNotFound{
+					Configs: []string{path},
+				}
 			}
 			return nil, err
 		}
diff --git a/server/services/config/forge.go b/server/services/config/forge.go
index 7413e783d41..9de4bfb3fc2 100644
--- a/server/services/config/forge.go
+++ b/server/services/config/forge.go
@@ -92,7 +92,7 @@ func (f *forgeFetcherContext) fetch(c context.Context, config string) ([]*types.
 
 		fileMetas, err := f.getFirstAvailableConfig(ctx, configs)
 		if err == nil {
-			return fileMetas, err
+			return fileMetas, nil
 		}
 
 		return nil, fmt.Errorf("user defined config '%s' not found: %w", config, err)
@@ -102,7 +102,7 @@ func (f *forgeFetcherContext) fetch(c context.Context, config string) ([]*types.
 	// for the order see shared/constants/constants.go
 	fileMetas, err := f.getFirstAvailableConfig(ctx, constant.DefaultConfigOrder[:])
 	if err == nil {
-		return fileMetas, err
+		return fileMetas, nil
 	}
 
 	select {


Test patch:
diff --git a/server/forge/bitbucket/bitbucket_test.go b/server/forge/bitbucket/bitbucket_test.go
index dd384296402..c9f878bd76f 100644
--- a/server/forge/bitbucket/bitbucket_test.go
+++ b/server/forge/bitbucket/bitbucket_test.go
@@ -217,14 +217,14 @@ func Test_bitbucket(t *testing.T) {
 
 		g.Describe("When requesting repo directory contents", func() {
 			g.It("Should return the details", func() {
-				files, err := c.Dir(ctx, fakeUser, fakeRepo, fakePipeline, "/dir")
+				files, err := c.Dir(ctx, fakeUser, fakeRepo, fakePipeline, "dir")
 				g.Assert(err).IsNil()
 				g.Assert(len(files)).Equal(3)
 				g.Assert(files[0].Name).Equal("README.md")
 				g.Assert(string(files[0].Data)).Equal("dummy payload")
 			})
 			g.It("Should handle not found errors", func() {
-				_, err := c.Dir(ctx, fakeUser, fakeRepo, fakePipeline, "dir_not_found/")
+				_, err := c.Dir(ctx, fakeUser, fakeRepo, fakePipeline, "dir_not_found")
 				g.Assert(err).IsNotNil()
 				g.Assert(errors.Is(err, &types.ErrConfigNotFound{})).IsTrue()
 			})