Repository: runatlantis/atlantis (language: go)

Issue:
Atlantis plans less projects than anticipated by modified files 
<!--- Please keep this note for the community --->

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
* If you are interested in working on this issue or have submitted a pull request, please leave a comment.

<!--- Thank you for keeping this note for the community --->

---

<!---
When filing a bug, please include the following headings if possible.
Any example text in this template can be deleted.
--->

### Overview of the Issue

<!---
Please describe the issue you are having and how you encountered the problem.
--->
I believe that the GetModifiedFiles function in [atlantis](https://github.com/runatlantis/atlantis/tree/main)/[server](https://github.com/runatlantis/atlantis/tree/main/server)/[events](https://github.com/runatlantis/atlantis/tree/main/server/events)/[vcs](https://github.com/runatlantis/atlantis/tree/main/server/events/vcs)/[azuredevops_client.go](https://github.com/runatlantis/atlantis/blob/main/server/events/vcs/azuredevops_client.go), is not returning all changes to the files. This is due to the pagination of the API being called from [go-azuredevops](https://github.com/mcdafydd/go-azuredevops/tree/master)/[azuredevops](https://github.com/mcdafydd/go-azuredevops/tree/master/azuredevops)/[git.go](https://github.com/mcdafydd/go-azuredevops/blob/master/azuredevops/git.go). 

See the reference to the API : https://learn.microsoft.com/en-us/rest/api/azure/devops/git/diffs/get?view=azure-devops-rest-5.1&tabs=HTTP

There is 2 parameters that should be used for pagination the $top and $skip. The $top default value is to return 100 elements. This results in getting less plans for projects than anticipated based on changes to files. 

### Reproduction Steps

<!--- 
In order to effectively and quickly resolve the issue, please provide exact steps that allow us the reproduce the problem. If no steps are provided, then it will likely take longer to get the issue resolved.
--->
To reproduce the issue, in an Azure DevOps git repository you need to create a large pull request involving touching more than 100 directories and files combined. Then get atlantis to plan, you should see less plans than what you expect based on the number of files modified. 

### Logs

<!---
Provide log files from Atlantis server

logs can be retrieved from the deployment or from atlantis comments by adding `--debug` such as `atlantis plan --debug`

<details>
  <summary>Logs</summary>

```
log output
```

</details>
--->
"caller":"events/project_command_builder.go:338","msg":"100 files were modified in this pull request. Modified files:


### Environment details

- Atlantis version: 0.28.5
<!---
If not already included, please provide the following:

- Atlantis version:
- Deployment method: ecs/eks/helm/tf module
- If not running the latest Atlantis version have you tried to reproduce this issue on the latest version: 
- Atlantis flags:

Atlantis server-side config file:

```yaml
# config file
```

Repo `atlantis.yaml` file:

```yaml
# config file
```

Any other information you can provide about the environment/deployment (efs/nfs, aws/gcp, k8s/fargate, etc)
--->

### Additional Context

<!---
Additional context on the problem. Docs, links to blogs, or other material that lead you to discover this issue or were helpful in troubleshooting the issue. 

Use a bulleted list to link to tickets
--->






Discussion / hints:

Fix patch:
diff --git a/go.mod b/go.mod
index 4337986c73..119a958c19 100644
--- a/go.mod
+++ b/go.mod
@@ -27,7 +27,7 @@ require (
 	github.com/hashicorp/terraform-config-inspect v0.0.0-20241129133400-c404f8227ea6
 	github.com/jpillora/backoff v1.0.0
 	github.com/kr/pretty v0.3.1
-	github.com/mcdafydd/go-azuredevops v0.12.1
+	github.com/drmaxgit/go-azuredevops v0.13.1
 	github.com/microcosm-cc/bluemonday v1.0.27
 	github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db
 	github.com/mitchellh/go-homedir v1.1.0
diff --git a/go.sum b/go.sum
index 577d442608..8394032195 100644
--- a/go.sum
+++ b/go.sum
@@ -114,6 +114,8 @@ github.com/davidmz/go-pageant v1.0.2 h1:bPblRCh5jGU+Uptpz6LgMZGD5hJoOt7otgT454Wv
 github.com/davidmz/go-pageant v1.0.2/go.mod h1:P2EDDnMqIwG5Rrp05dTRITj9z2zpGcD9efWSkTNKLIE=
 github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
 github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
+github.com/drmaxgit/go-azuredevops v0.13.1 h1:mqRxiwnbKraHAtqfTFa+xD75Atf84MgpjKJdA8hJeS4=
+github.com/drmaxgit/go-azuredevops v0.13.1/go.mod h1:m1pO2fW60I9FahzLHMmHYq3bM446ZMZKDpd8+AEKzxc=
 github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
 github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
 github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
diff --git a/server/controllers/events/azuredevops_request_validator.go b/server/controllers/events/azuredevops_request_validator.go
index 17e48fe7cb..4fa489c51f 100644
--- a/server/controllers/events/azuredevops_request_validator.go
+++ b/server/controllers/events/azuredevops_request_validator.go
@@ -5,7 +5,7 @@ import (
 	"io"
 	"net/http"
 
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 )
 
 //go:generate pegomock generate --package mocks -o mocks/mock_azuredevops_request_validator.go AzureDevopsRequestValidator
diff --git a/server/controllers/events/events_controller.go b/server/controllers/events/events_controller.go
index e3b2606332..1836795dd3 100644
--- a/server/controllers/events/events_controller.go
+++ b/server/controllers/events/events_controller.go
@@ -22,8 +22,8 @@ import (
 	"strconv"
 	"strings"
 
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/google/go-github/v68/github"
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
 	"github.com/microcosm-cc/bluemonday"
 	"github.com/pkg/errors"
 	"github.com/runatlantis/atlantis/server/events"
diff --git a/server/events/command_runner.go b/server/events/command_runner.go
index 30a82105a9..90870a467a 100644
--- a/server/events/command_runner.go
+++ b/server/events/command_runner.go
@@ -17,8 +17,8 @@ import (
 	"fmt"
 	"strconv"
 
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/google/go-github/v68/github"
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
 	"github.com/pkg/errors"
 	"github.com/runatlantis/atlantis/server/core/config/valid"
 	"github.com/runatlantis/atlantis/server/events/command"
diff --git a/server/events/event_parser.go b/server/events/event_parser.go
index c9cae1c828..c7c7117588 100644
--- a/server/events/event_parser.go
+++ b/server/events/event_parser.go
@@ -23,10 +23,10 @@ import (
 
 	giteasdk "code.gitea.io/sdk/gitea"
 
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/go-playground/validator/v10"
 	"github.com/google/go-github/v68/github"
 	lru "github.com/hashicorp/golang-lru/v2"
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
 	"github.com/pkg/errors"
 	"github.com/runatlantis/atlantis/server/events/command"
 	"github.com/runatlantis/atlantis/server/events/models"
diff --git a/server/events/mocks/mock_azuredevops_pull_getter.go b/server/events/mocks/mock_azuredevops_pull_getter.go
index 1ed64c3a10..2c298efa17 100644
--- a/server/events/mocks/mock_azuredevops_pull_getter.go
+++ b/server/events/mocks/mock_azuredevops_pull_getter.go
@@ -4,7 +4,7 @@
 package mocks
 
 import (
-	azuredevops "github.com/mcdafydd/go-azuredevops/azuredevops"
+	azuredevops "github.com/drmaxgit/go-azuredevops/azuredevops"
 	pegomock "github.com/petergtz/pegomock/v4"
 	models "github.com/runatlantis/atlantis/server/events/models"
 	logging "github.com/runatlantis/atlantis/server/logging"
diff --git a/server/events/mocks/mock_event_parsing.go b/server/events/mocks/mock_event_parsing.go
index e6b72acacd..eb22979a56 100644
--- a/server/events/mocks/mock_event_parsing.go
+++ b/server/events/mocks/mock_event_parsing.go
@@ -5,8 +5,8 @@ package mocks
 
 import (
 	gitea "code.gitea.io/sdk/gitea"
+	azuredevops "github.com/drmaxgit/go-azuredevops/azuredevops"
 	github "github.com/google/go-github/v68/github"
-	azuredevops "github.com/mcdafydd/go-azuredevops/azuredevops"
 	pegomock "github.com/petergtz/pegomock/v4"
 	models "github.com/runatlantis/atlantis/server/events/models"
 	gitea0 "github.com/runatlantis/atlantis/server/events/vcs/gitea"
diff --git a/server/events/vcs/azuredevops_client.go b/server/events/vcs/azuredevops_client.go
index fee01ff552..01cf1f669b 100644
--- a/server/events/vcs/azuredevops_client.go
+++ b/server/events/vcs/azuredevops_client.go
@@ -9,7 +9,7 @@ import (
 	"strings"
 	"time"
 
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/pkg/errors"
 	"github.com/runatlantis/atlantis/server/events/models"
 	"github.com/runatlantis/atlantis/server/events/vcs/common"
@@ -68,28 +68,40 @@ func (g *AzureDevopsClient) GetModifiedFiles(logger logging.SimpleLogging, repo
 	targetRefName := strings.Replace(pullRequest.GetTargetRefName(), "refs/heads/", "", 1)
 	sourceRefName := strings.Replace(pullRequest.GetSourceRefName(), "refs/heads/", "", 1)
 
-	r, resp, err := g.Client.Git.GetDiffs(g.ctx, owner, project, repoName, targetRefName, sourceRefName)
-	if err != nil {
-		return nil, errors.Wrap(err, "getting pull request")
-	}
-	if resp.StatusCode != http.StatusOK {
-		return nil, errors.Wrapf(err, "http response code %d getting diff %s to %s", resp.StatusCode, sourceRefName, targetRefName)
-	}
+	const pageSize = 100 // Number of files from diff call
+	var skip int
 
-	for _, change := range r.Changes {
-		item := change.GetItem()
-		// Convert the path to a relative path from the repo's root.
-		relativePath := filepath.Clean("./" + item.GetPath())
-		files = append(files, relativePath)
+	for {
+		r, resp, err := g.Client.Git.GetDiffs(g.ctx, owner, project, repoName, targetRefName, sourceRefName, &azuredevops.GitDiffListOptions{
+			Top:  pageSize,
+			Skip: skip,
+		})
+		if err != nil {
+			return nil, errors.Wrap(err, "getting pull request")
+		}
+		if resp.StatusCode != http.StatusOK {
+			return nil, errors.Wrapf(err, "http response code %d getting diff %s to %s", resp.StatusCode, sourceRefName, targetRefName)
+		}
 
-		// If the file was renamed, we'll want to run plan in the directory
-		// it was moved from as well.
-		changeType := azuredevops.Rename.String()
-		if change.ChangeType == &changeType {
+		for _, change := range r.Changes {
+			item := change.GetItem()
 			// Convert the path to a relative path from the repo's root.
-			relativePath = filepath.Clean("./" + change.GetSourceServerItem())
+			relativePath := filepath.Clean("./" + item.GetPath())
 			files = append(files, relativePath)
+
+			// If the file was renamed, we'll want to run plan in the directory
+			// it was moved from as well.
+			changeType := azuredevops.Rename.String()
+			if change.ChangeType == &changeType {
+				relativePath = filepath.Clean("./" + change.GetSourceServerItem())
+				files = append(files, relativePath)
+			}
+		}
+
+		if len(r.Changes) < pageSize {
+			break // Break if we have reached the end
 		}
+		skip += pageSize // Move to next page
 	}
 
 	return files, nil


Test patch:
diff --git a/server/controllers/events/events_controller_test.go b/server/controllers/events/events_controller_test.go
index f4c563552c..c2ba2a8f5a 100644
--- a/server/controllers/events/events_controller_test.go
+++ b/server/controllers/events/events_controller_test.go
@@ -25,8 +25,8 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/google/go-github/v68/github"
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
 	. "github.com/petergtz/pegomock/v4"
 	events_controllers "github.com/runatlantis/atlantis/server/controllers/events"
 	"github.com/runatlantis/atlantis/server/controllers/events/mocks"
diff --git a/server/events/event_parser_test.go b/server/events/event_parser_test.go
index 27515be71d..67c25e10b2 100644
--- a/server/events/event_parser_test.go
+++ b/server/events/event_parser_test.go
@@ -21,8 +21,8 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/google/go-github/v68/github"
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
 	"github.com/mohae/deepcopy"
 	"github.com/runatlantis/atlantis/server/events"
 	"github.com/runatlantis/atlantis/server/events/command"
diff --git a/server/events/vcs/azuredevops_client_test.go b/server/events/vcs/azuredevops_client_test.go
index a7095262d2..4cc4703dea 100644
--- a/server/events/vcs/azuredevops_client_test.go
+++ b/server/events/vcs/azuredevops_client_test.go
@@ -11,7 +11,7 @@ import (
 	"strings"
 	"testing"
 
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/runatlantis/atlantis/server/events/models"
 	"github.com/runatlantis/atlantis/server/events/vcs"
 	"github.com/runatlantis/atlantis/server/events/vcs/testdata"
@@ -273,7 +273,7 @@ func TestAzureDevopsClient_GetModifiedFiles(t *testing.T) {
 			case "/owner/project/_apis/git/repositories/repo/pullrequests/1?api-version=5.1-preview.1&includeWorkItemRefs=true":
 				w.Write([]byte(testdata.ADPullJSON)) // nolint: errcheck
 			// The second should hit this URL.
-			case "/owner/project/_apis/git/repositories/repo/diffs/commits?api-version=5.1&baseVersion=new_feature&targetVersion=npaulk/my_work":
+			case "/owner/project/_apis/git/repositories/repo/diffs/commits?%24top=100&api-version=5.1&baseVersion=new_feature&targetVersion=npaulk%2Fmy_work":
 				// We write a header that means there's an additional page.
 				w.Write([]byte(resp)) // nolint: errcheck
 				return
diff --git a/server/events/vcs/testdata/fixtures.go b/server/events/vcs/testdata/fixtures.go
index aa18059bac..638b1efbe6 100644
--- a/server/events/vcs/testdata/fixtures.go
+++ b/server/events/vcs/testdata/fixtures.go
@@ -21,9 +21,9 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/drmaxgit/go-azuredevops/azuredevops"
 	"github.com/golang-jwt/jwt/v5"
 	"github.com/google/go-github/v68/github"
-	"github.com/mcdafydd/go-azuredevops/azuredevops"
 )
 
 var PullEvent = github.PullRequestEvent{