Repository: port-labs/ocean
Pull request #1602: [Integration][Jira] Migrate Jira Integration from Deprecated search Endpoint to the GA endpoint Before May 2025 Deprecation
Author: login: emekanwaoma
Labels: name: size/M
Created: 2025-04-29T13:05:24Z | Merged: 2025-04-29T14:29:08Z

# Description

What - Update Jira integration to use new JQL search endpoint before May 1st, 2025 deprecation

Why - The current /rest/api/3/search endpoint will be deprecated and removed after May 1st, 2025. We need to migrate to the new /rest/api/3/search/jql endpoint to ensure continued functionality. [Deprecation Warning Link](https://developer.atlassian.com/changelog/?utm_campaign=sr_sr4jc_email_hs_ad-hoc-announcements_an_hs_aho_glob_all_en__&utm_medium=email&_hsmi=352955349&utm_source=hubspot#CHANGE-2046)

How - 

- Replace deprecated endpoint with new /rest/api/3/search/jql endpoint
- Update pagination to use nextPageToken instead of startAt
- Ensure fields parameter is always passed with *all default
- Update tests to reflect new endpoint behavior

## Type of change

Please leave one option from the following and delete the rest:

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] New Integration (non-breaking change which adds a new integration)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Non-breaking change (fix of existing functionality that will not change current behavior)
- [ ] Documentation (added/updated documentation)

<h4> All tests should be run against the port production environment(using a testing org). </h4>

### Core testing checklist

- [ ] Integration able to create all default resources from scratch
- [ ] Resync finishes successfully
- [ ] Resync able to create entities
- [ ] Resync able to update entities
- [ ] Resync able to detect and delete entities
- [ ] Scheduled resync able to abort existing resync and start a new one
- [ ] Tested with at least 2 integrations from scratch
- [ ] Tested with Kafka and Polling event listeners
- [ ] Tested deletion of entities that don't pass the selector


### Integration testing checklist

- [ ] Integration able to create all default resources from scratch
- [ ] Resync able to create entities
- [ ] Resync able to update entities
- [ ] Resync able to detect and delete entities
- [ ] Resync finishes successfully
- [ ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the `examples` folder in the integration directory.
- [ ] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
- [ ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
- [ ] Docs PR link [here](#)

### Preflight checklist

- [ ] Handled rate limiting
- [ ] Handled pagination
- [ ] Implemented the code in async
- [ ] Support Multi account

## Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

## API Documentation

Provide links to the API documentation used for this integration.

- [Jira Cloud Platform REST API - Search for issues using JQL](https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issue-search/#api-rest-api-3-search-jql-get)
- [Jira Cloud Platform REST API - Deprecation Notice](https://developer.atlassian.com/changelog/?utm_campaign=sr_sr4jc_email_hs_ad-hoc-announcements_an_hs_aho_glob_all_en__&utm_medium=email&_hsmi=352955349&utm_source=hubspot#CHANGE-2046)



Diff:
diff --git a/integrations/jira/CHANGELOG.md b/integrations/jira/CHANGELOG.md
index 769ea2430c..66ae0fe99e 100644
--- a/integrations/jira/CHANGELOG.md
+++ b/integrations/jira/CHANGELOG.md
@@ -6,6 +6,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 <!-- towncrier release notes start -->
+
+## 0.4.0 (2025-04-29)
+
+
+### Improvements
+
+- Updated Jira integration to use new JQL search endpoint (/rest/api/3/search/jql) in preparation for May 1st, 2025 deprecation
+- Added support for token-based pagination using nextPageToken
+- Ensured consistent field selection with *all default
+
+### Bug Fixes
+
+- Fixed issue where fields parameter wasn't being passed correctly in webhook processor
+- Updated tests to properly handle new endpoint parameters
+
+
 ## 0.3.18 (2025-04-27)
 
 ### Bug Fixes
diff --git a/integrations/jira/jira/client.py b/integrations/jira/jira/client.py
index febc17175b..ff6f439171 100644
--- a/integrations/jira/jira/client.py
+++ b/integrations/jira/jira/client.py
@@ -246,6 +246,32 @@ async def get_paginated_projects(
     async def get_single_issue(self, issue_key: str) -> dict[str, Any]:
         return await self._send_api_request("GET", f"{self.api_url}/issue/{issue_key}")
 
+    async def _get_paginated_data_using_next_page_token(
+        self,
+        url: str,
+        extract_key: str | None = None,
+        initial_params: dict[str, Any] | None = None,
+    ) -> AsyncGenerator[list[dict[str, Any]], None]:
+        """Get paginated data using token-based pagination for JQL endpoints."""
+        params = initial_params or {}
+        next_page_token = None
+
+        while True:
+            if next_page_token:
+                params["nextPageToken"] = next_page_token
+
+            response_data = await self._send_api_request("GET", url, params=params)
+            items = response_data.get(extract_key, []) if extract_key else response_data
+
+            if not items:
+                break
+
+            yield items
+
+            next_page_token = response_data.get("nextPageToken")
+            if not next_page_token:
+                break
+
     async def get_paginated_issues(
         self, params: dict[str, Any] | None = None
     ) -> AsyncGenerator[list[dict[str, Any]], None]:
@@ -254,8 +280,8 @@ async def get_paginated_issues(
         if "jql" in params:
             logger.info(f"Using JQL filter: {params['jql']}")
 
-        async for issues in self._get_paginated_data(
-            f"{self.api_url}/search", "issues", initial_params=params
+        async for issues in self._get_paginated_data_using_next_page_token(
+            f"{self.api_url}/search/jql", "issues", initial_params=params
         ):
             yield issues
 
diff --git a/integrations/jira/pyproject.toml b/integrations/jira/pyproject.toml
index b02cd15258..90fa64f462 100644
--- a/integrations/jira/pyproject.toml
+++ b/integrations/jira/pyproject.toml
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "jira"
-version = "0.3.18"
+version = "0.4.0"
 description = "Integration to bring information from Jira into Port"
 authors = ["Mor Paz <mor@getport.io>"]
 
diff --git a/integrations/jira/tests/test_client.py b/integrations/jira/tests/test_client.py
index fba6b1a664..a37acf1b8d 100644
--- a/integrations/jira/tests/test_client.py
+++ b/integrations/jira/tests/test_client.py
@@ -157,12 +157,8 @@ async def test_get_paginated_issues(mock_jira_client: JiraClient) -> None:
         # Verify params were passed correctly
         mock_request.assert_called_with(
             "GET",
-            f"{mock_jira_client.api_url}/search",
-            params={
-                "jql": "project = TEST",
-                "maxResults": PAGE_SIZE,
-                "startAt": 0,
-            },
+            f"{mock_jira_client.api_url}/search/jql",
+            params={"jql": "project = TEST"},
         )
 
 
diff --git a/integrations/jira/tests/test_processors.py b/integrations/jira/tests/test_processors.py
index ba54e8c603..320f02b8da 100644
--- a/integrations/jira/tests/test_processors.py
+++ b/integrations/jira/tests/test_processors.py
@@ -138,7 +138,7 @@ async def test_handleEvent_issueUpdated_noJqlFilterIssuesReturnedFromClient_upda
         async def mock_paginated_issues(
             *args: Any, **kwargs: Any
         ) -> AsyncGenerator[list[dict[str, Any]], None]:
-            assert args[0] == {"jql": "key = TEST-123"}
+            assert args[0] == {"jql": "key = TEST-123", "fields": "*all"}
             yield [mock_issue]
 
         mock_client.get_paginated_issues = mock_paginated_issues
@@ -171,7 +171,7 @@ async def test_handleEvent_issueUpdated_noJqlFilterIssuesNotReturnedFromClient_d
         async def mock_paginated_issues(
             *args: Any, **kwargs: Any
         ) -> AsyncGenerator[list[dict[str, Any]], None]:
-            assert args[0] == {"jql": "key = TEST-123"}
+            assert args[0] == {"jql": "key = TEST-123", "fields": "*all"}
             yield []
 
         mock_client.get_paginated_issues = mock_paginated_issues
@@ -206,7 +206,10 @@ async def test_handleEvent_issueUpdated_filterIssuesReturnedFromClient_updatedRa
         async def mock_paginated_issues(
             *args: Any, **kwargs: Any
         ) -> AsyncGenerator[list[dict[str, Any]], None]:
-            assert args[0] == {"jql": "project = TEST AND key = TEST-123"}
+            assert args[0] == {
+                "jql": "project = TEST AND key = TEST-123",
+                "fields": "*all",
+            }
             yield [mock_issue]
 
         mock_client.get_paginated_issues = mock_paginated_issues
@@ -237,7 +240,10 @@ async def test_handleEvent_issueUpdated_filterIssuesNotReturnedFromClient_delete
         async def mock_paginated_issues(
             *args: Any, **kwargs: Any
         ) -> AsyncGenerator[list[dict[str, Any]], None]:
-            assert args[0] == {"jql": "project = TEST AND key = TEST-123"}
+            assert args[0] == {
+                "jql": "project = TEST AND key = TEST-123",
+                "fields": "*all",
+            }
             yield []
 
         mock_client.get_paginated_issues = mock_paginated_issues
diff --git a/integrations/jira/webhook_processors/issue_webhook_processor.py b/integrations/jira/webhook_processors/issue_webhook_processor.py
index 2445111708..4590c58bf0 100644
--- a/integrations/jira/webhook_processors/issue_webhook_processor.py
+++ b/integrations/jira/webhook_processors/issue_webhook_processor.py
@@ -39,12 +39,13 @@ async def handle_event(
                 deleted_raw_results=[payload["issue"]],
             )
 
-        params = {}
+        params = {
+            "jql": f"key = {issue_key}",
+            "fields": config.selector.fields or "*all",
+        }
 
         if config.selector.jql:
-            params["jql"] = f"{config.selector.jql} AND key = {payload['issue']['key']}"
-        else:
-            params["jql"] = f"key = {payload['issue']['key']}"
+            params["jql"] = f"{config.selector.jql} AND key = {issue_key}"
 
         issues: list[dict[str, Any]] = []
         async for issues_batch in client.get_paginated_issues(params):


Reviews:
user: login: Tankilevitch
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T13:17:17Z
user: login: mk-armah
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-29T13:17:33Z
user: login: mk-armah
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-29T13:19:40Z
user: login: mk-armah
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-29T13:26:08Z
user: login: nivm-port
body: LGTM
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T13:46:57Z
user: login: Tankilevitch
body: Nice! LGTM
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T14:01:03Z
user: login: nivm-port
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T14:03:47Z
user: login: mk-armah
body: LGTM
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-29T14:28:57Z

Review comments:
diff_hunk: @@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 <!-- towncrier release notes start -->
+## 0.3.19 (2025-04-29)
+
+### Features
path: integrations/jira/CHANGELOG.md
user: login: mk-armah
body: ```suggestion
### Improvements
```
created_at: 2025-04-29T13:12:46Z
author_association: MEMBER
diff_hunk: @@ -1,6 +1,6 @@
 [tool.poetry]
 name = "jira"
-version = "0.3.18"
+version = "0.3.19"
path: integrations/jira/pyproject.toml
user: login: Tankilevitch
body: ```suggestion
version = "0.4.0"
```
created_at: 2025-04-29T13:12:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -246,6 +246,32 @@ async def get_paginated_projects(
     async def get_single_issue(self, issue_key: str) -> dict[str, Any]:
         return await self._send_api_request("GET", f"{self.api_url}/issue/{issue_key}")
 
+    async def _get_jql_paginated_data(
path: integrations/jira/jira/client.py
user: login: Tankilevitch
body: ```suggestion
    async def _get_paginated_data_using_next_page_token(
```
created_at: 2025-04-29T13:16:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 <!-- towncrier release notes start -->
+## 0.3.19 (2025-04-29)
path: integrations/jira/CHANGELOG.md
user: login: mk-armah
body: ```suggestion
## 0.4.0 (2025-04-29)
```
created_at: 2025-04-29T13:19:39Z
author_association: MEMBER
diff_hunk: @@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 <!-- towncrier release notes start -->
+## 0.4.0 (2025-04-29)
+
+### Improvements
+
+- Updated Jira integration to use new JQL search endpoint (/rest/api/3/search/jql) in preparation for May 1st, 2025 deprecation
+- Added support for token-based pagination using nextPageToken
+- Ensured consistent field selection with *all default
+
+### Bug Fixes
+
+- Fixed issue where fields parameter wasn't being passed correctly in webhook processor
+- Updated tests to properly handle new endpoint parameters
+
+
path: integrations/jira/CHANGELOG.md
user: login: mk-armah
body: ```suggestion

## 0.4.0 (2025-04-29)


### Improvements

- Updated Jira integration to use new JQL search endpoint (/rest/api/3/search/jql) in preparation for May 1st, 2025 deprecation
- Added support for token-based pagination using nextPageToken
- Ensured consistent field selection with *all default

### Bug Fixes

- Fixed issue where fields parameter wasn't being passed correctly in webhook processor
- Updated tests to properly handle new endpoint parameters


```
created_at: 2025-04-29T13:22:31Z
author_association: MEMBER

PR comments:
user: login: aws-amplify-eu-west-1[bot]
created_at: 2025-04-29T13:07:23Z
body: This pull request is automatically being deployed by Amplify Hosting ([learn more](https://aws.amazon.com/amplify/hosting/)).<br/><br/>Access this pull request here: https://pr-1602.d1ftd8v2gowp8w.amplifyapp.com
author_association: NONE
user: login: emekanwaoma
created_at: 2025-04-29T14:01:43Z
body: > You've changed the webhook code path but not the resync path on main.py (make sure that you send there the required params also)

The default value from JiraIssueSelector ensures we always have "*all"
author_association: CONTRIBUTOR
user: login: nivm-port
created_at: 2025-04-29T14:03:34Z
body: > > You've changed the webhook code path but not the resync path on main.py (make sure that you send there the required params also)
> 
> The default value from JiraIssueSelector ensures we always have "*all"

yep you are right sorry about it :)
author_association: CONTRIBUTOR