Repository: hartwork/wnpp.debian.net

Issue:
Forwarded (ITP or RFP) bugs are not found
Whilst doing some security tracker work, I checked wnpp.debian.net for freeswitch using both the "Project" and "Description" queries, looking for ITP or RFP. Nothing came up.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=389591 exists but is marked as Forwarded.

Other bugs similarly tagged "forwarded" also do not come up in wnpp.debian.net:
* freeswitch #389591
* visit #395573
* flex-sdk #592007
* xul-ext-pencil
* apostrophe 

16 packages in all.

https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=wnpp;dist=unstable
Forwarded (ITP or RFP) bugs are not found
Whilst doing some security tracker work, I checked wnpp.debian.net for freeswitch using both the "Project" and "Description" queries, looking for ITP or RFP. Nothing came up.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=389591 exists but is marked as Forwarded.

Other bugs similarly tagged "forwarded" also do not come up in wnpp.debian.net:
* freeswitch #389591
* visit #395573
* flex-sdk #592007
* xul-ext-pencil
* apostrophe 

16 packages in all.

https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=wnpp;dist=unstable


Discussion:
Hi @codehelp , when looking at the two freeswitch tickets I find that they do not have status "open".
In more detail:

```console
$ docker-compose run wnpp-debian-net ./manage.py shell
Creating wnpp-debian-net_wnpp-debian-net_run ... done
# wait-for-it --service postgres:5432
[*] Waiting 15 seconds for postgres:5432
[+] postgres:5432 is available after 0 seconds
# [[ 2 -gt 0 ]]
# [[ ./manage.py = test ]]
# exec ./manage.py shell
Python 3.9.1 (default, Feb  4 2021, 01:29:51) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.
[..]
In [9]: from wnpp_debian_net.debbugs import DebbugsWnppClient

In [10]: wnpp = DebbugsWnppClient()

In [11]: wnpp.connect()

In [12]: open_issues = wnpp.fetch_ids_of_open_issues()

In [13]: [(i, i in open_issues) for i in (389591, 513606)]
Out[13]: [(389591, False), (513606, False)]
```
And:

```console
$ docker-compose run wnpp-debian-net ./manage.py inspectdebbugs 389591 513606
Creating wnpp-debian-net_wnpp-debian-net_run ... done
# wait-for-it --service postgres:5432
[*] Waiting 15 seconds for postgres:5432
[+] postgres:5432 is available after 0 seconds
# [[ 4 -gt 0 ]]
# [[ ./manage.py = test ]]
# exec ./manage.py inspectdebbugs 389591 513606
{
  "389591": {
    "archived": "0",
    "bug_num": "389591",
    "date": "1159289291",
    "forwarded": "http://jira.freeswitch.org/browse/FS-353",
    "id": "389591",
    "keywords": "fixed-upstream",
    "last_modified": "1531157499",
    "location": "db-h",
    "log_modified": "1531157499",
    "mergedwith": "513606",
    "msgid": "<20060926164546.5409.38090.reportbug@localhost.localdomain>",
    "originator": "Juan Manuel Coronado Z\u00fa\u00f1iga <juan.coronado@avatar.com.co>",
    "package": "wnpp",
    "pending": "forwarded",
    "severity": "wishlist",
    "subject": "RFP: freeswitch -- Modular Media Switching Software Library and Soft-Switch Application.",
    "tags": "fixed-upstream",
    "unarchived": "1493760427"
  },
  "513606": {
    "archived": "0",
    "bug_num": "513606",
    "date": "1233336605",
    "forwarded": "http://jira.freeswitch.org/browse/FS-353",
    "id": "513606",
    "keywords": "fixed-upstream",
    "last_modified": "1531157500",
    "location": "db-h",
    "log_modified": "1531157500",
    "mergedwith": "389591",
    "msgid": "<20090130172905.7092.62408.reportbug@nicholas-hp.nicholasamorim.com.br>",
    "originator": "Nicholas Amorim <nicholas@montrealconsultoria.com.br>",
    "package": "wnpp",
    "pending": "forwarded",
    "severity": "wishlist",
    "subject": "RFP: freeswitch -- Modular Media Switching Software Library and Soft-Switch Application.",
    "tags": "fixed-upstream",
    "unarchived": "1493760428"
  }
}
```

@codehelp what is your understanding of the status of these tickets?
@codehelp any thoughts?
Closing because of no reply
I came across the same issue for `chezmoi` (bug 1012721). The ITP bug is forwarded to a GitHub issue, and the packager is still (actively) working on packaging the software. I feel that wnpp should include bugs like this.
I chatted with some DD, who believed that “Forwarded” is closer to “see also”; in this case, it essentially means something like

> You can find more updates and discussion in this GitHub issue, but it is still WIP, and the author still intend to fix it.

In addition, [my RFP request of the same package](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012832) was deduplicated onto bug 1012721. Both pieces of evidence make me believe that showing it in the search result is the right thing to do.
Hi @codehelp , when looking at the two freeswitch tickets I find that they do not have status "open".
In more detail:

```console
$ docker-compose run wnpp-debian-net ./manage.py shell
Creating wnpp-debian-net_wnpp-debian-net_run ... done
# wait-for-it --service postgres:5432
[*] Waiting 15 seconds for postgres:5432
[+] postgres:5432 is available after 0 seconds
# [[ 2 -gt 0 ]]
# [[ ./manage.py = test ]]
# exec ./manage.py shell
Python 3.9.1 (default, Feb  4 2021, 01:29:51) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.
[..]
In [9]: from wnpp_debian_net.debbugs import DebbugsWnppClient

In [10]: wnpp = DebbugsWnppClient()

In [11]: wnpp.connect()

In [12]: open_issues = wnpp.fetch_ids_of_open_issues()

In [13]: [(i, i in open_issues) for i in (389591, 513606)]
Out[13]: [(389591, False), (513606, False)]
```
And:

```console
$ docker-compose run wnpp-debian-net ./manage.py inspectdebbugs 389591 513606
Creating wnpp-debian-net_wnpp-debian-net_run ... done
# wait-for-it --service postgres:5432
[*] Waiting 15 seconds for postgres:5432
[+] postgres:5432 is available after 0 seconds
# [[ 4 -gt 0 ]]
# [[ ./manage.py = test ]]
# exec ./manage.py inspectdebbugs 389591 513606
{
  "389591": {
    "archived": "0",
    "bug_num": "389591",
    "date": "1159289291",
    "forwarded": "http://jira.freeswitch.org/browse/FS-353",
    "id": "389591",
    "keywords": "fixed-upstream",
    "last_modified": "1531157499",
    "location": "db-h",
    "log_modified": "1531157499",
    "mergedwith": "513606",
    "msgid": "<20060926164546.5409.38090.reportbug@localhost.localdomain>",
    "originator": "Juan Manuel Coronado Z\u00fa\u00f1iga <juan.coronado@avatar.com.co>",
    "package": "wnpp",
    "pending": "forwarded",
    "severity": "wishlist",
    "subject": "RFP: freeswitch -- Modular Media Switching Software Library and Soft-Switch Application.",
    "tags": "fixed-upstream",
    "unarchived": "1493760427"
  },
  "513606": {
    "archived": "0",
    "bug_num": "513606",
    "date": "1233336605",
    "forwarded": "http://jira.freeswitch.org/browse/FS-353",
    "id": "513606",
    "keywords": "fixed-upstream",
    "last_modified": "1531157500",
    "location": "db-h",
    "log_modified": "1531157500",
    "mergedwith": "389591",
    "msgid": "<20090130172905.7092.62408.reportbug@nicholas-hp.nicholasamorim.com.br>",
    "originator": "Nicholas Amorim <nicholas@montrealconsultoria.com.br>",
    "package": "wnpp",
    "pending": "forwarded",
    "severity": "wishlist",
    "subject": "RFP: freeswitch -- Modular Media Switching Software Library and Soft-Switch Application.",
    "tags": "fixed-upstream",
    "unarchived": "1493760428"
  }
}
```

@codehelp what is your understanding of the status of these tickets?
@codehelp any thoughts?
Closing because of no reply
I came across the same issue for `chezmoi` (bug 1012721). The ITP bug is forwarded to a GitHub issue, and the packager is still (actively) working on packaging the software. I feel that wnpp should include bugs like this.
I chatted with some DD, who believed that “Forwarded” is closer to “see also”; in this case, it essentially means something like

> You can find more updates and discussion in this GitHub issue, but it is still WIP, and the author still intend to fix it.

In addition, [my RFP request of the same package](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012832) was deduplicated onto bug 1012721. Both pieces of evidence make me believe that showing it in the search result is the right thing to do.

Resolving patch:
diff --git a/wnpp_debian_net/debbugs.py b/wnpp_debian_net/debbugs.py
--- a/wnpp_debian_net/debbugs.py
+++ b/wnpp_debian_net/debbugs.py
@@ -49,6 +49,11 @@ class IssueProperty(Enum):
     UNARCHIVED = 'unarchived'
 
 
+class IssueStatus(Enum):  # known to be incomplete
+    FORWARDED = 'forwarded'
+    OPEN = 'open'
+
+
 class DebbugsRequestError(Exception):
     pass
 
@@ -140,9 +145,9 @@ def _decode_base64_as_needed(candidate: str | None) -> str | None:
             return candidate
 
     @_wrap_exceptions
-    def fetch_ids_of_open_issues(self) -> list[int]:
+    def fetch_ids_of_issues_with_status(self, status: IssueStatus) -> list[int]:
         result: SimpleXMLElement = self._client.get_bugs(
-            **self._to_soap_kwargs('package', 'wnpp', 'status', 'open'))
+            **self._to_soap_kwargs('package', 'wnpp', 'status', status.value))
         return [
             int(item_element.firstChild.nodeValue)
             for item_element in result._element.getElementsByTagName('item')
diff --git a/wnpp_debian_net/management/commands/importdebbugs.py b/wnpp_debian_net/management/commands/importdebbugs.py
--- a/wnpp_debian_net/management/commands/importdebbugs.py
+++ b/wnpp_debian_net/management/commands/importdebbugs.py
@@ -4,6 +4,7 @@
 import datetime
 import re
 import sys
+from functools import partial
 from itertools import islice
 from signal import SIGINT
 from typing import Any
@@ -14,7 +15,8 @@
 from django.utils.text import Truncator
 from django.utils.timezone import now
 
-from ...debbugs import DebbugsRequestError, DebbugsRetry, DebbugsWnppClient, IssueProperty
+from ...debbugs import (DebbugsRequestError, DebbugsRetry, DebbugsWnppClient, IssueProperty,
+                        IssueStatus)
 from ...models import DebianLogIndex, DebianLogMods, DebianPopcon, DebianWnpp, EventKind
 from ._common import ReportingMixin
 
@@ -316,8 +318,11 @@ def handle(self, *args, **options):
         self._client.connect()
 
         try:
-            ids_of_remote_open_issues = DebbugsRetry(self._client.fetch_ids_of_open_issues,
-                                                     notify=self._notice)()
+            ids_of_remote_open_issues: set[int] = set()
+            for issue_status in (IssueStatus.FORWARDED, IssueStatus.OPEN):
+                fetch_ids = partial(self._client.fetch_ids_of_issues_with_status, issue_status)
+                issue_ids: list[int] = DebbugsRetry(fetch_ids, notify=self._notice)()
+                ids_of_remote_open_issues |= set(issue_ids)
 
             self._close_all_issues_but(ids_of_remote_open_issues)
             self._add_any_new_issues_from(ids_of_remote_open_issues)


Test patch:
diff --git a/wnpp_debian_net/management/commands/tests/test_importdebbugs.py b/wnpp_debian_net/management/commands/tests/test_importdebbugs.py
--- a/wnpp_debian_net/management/commands/tests/test_importdebbugs.py
+++ b/wnpp_debian_net/management/commands/tests/test_importdebbugs.py
@@ -15,7 +15,7 @@
 
 
 def _create_mock_debbugs_wnpp_client(issue_ids, properties_of_issues):
-    return Mock(fetch_ids_of_open_issues=Mock(return_value=issue_ids),
+    return Mock(fetch_ids_of_issues_with_status=Mock(return_value=issue_ids),
                 fetch_issues=Mock(return_value=properties_of_issues))
 
 
diff --git a/wnpp_debian_net/tests/cassettes/DebbugsWnppClientTest.test_fetch_ids_of_open_issues.yaml b/wnpp_debian_net/tests/cassettes/DebbugsWnppClientTest.test_fetch_ids_of_issues_with_status.yaml
similarity index 100%
rename from wnpp_debian_net/tests/cassettes/DebbugsWnppClientTest.test_fetch_ids_of_open_issues.yaml
rename to wnpp_debian_net/tests/cassettes/DebbugsWnppClientTest.test_fetch_ids_of_issues_with_status.yaml
diff --git a/wnpp_debian_net/tests/test_debbugs.py b/wnpp_debian_net/tests/test_debbugs.py
--- a/wnpp_debian_net/tests/test_debbugs.py
+++ b/wnpp_debian_net/tests/test_debbugs.py
@@ -3,7 +3,7 @@
 
 from vcr_unittest import VCRTestCase
 
-from ..debbugs import DebbugsWnppClient
+from ..debbugs import DebbugsWnppClient, IssueStatus
 
 ISSUE_IDS_OF_INTEREST = [
     748374,  # solo, not merged with any issues
@@ -73,8 +73,8 @@ def test_fetch_issues(self):
 
         self.assertEqual(properties_of_issue, EXPECTED_PROPERTIES_OF_ISSUE)
 
-    def test_fetch_ids_of_open_issues(self):
-        ids_of_open_issues = self.client.fetch_ids_of_open_issues()
+    def test_fetch_ids_of_issues_with_status(self):
+        ids_of_open_issues = self.client.fetch_ids_of_issues_with_status(IssueStatus.OPEN)
 
         self.assertEqual(len(ids_of_open_issues), 6256)
         self.assertEqual(ids_of_open_issues[0], 119911)