Repository: aj-white/piplexed

Issue:
Installed app not listed
I have installed the app 'tidal-dl-ng' normally with 'pipx install tidal-dl-ng[gui]', but i don't get it listed with 'piplexed list' - what could be the reason?

Discussion:
@aj-white: Hi, thanks for reaching out, do you get the app listed when you run `pipx list` ?
If it doesn't show then there maybe an issue with the `pipx install` ?

If it does show, then would you be able to provide a little more information so I can have a look.
- Version of piplexed (run `piplexed version`)
- Version of pipx (run `pipx --version`)
- Which operating system
- The location that pipx puts the apps in (PIPX_LOCAL_VENVS value from running `pipx environment`)

Thanks.@C-O-D: hi, thanks for responding - yes, the app is listed with `pipx list` and I would say it's installed from PyPI.
OS is Manjaro 24.1.2 and here's the additional info:

pipx list
venvs are in /home/cod/.local/share/pipx/venvs
apps are exposed on your $PATH at /home/cod/.local/bin
manual pages are exposed at /home/cod/.local/share/man
   package piplexed 0.6.1, installed using Python 3.12.6
    - piplexed
   package tidal-dl-ng 0.20.3, installed using Python 3.12.6
    - tdn
    - tdng
    - tidal-dl-ng
    - tidal-dl-ng-gui

piplexed version
0.6.1

pipx --version
1.6.0

pipx environment
Environment variables (set by user):

PIPX_HOME=
PIPX_BIN_DIR=
PIPX_MAN_DIR=
PIPX_SHARED_LIBS=
PIPX_DEFAULT_PYTHON=
PIPX_FETCH_MISSING_PYTHON=
USE_EMOJI=

Derived values (computed by pipx):

PIPX_HOME=/home/cod/.local/share/pipx
PIPX_BIN_DIR=/home/cod/.local/bin
PIPX_MAN_DIR=/home/cod/.local/share/man
PIPX_SHARED_LIBS=/home/cod/.local/share/pipx/shared
PIPX_LOCAL_VENVS=/home/cod/.local/share/pipx/venvs
PIPX_LOG_DIR=/home/cod/.local/state/pipx/log
PIPX_TRASH_DIR=/home/cod/.local/share/pipx/trash
PIPX_VENV_CACHEDIR=/home/cod/.cache/pipx
PIPX_STANDALONE_PYTHON_CACHEDIR=/home/cod/.local/share/pipx/py
PIPX_DEFAULT_PYTHON=/usr/bin/python3.12
USE_EMOJI=true@C-O-D: piplexed list
                 Pip❎ Packages                 
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Package Name ┃ Pipx Version ┃ Python Version ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│     piplexed │        0.6.1 │         3.12.6 │
└──────────────┴──────────────┴────────────────┘@aj-white: Thanks for that, the local venv path looks fine and given that `piplexed list` is returning piplexed, that means we are finding the venvs correctly.
My next thought would be, piplexed uses the *pipx_metadata.json* file, generated by pipx on install to get the python version, package version and determine if the package is installed from PyPI or not. Inside the `pipx/venvs` folder there will be the `tidal-dl-ng` folder and in that should be the *pipx_metadata.json* file, would you be able to post that here, as it maybe that there is a scenario piplexed is not accounting for.@C-O-D: No problem, here it is:
[pipx_metadata.json](https://github.com/user-attachments/files/17874242/pipx_metadata.json)
@C-O-D: Is something missing?@aj-white: I understand what's happened. Usually when a package is installed from pypi the `package` and `package_or_url` metadata is identical. Piplexed used this as a simple check to prevent non-pypi packages being checked for pypi details. It turns out that when a package is installed with optional extras, `package` is the *package_name* and `package_or_url` is *package_name[extras]*, as they don't match piplexed doesn't pick them up, which is what happened in this case.

Piplexed was originally designed for checking for newer pypi packages, howevere I think it is more sensible for `piplexed list` to show all packages installed and let the --outdated flag handle pypi only packages. I'll remove the check and handle non-pypi packages later in the process.
Hopefully get a fix out this week, thanks for raising the issue.@C-O-D: Sounds good, many thanks!

Solution patch:
diff --git a/docs/Command-line-options.md b/docs/Command-line-options.md
index 749e6bc..8ecc729 100644
--- a/docs/Command-line-options.md
+++ b/docs/Command-line-options.md
@@ -27,7 +27,7 @@ piplexed list <OPTIONS>
 
 |**Name**|**Type**|**Descrtiption**|**Default**|
 |---|---|---|---|
-`--outdated`, `-O`|boolean|Check PyPI for newer versions of installed packages/tools. If `False` list packages/tools installed with specified tool.| `False`
+`--outdated`, `-O`|boolean|Check PyPI for newer versions of installed packages/tools (*installed tools not available on PyPI are ignored during check*). If `False` list packages/tools installed with specified tool.| `False`
 `--pre`, `-P`|boolean|include pre-releases in check for newer versions|`False`
 `--tree`, `-T`|boolean|Show results in a tree format|`False`
 `--tool`|text|Choose which tool packages werre installed with. Options are `pipx`, `uv`, `all`|`pipx`
diff --git a/docs/How-piplexed-works.md b/docs/How-piplexed-works.md
index c5d3ce4..d744926 100644
--- a/docs/How-piplexed-works.md
+++ b/docs/How-piplexed-works.md
@@ -29,5 +29,8 @@ Because wheels are often platform specific, there is no guarantee that a suitabl
 
 **Piplexed** employs the strategy recommended in [PEP 700](https://peps.python.org/pep-0700/) to get the *latest* version, which utilises the [packaging](https://github.com/pypa/packaging) package to parse version numbers.
 
-!!! note
-    Most of this info was gleaned from looking at and researching [Bernát Gábor's](https://github.com/gaborbernat) package [pypi-changes](https://github.com/gaborbernat/pypi_changes), which also has heavily influenced this project.
\ No newline at end of file
+If any packages/tools have been installed that are not available on PyPI, such as local wheels or git repo, these will be ignored when checking for lastest PyPI version.
+
+/// note
+Most of this info was gleaned from looking at and researching [Bernát Gábor's](https://github.com/gaborbernat) package [pypi-changes](https://github.com/gaborbernat/pypi_changes), which also has heavily influenced this project.
+///
\ No newline at end of file
diff --git a/src/piplexed/pypi/pypi_info.py b/src/piplexed/pypi/pypi_info.py
index aab54fa..7a17bc2 100644
--- a/src/piplexed/pypi/pypi_info.py
+++ b/src/piplexed/pypi/pypi_info.py
@@ -7,6 +7,7 @@
 from packaging.version import InvalidVersion
 from packaging.version import Version
 from pypi_simple import DistributionPackage
+from pypi_simple import NoSuchProjectError
 from pypi_simple import PyPISimple
 from rich.progress import BarColumn
 from rich.progress import Progress
@@ -72,7 +73,11 @@ def find_most_recent_version_on_pypi(*, venvs: list[PackageInfo], is_prerelease:
 
         updates = []
         for future in as_completed(results):
-            result = future.result()
+            try:
+                result = future.result()
+            except NoSuchProjectError:
+                continue
+
             if result.newer_pypi_version():
                 updates.append(result)
             progress_bar.update(task, advance=1)
diff --git a/src/piplexed/venvs/pipx_venvs.py b/src/piplexed/venvs/pipx_venvs.py
index 3a6a084..3e6dc23 100644
--- a/src/piplexed/venvs/pipx_venvs.py
+++ b/src/piplexed/venvs/pipx_venvs.py
@@ -67,13 +67,11 @@ def installed_pipx_tools(venv_dir: Path | None = PIPX_LOCAL_VENVS) -> list[Packa
                             stacklevel=2,
                             category=UserWarning,
                         )
-                    # packages installed from pypi have the same package and package_or_url
-                    if data["main_package"]["package"] == data["main_package"]["package_or_url"]:
-                        pkg_data = PackageInfo(
-                            name=canonicalize_name(data["main_package"]["package"]),
-                            version=Version(data["main_package"]["package_version"]),
-                            python=data["python_version"].split()[-1],
-                        )
-                        venvs.append(pkg_data)
+                    pkg_data = PackageInfo(
+                        name=canonicalize_name(data["main_package"]["package"]),
+                        version=Version(data["main_package"]["package_version"]),
+                        python=data["python_version"].split()[-1],
+                    )
+                    venvs.append(pkg_data)
 
     return venvs


Test patch:
diff --git a/tests/test_pipx_venvs.py b/tests/test_pipx_venvs.py
index 6ac197b..37ac541 100644
--- a/tests/test_pipx_venvs.py
+++ b/tests/test_pipx_venvs.py
@@ -105,7 +105,10 @@ def test_pipx_metadata(venv_dir_test_setup, pipx_metadata_version):
     pypi_json = venv_dir_test_setup / "pypi_package" / "pipx_metadata.json"
     local_json = venv_dir_test_setup / "local_package" / "pipx_metadata.json"
 
-    expected = [PackageInfo(name="testy-mctestface", version=Version("23.1.0"), python="3.11.2")]
+    expected = [
+        PackageInfo(name="local-package", version=Version("23.1.0"), python="3.11.2"),
+        PackageInfo(name="testy-mctestface", version=Version("23.1.0"), python="3.11.2"),
+    ]
 
     with open(pypi_json, "w") as f:
         pipx_metadata = mock_metadata(metadata_version=pipx_metadata_version)
@@ -115,7 +118,10 @@ def test_pipx_metadata(venv_dir_test_setup, pipx_metadata_version):
         pipx_metadata = mock_metadata(metadata_version=pipx_metadata_version, pypi_package=False)
         json.dump(pipx_metadata, f)
 
-    assert installed_pipx_tools(venv_dir_test_setup) == expected
+    results = installed_pipx_tools(venv_dir_test_setup)
+    assert expected[0] in results
+    assert expected[1] in results
+    assert len(results) == len(expected)
 
 
 def test_multiple_json_files_in_venv(venv_dir_test_setup):
diff --git a/tests/test_pypi_info.py b/tests/test_pypi_info.py
index a0d31ce..aae0307 100644
--- a/tests/test_pypi_info.py
+++ b/tests/test_pypi_info.py
@@ -4,11 +4,13 @@
 import pytest
 from packaging.version import Version
 from pypi_simple import DistributionPackage
+from pypi_simple import NoSuchProjectError
 from pypi_simple.client import PyPISimple
 
 from piplexed.pypi.pypi_info import find_most_recent_version_on_pypi
 from piplexed.pypi.pypi_info import get_pypi_versions
 from piplexed.pypi.pypi_info import latest_pypi_version
+from piplexed.pypi.pypi_info import pypi_package_info
 from piplexed.venvs.pipx_venvs import PackageInfo
 
 
@@ -213,6 +215,7 @@ def mocked_venvs():
     return [
         PackageInfo(name="package1", version="1.0.0"),
         PackageInfo(name="package2", version="2.0.0"),
+        PackageInfo(name="nonpypi-package", version="1.5.1"),
     ]
 
 
@@ -230,8 +233,10 @@ def side_effect(client, pkg, is_prerelease):  # noqa: ARG001
                 return PackageInfo(
                     name="package1", version="1.0.0", latest_pypi_version="1.1.0" if not is_prerelease else "1.2.0-beta"
                 )
-            else:
+            elif pkg.name == "package2":
                 return PackageInfo(name="package2", version="2.0.0", latest_pypi_version="2.0.0")
+            else:
+                raise NoSuchProjectError(project="nonpypi-package", url="mock/url")
 
         mock.side_effect = side_effect
         yield mock
@@ -243,7 +248,7 @@ def test_find_most_recent_version_on_pypi(mocked_venvs, mock_get_pypi_versions):
     assert result[0].name == "package1"
     assert result[0].version == "1.0.0"
     assert result[0].latest_pypi_version == "1.1.0"
-    assert mock_get_pypi_versions.call_count == 2
+    assert mock_get_pypi_versions.call_count == 3
 
 
 def test_find_most_recent_version_on_pypi_is_prerelease(mocked_venvs, mock_get_pypi_versions):
@@ -252,7 +257,7 @@ def test_find_most_recent_version_on_pypi_is_prerelease(mocked_venvs, mock_get_p
     assert result[0].name == "package1"
     assert result[0].version == "1.0.0"
     assert result[0].latest_pypi_version == "1.2.0-beta"
-    assert mock_get_pypi_versions.call_count == 2
+    assert mock_get_pypi_versions.call_count == 3
 
 
 def test_find_most_recent_version_on_pypi_no_updates(mocked_venvs, mock_get_pypi_versions):
@@ -264,7 +269,7 @@ def test_find_most_recent_version_on_pypi_no_updates(mocked_venvs, mock_get_pypi
 
     result = find_most_recent_version_on_pypi(venvs=mocked_venvs, is_prerelease=False)
     assert not result
-    assert mock_get_pypi_versions.call_count == 2
+    assert mock_get_pypi_versions.call_count == 3
 
 
 def test_find_most_recent_version_on_pypi_unstable_no_updates(mocked_venvs, mock_get_pypi_versions):
@@ -276,4 +281,11 @@ def test_find_most_recent_version_on_pypi_unstable_no_updates(mocked_venvs, mock
 
     result = find_most_recent_version_on_pypi(venvs=mocked_venvs, is_prerelease=True)
     assert not result
-    assert mock_get_pypi_versions.call_count == 2
+    assert mock_get_pypi_versions.call_count == 3
+
+
+def test_pypi_package_info_non_pypi_project():
+    mock_client = create_autospec(PyPISimple, instance=True)
+    mock_client.get_project_page.side_effect = NoSuchProjectError("jeff", "mock/url")
+    with pytest.raises(NoSuchProjectError):
+        pypi_package_info(client=mock_client, package_name="jeff")