Repository: twisted/towncrier

Issue:
Get the version from pyproject.toml [tool.poetry] section
Currently using towncrier with a project that uses poetry as package manager. The docs says that you can add towncrier's config to `pyproject.toml`, and poetry has metadata version like this

```
[tool.poetry]
...
version = "0.1.0"
```

if adding towncrier metadata to the toml like this
```
[tool.towncrier]
directory = "changes"
package_dir = "."
filename = "NEWS.rst"
```
it complains about lacking the version. Can't towncrier parse the toml looking for version instead of repeating version variables?

Discussion:
agriyakhetarpal: Some tests in `VersionFetchingTests` are currently failing locally because `importlib.metadata.version()` requires a package to be either installed or editable-installed, and the tests do not do so currently when creating a temporary project. How may I proceed? I have caught `PackageNotFoundError` in a try-except block for now so that the functionality is retained

Alternatives are to try to use `importlib.metadata.distribution` to get the version, or to just parse `pyproject.toml` — the latter is not in consideration since it is neither the current standard nor Pythonic.
SmileyChris: See also: #491 
hynek: I'm not sure how I feel about this whole"install towncrier along with your package" -- it kinda seems like a misfeature / attractive nuisance for the reasons we've just seen. :-/ 

I would argue for deprecation and adding ways to extract the version from the _outside_ (pyproject.toml / extract using regexp from a file / run a command [e.g. hatch]).
agriyakhetarpal: It would be much easier to use a `toml` parser library to extract package versions from `pyproject.toml`. With hatch as the build backend, it is just as easy to get it just as it is to update it, –from `hatch version`, which also might be extracting it from `[tool.hatch.version]` or `[project]`, I am not sure.
agriyakhetarpal: It would be much easier to use a `toml` parser library to extract package versions from `pyproject.toml`. With hatch as the build backend, it is just as easy to get it just as it is to update it – from `hatch version`, which also might be extracting it from `[tool.hatch.version]` or `[project]`, I am not sure.
hynek: It’s not that simple: hatch supports dynamic versions using plugins like `hatch-vcs` that use git metadata. Running `hatch` would be probably the most correct but also the most invasive approach.

I for one would claim that the user can run `towncrier build --version $
(hatch version)` themselves just fine?

Don’t y’all feel too that these features are children of a bygone Python packaging era when things were simpler and more homogenous?
adiroiban: I think that `towncrier build --version $(hatch version)` already works.

At the same time, I think that we should make simple things simple.

So if the version is already defined as a simple text value inside the .toml file, I think that it makes sense to accept a PR in which the version is automatically extracted.

We can document that this only works for limited use cases.

----------

I prefer to have all the common commands stored in file.
So maybe `towncrier build --version $(hatch version)` can be implemented as

If towncrier `version` starts with `shell`, then we will call an external command.
I think that we already are calling `git` commands

```
[tool.towncrier]
version = "shell:hatch version" 
```

or if someone already has the package installed, they can use something like

```
[tool.towncrier]
version = "importlib.metadata" 
```


----------

These options will not work for any case... but I don't think that we should reject a PR adding support for a specific use case.

I expect that the majority of project will try to keep things simple, and have the version as a string inside the `.toml` file.


----------

Just my feedback
hynek: Hmm either way I'm sorry, I think the approach in this PR is the more correct one even though I disagree with the feature as a whole.

It will need something like [this](https://github.com/hynek/structlog/blob/ade1f4b10f361a7ba126e72111d455c1c50779e6/src/structlog/__init__.py#L107-L110) and a conditional dependency `'importlib_metadata; python_version<'3.8'"` to work on 3.7, though.

Please consider this un-bikeshedded otherwise.
adiroiban: So for this PR to be merged  it needs:

* newsfragment / release notes to advertise this feature
* (nice to have) Update the existing documentation to mention that if you have your project installed, towncrier can extract the version
* automated tests to check the new feature - for this, we might want automatically create a super simple package as part of the test suite, install that package and check that version is detected.

I think there are already some tests generating `setup.cfg/pyproject.toml` on the fly and executing towncrier.

With the automated tests, there is one big problem, if you start a python interpreter/process and while the process is running you install packages... the metadata of the new pacage might not be available.

For my code sometimes I am calling `pip` like this... and I get a warning that this usage is not supported:

```
from pip import main
pip_arguments = ['install', 'some-test-package']
result = main(args=pip_arguments)
```

It works... if in the same process I call `uninstall` pip complains that the package doesn't exist.

So just something to take into consideration when testing.
adiroiban: I triggered the current test suite to see if there are any regression that might be introduced by this change.

To me, this approach is harmeless ... if you don't have the package install it should work as before.

Also, I don't know if we should worry about python 3.7 ... It will be EOL soon.

hynek: How about we (read: you 😇) cut a release and drop 3.7 afterwards?

(Pls someone merge my typo PR first tho 😅)
agriyakhetarpal: Dropping Python 3.7 before this makes a lot of sense, though the backport of `importlib.metadata` works fine as well. I was waiting on a consensus before working further and it shall hopefully be fine using a `toml` parser rather than using regex, just for the sake of convenience
adiroiban: Well...  I think that the  overall feedback is that people are expecting from `towncrier` to auto-magically detect the version of their project.

Using `importlib.metadata_version` should be the most `standard` way of doing this.

So the direction of this PR is good.

But in order to merge this PR, we need all tests to be green.
agriyakhetarpal: I realise that this PR has been lost in the hallows of my GitHub account... and it has taken almost an year for me to get back to that following the notification for the received review, thanks! 🙂

I will shall look to rebase this branch soon and finish this up, because it looks like other PRs are now depending on this (such as #491).
SmileyChris: So something like my last commit, @RazerM ?

Patch:
diff --git a/pyproject.toml b/pyproject.toml
index 245e7dd..d6f8fa6 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -31,6 +31,7 @@ requires-python = ">=3.8"
 dependencies = [
     "click",
     "importlib-resources>=5; python_version<'3.10'",
+    "importlib-metadata>=4.6; python_version<'3.10'",
     "incremental",
     "jinja2",
     "tomli; python_version<'3.11'",
diff --git a/src/towncrier/_project.py b/src/towncrier/_project.py
index cfaa11f..53e72e9 100644
--- a/src/towncrier/_project.py
+++ b/src/towncrier/_project.py
@@ -5,17 +5,24 @@
 Responsible for getting the version and name from a project.
 """
 
-
 from __future__ import annotations
 
 import sys
 
 from importlib import import_module
+from importlib.metadata import version as metadata_version
 from types import ModuleType
+from typing import Any
 
 from incremental import Version as IncrementalVersion
 
 
+if sys.version_info >= (3, 10):
+    from importlib.metadata import packages_distributions
+else:
+    from importlib_metadata import packages_distributions  # type: ignore
+
+
 def _get_package(package_dir: str, package: str) -> ModuleType:
     try:
         module = import_module(package)
@@ -37,13 +44,40 @@ def _get_package(package_dir: str, package: str) -> ModuleType:
     return module
 
 
+def _get_metadata_version(package: str) -> str | None:
+    """
+    Try to get the version from the package metadata.
+    """
+    distributions = packages_distributions()
+    distribution_names = distributions.get(package)
+    if not distribution_names or len(distribution_names) != 1:
+        # We can only determine the version if there is exactly one matching distribution.
+        return None
+    return metadata_version(distribution_names[0])
+
+
 def get_version(package_dir: str, package: str) -> str:
+    """
+    Get the version of a package.
+
+    Try to extract the version from the distribution version metadata that matches
+    `package`, then fall back to looking for the package in `package_dir`.
+    """
+    version: Any
+
+    # First try to get the version from the package metadata.
+    if version := _get_metadata_version(package):
+        return version
+
+    # When no version if found, fall back to looking for the package in `package_dir`.
     module = _get_package(package_dir, package)
 
     version = getattr(module, "__version__", None)
 
     if not version:
-        raise Exception("No __version__, I don't know how else to look")
+        raise Exception(
+            f"No __version__ or metadata version info for the '{package}' package."
+        )
 
     if isinstance(version, str):
         return version.strip()
diff --git a/src/towncrier/newsfragments/432.feature.rst b/src/towncrier/newsfragments/432.feature.rst
new file mode 100644
index 0000000..94f19ea
--- /dev/null
+++ b/src/towncrier/newsfragments/432.feature.rst
@@ -0,0 +1,1 @@
+Inferring the version of a Python package now tries to use the metadata of the installed package before importing the package explicitly (which only looks for ``[package].__version__``).


Test patch:
diff --git a/src/towncrier/test/test_project.py b/src/towncrier/test/test_project.py
index 4587568..62ec3d7 100644
--- a/src/towncrier/test/test_project.py
+++ b/src/towncrier/test/test_project.py
@@ -4,7 +4,8 @@
 import os
 import sys
 
-from unittest import skipIf
+from importlib.metadata import version as metadata_version
+from unittest import mock
 
 from click.testing import CliRunner
 from twisted.trial.unittest import TestCase
@@ -14,12 +15,6 @@ from .._shell import cli as towncrier_cli
 from .helpers import write
 
 
-try:
-    from importlib.metadata import version as metadata_version
-except ImportError:
-    metadata_version = None
-
-
 towncrier_cli.name = "towncrier"
 
 
@@ -50,7 +45,6 @@ class VersionFetchingTests(TestCase):
         version = get_version(temp, "mytestproja")
         self.assertEqual(version, "1.3.12")
 
-    @skipIf(metadata_version is None, "Needs importlib.metadata.")
     def test_incremental(self):
         """
         An incremental Version __version__  is picked up.
@@ -60,6 +54,14 @@ class VersionFetchingTests(TestCase):
         with self.assertWarnsRegex(
             DeprecationWarning, "Accessing towncrier.__version__ is deprecated.*"
         ):
+            # Previously this triggered towncrier.__version__ but now the first version
+            # check is from the package metadata. Let's mock out that part to ensure we
+            # can get incremental versions from __version__ still.
+            with mock.patch(
+                "towncrier._project._get_metadata_version", return_value=None
+            ):
+                version = get_version(pkg, "towncrier")
+
             version = get_version(pkg, "towncrier")
 
         with self.assertWarnsRegex(
@@ -70,6 +72,13 @@ class VersionFetchingTests(TestCase):
         self.assertEqual(metadata_version("towncrier"), version)
         self.assertEqual("towncrier", name)
 
+    def test_version_from_metadata(self):
+        """
+        A version from package metadata is picked up.
+        """
+        version = get_version(".", "towncrier")
+        self.assertEqual(metadata_version("towncrier"), version)
+
     def _setup_missing(self):
         """
         Create a minimalistic project with missing metadata in a temporary
@@ -91,10 +100,12 @@ class VersionFetchingTests(TestCase):
         tmp_dir = self._setup_missing()
 
         with self.assertRaises(Exception) as e:
+            # The 'missing' package has no __version__ string.
             get_version(tmp_dir, "missing")
 
         self.assertEqual(
-            ("No __version__, I don't know how else to look",), e.exception.args
+            ("No __version__ or metadata version info for the 'missing' package.",),
+            e.exception.args,
         )
 
     def test_missing_version_project_name(self):