Repository: hyperledger/aries-cloudagent-python

Issue:
Transition ACA-Py to use poetry
With #2247, we're now running on Python 3.9 and there are a number of dependency updates that I'm planning to make. This could be a good opportunity to make some improvements to our dependency management.

I am personally a fan of poetry -- it's dependency resolution is superior to pip's and it's got several nice-to-haves like builtin virtual environment handling and build and publish commands. There are still some limitations; it doesn't support platform specific builds and bundling binaries so it is unsuitable for projects like Askar right now but I think ACA-Py fits well into poetry's strengths. Externally, this transition should be seamless for ACA-Py users, whether they're consuming ACA-Py via PyPI or container image.

I'm content with our current `setup.py` and `requirements.*` files but my team has been pleased with poetry in our projects over the past couple of years.

Thoughts?

@swcurran @usingtechnology @ianco @TimoGlastra

https://python-poetry.org/


Discussion:
I like poetry, particularly for development I prefer it for dependency management. I would not be opposed to bringing it into ACA-Py.

I'm not sure how the various requirements files (askar, bbs, indy, dev) would work. With poetry, i assume they would be in [dependency groups](https://python-poetry.org/docs/pyproject/#dependencies-and-dependency-groups), but a developer without poetry (or who does not want to use it) can't leverage those groups through `pip`? 

Maybe a few things to prove out, but certainly onboard with supporting poetry.
The askar, bbs, and indy requirements I think would make sense as an "extra;" these behave like pip extras and enable the user to install them using commands like `pip install aries-cloudagent[askar]` (which I believe is how the `setup.py` is configured currently anyways). The dev dependencies would be a dependency group; developers working on ACA-Py would need to use poetry (there might be some workarounds to continue using pip but install deps from a poetry.lock) and use it to install those dependencies.
when this work is tackled, can we do our best to update dependencies? I recently (using acapy 0.8.2) had issues doing plugin updates where packaging was in conflict with (i think black?) and i also had to specify an explicit version for typing-extensions to resolve. i know we can't possibly know all of what is out there, but it would be nice if we could reduce the potential for conflicts.
@usingtechnology absolutely; I agree that this is sorely needed. We've run into similar issues.
Added high priority label; I think updating dependencies is becoming increasingly urgent and having this in place would be helpful.
@gavinok
Assigned to Gavin 

Resolving patch:
diff --git a/aries_cloudagent/__main__.py b/aries_cloudagent/__main__.py
--- a/aries_cloudagent/__main__.py
+++ b/aries_cloudagent/__main__.py
@@ -64,11 +64,16 @@ def run(args):
     run_command(command, args)
 
 
+def script_main():
+    """Run the main function as a script for poetry."""
+    main(sys.argv)
+
+
 def main(args):
     """Execute default entry point."""
-    if __name__ == "__main__":
-        init_debug(args)
-        run(args)
+    init_debug(args)
+    run(args)
 
 
-main(sys.argv)
+if __name__ == "__main__":
+    script_main()
diff --git a/setup.py b/setup.py
deleted file mode 100644
--- a/setup.py
+++ /dev/null
@@ -1,51 +0,0 @@
-"""Module setup."""
-
-import os
-import runpy
-from setuptools import setup, find_packages
-
-PACKAGE_NAME = "aries_cloudagent"
-version_meta = runpy.run_path("./{}/version.py".format(PACKAGE_NAME))
-VERSION = version_meta["__version__"]
-
-
-with open(os.path.abspath("./README.md"), "r") as fh:
-    long_description = fh.read()
-
-
-def parse_requirements(filename):
-    """Load requirements from a pip requirements file."""
-    lineiter = (line.strip() for line in open(filename))
-    return [
-        line
-        for line in lineiter
-        if line and not line.startswith("#") and not line.startswith("git+")
-    ]
-
-
-if __name__ == "__main__":
-    setup(
-        name=PACKAGE_NAME,
-        version=VERSION,
-        long_description=long_description,
-        long_description_content_type="text/markdown",
-        url="https://github.com/hyperledger/aries-cloudagent-python",
-        packages=find_packages(),
-        include_package_data=True,
-        package_data={"aries_cloudagent": ["requirements.txt"]},
-        install_requires=parse_requirements("requirements.txt"),
-        tests_require=parse_requirements("requirements.dev.txt"),
-        extras_require={
-            "askar": parse_requirements("requirements.askar.txt"),
-            "indy": parse_requirements("requirements.indy.txt"),
-            "bbs": parse_requirements("requirements.bbs.txt"),
-            "uvloop": {"uvloop": "^=0.14.0"},
-        },
-        python_requires=">=3.9",
-        classifiers=[
-            "Programming Language :: Python :: 3",
-            "License :: OSI Approved :: Apache Software License",
-            "Operating System :: OS Independent",
-        ],
-        scripts=["bin/aca-py"],
-    )


Test patch:
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -22,14 +22,9 @@ jobs:
           python-version: ${{ inputs.python-version }}
           cache: 'pip'
           cache-dependency-path: 'requirements*.txt'
-      - name: Install dependencies
+      - name: Install the project dependencies
         run: |
-          python -m pip install --upgrade pip
-          pip3 install --no-cache-dir \
-            -r requirements.txt \
-            -r requirements.askar.txt \
-            -r requirements.bbs.txt \
-            -r requirements.dev.txt
+          pip install poetry
+          poetry install -E "askar bbs"
       - name: Tests
-        run: |
-          pytest
+        run: poetry run pytest
diff --git a/docker/Dockerfile.test b/docker/Dockerfile.test
--- a/docker/Dockerfile.test
+++ b/docker/Dockerfile.test
@@ -9,14 +9,12 @@ RUN apt-get update -y && \
 
 WORKDIR /usr/src/app
 
-ADD requirements*.txt ./
+RUN pip install --no-cache-dir poetry
 
-RUN pip3 install --no-cache-dir \
-	-r requirements.txt \
-	-r requirements.askar.txt \
-	-r requirements.bbs.txt \
-	-r requirements.dev.txt
+ADD ./README.md pyproject.toml ./poetry.lock ./
+
+RUN poetry install --no-root --no-directory -E "askar bbs"
 
 ADD . .
 
-ENTRYPOINT ["/bin/bash", "-c", "pytest \"$@\"", "--"]
+ENTRYPOINT ["/bin/bash", "-c", "poetry run pytest \"$@\"", "--"]