Repository: Qiskit/qiskit

Issue:
Avoid dependencies duplicity
<!-- ⚠️ If you do not respect this template, your issue will be closed -->
<!-- ⚠️ Make sure to browse the opened and closed issues to confirm this idea does not exist. -->

### What is the expected enhancement?
Currently, when you want to add or update a dependency, you need to do that in the `requirements.txt` and `setup.py` files. That is really error-prone.

It would be nice to avoid that situation and make changes only in one of the files when a dependency is added or updated.



Discussion:
Thanks for the PR, @iblancasa - this topic has been touched in the past (with the sequence #153 -> #163 -> #201, ultimately reverting in #206; and the slightly related #343), and it has some more depth than apparent.

In theory, [each file serves a different purpose](https://packaging.python.org/discussions/install-requires-vs-requirements/) - there is a bunch of information (and opinions!) on the topic, but a brief, loose and _personal_ summary would be that "`install_requires` describes what `setuptools` should do when installing a package, and `requirements.txt` describes what developers should install for working on the package". They also vary a bit in term of "standard-ness" - while `install_requires` is a `setuptools` standard argument, `requirements.txt` is more of a convention, not strongly defined (and it assumes that the developers will fall into a pretty common but not really standardized behaviour, ie `pip install -r requirements.txt`).

In practice, we are probably not at the point where we need to take advantage of the "abstract vs concrete" distinction - that's why both dependency sets are equal (actually it is a bit arguable - we do use `requirements-dev.txt` for "what developers should install for an enhanced experience :tm:" ). So I think the main questions are:
* how taxing is having the dependencies declared in two places (we do tend to forget to keep them in sync)?
* if it is indeed taxing, what is the best way to solve it?

If we think it is really taxing, I'd rather try to find a way that preserves the dependencies explicit in `setup.py`, allowing it to act as the ultimate source of truth - #1587 reverses that dependency, and forces us to reference the file correctly and include it in the final packages which might lead to subtle annoyances such as #201). I'm not sure a totally clean way without side effects (extra dependencies, tweaks to the flow, etc) exists - the closest one is probably using `.` in the requirements.txt, but that involves having a `pip install -e thispackage` under the hood, which might include some issues and edge cases of its own.

All things being equal, I'd _personally_ try to minimize the risk of them falling out of sync by trusting the reviewers to nag people a bit when a PR that updates the requirements arises - but whatever the team feels more convenient! @Qiskit/qiskit-terra : any thoughts?

I think removing the duplication provides a lot of value. Having 2 spots provides a place for mistakes to come up or slip in. We actually saw this just yesterday with the marshmallow bump, so I'm very much in favor of this. I don't think there is any issue with relying on a requirements.txt file to be the source of truth, this is actually what I'm used to (although it differs from the recommendations from pypa, they're moving to defining things in a config file eventually too) but having it just be in setup.py without a requirements file would be fine. But what we have now with 2 places just causes confusion and mistakes.

Another option is now that terra will just be python code we could rely on a library like pbr [1] which is how I normally handle setup.py in other projects. It automates and makes things a lot simpler and less error prone. (there are other libs out there that do similar things)

[1] https://docs.openstack.org/pbr/latest/
Having to introduce an extra library for this purpose seems overkill to me - kind of a solution looking for a problem, even if we hypothetically discount the fact that each files serves a different purpose, and the standard-ness described on the first paragraph. In general, I think we are better off requiring strong justifications for introducing a dependency, as each one introduces another potential point of failure and similar potential burden.

Actually, I think the marshmallow bump that you mention seems to me a good argument for:

> All things being equal, I'd personally try to minimize the risk of them falling out of sync by trusting the reviewers to nag people a bit when a PR that updates the requirements arises

I don't think it is too demanding for a reviewer to remember `if requirements.txt touched -> check setup.py`, on the not so frequent occasions such changes occur - we could actually take the chance to formalize a bit a minimal checklist for reviewers, as there are some recurring situations that can probably be avoided by catching them at review time (missing changelog entry, copyright year, docstrings, etc) and optimize our efforts.
If we are not leveraging the semantic difference between them, then I think the duplication and the need of remembering to review both files is a problem we should fix. I agree with @diego-plan9 and think `setup.py` should be the source of truth since we are creating a library, this following the general advice on `setup.py` vs `requirements.txt`. The low hanging fruit is to replace the contents of `requirements.txt` with:

```
-e .
```

I'm not against a tool automating this synchronization and that could be a solution which removes the need of manually reviewing these files and also avoids duplication but I don't want to introduce a new way of handling dependencies away from the common ones of `setup.py` or `requirements.txt`.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -22,20 +22,8 @@
     subprocess.call([sys.executable, '-m', 'pip', 'install', 'Cython>=0.27.1'])
     from Cython.Build import cythonize
 
-REQUIREMENTS = [
-    "contextvars>=2.4;python_version<'3.7'",
-    "jsonschema>=2.6",
-    "retworkx>=0.7.0",
-    "numpy>=1.17",
-    "ply>=3.10",
-    "psutil>=5",
-    "scipy>=1.4",
-    "sympy>=1.3",
-    "dill>=0.3",
-    "fastjsonschema>=2.10",
-    "python-constraint>=1.4",
-    "python-dateutil>=2.8.0",
-]
+with open('requirements.txt') as f:
+    REQUIREMENTS = f.read().splitlines()
 
 # Add Cython extensions here
 CYTHON_EXTS = ['utils', 'swap_trial']


Test patch: