Repository: readthedocs/readthedocs.org

Issue:
Support setuptools/distribute extras dependencies
For example, I have in one of my projects:

```
  extras_require={
      'test': ['webtest', 'nose', 'coverage'],
      'develop': ['bpython', 'z3c.checkversions [buildout]'],
  },
```

For API documentation, 'test' packages need to be installed, would be nice if rtd supported this feature :)

Support setuptools/distribute extras dependencies
For example, I have in one of my projects:

```
  extras_require={
      'test': ['webtest', 'nose', 'coverage'],
      'develop': ['bpython', 'z3c.checkversions [buildout]'],
  },
```

For API documentation, 'test' packages need to be installed, would be nice if rtd supported this feature :)



Discussion:
Is there a concept of documentation as a key here? I don't think it makes sense to install necessarily the test dependencies, but if there is one for creating the documentation, I would get behind that.

Extras can be anything. Some people do like to put 'docs' extras, where they enumerate all dependencies needed to be installed for documentation generation.

Closing this for now. I don't really see how it makes sense for RTD to support. If there was some kind of standard for "things I need to install to build docs", we could support that. Something in distribute, or a documentation_requirements.txt in the top-level or requirements/documentation.txt for pip. RTD might be able to take the lead in adding that, but this ticket doesn't really mean that. So feel free to re-open with a proposal for something awesome like that :)

I would like to reopen this issue for discussion. Using setuptools and specifying `extras_require` has become pretty standard. Pip supports installing these extras using

``` sh
pip install project\[docs,other,extra\]
```

Using `extras_require` instead of requirements prevent cluttering the root directory of a project.

Since a requirements file can already be specified for RTD, I think 'extras' can be specified just as easy.

Hello from the future! I agree with @remcohaszing, `extras_require` is the standard approach, can someone from rtfd reopen this issue?

What do you want to be able to do that you can't do now? Can't you just specify the extras_require in your requirements file?

hi @ericholscher!

Instead of `requirements.txt`, I'd like to use my `setup.py` to specify dependencies to build the docs. Right now in our project we have duplication between [`setup.py`](https://github.com/bigchaindb/bigchaindb/blob/develop/setup.py#L25) and [`requirements.txt`](https://github.com/bigchaindb/bigchaindb/blob/develop/docs/requirements.txt). Yes, it's not duplicated anymore if we remove the dependencies from `setup.py`, but I really like the idea to keep everything in a single place.

In the advanced settings I can only set the path to the `requirements.txt`, but it'd be a nice idea to support also dependencies specified in `extras_require`.

![image](https://cloud.githubusercontent.com/assets/134680/13524404/4628f608-e1fb-11e5-87b7-3e50ca108bca.png)

I'm going to reopen this. I think stronger integration with `setup.py` would be a good thing for module maintainers, and other folks that have no other need for requirements files. This functionality likely won't make it into project options, but would be a good addition to our `readthedocs.yaml` and https://github.com/rtfd/readthedocs-build

Leaving it open here for now, though it will likely require work in multiple repos. If anyone wants to tackle this, there is some previous work adding similar options to the yaml spec here and in readthedocs-build.

I'm not a fan of the setuptools extensions of setup.py but it would be nice to see `pip install -e .` being used as this will pickup any dependencies required where `python setup.py install` will not. :-(

Is there a concept of documentation as a key here? I don't think it makes sense to install necessarily the test dependencies, but if there is one for creating the documentation, I would get behind that.

Extras can be anything. Some people do like to put 'docs' extras, where they enumerate all dependencies needed to be installed for documentation generation.

Closing this for now. I don't really see how it makes sense for RTD to support. If there was some kind of standard for "things I need to install to build docs", we could support that. Something in distribute, or a documentation_requirements.txt in the top-level or requirements/documentation.txt for pip. RTD might be able to take the lead in adding that, but this ticket doesn't really mean that. So feel free to re-open with a proposal for something awesome like that :)

I would like to reopen this issue for discussion. Using setuptools and specifying `extras_require` has become pretty standard. Pip supports installing these extras using

``` sh
pip install project\[docs,other,extra\]
```

Using `extras_require` instead of requirements prevent cluttering the root directory of a project.

Since a requirements file can already be specified for RTD, I think 'extras' can be specified just as easy.

Hello from the future! I agree with @remcohaszing, `extras_require` is the standard approach, can someone from rtfd reopen this issue?

What do you want to be able to do that you can't do now? Can't you just specify the extras_require in your requirements file?

hi @ericholscher!

Instead of `requirements.txt`, I'd like to use my `setup.py` to specify dependencies to build the docs. Right now in our project we have duplication between [`setup.py`](https://github.com/bigchaindb/bigchaindb/blob/develop/setup.py#L25) and [`requirements.txt`](https://github.com/bigchaindb/bigchaindb/blob/develop/docs/requirements.txt). Yes, it's not duplicated anymore if we remove the dependencies from `setup.py`, but I really like the idea to keep everything in a single place.

In the advanced settings I can only set the path to the `requirements.txt`, but it'd be a nice idea to support also dependencies specified in `extras_require`.

![image](https://cloud.githubusercontent.com/assets/134680/13524404/4628f608-e1fb-11e5-87b7-3e50ca108bca.png)

I'm going to reopen this. I think stronger integration with `setup.py` would be a good thing for module maintainers, and other folks that have no other need for requirements files. This functionality likely won't make it into project options, but would be a good addition to our `readthedocs.yaml` and https://github.com/rtfd/readthedocs-build

Leaving it open here for now, though it will likely require work in multiple repos. If anyone wants to tackle this, there is some previous work adding similar options to the yaml spec here and in readthedocs-build.

I'm not a fan of the setuptools extensions of setup.py but it would be nice to see `pip install -e .` being used as this will pickup any dependencies required where `python setup.py install` will not. :-(


Resolving patch:
diff --git a/readthedocs/doc_builder/config.py b/readthedocs/doc_builder/config.py
--- a/readthedocs/doc_builder/config.py
+++ b/readthedocs/doc_builder/config.py
@@ -39,6 +39,14 @@ def install_project(self):
         else:
             return self._project.install_project
 
+    @property
+    def extra_requirements(self):
+        if self.pip_install and 'extra_requirements' in self._yaml_config.get(
+                'python', {}):
+            return self._yaml_config['python']['extra_requirements']
+        else:
+            return []
+
     @property
     def python_interpreter(self):
         if 'version' in self._yaml_config.get('python', {}):
diff --git a/readthedocs/doc_builder/python_environments.py b/readthedocs/doc_builder/python_environments.py
--- a/readthedocs/doc_builder/python_environments.py
+++ b/readthedocs/doc_builder/python_environments.py
@@ -44,6 +44,10 @@ def install_package(self):
         setup_path = os.path.join(self.checkout_path, 'setup.py')
         if os.path.isfile(setup_path) and self.config.install_project:
             if self.config.pip_install or getattr(settings, 'USE_PIP_INSTALL', False):
+                extra_req_param = ''
+                if self.config.extra_requirements:
+                    extra_req_param = '[{0}]'.format(
+                        ','.join(self.config.extra_requirements))
                 self.build_env.run(
                     'python',
                     self.venv_bin(filename='pip'),
@@ -51,7 +55,7 @@ def install_package(self):
                     '--ignore-installed',
                     '--cache-dir',
                     self.project.pip_cache_path,
-                    '.',
+                    '.{0}'.format(extra_req_param),
                     cwd=self.checkout_path,
                     bin_path=self.venv_bin()
                 )


Test patch:
diff --git a/readthedocs/rtd_tests/tests/test_config_wrapper.py b/readthedocs/rtd_tests/tests/test_config_wrapper.py
--- a/readthedocs/rtd_tests/tests/test_config_wrapper.py
+++ b/readthedocs/rtd_tests/tests/test_config_wrapper.py
@@ -55,6 +55,28 @@ def test_install_project(self):
         config = ConfigWrapper(version=self.version, yaml_config=yaml_config)
         self.assertEqual(config.install_project, False)
 
+    def test_extra_requirements(self):
+        yaml_config = get_build_config({'python': {
+            'pip_install': True,
+            'extra_requirements': ['tests', 'docs']}})
+        config = ConfigWrapper(version=self.version, yaml_config=yaml_config)
+        self.assertEqual(config.extra_requirements, ['tests', 'docs'])
+
+        yaml_config = get_build_config({'python': {
+            'extra_requirements': ['tests', 'docs']}})
+        config = ConfigWrapper(version=self.version, yaml_config=yaml_config)
+        self.assertEqual(config.extra_requirements, [])
+
+        yaml_config = get_build_config({})
+        config = ConfigWrapper(version=self.version, yaml_config=yaml_config)
+        self.assertEqual(config.extra_requirements, [])
+
+        yaml_config = get_build_config({'python': {
+            'setup_py_install': True,
+            'extra_requirements': ['tests', 'docs']}})
+        config = ConfigWrapper(version=self.version, yaml_config=yaml_config)
+        self.assertEqual(config.extra_requirements, [])
+
     def test_conda(self):
         to_find = 'urls.py'
         yaml_config = get_build_config({'conda': {'file': to_find}})