Repository: readthedocs/readthedocs.org

Issue:
Design around multiple configurations in one file
Related code https://github.com/rtfd/readthedocs.org/blob/a98441717ea50b19fe6bb0f1ea52df092dbc8190/readthedocs/config/config.py#L988-L988

I'm not sure what use case has. Maybe a monorepo? But still, rtd doesn't support that workflow (more the one doc per project).


Discussion:
I'm +1 on getting rid of the logic that searches for a readthedocs.yml file. I think we should only support one top-level config file, all of the other UX patterns are weird.

* Config in some other directory: why not just put it top level?
* Multiple configs: which one are we supposed to use?
* Config in submodule or something: why would you want to use it?
Oops, I think I put the wrong title here, for multiple configurations files the issue is #4669. This is for a configuration file with multiple configurations (yaml support having a top level dict)
@stsewd I found this issue a bit confusing. Can you expand on the purpose of it and what it's needed to do/decide here?
Our code is designed to support several configurations in one yaml file (yaml allows to have more than one top dict), but in reality we only use the _first_ one, so we have some dead code here, and maybe we aren't going to use it (v2 uses only one too).
I see...

So,

> But still, rtd doesn't support that workflow (more the one doc per project).

RTD does not support more than one doc per project, but if the repository has multiple translations the use will import the same repo many times as different (sub)projects. In that case, we may want to allow:

1. multiple YAML files
2. multiple configs on the same YAML file
3. none of the above and force to build all the projects with the same YAML since the only thing that should change on each different translation are the `.po` files.

I'm mentioning 3) even though it's not a problem in that particular case, but as an example that it _could_ be useful in some weird case, maybe.

At first, it seems we could kill that code.
I think our translations workflow needs to be ripped apart eventually. Adding a translation should not require a separate file on the repo. However, currently, you shouldn't need a separate config file for this, the language selector is in the admin UI.

We've discussed in other tickets, but translations will eventually just be another project setting (ie, just enable "Español" translation to your project like you do versions, not add a separate project).

I'm still -1 on multiple config files I think, this is in line with other services like travis and circleci
Just to be clear, this is about having multiple configs in **one file** p: anyway, looks like we don't have a use case for this is kind of the same as having multiples files.

Resolving patch:
diff --git a/readthedocs/config/config.py b/readthedocs/config/config.py
--- a/readthedocs/config/config.py
+++ b/readthedocs/config/config.py
@@ -36,7 +36,6 @@
     'ConfigError',
     'ConfigOptionNotSupportedError',
     'InvalidConfig',
-    'ProjectConfig',
 )
 
 ALL = 'all'
@@ -110,12 +109,10 @@ class InvalidConfig(ConfigError):
 
     message_template = 'Invalid "{key}": {error}'
 
-    def __init__(self, key, code, error_message, source_file=None,
-                 source_position=None):
+    def __init__(self, key, code, error_message, source_file=None):
         self.key = key
         self.code = code
         self.source_file = source_file
-        self.source_position = source_position
         message = self.message_template.format(
             key=key,
             code=code,
@@ -144,11 +141,10 @@ class BuildConfigBase(object):
 
     version = None
 
-    def __init__(self, env_config, raw_config, source_file, source_position):
+    def __init__(self, env_config, raw_config, source_file):
         self.env_config = env_config
         self.raw_config = raw_config
         self.source_file = source_file
-        self.source_position = source_position
         if os.path.isdir(self.source_file):
             self.base_path = self.source_file
         else:
@@ -160,10 +156,7 @@ def __init__(self, env_config, raw_config, source_file, source_position):
     def error(self, key, message, code):
         """Raise an error related to ``key``."""
         if not os.path.isdir(self.source_file):
-            source = '{file} [{pos}]'.format(
-                file=os.path.relpath(self.source_file, self.base_path),
-                pos=self.source_position,
-            )
+            source = os.path.relpath(self.source_file, self.base_path)
             error_message = '{source}: {message}'.format(
                 source=source,
                 message=message,
@@ -175,7 +168,6 @@ def error(self, key, message, code):
             code=code,
             error_message=error_message,
             source_file=self.source_file,
-            source_position=self.source_position,
         )
 
     @contextmanager
@@ -189,7 +181,6 @@ def catch_validation_error(self, key):
                 code=error.code,
                 error_message=str(error),
                 source_file=self.source_file,
-                source_position=self.source_position,
             )
 
     def pop(self, name, container, default, raise_ex):
@@ -1043,16 +1034,6 @@ def submodules(self):
         return Submodules(**self._config['submodules'])
 
 
-class ProjectConfig(list):
-
-    """Wrapper for multiple build configs."""
-
-    def validate(self):
-        """Validates each configuration build."""
-        for build in self:
-            build.validate()
-
-
 def load(path, env_config):
     """
     Load a project configuration and the top-most build config for a given path.
@@ -1068,10 +1049,9 @@ def load(path, env_config):
             'No configuration file found',
             code=CONFIG_REQUIRED
         )
-    build_configs = []
     with open(filename, 'r') as configuration_file:
         try:
-            configs = parse(configuration_file.read())
+            config = parse(configuration_file.read())
         except ParseError as error:
             raise ConfigError(
                 'Parse error in {filename}: {message}'.format(
@@ -1080,23 +1060,19 @@ def load(path, env_config):
                 ),
                 code=CONFIG_SYNTAX_INVALID,
             )
-        for i, config in enumerate(configs):
-            allow_v2 = env_config.get('allow_v2')
-            if allow_v2:
-                version = config.get('version', 1)
-            else:
-                version = 1
-            build_config = get_configuration_class(version)(
-                env_config,
-                config,
-                source_file=filename,
-                source_position=i,
-            )
-            build_configs.append(build_config)
+        allow_v2 = env_config.get('allow_v2')
+        if allow_v2:
+            version = config.get('version', 1)
+        else:
+            version = 1
+        build_config = get_configuration_class(version)(
+            env_config,
+            config,
+            source_file=filename,
+        )
 
-    project_config = ProjectConfig(build_configs)
-    project_config.validate()
-    return project_config
+    build_config.validate()
+    return build_config
 
 
 def get_configuration_class(version):
diff --git a/readthedocs/config/parser.py b/readthedocs/config/parser.py
--- a/readthedocs/config/parser.py
+++ b/readthedocs/config/parser.py
@@ -17,18 +17,17 @@ class ParseError(Exception):
 
 def parse(stream):
     """
-    Take file-like object and return a list of project configurations.
+    Take file-like object and return a project configuration.
 
-    The files need be valid YAML and only contain mappings as documents.
+    The file need be valid YAML and only contain mappings as document.
     Everything else raises a ``ParseError``.
     """
     try:
-        configs = list(yaml.safe_load_all(stream))
+        config = yaml.safe_load(stream)
     except yaml.YAMLError as error:
         raise ParseError('YAML: {message}'.format(message=error))
-    if not configs:
+    if not isinstance(config, dict):
+        raise ParseError('Expected mapping')
+    if not config:
         raise ParseError('Empty config')
-    for config in configs:
-        if not isinstance(config, dict):
-            raise ParseError('Expected mapping')
-    return configs
+    return config
diff --git a/readthedocs/doc_builder/config.py b/readthedocs/doc_builder/config.py
--- a/readthedocs/doc_builder/config.py
+++ b/readthedocs/doc_builder/config.py
@@ -65,7 +65,7 @@ def load_yaml_config(version):
         config = load_config(
             path=checkout_path,
             env_config=env_config,
-        )[0]
+        )
     except InvalidConfig:
         # This is a subclass of ConfigError, so has to come first
         raise
@@ -74,7 +74,6 @@ def load_yaml_config(version):
             env_config=env_config,
             raw_config={},
             source_file=checkout_path,
-            source_position=0,
         )
         config.validate()
     return config


Test patch:
diff --git a/readthedocs/config/tests/test_config.py b/readthedocs/config/tests/test_config.py
--- a/readthedocs/config/tests/test_config.py
+++ b/readthedocs/config/tests/test_config.py
@@ -17,7 +17,6 @@
     ConfigError,
     ConfigOptionNotSupportedError,
     InvalidConfig,
-    ProjectConfig,
     load,
 )
 from readthedocs.config.config import (
@@ -81,13 +80,11 @@
 }
 
 
-def get_build_config(config, env_config=None, source_file='readthedocs.yml',
-                     source_position=0):
+def get_build_config(config, env_config=None, source_file='readthedocs.yml'):
     return BuildConfigV1(
         env_config or {},
         config,
         source_file=source_file,
-        source_position=source_position,
     )
 
 
@@ -131,10 +128,7 @@ def test_load_empty_config_file(tmpdir):
 def test_minimal_config(tmpdir):
     apply_fs(tmpdir, minimal_config_dir)
     base = str(tmpdir)
-    config = load(base, env_config)
-    assert isinstance(config, ProjectConfig)
-    assert len(config) == 1
-    build = config[0]
+    build = load(base, env_config)
     assert isinstance(build, BuildConfigV1)
 
 
@@ -145,10 +139,7 @@ def test_load_version1(tmpdir):
         ''')
     })
     base = str(tmpdir)
-    config = load(base, get_env_config({'allow_v2': True}))
-    assert isinstance(config, ProjectConfig)
-    assert len(config) == 1
-    build = config[0]
+    build = load(base, get_env_config({'allow_v2': True}))
     assert isinstance(build, BuildConfigV1)
 
 
@@ -159,10 +150,7 @@ def test_load_version2(tmpdir):
         ''')
     })
     base = str(tmpdir)
-    config = load(base, get_env_config({'allow_v2': True}))
-    assert isinstance(config, ProjectConfig)
-    assert len(config) == 1
-    build = config[0]
+    build = load(base, get_env_config({'allow_v2': True}))
     assert isinstance(build, BuildConfigV2)
 
 
@@ -183,31 +171,18 @@ def test_yaml_extension(tmpdir):
     apply_fs(tmpdir, yaml_extension_config_dir)
     base = str(tmpdir)
     config = load(base, env_config)
-    assert len(config) == 1
+    assert isinstance(config, BuildConfigV1)
 
 
 def test_build_config_has_source_file(tmpdir):
     base = str(apply_fs(tmpdir, minimal_config_dir))
-    build = load(base, env_config)[0]
+    build = load(base, env_config)
     assert build.source_file == os.path.join(base, 'readthedocs.yml')
-    assert build.source_position == 0
-
-
-def test_build_config_has_source_position(tmpdir):
-    base = str(apply_fs(tmpdir, multiple_config_dir))
-    builds = load(base, env_config)
-    assert len(builds) == 2
-    first, second = filter(
-        lambda b: not b.source_file.endswith('nested/readthedocs.yml'),
-        builds,
-    )
-    assert first.source_position == 0
-    assert second.source_position == 1
 
 
 def test_build_config_has_list_with_single_empty_value(tmpdir):
     base = str(apply_fs(tmpdir, config_with_explicit_empty_list))
-    build = load(base, env_config)[0]
+    build = load(base, env_config)
     assert isinstance(build, BuildConfigV1)
     assert build.formats == []
 
@@ -217,7 +192,6 @@ def test_config_requires_name():
         {'output_base': ''},
         {},
         source_file='readthedocs.yml',
-        source_position=0,
     )
     with raises(InvalidConfig) as excinfo:
         build.validate()
@@ -230,7 +204,6 @@ def test_build_requires_valid_name():
         {'output_base': ''},
         {'name': 'with/slashes'},
         source_file='readthedocs.yml',
-        source_position=0,
     )
     with raises(InvalidConfig) as excinfo:
         build.validate()
@@ -554,7 +527,6 @@ def test_valid_build_config():
         env_config,
         minimal_config,
         source_file='readthedocs.yml',
-        source_position=0,
     )
     build.validate()
     assert build.name == 'docs'
@@ -576,7 +548,6 @@ def it_validates_to_abspath(tmpdir):
                 get_env_config(),
                 {'base': '../docs'},
                 source_file=source_file,
-                source_position=0,
             )
             build.validate()
             assert build.base == str(tmpdir.join('docs'))
@@ -597,7 +568,6 @@ def it_fails_if_base_is_not_a_string(tmpdir):
                 get_env_config(),
                 {'base': 1},
                 source_file=str(tmpdir.join('readthedocs.yml')),
-                source_position=0,
             )
             with raises(InvalidConfig) as excinfo:
                 build.validate()
@@ -610,7 +580,6 @@ def it_fails_if_base_does_not_exist(tmpdir):
             get_env_config(),
             {'base': 'docs'},
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         with raises(InvalidConfig) as excinfo:
             build.validate()
@@ -626,7 +595,6 @@ def it_fails_if_build_is_invalid_option(tmpdir):
             get_env_config(),
             {'build': {'image': 3.0}},
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         with raises(InvalidConfig) as excinfo:
             build.validate()
@@ -642,7 +610,6 @@ def it_fails_on_python_validation(tmpdir):
                 'python': {'version': '3.3'},
             },
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         build.validate_build()
         with raises(InvalidConfig) as excinfo:
@@ -659,7 +626,6 @@ def it_works_on_python_validation(tmpdir):
                 'python': {'version': '3.3'},
             },
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         build.validate_build()
         build.validate_python()
@@ -670,7 +636,6 @@ def it_works(tmpdir):
             get_env_config(),
             {'build': {'image': 'latest'}},
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         build.validate()
         assert build.build.image == 'readthedocs/build:latest'
@@ -681,7 +646,6 @@ def default(tmpdir):
             get_env_config(),
             {},
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         build.validate()
         assert build.build.image == 'readthedocs/build:2.0'
@@ -697,7 +661,6 @@ def it_priorities_image_from_env_config(tmpdir, image):
             get_env_config({'defaults': defaults}),
             {'build': {'image': 'latest'}},
             source_file=str(tmpdir.join('readthedocs.yml')),
-            source_position=0,
         )
         build.validate()
         assert build.build.image == image
@@ -787,7 +750,6 @@ def test_build_validate_calls_all_subvalidators(tmpdir):
         {},
         {},
         source_file=str(tmpdir.join('readthedocs.yml')),
-        source_position=0,
     )
     with patch.multiple(
             BuildConfigV1,
@@ -803,20 +765,6 @@ def test_build_validate_calls_all_subvalidators(tmpdir):
         BuildConfigV1.validate_output_base.assert_called_with()
 
 
-def test_validate_project_config():
-    with patch.object(BuildConfigV1, 'validate') as build_validate:
-        project = ProjectConfig([
-            BuildConfigV1(
-                env_config,
-                minimal_config,
-                source_file='readthedocs.yml',
-                source_position=0,
-            ),
-        ])
-        project.validate()
-        assert build_validate.call_count == 1
-
-
 def test_load_calls_validate(tmpdir):
     apply_fs(tmpdir, minimal_config_dir)
     base = str(tmpdir)
@@ -843,13 +791,12 @@ def test_config_filenames_regex(correct_config_filename):
 
 class TestBuildConfigV2(object):
 
-    def get_build_config(self, config, env_config=None,
-                         source_file='readthedocs.yml', source_position=0):
+    def get_build_config(
+            self, config, env_config=None, source_file='readthedocs.yml'):
         return BuildConfigV2(
             env_config or {},
             config,
             source_file=source_file,
-            source_position=source_position,
         )
 
     def test_version(self):
diff --git a/readthedocs/config/tests/test_parser.py b/readthedocs/config/tests/test_parser.py
--- a/readthedocs/config/tests/test_parser.py
+++ b/readthedocs/config/tests/test_parser.py
@@ -28,36 +28,35 @@ def test_parse_bad_type():
 def test_parse_single_config():
     buf = StringIO(u'base: path')
     config = parse(buf)
-    assert isinstance(config, list)
-    assert len(config) == 1
-    assert config[0]['base'] == 'path'
+    assert isinstance(config, dict)
+    assert config['base'] == 'path'
 
 
 def test_parse_null_value():
     buf = StringIO(u'base: null')
     config = parse(buf)
-    assert config[0]['base'] is None
+    assert config['base'] is None
 
 
 def test_parse_empty_value():
     buf = StringIO(u'base:')
     config = parse(buf)
-    assert config[0]['base'] is None
+    assert config['base'] is None
 
 
 def test_parse_empty_string_value():
     buf = StringIO(u'base: ""')
     config = parse(buf)
-    assert config[0]['base'] == ''
+    assert config['base'] == ''
 
 
 def test_parse_empty_list():
     buf = StringIO(u'base: []')
     config = parse(buf)
-    assert config[0]['base'] == []
+    assert config['base'] == []
 
 
-def test_parse_multiple_configs_in_one_file():
+def test_do_not_parse_multiple_configs_in_one_file():
     buf = StringIO(
         u'''
 base: path
@@ -67,8 +66,5 @@ def test_parse_multiple_configs_in_one_file():
 nested:
     works: true
         ''')
-    configs = parse(buf)
-    assert isinstance(configs, list)
-    assert len(configs) == 2
-    assert configs[0]['base'] == 'path'
-    assert configs[1]['nested'] == {'works': True}
+    with raises(ParseError):
+        parse(buf)
diff --git a/readthedocs/rtd_tests/tests/test_config_integration.py b/readthedocs/rtd_tests/tests/test_config_integration.py
--- a/readthedocs/rtd_tests/tests/test_config_integration.py
+++ b/readthedocs/rtd_tests/tests/test_config_integration.py
@@ -13,7 +13,7 @@
 from mock import MagicMock, PropertyMock, patch
 
 from readthedocs.builds.models import Version
-from readthedocs.config import ALL, BuildConfigV1, InvalidConfig, ProjectConfig
+from readthedocs.config import ALL, BuildConfigV1, InvalidConfig
 from readthedocs.config.tests.utils import apply_fs
 from readthedocs.doc_builder.config import load_yaml_config
 from readthedocs.doc_builder.environments import LocalBuildEnvironment
@@ -27,9 +27,7 @@ def create_load(config=None):
     """
     Mock out the function of the build load function.
 
-    This will create a ProjectConfig list of BuildConfigV1 objects and validate
-    them. The default load function iterates over files and builds up a list of
-    objects. Instead of mocking all of this, just mock the end result.
+    This will create a BuildConfigV1 object and validate it.
     """
     if config is None:
         config = {}
@@ -41,14 +39,11 @@ def inner(path=None, env_config=None):
         }
         if env_config is not None:
             env_config_defaults.update(env_config)
-        yaml_config = ProjectConfig([
-            BuildConfigV1(
-                env_config_defaults,
-                config,
-                source_file='readthedocs.yml',
-                source_position=0,
-            ),
-        ])
+        yaml_config = BuildConfigV1(
+            env_config_defaults,
+            config,
+            source_file='readthedocs.yml',
+        )
         yaml_config.validate()
         return yaml_config