Repository: ARMmbed/mbed-tools

Issue:
Mbed OS path not searched when processing config
### Description
When we added the `--mbed-os-path` option to `configure` and `compile` we didn't add it to the list of search paths for the config processing tool. Currently, the tool always searches for config files from the project root down, and `mbed-os-path` could be outside of the project root.

This causes issues with Greentea tests, as the Greentea test "project root" is a subdirectory of `mbed-os`, so the tool fails to find any `mbed_lib.json` files as it's only searching from the "project root" down.

We should pass this `mbed-os-path` argument to the config processing tool so it can search mbed-os for config files even when mbed-os is outside of the "project root".
<!--
A detailed description of what is being reported. Please include steps to reproduce the problem.

Things to consider sharing:
- What version of the package is being used (pip show mbed-tools)?
- What is the host platform and version (e.g. macOS 10.15.2, Windows 10, Ubuntu 18.04 LTS)?
-->



### Issue request type

<!--
Please add only one `x` to one of the following types. Do not fill multiple types (split the issue otherwise).


For questions please use https://forums.mbed.com/
-->

- [ ] Enhancement
- [x] Bug


Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207?src=pr&el=h1) Report
> Merging [#207](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207?src=pr&el=desc) (ce44d6c) into [master](https://codecov.io/gh/ARMmbed/mbed-tools/commit/e9710eabb27a4f65e7c0e799224c3f8bcd2b4881?el=desc) (e9710ea) will **increase** coverage by `0.07%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207/graphs/tree.svg?width=650&height=150&src=pr&token=QFRIdRv91G)](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #207      +/-   ##
==========================================
+ Coverage   96.63%   96.71%   +0.07%     
==========================================
  Files          94       94              
  Lines        2674     2675       +1     
==========================================
+ Hits         2584     2587       +3     
+ Misses         90       88       -2     
```


| [Impacted Files](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...ls/build/\_internal/config/assemble\_build\_config.py](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207/diff?src=pr&el=tree#diff-c3JjL21iZWRfdG9vbHMvYnVpbGQvX2ludGVybmFsL2NvbmZpZy9hc3NlbWJsZV9idWlsZF9jb25maWcucHk=) | `100.00% <100.00%> (ø)` | |
| [src/mbed\_tools/build/config.py](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207/diff?src=pr&el=tree#diff-c3JjL21iZWRfdG9vbHMvYnVpbGQvY29uZmlnLnB5) | `100.00% <100.00%> (ø)` | |
| [src/mbed\_tools/project/\_internal/project\_data.py](https://codecov.io/gh/ARMmbed/mbed-tools/pull/207/diff?src=pr&el=tree#diff-c3JjL21iZWRfdG9vbHMvcHJvamVjdC9faW50ZXJuYWwvcHJvamVjdF9kYXRhLnB5) | `100.00% <0.00%> (+2.81%)` | :arrow_up: |


Patch:
diff --git a/news/20210302104001.bugfix b/news/20210302104001.bugfix
new file mode 100644
index 0000000..398a80b
--- /dev/null
+++ b/news/20210302104001.bugfix
@@ -0,0 +1,1 @@
+Search mbed-os for config files when mbed-os-path is used and mbed-os is not a subdirectory of the project.
diff --git a/src/mbed_tools/build/_internal/config/assemble_build_config.py b/src/mbed_tools/build/_internal/config/assemble_build_config.py
index 805a6d0..676bc4a 100644
--- a/src/mbed_tools/build/_internal/config/assemble_build_config.py
+++ b/src/mbed_tools/build/_internal/config/assemble_build_config.py
@@ -3,6 +3,8 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 """Configuration assembly algorithm."""
+import itertools
+
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Iterable, List, Optional, Set
@@ -12,7 +14,7 @@ from mbed_tools.build._internal.config import source
 from mbed_tools.build._internal.find_files import LabelFilter, RequiresFilter, filter_files, find_files
 
 
-def assemble_config(target_attributes: dict, mbed_program_directory: Path, mbed_app_file: Optional[Path]) -> Config:
+def assemble_config(target_attributes: dict, search_paths: Iterable[Path], mbed_app_file: Optional[Path]) -> Config:
     """Assemble config for given target and program directory.
 
     Mbed library and application specific config parameters are parsed from mbed_lib.json and mbed_app.json files
@@ -24,8 +26,15 @@ def assemble_config(target_attributes: dict, mbed_program_directory: Path, mbed_
     Unfortunately, mbed_app.json may also contain filter labels to tell us which mbed libs we're depending on.
     This means we have to collect the filter labels from mbed_app.json before parsing any other config files.
     Then we parse all the required mbed_lib config and finally apply the app overrides.
+
+    Args:
+        target_attributes: Mapping of target specific config parameters.
+        search_paths: Iterable of paths to search for mbed_lib.json files.
+        mbed_app_file: The path to mbed_app.json. This can be None.
     """
-    mbed_lib_files = find_files("mbed_lib.json", mbed_program_directory)
+    mbed_lib_files = list(
+        set(itertools.chain.from_iterable(find_files("mbed_lib.json", path) for path in search_paths))
+    )
     return _assemble_config_from_sources(target_attributes, mbed_lib_files, mbed_app_file)
 
 
diff --git a/src/mbed_tools/build/config.py b/src/mbed_tools/build/config.py
index 90a1a77..620e1fe 100644
--- a/src/mbed_tools/build/config.py
+++ b/src/mbed_tools/build/config.py
@@ -31,7 +31,9 @@ def generate_config(target_name: str, toolchain: str, program: MbedProgram) -> p
     """
     targets_data = _load_raw_targets_data(program)
     target_build_attributes = get_target_by_name(target_name, targets_data)
-    config = assemble_config(target_build_attributes, program.root, program.files.app_config_file)
+    config = assemble_config(
+        target_build_attributes, [program.root, program.mbed_os.root], program.files.app_config_file
+    )
     cmake_file_contents = render_mbed_config_cmake_template(
         target_name=target_name, config=config, toolchain_name=toolchain,
     )


Test patch:
diff --git a/tests/build/test_generate_config.py b/tests/build/test_generate_config.py
index 362b020..b2e7008 100644
--- a/tests/build/test_generate_config.py
+++ b/tests/build/test_generate_config.py
@@ -71,6 +71,19 @@ def program(tmp_path):
     return prog
 
 
+@pytest.fixture
+def program_in_mbed_os_subdir(tmp_path):
+    mbed_os_path = tmp_path / "mbed-os"
+    targets_json = mbed_os_path / "targets" / "targets.json"
+    program_root = mbed_os_path / "test-prog"
+    build_subdir = program_root / "__build" / "k64f"
+    program_root.mkdir(parents=True, exist_ok=True)
+    # Create program mbed-os directory and fake targets.json
+    targets_json.parent.mkdir(exist_ok=True, parents=True)
+    targets_json.write_text(json.dumps({target: TARGET_DATA for target in TARGETS}))
+    return MbedProgram.from_existing(program_root, build_subdir, mbed_os_path=mbed_os_path)
+
+
 @pytest.fixture(
     params=[(TARGETS[0], TARGETS[0]), (TARGETS[1], TARGETS[1]), (TARGETS[0], "*")],
     ids=lambda fixture_val: f"target: {fixture_val[0]}, filter: {fixture_val[1]}",
@@ -456,3 +469,23 @@ def test_target_requires_config_option(program):
 
     assert "MBED_CONF_PLATFORM_STDIO_BAUD_RATE=9600" in config_text
     assert "MBED_LFS_READ_SIZE=64" not in config_text
+
+
+def test_config_parsed_when_mbed_os_outside_project_root(program_in_mbed_os_subdir, matching_target_and_filter):
+    program = program_in_mbed_os_subdir
+    target, target_filter = matching_target_and_filter
+    create_mbed_lib_json(
+        program.mbed_os.root / "mbed_lib.json", "platform", config={"stdio-baud-rate": {"value": 9600}},
+    )
+    create_mbed_lib_json(
+        program.mbed_os.root / "storage" / "mbed_lib.json",
+        "filesystem",
+        config={"read_size": {"macro_name": "MBED_LFS_READ_SIZE", "value": 64}},
+    )
+
+    generate_config("K64F", "GCC_ARM", program)
+
+    config_text = (program.files.cmake_build_dir / CMAKE_CONFIG_FILE).read_text()
+
+    assert "MBED_CONF_PLATFORM_STDIO_BAUD_RATE=9600" in config_text
+    assert "MBED_LFS_READ_SIZE=64" in config_text