Repository: facebookresearch/CompilerGym

Issue:
Running autotuning fails with `Primary config directory not found.`
## 🐛 Bug

When I tried running the auto tuning script, I get an error saying `Primary config directory not found`. See details below.

## To Reproduce

Steps to reproduce the behavior:

1. Run `make install` from base directory to install from source
2. Run `python setup.py install` from `examples/` to install scripts.
3. Run the following command to run the script

```
HYDRA_FULL_ERROR=1 python -m llvm_autotuning.tune -m \
  experiment=my-exp \
  outputs=/tmp/logs \
  executor.cpus=32 \
  num_replicas=1 \
  autotuner=nevergrad \
  autotuner.optimization_target=runtime \      
  autotuner.search_time_seconds=600
```

The full error message is:

```
Traceback (most recent call last):
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym_examples-0.2.4-py3.8.egg/llvm_autotuning/tune.py", line 37, in <module>
    main()
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
    _run_hydra(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 375, in _run_hydra
    run_and_report(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/utils.py", line 376, in <lambda>
    lambda: hydra.multirun(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 121, in multirun
    cfg = self.compose_config(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 564, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 146, in load_configuration
    return self._load_configuration_impl(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 227, in _load_configuration_impl
    self.ensure_main_config_source_available()
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 134, in ensure_main_config_source_available
    self._missing_config_error(
  File "/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 108, in _missing_config_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: Primary config directory not found.
Check that the config directory '/Users/qingweilan/miniconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym_examples-0.2.4-py3.8.egg/llvm_autotuning/config' exists and readable
```

## Expected behavior

The script shouldn't fail.

## Environment

Please fill in this checklist:

- CompilerGym: latest development branch
- How you installed CompilerGym (pip, source): source
- OS: MacOS
- Python version: 3.7
- Build command you used (if compiling from source): make install
- GCC/clang version (if compiling from source):
- Versions of any other relevant libraries:

You may use the
[environment collection script](https://raw.githubusercontent.com/facebookresearch/CompilerGym/stable/build_tools/collect_env.py)
to generate most of this information. You can get the script and run it with:

```sh
wget https://raw.githubusercontent.com/facebookresearch/CompilerGym/stable/build_tools/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```



Discussion:

Resolving patch:
diff --git a/examples/setup.py b/examples/setup.py
--- a/examples/setup.py
+++ b/examples/setup.py
@@ -31,10 +31,14 @@
         "llvm_rl.model",
     ],
     package_data={
+        "llvm_autotuning": [
+            "config/*.yaml",
+            "config/**/*.yaml",
+        ],
         "llvm_rl": [
             "config/*.yaml",
             "config/**/*.yaml",
-        ]
+        ],
     },
     python_requires=">=3.8",
     platforms=[distutils.util.get_platform()],


Test patch:
diff --git a/examples/llvm_autotuning/config/benchmarks/single_benchmark_for_testing.yaml b/examples/llvm_autotuning/config/benchmarks/single_benchmark_for_testing.yaml
new file mode 100644
--- /dev/null
+++ b/examples/llvm_autotuning/config/benchmarks/single_benchmark_for_testing.yaml
@@ -0,0 +1,5 @@
+---
+# A single benchmark for
+benchmarks:
+    - uris:
+          - benchmark://cbench-v1/crc32
diff --git a/examples/llvm_autotuning/tests/integration_test.py b/examples/llvm_autotuning/tests/integration_test.py
new file mode 100644
--- /dev/null
+++ b/examples/llvm_autotuning/tests/integration_test.py
@@ -0,0 +1,28 @@
+# Copyright (c) Facebook, Inc. and its affiliates.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+import subprocess
+import sys
+from pathlib import Path
+
+
+def test_llvm_autotuner_integration_test(tmp_path: Path):
+    subprocess.check_call(
+        [
+            sys.executable,
+            "-m",
+            "llvm_autotuning.tune",
+            "-m",
+            "experiment=my-exp",
+            f"outputs={tmp_path}/llvm_autotuning",
+            "executor.cpus=1",
+            "num_replicas=1",
+            "autotuner=nevergrad",
+            "autotuner.optimization_target=codesize",
+            "autotuner.search_time_seconds=3",
+            "autotuner.algorithm_config.episode_length=5",
+            "benchmarks=single_benchmark_for_testing",
+        ]
+    )
+    assert (Path(tmp_path) / "llvm_autotuning/my-exp").is_dir()