Repository: ndsev/mapget (language: python)

Issue:
Arguments to mapget can be specified in a configuration file
Currently, starting mapget with custom data sources requires lengthy command line invocations. CLI11, the command line argument parser used by mapget, can be used to parse config files containing the arguments as documented [here](https://cliutils.github.io/CLI11/book/chapters/config.html).

The goal is for mapget to support reading arguments from a configuration file in addition to the command line, and to document this option. Existing integration tests triggered by CLI arguments should be extended to include one example config file.


Discussion / hints:

Fix patch:
diff --git a/README.md b/README.md
index 24d977e5..348c4766 100644
--- a/README.md
+++ b/README.md
@@ -15,9 +15,30 @@
 
 The `mapget` package is deployed to PyPI for any Python version between 3.8 and 3.11. Simply running `pip install mapget` is enough to get you started:
 
-* **`python -m mapget serve`** will run a server. Check out `--help` for further assistance.
-* **`python -m mapget fetch`** allows you to talk to a remote server. Check out `--help` for further assistance.
-* You can also use the Python package to write a data source, as documented [here](#implementing-a-data-source).
+* **`python -m mapget serve`** will run a server,
+* **`python -m mapget fetch`** allows you to talk to a remote server,
+* you can also use the Python package to write a data source, as documented [here](#implementing-a-data-source).
+
+If you build `mapget` from source as described below, you obtain an executable that can be used analogously to the Python package with `mapget serve` or `mapget fetch`. 
+
+## Configuration
+
+The command line parameters for `mapget` and its subcommands can be viewed with:
+
+```bash
+mapget --help
+mapget fetch --help
+mapget serve --help
+```
+
+(or `python -m mapget --help` for the Python package).
+
+The `mapget` executable can parse a config file with arguments supported by the command line interface. The path to the config file can be provided to `mapget` via command line by specifying the `--config` parameter.
+
+Sample configuration files can be found under `examples/config`:
+
+- [sample-first-datasource.toml](examples/config/sample-first-datasource.toml) and [sample-second-datasource.toml](examples/config/sample-second-datasource.toml) will configure mapget to run a simple datasource with sample data. Note: the two formats in config files for subcommand parameters can be used interchangeably.
+- [sample-service.toml](examples/config/sample-service.toml) to execute the `mapget serve` command. The instance will fetch and serve data from sources started with `sample-*-datasource.toml` configs above.
 
 ## Map Data Sources
 
@@ -243,16 +264,3 @@ void main(int argc, char const *argv[])
 ```
 
 Keep in mind, that you can also run a `mapget` service without any RPCs in your application. Check out [`examples/cpp/local-datasource`](examples/cpp/local-datasource/main.cpp) on how to do that.
-
-## Configuration
-
-*TODO: Implement*
-
-The `mapget::HttpService` class can parse a config file, from which it knows about available data source
-endpoints. The path to this config file may be provided either via the `MAPGET_CONFIG_FILE` env.
-var, or via a service command-line/constructor parameter. The structure of the file is as follows:
-
-```yaml
-sources:
-   - <url>  # e.g. http://localhost:12345
-```
diff --git a/examples/config/sample-first-datasource.toml b/examples/config/sample-first-datasource.toml
new file mode 100644
index 00000000..9da7d419
--- /dev/null
+++ b/examples/config/sample-first-datasource.toml
@@ -0,0 +1,13 @@
+# mapget: A client/server application for map data retrieval.
+
+# From: trace, debug, info, warn, error, critical. Overrides MAPGET_LOG_LEVEL.
+log-level="trace"
+
+# Server to connect to in format <host:port>.
+fetch.server="127.0.0.1:61852"
+# Map to retrieve.
+fetch.map="Tropico"
+# Layer of the map to retrieve.
+fetch.layer="WayLayer"
+# Tile of the map to retrieve. Can be specified multiple times.
+fetch.tile="12345"
diff --git a/examples/config/sample-second-datasource.toml b/examples/config/sample-second-datasource.toml
new file mode 100644
index 00000000..003e5b38
--- /dev/null
+++ b/examples/config/sample-second-datasource.toml
@@ -0,0 +1,14 @@
+# mapget: A client/server application for map data retrieval.
+
+# From: trace, debug, info, warn, error, critical. Overrides MAPGET_LOG_LEVEL.
+log-level="trace"
+
+[fetch]
+# Server to connect to in format <host:port>.
+server="127.0.0.1:61852"
+# Map to retrieve.
+map="TestMap"
+# Layer of the map to retrieve.
+layer="WayLayer"
+# Tile of the map to retrieve. Can be specified multiple times.
+tile="12345"
diff --git a/examples/config/sample-service.toml b/examples/config/sample-service.toml
new file mode 100644
index 00000000..94d45f5a
--- /dev/null
+++ b/examples/config/sample-service.toml
@@ -0,0 +1,16 @@
+# mapget: A client/server application for map data retrieval.
+
+# From: trace, debug, info, warn, error, critical. Overrides MAPGET_LOG_LEVEL.
+log-level="trace"
+
+# Port to start the server on. Default is 0.
+serve.port=61852
+
+# Datasources for the server in format <host:port>. Can be specified multiple times.
+serve.datasource-host=["127.0.0.1:61853", "127.0.0.1:61854"]
+
+# Datasource executable paths, including arguments, for the server. Can be specified multiple times.
+# serve.datasource-exe=
+
+# Serve a static web application, in the format [<url-scope>:]<filesystem-path>.
+# serve.webapp=
diff --git a/libs/http-datasource/src/http-server.cpp b/libs/http-datasource/src/http-server.cpp
index 213ddae5..9d937c99 100644
--- a/libs/http-datasource/src/http-server.cpp
+++ b/libs/http-datasource/src/http-server.cpp
@@ -70,7 +70,7 @@ void HttpServer::go(
         [this, interfaceAddr]
         {
             if (impl_->printPortToStdout_)
-                std::cout << "====== Running on port " << impl_->port_ << "======" << std::endl;
+                std::cout << "====== Running on port " << impl_->port_ << " ======" << std::endl;
             else
                 log().info("====== Running on port {} ======", impl_->port_);
             impl_->server_.listen_after_bind();
diff --git a/libs/http-service/src/cli.cpp b/libs/http-service/src/cli.cpp
index 7aa00561..90ef4e07 100644
--- a/libs/http-service/src/cli.cpp
+++ b/libs/http-service/src/cli.cpp
@@ -127,7 +127,9 @@ int runFromCommandLine(std::vector<std::string> args)
         log_level_,
         "From: trace, debug, info, warn, error, critical. Overrides MAPGET_LOG_LEVEL."
     )->default_val("");
-    app.require_subcommand(1); // Require at least one subcommand.
+    app.set_config("--config", "", "Optional path to a file with configuration arguments for mapget.");
+
+    app.require_subcommand(1);
 
     if (!log_level_.empty()) {
         mapget::setLogLevel(log_level_, log());


Test patch:
diff --git a/test/integration/CMakeLists.txt b/test/integration/CMakeLists.txt
index f8c825a3..8937a31f 100644
--- a/test/integration/CMakeLists.txt
+++ b/test/integration/CMakeLists.txt
@@ -60,3 +60,43 @@ add_wheel_test(test-cli-python
     # Request from py datasource
     -f "python -m mapget --log-level trace fetch -s 127.0.0.1:${MAPGET_SERVER_PORT} -m TestMap -l WayLayer -t 12345"
   )
+
+add_wheel_test(test-config-cpp
+  WORKING_DIRECTORY
+    "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"
+  COMMANDS
+    # Run Python datasource
+    -b "python ${CMAKE_CURRENT_LIST_DIR}/../../examples/python/datasource.py ${DATASOURCE_PY_PORT}"
+
+    # Run C++ datasource
+    -b "./cpp-sample-http-datasource ${DATASOURCE_CPP_PORT}"
+
+    # Run service
+    -b "./mapget --config ${CMAKE_CURRENT_LIST_DIR}/../../examples/config/sample-service.toml serve"
+
+    # Request from py datasource
+    -f "./mapget --config ${CMAKE_CURRENT_LIST_DIR}/../../examples/config/sample-second-datasource.toml fetch"
+
+    # Request from cpp datasource
+    -f "./mapget --config ${CMAKE_CURRENT_LIST_DIR}/../../examples/config/sample-first-datasource.toml fetch"
+  )
+
+add_wheel_test(test-config-py
+  WORKING_DIRECTORY
+    "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"
+  COMMANDS
+    # Run Python datasource
+    -b "python ${CMAKE_CURRENT_LIST_DIR}/../../examples/python/datasource.py ${DATASOURCE_PY_PORT}"
+
+    # Run C++ datasource
+    -b "./cpp-sample-http-datasource ${DATASOURCE_CPP_PORT}"
+
+    # Run service
+    -b "python -m mapget --config ${CMAKE_CURRENT_LIST_DIR}/../../examples/config/sample-service.toml serve"
+
+    # Request from py datasource
+    -f "python -m mapget --config ${CMAKE_CURRENT_LIST_DIR}/../../examples/config/sample-second-datasource.toml fetch"
+
+    # Request from cpp datasource
+    -f "python -m mapget --config ${CMAKE_CURRENT_LIST_DIR}/../../examples/config/sample-first-datasource.toml fetch"
+)