Repository: mlcommons/inference
Issue: Build issue with the last commit (5e09cba298a3a35d21cf1c45e3a31d9c0f068206)
Root cause: Interface Contract Mismatch

Start test has been modified to accept 5 parameters, whereas not all callers are modified.

@shabanim  Can you add the link to the file where you encountered this and any other details?

[This commit](https://github.com/mlcommons/inference/commit/5e09cba298a3a35d21cf1c45e3a31d9c0f068206).

@pgmpablo157321 

I confirm that building LoadGen fails as follows:

<pre><b>/home/anton/CK-TOOLS/mlperf-inference-upstream.master/inference/loadgen/benchmark/repro.cpp:</b> In function ‘<b>int main(int, char**)</b>’:
<b>/home/anton/CK-TOOLS/mlperf-inference-upstream.master/inference/loadgen/benchmark/repro.cpp:297:63:</b> <font color="#DC322F"><b>error: </b></font>too few arguments to function ‘<b>void mlperf::StartTest(mlperf::SystemUnderTest*, mlperf::QuerySampleLibrary*, const mlperf::TestSettings&amp;, const mlperf::LogSettings&amp;, const string&amp;)</b>’
  297 |   mlperf::StartTest(sut.get(), &amp;qsl, testSettings, logSettings<font color="#DC322F"><b>)</b></font>;
      |                                                               <font color="#DC322F"><b>^</b></font>
In file included from <b>/home/anton/CK-TOOLS/mlperf-inference-upstream.master/inference/loadgen/benchmark/repro.cpp:25</b>:
<b>/home/anton/CK-TOOLS/mlperf-inference-upstream.master/inference/loadgen/loadgen.h:70:6:</b> <font color="#2AA198"><b>note: </b></font>declared here
   70 | void <font color="#2AA198"><b>StartTest</b></font>(SystemUnderTest* sut, QuerySampleLibrary* qsl,
      |      <font color="#2AA198"><b>^~~~~~~~~</b></font>
make[2]: *** [CMakeFiles/benchmark.dir/build.make:76: CMakeFiles/benchmark.dir/benchmark/repro.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:85: CMakeFiles/benchmark.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

</pre>

One quick way to fix it:
<pre><font color="#859900"><b>anton@krai</b></font>:<font color="#268BD2"><b>~/CK-TOOLS/mlperf-inference-upstream.master/inference</b></font>$ git diff
<b>diff --git a/loadgen/benchmark/repro.cpp b/loadgen/benchmark/repro.cpp</b>
<b>index 8b4bc8a..a098829 100644</b>
<b>--- a/loadgen/benchmark/repro.cpp</b>
<b>+++ b/loadgen/benchmark/repro.cpp</b>
<font color="#2AA198">@@ -294,7 +294,7 @@</font> int main(int argc, char** argv) {
 
   // Start test
   std::cout &lt;&lt; &quot;Start test...&quot; &lt;&lt; std::endl;
<font color="#DC322F">-  mlperf::StartTest(sut.get(), &amp;qsl, testSettings, logSettings);</font>
<font color="#859900">+  mlperf::StartTest(sut.get(), &amp;qsl, testSettings, logSettings, &quot;&quot;);</font>
   std::cout &lt;&lt; &quot;Test done. Clean up SUT...&quot; &lt;&lt; std::endl;
   sut.reset();
   std::cout &lt;&lt; &quot;Done!&quot; &lt;&lt; std::endl;
</pre>

Another:
<pre><font color="#859900"><b>anton@krai</b></font>:<font color="#268BD2"><b>~/CK-TOOLS/mlperf-inference-upstream.master/inference</b></font>$ git diff
<b>diff --git a/loadgen/loadgen.cc b/loadgen/loadgen.cc</b>
<b>index f709ab3..56ec478 100644</b>
<b>--- a/loadgen/loadgen.cc</b>
<b>+++ b/loadgen/loadgen.cc</b>
<font color="#2AA198">@@ -1535,7 +1535,7 @@</font> struct RunFunctions {
 void StartTest(SystemUnderTest* sut, QuerySampleLibrary* qsl,
                const TestSettings&amp; requested_settings,
                const LogSettings&amp; log_settings,
<font color="#DC322F">-               const std::string&amp; audit_config_filename) {</font>
<font color="#859900">+               const std::string audit_config_filename) {</font>
   GlobalLogger().StartIOThread();
 
   const std::string test_date_time = CurrentDateTimeISO8601();
<b>diff --git a/loadgen/loadgen.h b/loadgen/loadgen.h</b>
<b>index 3294764..1b71ad1 100644</b>
<b>--- a/loadgen/loadgen.h</b>
<b>+++ b/loadgen/loadgen.h</b>
<font color="#2AA198">@@ -70,7 +70,7 @@</font> void QuerySamplesComplete(QuerySampleResponse* responses, size_t response_count,
 void StartTest(SystemUnderTest* sut, QuerySampleLibrary* qsl,
                const TestSettings&amp; requested_settings,
                const LogSettings&amp; log_settings,
<font color="#DC322F">-               const std::string&amp; audit_config_filename);</font>
<font color="#859900">+               const std::string audit_config_filename=&quot;&quot;);</font>
 
 ///
 /// \brief Aborts the running test.
</pre>

With a simple StartTest search, you will find all instances of  StartTest.
Defined at loadgen.h/loadgen.cc (Binding files: c_api.h, x_api.cc, python_api.cc)
 No need to change above files though.

Need to change some callers in the followings:
perftests_null_sut.cc
basic.cc
repro.cpp

Hope this is of help. 

Yes, I noticed that I only modified some callers. I only modified the python and c apis

I think something like what Anton proposes will fix all the calls:

> Another:
> 
> anton@krai:~/CK-TOOLS/mlperf-inference-upstream.master/inference$ git diff
> diff --git a/loadgen/loadgen.cc b/loadgen/loadgen.cc
> index f709ab3..56ec478 100644
> --- a/loadgen/loadgen.cc
> +++ b/loadgen/loadgen.cc
> @@ -1535,7 +1535,7 @@ struct RunFunctions {
>  void StartTest(SystemUnderTest* sut, QuerySampleLibrary* qsl,
>                 const TestSettings& requested_settings,
>                 const LogSettings& log_settings,
> -               const std::string& audit_config_filename) {
> +               const std::string audit_config_filename) {
>    GlobalLogger().StartIOThread();
>  
>    const std::string test_date_time = CurrentDateTimeISO8601();
> diff --git a/loadgen/loadgen.h b/loadgen/loadgen.h
> index 3294764..1b71ad1 100644
> --- a/loadgen/loadgen.h
> +++ b/loadgen/loadgen.h
> @@ -70,7 +70,7 @@ void QuerySamplesComplete(QuerySampleResponse* responses, size_t response_count,
>  void StartTest(SystemUnderTest* sut, QuerySampleLibrary* qsl,
>                 const TestSettings& requested_settings,
>                 const LogSettings& log_settings,
> -               const std::string& audit_config_filename);
> +               const std::string audit_config_filename="");
>  
>  ///
>  /// \brief Aborts the running test.

However, the default should be "audit.config" so that it works like it did before it had the parameter. I will make the necessary changes


merged now