Repository: mlcommons/inference
Issue: Determine whether mlperf.conf contains prescriptive or submitter-set values
Root cause: Documentation Deficiency

The current sample mlperf.conf contains both rule-prescribed settings (such as SingleStream.target_latency_percentile) and submitter-set values (such as SingleStream.target_latency) mixed together in the same file.

Possible solutions: "do not modify this section", two files, ...

Since we are tight on schedule, I propose we add the following comment and keep the file as it is.
What do you think @guschmue @briandersn although there is one item I'm not sure.

*.SingleStream.target_latency = 10 # user set to ~1000/latency in milli sec
*.SingleStream.target_latency_percentile = 90 # do not change this
*.SingleStream.min_duration = 60000 # do not change this
*.SingleStream.min_query_count = 1024 # do not change this

*.MultiStream.target_qps = 1.0 # user set to ~1000/latency in milli sec
*.MultiStream.target_latency_percentile = 99 # do not change this
*.MultiStream.samples_per_query = 4 # user input
*.MultiStream.max_async_queries = 1 # do not change this
*.MultiStream.target_latency = 50 # do not change this
*.MultiStream.min_duration = 60000 # do not change this
*.MultiStream.min_query_count = 270336 # do not change this
ssd-resnet34.MultiStream.target_latency = 66 # do not change this
gnmt.MultiStream.min_query_count = 90112 # do not change this
gnmt.MultiStream.target_latency = 100 #  do not change this
gnmt.MultiStream.target_latency_percentile = 97 # do not change this

*.Server.target_qps = 1.0  # user set to ~1000/latency in milli sec
*.Server.target_latency = 10 # (not sure)
*.Server.target_latency_percentile = 99 # do not change this
*.Server.target_duration = 0 # do not change this
*.Server.min_duration = 60000 # do not change this
*.Server.min_query_count = 270336 # do not change this
resnet50.Server.target_latency = 15 # do not change this
ssd-resnet34.Server.target_latency = 100 # do not change this
gnmt.Server.min_query_count = 90112 # do not change this
gnmt.Server.target_latency = 250 # do not change this
gnmt.Server.target_latency_percentile = 97 # do not change this

*.Offline.target_qps = 1.0 # user set to ~1000/latency in milli sec
*.Offline.target_latency_percentile = 90 # do not change this
*.Offline.min_duration = 60000 # do not change this
*.Offline.min_query_count = 1 # do not change this


shall we just use mlperf.conf as proposed above instead of introducing mlperf.conf + user.conf. for v0.5 only?

It works today to use mlperf.conf and load a user.conf that overrides sections of it. The reference app does not make use of this because it overrides it from the command line.
The overrides would be all none compliant and for testing I think.

@guschmue how to use 2 conf files in cpp code? Is below order correct?
```
mlperf_args.FromConfig(mperf_config_file_name, model_name, scenario_name);
mlperf_args.FromConfig(user_config_file_name, model_name, scenario_name);
```

@guschmue and shall the one at [here](https://github.com/mlperf/inference/blob/master/v0.5/mlperf.conf) be used for submission?

yes, the order is correct, every FromConfig() will just override the field found in the specific config file.
I'd use the unmodified mlperf.conf and a user.conf with your local changes and include both in the submission. 

Close w/ recommendation to load a second config file in your SUT-LoadGen integration