Repository: mlcommons/inference
Issue: I try to run mlperf on GPU
Root cause: Environment Incompatibility

./run_local.sh gpu backend model

It does run but I saw the result and I saw nvidia-smi.
Result is same as CPU.
On nvidia-smi, there is no utilization of GPU.

I think even thought I put GPU in command line. It doesn't run with GPU.

Can you guys know why?

Which benchmark is this referring to?

@nvpohanh What do you mean which benchmark? 

I did tf with tensorflow backend~

There are two `run_local.sh` files...
https://github.com/mlperf/inference/blob/master/v0.5/recommendation/run_local.sh
https://github.com/mlperf/inference/blob/master/v0.5/classification_and_detection/run_local.sh

> There are two `run_local.sh` files...
> https://github.com/mlperf/inference/blob/master/v0.5/recommendation/run_local.sh
> https://github.com/mlperf/inference/blob/master/v0.5/classification_and_detection/run_local.sh

Hi~ I got the same problem
I want to run inference/recommendation
I ran this code：
./run_local.sh pytorch dlrm terabyte gpu --max-in-range=10000000
and I got this:
python3 python/main.py --profile dlrm-terabyte-pytorch --config ../mlperf.conf --model dlrm --model-path /root/mlperf/.5/recommendation/model/dlrm_terabyte.pytorch --dataset terabyte --dataset-path /root/mlperf/mlperf_inference/v0.5/recomeo --output /root/mlperf/mlperf_inference/v0.5/recommendation/output/pytorch-gpu/dlrm --use-gpu --max-ind-range=10000000
INFO:main:Namespace(accuracy=False, backend='pytorch-native', cache=0, config='../mlperf.conf', count_queries=None, couna_sub_sample_rate=0.0, dataset='terabyte', dataset_path='/root/mlperf/mlperf_inference/v0.5/recommendation/fake_criteo',d_peak_performance=False, inputs=['continuous and categorical features'], max_batchsize=2048, max_ind_range=10000000, ma_samples_to_aggregate=None, min_samples_to_aggregate=None, mlperf_bin_loader=False, model='dlrm', model_path='/root/mlpe/v0.5/recommendation/model/dlrm_terabyte.pytorch', numpy_rand_seed=123, output='/root/mlperf/mlperf_inference/v0.5/recomtorch-gpu/dlrm', outputs=['probability'], profile='dlrm-terabyte-pytorch', samples_per_query=None, samples_to_aggregate=gregate_quantile_file=None, samples_to_aggregate_trace_file='dlrm_trace_of_aggregated_samples.txt', scenario='SingleStree, test_num_workers=0, threads=8, use_gpu=True)
Using CPU...
How do I run it using a GPU?

I am facing the same issue. 
I ran mlperf with ```./run_local.sh onnxruntime resnet50 gpu --scenario SingleStream``` following this doc https://github.com/mlcommons/inference/blob/r0.5/v0.5/classification_and_detection/GettingStarted.ipynb.
But, still it running in CPU i found out from htop. 
Does some extra configuration required for running in GPU?

> I am facing the same issue.
> I ran mlperf with `./run_local.sh onnxruntime resnet50 gpu --scenario SingleStream` following this doc https://github.com/mlcommons/inference/blob/r0.5/v0.5/classification_and_detection/GettingStarted.ipynb.
> But, still it running in CPU i found out from htop.
> Does some extra configuration required for running in GPU?

I got the same issue. However, it was solved in the following way.
I installed the CPU version.So,  I tried uninstalling onnxruntime and install GPU version.
```
$ pip uninstall onnxruntime
$ pip install onnxruntime-gpu
```

Test
```
>>> import onnxruntime as ort
>>> ort.get_device()
'GPU'
```