Repository: mlcommons/inference
Issue: CM error: no scripts were found with above tags and variations
Root cause: Unimplemented Feature Gap

Hi, I am kind of new to MLPerf and trying to use MLPerf Inference v4.0 NVIDIA-Optimized Implementations on a gh200.

Steps:
python3 -m pip install cmind -U
cm test core
cm pull repo cknowledge@cm4mlops --branch=mlperf-inference
cm rm cache -f

  cm run script --tags=run-mlperf,inference,_performance-only,_short     --division=closed    --category=datacenter    --device=cuda    --model=gptj-99    --implementation=nvidia    --backend=tensorrt    --scenario=Offline    --execution_mode=test    --power=no    --adr.python.version_min=3.8    --clean    --compliance=no    --quiet    --time

Keep getting this error:
 
  * cm run script "reproduce mlperf nvidia inference _run_harness _offline _gptj-99 _cuda _tensorrt _gpu_memory.96"

CM error: no scripts were found with above tags and variations

variation tags ['run_harness', 'cuda', 'gptj-99', 'offline', 'tensorrt', 'gpu_memory.96'] are not matching for the found script app-mlperf-inference-nvidia with variations dict_keys(['v4.0', 'v3.1', 'cpu', 'cuda', 'tensorrt', 'resnet50', 'retinanet', 'sdxl', 'bert_', 'bert-99', 'bert-99.9', '3d-unet_', '3d-unet-99', '3d-unet-99.9', 'rnnt', 'dlrm_', 'dlrm-v2-99', 'dlrm-v2-99.9', 'gptj_', 'gptj_,build', 'gptj_,build_engine', 'gptj-99', 'gptj-99.9', 'batch_size.#', 'dla_batch_size.#', 'use_triton', 'prebuild', 'build', 'maxq', 'maxn', 'preprocess-data', 'preprocess_data', 'download-model', 'download_model', 'calibrate', 'build-engine', 'build_engine', 'singlestream', 'multistream', 'offline', 'server', 'run-harness', 'run_harness', 'build_engine_options.#', 'gpu_memory.16', 'gpu_memory.24', 'gpu_memory.8', 'gpu_memory.32', 'gpu_memory.40', 'gpu_memory.48', 'gpu_memory.80', 'singlestream,resnet50', 'multistream,resnet50', 'singlestream,run_harness', 'gptj_,run_harness', 'gpu_memory.16,gptj_,offline,run_harness', 'gpu_memory.24,gptj_,offline,run_harness', 'gpu_memory.32,gptj_,offline,run_harness', 'gpu_memory.48,gptj_,offline,run_harness', 'gpu_memory.40,gptj_,offline,run_harness', 'gpu_memory.80,gptj_,offline,run_harness', 'gpu_memory.16,sdxl,offline,run_harness', 'gpu_memory.24,sdxl,offline,run_harness', 'gpu_memory.32,sdxl,offline,run_harness', 'gpu_memory.80,sdxl,offline,run_harness', 'gpu_memory.96,sdxl,offline,run_harness', 'gpu_memory.96,sdxl,server,run_harness', 'gpu_memory.80,sdxl,server,run_harness', 'gpu_memory.140,sdxl,offline,run_harness', 'gpu_memory.16,bert_,offline,run_harness', 'gpu_memory.24,bert_,offline,run_harness', 'gpu_memory.32,bert_,offline,run_harness', 'gpu_memory.48,bert_,offline,run_harness', 'gpu_memory.40,bert_,offline,run_harness', 'gpu_memory.80,bert_,server,run_harness', 'gpu_memory.16,resnet50,offline,run_harness', 'gpu_memory.40,resnet50,offline,run_harness', 'gpu_memory.24,resnet50,offline,run_harness', 'gpu_memory.32,resnet50,offline,run_harness', 'gpu_memory.48,resnet50,offline,run_harness', 'gpu_memory.80,resnet50,offline,run_harness', 'num-gpus.#', 'num-gpus.1', 'resnet50,server,run_harness', 'resnet50,multistream,run_harness,num-gpus.1', 'resnet50,multistream,run_harness,num-gpus.2', 'retinanet,multistream,run_harness', 'gpu_memory.16,retinanet,offline,run_harness', 'gpu_memory.40,retinanet,offline,run_harness', 'gpu_memory.32,retinanet,offline,run_harness', 'gpu_memory.48,retinanet,offline,run_harness', 'gpu_memory.24,retinanet,offline,run_harness', 'gpu_memory.80,retinanet,offline,run_harness', 'retinanet,server,run_harness', 'gpu_memory.16,rnnt,offline,run_harness', 'gpu_memory.40,rnnt,offline,run_harness', 'gpu_memory.24,rnnt,offline,run_harness', 'gpu_memory.32,rnnt,offline,run_harness', 'gpu_memory.48,rnnt,offline,run_harness', 'gpu_memory.80,rnnt,offline,run_harness', 'gpu_memory.16,3d-unet_,offline,run_harness', 'gpu_memory.40,3d-unet_,offline,run_harness', 'gpu_memory.24,3d-unet_,offline,run_harness', 'gpu_memory.80,3d-unet_,offline,run_harness', 'gpu_memory.32,3d-unet_,offline,run_harness', 'gpu_memory.48,3d-unet_,offline,run_harness', 'gpu_memory.16,dlrm_,offline,run_harness', 'gpu_memory.40,dlrm_,offline,run_harness', 'gpu_memory.24,dlrm_,offline,run_harness', 'gpu_memory.32,dlrm_,offline,run_harness', 'gpu_memory.48,dlrm_,offline,run_harness', 'gpu_memory.80,dlrm_,offline,run_harness', 'orin', 'orin,rnnt,singlestream,run_harness', 'orin,sdxl,offline,run_harness', 'rtx_4090', 'rtx_4090,sdxl,offline,run_harness', 'rtx_4090,sdxl,server,run_harness', 'rtx_4090,resnet50,offline,run_harness', 'rtx_4090,resnet50,server,run_harness', 'rtx_4090,retinanet,offline,run_harness', 'rtx_4090,retinanet,server,run_harness', 'rtx_4090,bert_,offline,run_harness', 'rtx_4090,bert_,server,run_harness', 'rtx_4090,3d-unet_,offline,run_harness', 'rtx_4090,3d-unet_,server,run_harness', 'rtx_4090,rnnt,offline,run_harness', 'rtx_4090,rnnt,server,run_harness', 'rtx_4090,gptj_,offline,run_harness', 'rtx_4090,gptj_,server,run_harness', 'rtx_4090,dlrm_,offline,run_harness', 'a6000', 'rtx_a6000,resnet50,offline,run_harness', 'rtx_a6000,resnet50,server,run_harness', 'rtx_a6000,retinanet,offline,run_harness', 'rtx_a6000,retinanet,server,run_harness', 'rtx_a6000,bert_,offline,run_harness', 'rtx_a6000,bert_,server,run_harness', 'rtx_a6000,3d-unet_,offline,run_harness', 'rtx_a6000,3d-unet_,server,run_harness', 'rtx_a6000,rnnt,offline,run_harness', 'rtx_a6000,rnnt,server,run_harness', 'rtx_a6000,dlrm_,offline,run_harness', 'rtx_6000_ada', 'rtx_6000_ada,resnet50,offline,run_harness', 'rtx_6000_ada,resnet50,server,run_harness', 'rtx_6000_ada,retinanet,offline,run_harness', 'rtx_6000_ada,retinanet,server,run_harness', 'rtx_6000_ada,bert_,offline,run_harness', 'rtx_6000_ada,bert_,server,run_harness', 'rtx_6000_ada,3d-unet_,offline,run_harness', 'rtx_6000_ada,3d-unet_,server,run_harness', 'rtx_6000_ada,rnnt,offline,run_harness', 'rtx_6000_ada,rnnt,server,run_harness', 'rtx_6000_ada,dlrm_,offline,run_harness', 'l4', 'l4,sdxl,offline,run_harness', 'l4,sdxl,offline,run_harness,num-gpu.8', 'l4,sdxl,server,run_harness,num-gpu.1', 'l4,sdxl,server,run_harness,num-gpu.8', 'l4,resnet50', 'l4,resnet50,offline,run_harness', 'l4,resnet50,server,run_harness', 'l4,retinanet,offline,run_harness', 'l4,retinanet,server,run_harness', 'l4,bert_,offline,run_harness', 'l4,bert_,server,run_harness', 'l4,3d-unet_,offline,run_harness', 'l4,rnnt,offline,run_harness', 'l4,rnnt,server,run_harness', 'l4,dlrm_,offline,run_harness', 't4', 't4,resnet50', 't4,resnet50,offline,run_harness', 't4,resnet50,server,run_harness', 't4,retinanet,offline,run_harness', 't4,retinanet,server,run_harness', 't4,bert_,offline,run_harness', 't4,bert_,server,run_harness', 't4,3d-unet_,offline,run_harness', 't4,rnnt,offline,run_harness', 't4,rnnt,server,run_harness', 't4,dlrm_,offline,run_harness', 'pcie', 'sxm', 'custom', 'a100', 'a100,sxm,resnet50,offline,run_harness', 'a100,sxm,retinanet,offline,run_harness', 'a100,sxm,bert_,offline,run_harness', 'a100,sxm,3d-unet_,offline,run_harness', 'a100,sxm,rnnt,offline,run_harness', 'a100,sxm,dlrm_,offline,run_harness'])

Thanks for the help.

Hi @kraza8 can you please retry this now? We have just added the support for 96GB Nvidia GPUs. 

Please do `cm pull repo` to get the updated changes and add `--docker_cache=no` to the cm run command. 

Thanks Arjun, will give that a try.