Repository: pytorch/benchmark
Issue: Ask help: fail to run benchmark with model yolov3 on cpu device and torchscript
Root cause: Validation Gap

(torchbenchmark) root@ip-172-31-0-218:~/benchmark# name="yolov3"
(torchbenchmark) root@ip-172-31-0-218:~/benchmark# python3 run_benchmark.py cpu --model ${name} --test eval --metrics=latencies,throughputs --backend=torchscript

Running benchmark: /root/miniconda3/envs/torchbenchmark/bin/python3 /root/benchmark/userbenchmark/cpu/run_config.py -m yolov3 -d cpu -t eval --backend=torchscript --metrics latencies,throughputs --nwarmup 20 --niter 50 -o /root/benchmark/.userbenchmark/cpu/cpu-20231124034334
/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:3526.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Running TorchBenchModelConfig(name='yolov3', test='eval', device='cpu', batch_size=None, extra_args=['--backend=torchscript'], extra_env=None) ...Traceback (most recent call last):
  File "/root/benchmark/userbenchmark/cpu/run_config.py", line 99, in <module>
    run(args, extra_args)
  File "/root/benchmark/userbenchmark/cpu/run_config.py", line 77, in run
    metrics_res = run_config(config, metrics, nwarmup=int(args.nwarmup), niter=int(args.niter), dryrun=args.dryrun)
  File "/root/benchmark/userbenchmark/cpu/run_config.py", line 55, in run_config
    model = load_model(config)
  File "/root/benchmark/torchbenchmark/util/experiment/instantiator.py", line 50, in load_model
    model_instance = Model(test=config.test, device=config.device, batch_size=config.batch_size, extra_args=config.extra_args)
  File "/root/benchmark/torchbenchmark/util/model.py", line 25, in __call__
    obj.__post__init__()
  File "/root/benchmark/torchbenchmark/util/model.py", line 136, in __post__init__
    apply_opt_args(self, self.opt_args)
  File "/root/benchmark/torchbenchmark/util/extra_args.py", line 139, in apply_opt_args
    model._enable_backend()
  File "/root/benchmark/torchbenchmark/util/backends/jit.py", line 36, in _torchscript
    module = torch.jit.script(module, example_inputs=[example_inputs, ])
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_script.py", line 1324, in script
    return torch.jit._recursive.create_script_module(
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 559, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 632, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_script.py", line 639, in _construct
    init_fn(script_module)
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 608, in init_fn
    scripted = create_script_module_impl(
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 632, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_script.py", line 639, in _construct
    init_fn(script_module)
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 608, in init_fn
    scripted = create_script_module_impl(
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 636, in create_script_module_impl
    create_methods_and_properties_from_stubs(
  File "/root/miniconda3/envs/torchbenchmark/lib/python3.10/site-packages/torch/jit/_recursive.py", line 469, in create_methods_and_properties_from_stubs
    concrete_type._create_methods_and_properties(
RuntimeError: 
Module 'WeightedFeatureFusion' has no attribute 'w' :
  File "/root/benchmark/torchbenchmark/models/yolov3/yolo_utils/layers.py", line 50
        # Weights
        if self.weight:
            w = torch.sigmoid(self.w) * (2 / self.n)  # sigmoid weights (0-1)
                              ~~~~~~ <--- HERE
            x = x * w[0]
    

(torchbenchmark) root@ip-172-31-0-218:~/benchmark# 

Duplicate with https://github.com/pytorch/benchmark/issues/2056, we do not guarantee the enablement of torchscript across all models. We only ensure the eager mode works.

Closing it, feel free to re-open if you still have questions.