Repository: mlcommons/inference
Issue: BERT make build_docker fails due to pybind dependency
Root cause: External Dependency Breakage

### BERT Docker file:
https://github.com/mlcommons/inference/blob/0ed5190e82e1ca1cf95626e50e61cb6678c7f2ff/language/bert/Dockerfile#L23



The above file expects the pybind to be listed in the submodules for git but this was removed as part of this MR:
https://github.com/mlcommons/inference/pull/1501


Yes, that's correct - those Dockerfiles are outdated. If you're trying the reference implementation it is better to follow the [CM README](https://github.com/mlcommons/inference/blob/master/language/bert/README_cm.md)

Thanks for the quick response @arjunsuresh. Does cm provide a way to run network benchmarks as well? From the documentation I have seen open and closed divisions.

You're welcome @rgandikota . Yes `cm` does support network mode but the network code for `bert` was added recently and so is not documented yet in the `CM readme`. But you can find the `cm` commands to try network mode [here](https://github.com/mlcommons/inference/tree/master/language/bert#loadgen-over-the-network). Reference network code is currently available only for bert-onnxruntime and bert-pytorch and should run on x86 or ARM CPUs, Nvidia and AMD GPUs.

Also, from `v4.0` (submission deadline on February 23, 2024) `network` will no longer be a separate division but will be another `scenario` like 'Offline' and 'Server'. 

Thank you @arjunsuresh, i'll give it a shot.
Also, when I try to build using this cm: https://github.com/mlcommons/inference/tree/master/language/bert#loadgen-over-the-network
I keep running into this issue with loadgen. Is this something you have seen before?

running bdist_wheel
running build
running build_ext
/opt/nvidia/hpc_sdk/Linux_x86_64/23.11/compilers/bin/nvc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -std=c++14 -O3 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/mnt/nfsshare/tools/ram/inference/language/bert/cm/include -I/usr/include/python3.10 -c flagcheck.cpp -o flagcheck.o -std=c++17
nvc-Error-Unknown switch: -Wno-unused-result
nvc-Error-Unknown switch: -fwrapv
nvc-Error-Unknown switch: -fstack-protector-strong
nvc-Error-Unknown switch: -fwrapv
nvc-Error-Unknown switch: -Wdate-time
/opt/nvidia/hpc_sdk/Linux_x86_64/23.11/compilers/bin/nvc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -std=c++14 -O3 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/mnt/nfsshare/tools/ram/inference/language/bert/cm/include -I/usr/include/python3.10 -c flagcheck.cpp -o flagcheck.o -std=c++14
nvc-Error-Unknown switch: -Wno-unused-result
nvc-Error-Unknown switch: -fwrapv
nvc-Error-Unknown switch: -fstack-protector-strong
nvc-Error-Unknown switch: -fwrapv
nvc-Error-Unknown switch: -Wdate-time
/opt/nvidia/hpc_sdk/Linux_x86_64/23.11/compilers/bin/nvc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -std=c++14 -O3 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/mnt/nfsshare/tools/ram/inference/language/bert/cm/include -I/usr/include/python3.10 -c flagcheck.cpp -o flagcheck.o -std=c++11
nvc-Error-Unknown switch: -Wno-unused-result
nvc-Error-Unknown switch: -fwrapv
nvc-Error-Unknown switch: -fstack-protector-strong
nvc-Error-Unknown switch: -fwrapv
nvc-Error-Unknown switch: -Wdate-time
Traceback (most recent call last):
  File "/home/rakesh/CM/repos/local/cache/86c56a07b4df430d/inference/loadgen/setup.py", line 84, in <module>
    setup(name="mlperf_loadgen",
  File "/mnt/nfsshare/tools/ram/inference/language/bert/cm/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/mnt/nfsshare/tools/ram/inference/language/bert/cm/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 368, in run
    self.run_command("build")
  File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/mnt/nfsshare/tools/ram/inference/language/bert/cm/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/mnt/nfsshare/tools/ram/inference/language/bert/cm/lib/python3.10/site-packages/pybind11/setup_helpers.py", line 283, in build_extensions
    ext.cxx_std = auto_cpp_level(self.compiler)
  File "/mnt/nfsshare/tools/ram/inference/language/bert/cm/lib/python3.10/site-packages/pybind11/setup_helpers.py", line 266, in auto_cpp_level
    raise RuntimeError(msg)
RuntimeError: Unsupported compiler -- at least C++11 support is needed!

Actually no. Is this happening inside Nvidia docker container? You can add `--adr.compiler.tags=gcc` to pick the system installed gcc.