Repository: google/fuzzbench

Issue:
expose BENCHMARK env variable to builder container
It would be useful to expose a BENCHMARK env variable to the runner. This way fuzzer.py can use it adjust the building code (fuzzer.py). For example: currently lafintel uses the PWD env variable to detect the php benchmark. Cleaner to have a BENCHMARK variable


Discussion:
I think we do provide it to the runner container, just not to the fuzzer.py when run in the *builder*.
Should we give oss-fuzz.yaml's project name instead of BENCHMARK variable ? Also, how to get right project name for non-oss-fuzz benchmark. i think this issue is just https://github.com/google/fuzzbench/issues/45
>Should we give oss-fuzz.yaml's project name instead of BENCHMARK variable? Also, how to get right project name for non-oss-fuzz benchmark.

Right, this would mean inconsistent behavior between OSS-Fuzz benchmarks and regular ones so I don't think there's a reason to give project name instead of the benchmark.

>Also, how to get right project name for non-oss-fuzz benchmark. i think this issue is just #45

I don't think it's quite the same though I don't really understand #45. We do give $BENCHMARK in the runner, we don't give it in the builder.
I think I made a mistake. I meant expose BENCHMARK in the builder.

>

also this needs fixing - https://github.com/google/fuzzbench/pull/193/files#diff-2549469637697094671972023bf99edd to not use PWD.
isnt this similar to #227 ? so maybe close 227?
@vanhauser-thc , #227 is about exposing fuzz target / fuzzer path as FUZZ_TARGET env variable. This one is about exposing benchmark name as BENCHMARK. Similar small bugs, but not same.
@inferno-chromium ok then what is which value? I would expect FUZZ_TARGET is libpng-1.2.56 and BENCHMARK is 2020-04-20-aflplusplus, but from the conversation it seems to be the opposite?

And are both variables already available?

FUZZ_TARGET is actually full path to fuzz target binary.
/out/<fuzzer_name>

BENCHMARK is like libpng-1.2.56

Some of these vars are not available during build process. And some like FUZZ_TARGET is only set for non oss-fuzz benchmarks. will try to fix this soon to avoid confusion.
This is already exposed to runner, see
https://github.com/google/fuzzbench/blob/master/experiment/resources/runner-startup-script-template.sh#L28

Need to fix builder.

Resolving patch:
diff --git a/experiment/build/gcb_build.py b/experiment/build/gcb_build.py
--- a/experiment/build/gcb_build.py
+++ b/experiment/build/gcb_build.py
@@ -66,6 +66,7 @@ def _build_oss_fuzz_project_fuzzer(benchmark: str,
     oss_fuzz_builder_hash = benchmark_utils.get_oss_fuzz_builder_hash(benchmark)
     substitutions = {
         '_OSS_FUZZ_PROJECT': project,
+        '_BENCHMARK': benchmark,
         '_FUZZER': fuzzer,
         '_OSS_FUZZ_BUILDER_HASH': oss_fuzz_builder_hash,
     }
@@ -99,6 +100,7 @@ def _build_oss_fuzz_project_coverage(benchmark: str) -> Tuple[int, str]:
     substitutions = {
         '_GCS_COVERAGE_BINARIES_DIR': coverage_binaries_dir,
         '_OSS_FUZZ_PROJECT': project,
+        '_BENCHMARK': benchmark,
         '_OSS_FUZZ_BUILDER_HASH': oss_fuzz_builder_hash,
     }
     config_file = get_build_config_file('oss-fuzz-coverage.yaml')
diff --git a/fuzzers/utils.py b/fuzzers/utils.py
--- a/fuzzers/utils.py
+++ b/fuzzers/utils.py
@@ -45,7 +45,10 @@ def build_benchmark(env=None):
     else:
         build_script = os.path.join('benchmark', 'build.sh')
 
-    print('Building benchmark')
+    benchmark = os.getenv('BENCHMARK')
+    fuzzer = os.getenv('FUZZER')
+    print('Building benchmark {benchmark} with fuzzer {fuzzer}'.format(
+        benchmark=benchmark, fuzzer=fuzzer))
     subprocess.check_call(['/bin/bash', '-ex', build_script], env=env)
 
 


Test patch: