Repository: bazelbuild/bazel
Issue: incompatible_sandbox_hermetic_tmp
Labels: type: process
incompatible-change
team-Local-Exec
migration-ready
breaking-change-7.0

Historically, on Linux, Bazel mounted the host machine's `/tmp` directory into each sandbox as `/tmp`. Since each sandbox maintains its own PID namespace, this [causes problems](https://github.com/bazelbuild/bazel/issues/3236) with actions that create files in `/tmp` using only the PID as a distinguishing element of the filename, e.g. well-known sockets.

With `--incompatible_sandbox_hermetic_tmp`, Bazel creates and later cleans up a dedicated, initially empty temporary directory for each sandboxed action on Linux.

**Migration:**  
If any actions in your build depend on access to the host's `/tmp` directory, for example to exchange data with non-hermetic daemons running on the host, you can either temporarily disable this new behavior via `--noincompatible_sandbox_hermetic_tmp` (not recommended as the flag will be removed in the future) or explicitly mount the host temporary directory via `--sandbox_add_mount_pair=/tmp`.

Issue comments:
@lberki I created this issue to get downstream coverage. Assuming CI is green, would you support flipping this for 7.0.0?
Flipping this flag SGTM. It looks like the incompatible changes pipeline didn't find any incompatibilities.

Pull request review comments:

Patch that resolved the issue:
diff --git a/src/main/java/com/google/devtools/build/lib/sandbox/SandboxOptions.java b/src/main/java/com/google/devtools/build/lib/sandbox/SandboxOptions.java
index 768c07511fd795..8833e8615d53a3 100644
--- a/src/main/java/com/google/devtools/build/lib/sandbox/SandboxOptions.java
+++ b/src/main/java/com/google/devtools/build/lib/sandbox/SandboxOptions.java
@@ -350,12 +350,12 @@ public ImmutableSet<Path> getInaccessiblePaths(FileSystem fs) {
 
   @Option(
       name = "incompatible_sandbox_hermetic_tmp",
-      defaultValue = "false",
+      defaultValue = "true",
       documentationCategory = OptionDocumentationCategory.EXECUTION_STRATEGY,
       effectTags = {OptionEffectTag.EXECUTION},
       help =
           "If set to true, each Linux sandbox will have its own dedicated empty directory mounted"
-              + " as /tmp rather thansharing /tmp with the host filesystem. Use"
+              + " as /tmp rather than sharing /tmp with the host filesystem. Use"
               + " --sandbox_add_mount_pair=/tmp to keep seeing the host's /tmp in all sandboxes.")
   public boolean sandboxHermeticTmp;
 


Test patch:
diff --git a/src/test/java/com/google/devtools/build/lib/buildtool/EditDuringBuildTest.java b/src/test/java/com/google/devtools/build/lib/buildtool/EditDuringBuildTest.java
index e7b1dc8a677bbc..b5d194d86c2db4 100644
--- a/src/test/java/com/google/devtools/build/lib/buildtool/EditDuringBuildTest.java
+++ b/src/test/java/com/google/devtools/build/lib/buildtool/EditDuringBuildTest.java
@@ -44,9 +44,8 @@ public void testEditDuringBuild() throws Exception {
     Path in = write("edit/in", "line1");
     in.setLastModifiedTime(123456789);
 
-    // Make in writable from sandbox (in case sandbox strategy is used).
-    String absoluteInPath = in.getPathString();
-    addOptions("--sandbox_writable_path=" + absoluteInPath);
+    // Modify the actual source file, not a sandboxed copy.
+    addOptions("--spawn_strategy=local");
 
     // The "echo" effects editing of the source file during the build:
     write("edit/BUILD",
diff --git a/src/test/shell/bazel/bazel_sandboxing_networking_test.sh b/src/test/shell/bazel/bazel_sandboxing_networking_test.sh
index 51b9fd8cd944d9..6a32f51bc64395 100755
--- a/src/test/shell/bazel/bazel_sandboxing_networking_test.sh
+++ b/src/test/shell/bazel/bazel_sandboxing_networking_test.sh
@@ -36,6 +36,8 @@ source ${CURRENT_DIR}/remote_helpers.sh \
 function set_up() {
   add_to_bazelrc "build --spawn_strategy=sandboxed"
   add_to_bazelrc "build --genrule_strategy=sandboxed"
+  # Allow the network socket to be seen in the sandbox.
+  add_to_bazelrc "build --sandbox_add_mount_pair=/tmp"
 
   sed -i.bak '/sandbox_tmpfs_path/d' $TEST_TMPDIR/bazelrc
 }
diff --git a/src/test/shell/integration/execution_strategies_test.sh b/src/test/shell/integration/execution_strategies_test.sh
index 94109cba33b919..6dc1e4f6164945 100755
--- a/src/test/shell/integration/execution_strategies_test.sh
+++ b/src/test/shell/integration/execution_strategies_test.sh
@@ -223,6 +223,7 @@ EOF
   bazel build --internal_spawn_scheduler --genrule_strategy=dynamic \
     --dynamic_remote_strategy=sandboxed \
     --dynamic_local_strategy=standalone \
+    --noincompatible_sandbox_hermetic_tmp \
     --experimental_dynamic_ignore_local_signals=8,9,10 \
     --experimental_local_lockfree_output \
     --experimental_local_execution_delay=0 \
diff --git a/src/test/shell/integration/sandboxing_test.sh b/src/test/shell/integration/sandboxing_test.sh
index f23d95c9977252..b6723aabe37804 100755
--- a/src/test/shell/integration/sandboxing_test.sh
+++ b/src/test/shell/integration/sandboxing_test.sh
@@ -735,6 +735,7 @@ EOF
 
   touch "${temp_dir}/file"
   bazel test //pkg:tmp_test \
+    --sandbox_add_mount_pair=/tmp \
     --test_output=errors &>$TEST_log || fail "Expected test to pass"
 }
 
@@ -812,6 +813,7 @@ EOF
   chmod +x pkg/tmp_test.sh
 
   bazel test //pkg:tmp_test \
+    --sandbox_add_mount_pair=/tmp \
     --test_output=errors &>$TEST_log || fail "Expected test to pass"
   [[ -f "${temp_dir}/file" ]] || fail "Expected ${temp_dir}/file to exist"
 }