Repository: woocommerce/woocommerce-android
Pull request #14034: [Tooling] Skip build / validation jobs depending on the PR changes
Author: login: iangmaia
Labels: name: type: enhancement
description: A request for an enhancement.
name: category: tooling
description: Anything that involves building & maintaining the project, including scripts, `Fastfile`, etc.
Created: 2025-05-09T17:48:10Z | Merged: 2025-05-15T14:17:18Z

Internal reference: paaHJt-7Ss-p2

The main goal of this PR is to optimize the wait time for PRs with minor, non-code related changes, specially the ones related to the release process.

I've updated the original script to cover build, validation and localization jobs:
- `should-skip-job.sh --job-type [build|lint]`: skips if the changes are limited to documentation, tooling, metadata, etc
- `should-skip-job.sh --job-type validation`: same as above, but also skip on localization changes. The idea is that it can be valuable to run things as a Prototype build when localization changes

## Testing

I've created the PR https://github.com/woocommerce/woocommerce-android/pull/14035 to run a few tests in CI:
- Version bump https://buildkite.com/automattic/woocommerce-android/builds/29095
- Update release notes https://buildkite.com/automattic/woocommerce-android/builds/29098
- Update localized strings https://buildkite.com/automattic/woocommerce-android/builds/29100
- **Update**: Same as above, but with the latest changes adding `--job-type` https://buildkite.com/automattic/woocommerce-android/builds/29137


Diff:
diff --git a/.buildkite/commands/diff-merged-manifest.sh b/.buildkite/commands/diff-merged-manifest.sh
index 23bef5a71398..43d5582aebdf 100755
--- a/.buildkite/commands/diff-merged-manifest.sh
+++ b/.buildkite/commands/diff-merged-manifest.sh
@@ -2,6 +2,11 @@
 
 set -euo pipefail
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --job-type validation; then
+  exit 0
+fi
+
 "$(dirname "${BASH_SOURCE[0]}")/restore-cache.sh"
 
 BUILD_VARIANT=$1
diff --git a/.buildkite/commands/gradle-cache-build.sh b/.buildkite/commands/gradle-cache-build.sh
index 71eb1ac597d3..322de2857604 100755
--- a/.buildkite/commands/gradle-cache-build.sh
+++ b/.buildkite/commands/gradle-cache-build.sh
@@ -1,5 +1,10 @@
 #!/bin/bash -eu
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --job-type build; then
+  exit 0
+fi
+
 # This script is used to populate Gradle's build cache with task outputs that can be reused
 # by the local machine.
 
diff --git a/.buildkite/commands/lint.sh b/.buildkite/commands/lint.sh
index 4e68a7857e08..90ee1a3da192 100755
--- a/.buildkite/commands/lint.sh
+++ b/.buildkite/commands/lint.sh
@@ -1,5 +1,10 @@
 #!/bin/bash -u
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --job-type lint; then
+  exit 0
+fi
+
 "$(dirname "${BASH_SOURCE[0]}")/restore-cache.sh"
 
 echo "--- 🧹 Linting"
diff --git a/.buildkite/commands/prototype-build.sh b/.buildkite/commands/prototype-build.sh
index ad9b8f7cf19c..c4054f8ba101 100755
--- a/.buildkite/commands/prototype-build.sh
+++ b/.buildkite/commands/prototype-build.sh
@@ -1,5 +1,10 @@
 #!/bin/bash -eu
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --job-type build; then
+  exit 0
+fi
+
 "$(dirname "${BASH_SOURCE[0]}")/restore-cache.sh"
 
 APP_TO_BUILD="${1?You need to specify the app to build, WooCommerce or WooCommerce-Wear}"
diff --git a/.buildkite/commands/run-instrumented-tests.sh b/.buildkite/commands/run-instrumented-tests.sh
index aba1ddde8771..a4dbd8f51738 100755
--- a/.buildkite/commands/run-instrumented-tests.sh
+++ b/.buildkite/commands/run-instrumented-tests.sh
@@ -1,5 +1,11 @@
 #!/bin/bash -eu
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --job-type validation; then
+  mkdir -p WooCommerce/build/buildkite-test-analytics && touch WooCommerce/build/buildkite-test-analytics/empty.xml
+  exit 0
+fi
+
 "$(dirname "${BASH_SOURCE[0]}")/restore-cache.sh"
 
 echo "--- :rubygems: Setting up Gems"
diff --git a/.buildkite/commands/run-unit-tests.sh b/.buildkite/commands/run-unit-tests.sh
index ba9413a8f32f..e2e3fce5016a 100755
--- a/.buildkite/commands/run-unit-tests.sh
+++ b/.buildkite/commands/run-unit-tests.sh
@@ -1,5 +1,11 @@
 #!/bin/bash -eu
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --job-type validation; then
+  mkdir -p WooCommerce/build/buildkite-test-analytics && touch WooCommerce/build/buildkite-test-analytics/empty.xml
+  exit 0
+fi
+
 "$(dirname "${BASH_SOURCE[0]}")/restore-cache.sh"
 
 echo "--- :rubygems: Setting up Gems"
diff --git a/.buildkite/commands/should-skip-job.sh b/.buildkite/commands/should-skip-job.sh
new file mode 100755
index 000000000000..166750e04f5f
--- /dev/null
+++ b/.buildkite/commands/should-skip-job.sh
@@ -0,0 +1,74 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|lint|build]
+# --job-type validation: For jobs like unit/instrumented tests, manifest validation…
+#     Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --job-type lint: For jobs doing linting, especially ones that might include linting of localization files
+#     Skip when changes are limited to documentation, tooling, and non-code files.
+#     Does not skip if localization files changed (so that linter can run to report translation inconsistencies)
+# --job-type build: For jobs building an apk binary, e.g. Assemble, Prototype Builds…
+#     Skip when changes are limited to documentation, tooling, and non-code files
+#
+# Return codes:
+# 0 - Job should be skipped (script also handles displaying the message and annotation)
+# 1 - Job should not be skipped
+# 15 - Error in script parameters
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+# Check if arguments are valid
+if [ -z "${1:-}" ] || [ "$1" != "--job-type" ] || [ -z "${2:-}" ]; then
+  echo "Error: Must specify --job-type [validation|build|lint]"
+  buildkite-agent step cancel
+  exit 15
+fi
+
+# Function to display skip message and create annotation
+show_skip_message() {
+  local job_type=$1
+  local message="Skipping ${BUILDKITE_LABEL:-Job} - no relevant files changed"
+  local context="skip-$(echo "${BUILDKITE_LABEL:-$job_type}" | sed -E -e 's/[^[:alnum:]]+/-/g' | tr A-Z a-z)"
+  
+  echo "$message" | buildkite-agent annotate --style "info" --context "$context"
+  echo "$message"
+}
+
+job_type="$2"
+case "$job_type" in
+  "validation")
+    # We should skip if changes are limited to documentation, tooling, non-code files, and localization files
+    PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+    if pr_changed_files --all-match "${PATTERNS[@]}"; then
+      show_skip_message "$job_type"
+      exit 0
+    fi
+    exit 1
+    ;;
+  "build"|"lint")
+    # We should skip if changes are limited to documentation, tooling, and non-code files
+    # We'll let the job run (won't skip) if PR includes changes in localization files though
+    PATTERNS=("${COMMON_PATTERNS[@]}")
+    if pr_changed_files --all-match "${PATTERNS[@]}"; then
+      show_skip_message "$job_type"
+      exit 0
+    fi
+    exit 1
+    ;;
+  *)
+    echo "Error: Job type must be either 'validation', 'build', or 'lint'"
+    buildkite-agent step cancel
+    exit 15
+    ;;
+esac
diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml
index 5547a08e657f..a91c9a4ced48 100644
--- a/.buildkite/pipeline.yml
+++ b/.buildkite/pipeline.yml
@@ -35,7 +35,11 @@ steps:
           queue: "linter"
 
       - label: "detekt"
-        command: ./gradlew detektAll
+        command: |
+          if .buildkite/commands/should-skip-job.sh --job-type validation; then
+            exit 0
+          fi
+          ./gradlew detektAll
         plugins: [$CI_TOOLKIT]
         artifact_paths:
           - "**/build/reports/detekt/detekt.html"
@@ -100,5 +104,5 @@ steps:
 
 
   - label: "🐘 Populate Gradle build cache"
-    command: ".buildkite/commands/gradle-cache-build.sh"
+    command: .buildkite/commands/gradle-cache-build.sh
     plugins: [$CI_TOOLKIT]
diff --git a/.buildkite/shared-pipeline-vars b/.buildkite/shared-pipeline-vars
index 5ba965925834..4b323dab8031 100644
--- a/.buildkite/shared-pipeline-vars
+++ b/.buildkite/shared-pipeline-vars
@@ -3,5 +3,5 @@
 # This file is `source`'d before calling `buildkite-agent pipeline upload`, and can be used
 # to set up some variables that will be interpolated in the `.yml` pipeline before uploading it.
 
-export CI_TOOLKIT="automattic/a8c-ci-toolkit#5.1.3"
+export CI_TOOLKIT="automattic/a8c-ci-toolkit#5.3.1"
 export TEST_COLLECTOR="test-collector#v1.10.1"


Reviews:
user: login: AliSoftware
body: Overall LGTM, my only notes are on the naming and how we could make the parameters more clear what they are for to make reading the logic less potentially ambiguous.

I'd vote for `--job-type validation` vs `--job-type build` for added clarity, but I've left a couple of other options and brain dump to consider in PR comments.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T12:54:43Z
user: login: iangmaia
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T13:49:52Z
user: login: iangmaia
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T13:53:55Z
user: login: iangmaia
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T15:10:03Z
user: login: iangmaia
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T15:12:06Z
user: login: iangmaia
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T15:17:57Z
user: login: AliSoftware
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T16:22:15Z
user: login: iangmaia
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T18:48:18Z
user: login: AliSoftware
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T19:26:09Z

Review comments:
diff_hunk: @@ -1,5 +1,13 @@
 #!/bin/bash -u
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --validation; then
path: .buildkite/commands/lint.sh
user: login: AliSoftware
body: This is debatable, depending on what the `lint` gradle task does.

In particular, if as part of its linting, it reports warnings and/or errors on localization inconsistencies (e.g. a key using `%d` in `values/strings.xml` but `%f` in `values-pt/strings.xml`…)… then we'll probably still want to run this on PRs which integrate new translations, i.e. PRs in which the only changes are `**/strings.xml` files
created_at: 2025-05-12T12:26:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,32 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh [--validation|--build]
+# --validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --build: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+if [ "$1" = "--validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
+  PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+elif [ "$1" = "--build" ]; then
+  # Check if changes are limited to documentation, tooling, and non-code files (NOT localization files)
+  PATTERNS=("${COMMON_PATTERNS[@]}")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+else
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: 🧠  Brainstorming 

My non-caffeinated brain is having a hard time making sense of all the positive/negative logic here between what the exit code of `pr_changed_files` is vs the scripts' exit code vs if that means skip or don't skip…

I mean overall it makes sense, the script is called `should-skip-job` so it will `exit 0` (= "true") if it needs to skip the job, and that `exit 0` will happen when `pr_changed_files` itself exits 0… which will happen if all the files in the PR match the pattern… which can be similarly interpreted as "when the PR does NOT have any file that is outside of the list of provided patterns"…
…and at that point in this sentence there's already too many double-negations here between `0 = true` to "skip" vs "don't skip" to "all files in PR match X" == "the PR doesn't contain any file not matching X" that made me lose track 😅 

I think in a way that sort of mix of vocabulary of positive and negative terms in the implementation and naming is a bit inevitable (after all, the implementation is correct and what it should be). But maybe even more details in the code comments could help anyone who'd want to try to read the implementation again later to try to better understand the logic?

---

Especially since I can also imagine the parameter `--validation`/`--build` to not always be clear at first glance (and that's one of the reason why I looked at the script's implementation to better understand what it was doing under the hood and thus how to interpret those params in the context of the commands that called them). I'm not sure we can solve that either (because naming is hard and I don't have a good suggestion of another name for those params that would be any clearer anyway) but maybe we could provide more doc details to help then.

---

Though maybe one alternative to help with those parameters' names could be `--for-validation-job`/`--for-build-job`, or `--job-type=validation`/`--job-type=build`, to at least make it clear at call site that those parameters are for indicating which type of job we're talking about to then decide the condition for skipping, as opposed to e.g. `--validation` being to do a dry-run of the skip test / "validate if the job should be skipped or not"?

Alternatively, the parameter could just be more explicit at to it including the condition for localizations or not, e.g. `should-skip-job.sh --translation-updates-should-skip` or `--ignore-translation-files` (though that last one brings back the potential issue of being a headache to interpret due to double-negative, and thus probably bringing more confusion than being helpful)… naming is hard…
created_at: 2025-05-12T12:39:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,32 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh [--validation|--build]
+# --validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --build: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+if [ "$1" = "--validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
+  PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+elif [ "$1" = "--build" ]; then
+  # Check if changes are limited to documentation, tooling, and non-code files (NOT localization files)
+  PATTERNS=("${COMMON_PATTERNS[@]}")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+else
+  echo "Error: Must specify either --validation or --build"
+  exit 1
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: This line is one example of where my brain had to work a little extra to reason about this and validate that this would have the expected behavior 😅  But I believe this is correct.

Indeed, if one accidentally calls `should-skip-job.sh` without one of the 2 valid parameters, this `exit 1` will essentially make the script answer "NO" to the question "should we skip this job?", and would thus run the job anyway, never skipping it.

It'd probably be even better for such a situation to make the whole CI job that made such invocation be marked as failed itself and stop the job to mark it as red at that point—as opposed to such a mistake/typo in the script easily being unnoticed that it'd make the logic basically be a no-op and the job still run.
I don't think that's possible with just an exit code, since the exit code will be consumed by the `if …` expression that called the script, not the parent script of the job… and maybe using `buildkite-agent step cancel` could be an option to explore, though it'd mark the job as canceled, not failed… 🤔 
created_at: 2025-05-12T12:48:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,5 +1,13 @@
 #!/bin/bash -u
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --validation; then
path: .buildkite/commands/lint.sh
user: login: iangmaia
body: Good point -- on a second thought, this is Android Lint, which **will** run a few checks related to translations.
Maybe it's good to start on the safe side and optimize later (and this thought can probably serve for other cases as well) with the feedback from the teams. I think I'll remove this skip check 👍 
created_at: 2025-05-12T13:49:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,32 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh [--validation|--build]
+# --validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --build: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+if [ "$1" = "--validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
+  PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+elif [ "$1" = "--build" ]; then
+  # Check if changes are limited to documentation, tooling, and non-code files (NOT localization files)
+  PATTERNS=("${COMMON_PATTERNS[@]}")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+else
path: .buildkite/commands/should-skip-job.sh
user: login: iangmaia
body: > Especially since I can also imagine the parameter --validation/--build to not always be clear at first glance

Agreed! I think it adds another dimension to the "should skip tests?" question without really having to, and I agree on something like `--job-type validation` would make things clearer! Great idea 👍 
created_at: 2025-05-12T13:53:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,32 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh [--validation|--build]
+# --validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --build: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+if [ "$1" = "--validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
+  PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+elif [ "$1" = "--build" ]; then
+  # Check if changes are limited to documentation, tooling, and non-code files (NOT localization files)
+  PATTERNS=("${COMMON_PATTERNS[@]}")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+else
path: .buildkite/commands/should-skip-job.sh
user: login: iangmaia
body: Updated on 1fdcfb79bd3a4dc937e5c054f0ae1b397f60008d.
created_at: 2025-05-12T15:10:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,5 +1,13 @@
 #!/bin/bash -u
 
+# Check if we can skip this job based on PR changes
+if .buildkite/commands/should-skip-job.sh --validation; then
path: .buildkite/commands/lint.sh
user: login: iangmaia
body: For now, I went with a separate `lint` job type ran with the same skip patterns as `build`. Updated on fc0e0074ba636468f0476e18c07df26d10998107.
created_at: 2025-05-12T15:12:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,32 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh [--validation|--build]
+# --validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --build: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+if [ "$1" = "--validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
+  PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+elif [ "$1" = "--build" ]; then
+  # Check if changes are limited to documentation, tooling, and non-code files (NOT localization files)
+  PATTERNS=("${COMMON_PATTERNS[@]}")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+else
+  echo "Error: Must specify either --validation or --build"
+  exit 1
path: .buildkite/commands/should-skip-job.sh
user: login: iangmaia
body: I've added better error handling and `buildkite-agent step cancel` (it [worked fine](https://buildkite.com/automattic/woocommerce-android/builds/29136) -- it cancels the job, but IMO it also makes it very clear that things aren't right) on 1fdcfb79bd3a4dc937e5c054f0ae1b397f60008d / eb1498dd8c5336f85c1007a7d14f4b7fd37688b9.
created_at: 2025-05-12T15:17:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,41 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|build|lint]
+# --job-type validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --job-type build: Skip when changes are limited to documentation, tooling, and non-code files
+# --job-type lint: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+# Check if arguments are valid
+if [ -z "${1:-}" ] || [ "$1" != "--job-type" ] || [ -z "${2:-}" ]; then
+  echo "Error: Must specify --job-type [validation|build|lint]"
+  buildkite-agent step cancel
+  exit 1
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: I wonder if a more differentiating exit code would be beneficial for that (e.g. `127` or `42` or whatever, anything arbitrary but `1`), to differentiate it from the `exit 1` that `pr_changed_files` would use to return an answer of "false"? 🤷 

```suggestion
  exit 15
```
And same for line 40
created_at: 2025-05-12T16:16:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,41 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|build|lint]
+# --job-type validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --job-type build: Skip when changes are limited to documentation, tooling, and non-code files
+# --job-type lint: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+# Check if arguments are valid
+if [ -z "${1:-}" ] || [ "$1" != "--job-type" ] || [ -z "${2:-}" ]; then
+  echo "Error: Must specify --job-type [validation|build|lint]"
+  buildkite-agent step cancel
+  exit 1
+fi
+
+if [ "$2" = "validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: ```suggestion
  # We should skip if changes are limited to documentation, tooling, non-code files, and localization files
```
created_at: 2025-05-12T16:17:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,41 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|build|lint]
+# --job-type validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --job-type build: Skip when changes are limited to documentation, tooling, and non-code files
+# --job-type lint: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+# Check if arguments are valid
+if [ -z "${1:-}" ] || [ "$1" != "--job-type" ] || [ -z "${2:-}" ]; then
+  echo "Error: Must specify --job-type [validation|build|lint]"
+  buildkite-agent step cancel
+  exit 1
+fi
+
+if [ "$2" = "validation" ]; then
+  # Check if changes are limited to documentation, tooling, non-code files, and localization files
+  PATTERNS=("${COMMON_PATTERNS[@]}" "**/strings.xml")
+  pr_changed_files --all-match "${PATTERNS[@]}"
+elif [ "$2" = "build" ] || [ "$2" = "lint" ]; then
+  # Check if changes are limited to documentation, tooling, and non-code files (NOT localization files)
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: ```suggestion
  # We should if changes are limited to documentation, tooling, and non-code files
  # We'll let the job run (won't skip) if PR includes changes in localization files though
```
created_at: 2025-05-12T16:18:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,41 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|build|lint]
+# --job-type validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --job-type build: Skip when changes are limited to documentation, tooling, and non-code files
+# --job-type lint: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+# Check if arguments are valid
+if [ -z "${1:-}" ] || [ "$1" != "--job-type" ] || [ -z "${2:-}" ]; then
+  echo "Error: Must specify --job-type [validation|build|lint]"
+  buildkite-agent step cancel
+  exit 1
+fi
+
+if [ "$2" = "validation" ]; then
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: 💄 Nitpick: wdyt about using the `switch/case` syntax of bash instead of `if…elsif…else…fi` syntax? 🤔 

```bash
case "$2" in
  validation)
    …
    ;;
  lint | build)
    …
    ;;
  *)
    # handle error
esac
```
created_at: 2025-05-12T16:21:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,41 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|build|lint]
+# --job-type validation: Skip when changes are limited to documentation, tooling, non-code files, and localization files
+# --job-type build: Skip when changes are limited to documentation, tooling, and non-code files
+# --job-type lint: Skip when changes are limited to documentation, tooling, and non-code files
+
+COMMON_PATTERNS=(
+  "*.md"
+  "*.pot"
+  "*.txt"
+  ".gitignore"
+  "config/**"
+  "docs/**"
+  "fastlane/**"
+  "Gemfile"
+  "Gemfile.lock"
+  "gradle/**"
+  "version.properties"
+)
+
+# Check if arguments are valid
+if [ -z "${1:-}" ] || [ "$1" != "--job-type" ] || [ -z "${2:-}" ]; then
+  echo "Error: Must specify --job-type [validation|build|lint]"
+  buildkite-agent step cancel
+  exit 1
+fi
+
+if [ "$2" = "validation" ]; then
path: .buildkite/commands/should-skip-job.sh
user: login: iangmaia
body: Yep I like it, reads a bit better. Updated on 6f635b603a40310bb199a2de5b0c1cae5139c0a9.
created_at: 2025-05-12T18:48:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,74 @@
+#!/bin/bash -eu
+
+# Usage: should-skip-job.sh --job-type [validation|build|lint]
path: .buildkite/commands/should-skip-job.sh
user: login: AliSoftware
body: For consistency with order in which they're documented below

```suggestion
# Usage: should-skip-job.sh --job-type [validation|lint|build]
```
created_at: 2025-05-14T19:25:24Z
author_association: CONTRIBUTOR

PR comments:
user: login: wpmobilebot
created_at: 2025-05-09T17:53:21Z
body: <!-- REUSE_ID: WooCommerce-Wear-prototype-build-link -->

<details><summary>📲 You can test the changes from this Pull Request in <b>WooCommerce-Wear Android</b> by scanning the QR code below to install the corresponding build.</summary>
<table>
<tr>
  <td rowspan='6' width='260px'><img src='https://api.qrserver.com/v1/create-qr-code/?size=500x500&qzone=4&data=https%3A%2F%2Fcdn.a8c-ci.services%2F70ad533e752cbd4dcd515ed6cfb9f823d6f9f819%2Fwoocommerce-wear-prototype-build-pr14034-2aaf294.apk' width='250' height='250' /></td>
  <td><b>App Name</b></td><td> WooCommerce-Wear Android</td>
</tr>
<tr><td><b>Platform</b></td><td>⌚️ Wear OS</td></tr>
<tr><td><b>Flavor</b></td><td>Jalapeno</td></tr>
<tr><td><b>Build Type</b></td><td>Debug</td></tr>
<tr><td><b>Commit</b></td><td>2aaf2947c016c805c726679e76258ab2c47aed15</td></tr>
<tr><td><b>Direct Download</b></td><td><a href='https://cdn.a8c-ci.services/70ad533e752cbd4dcd515ed6cfb9f823d6f9f819/woocommerce-wear-prototype-build-pr14034-2aaf294.apk'><code>woocommerce-wear-prototype-build-pr14034-2aaf294.apk</code></a></td></tr>
</table>

</details>

author_association: COLLABORATOR
user: login: wpmobilebot
created_at: 2025-05-09T17:54:17Z
body: <!-- REUSE_ID: WooCommerce-prototype-build-link -->

<p>📲 You can test the changes from this Pull Request in <b>WooCommerce Android</b> by scanning the QR code below to install the corresponding build.</p>
<table>
<tr>
  <td rowspan='6' width='260px'><img src='https://api.qrserver.com/v1/create-qr-code/?size=500x500&qzone=4&data=https%3A%2F%2Fcdn.a8c-ci.services%2Fa8b6e90d9d969f1b573b3c0dd090796761ee106e%2Fwoocommerce-prototype-build-pr14034-2aaf294.apk' width='250' height='250' /></td>
  <td><b>App Name</b></td><td> WooCommerce Android</td>
</tr>
<tr><td><b>Platform</b></td><td>📱 Mobile</td></tr>
<tr><td><b>Flavor</b></td><td>Jalapeno</td></tr>
<tr><td><b>Build Type</b></td><td>Debug</td></tr>
<tr><td><b>Commit</b></td><td>2aaf2947c016c805c726679e76258ab2c47aed15</td></tr>
<tr><td><b>Direct Download</b></td><td><a href='https://cdn.a8c-ci.services/a8b6e90d9d969f1b573b3c0dd090796761ee106e/woocommerce-prototype-build-pr14034-2aaf294.apk'><code>woocommerce-prototype-build-pr14034-2aaf294.apk</code></a></td></tr>
</table>


author_association: COLLABORATOR
user: login: codecov-commenter
created_at: 2025-05-14T16:49:42Z
body: ## [Codecov](https://app.codecov.io/gh/woocommerce/woocommerce-android/pull/14034?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=woocommerce) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 38.39%. Comparing base [(`49082c4`)](https://app.codecov.io/gh/woocommerce/woocommerce-android/commit/49082c44dca367c4a4dda7a95264213270794490?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=woocommerce) to head [(`c9d1143`)](https://app.codecov.io/gh/woocommerce/woocommerce-android/commit/c9d11431acb854bce35890b0fef3ee88a4297dbc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=woocommerce).
> Report is 2 commits behind head on trunk.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff              @@
##              trunk   #14034      +/-   ##
============================================
- Coverage     38.39%   38.39%   -0.01%     
+ Complexity     9606     9605       -1     
============================================
  Files          2125     2125              
  Lines        117060   117060              
  Branches      15030    15030              
============================================
- Hits          44948    44947       -1     
  Misses        67994    67994              
- Partials       4118     4119       +1     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/woocommerce/woocommerce-android/pull/14034?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=woocommerce).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=woocommerce).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE