Repository: benchopt/benchopt
Pull request #728 (2024-06-03 20:48:13)

MNT Execute only the last push commit in CI
In a PR, each push executes continuous integration tests; however, if you commit and push multiple times within a PR, you create a queue of jobs and testing.

If you have tests that take longer, this ends up generating a long queue to have one test done. This is meaningless, as we always want to test only the last commit.

In this PR, we ensure that CI stops the tests for the previous push and runs the test only for the last one.


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/benchopt/benchopt/pull/728?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=benchopt) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 80.87%. Comparing base [(`5fc3e3f`)](https://app.codecov.io/gh/benchopt/benchopt/commit/5fc3e3fd7629e85bca437c27066e4811985f5370?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=benchopt) to head [(`5c94ae0`)](https://app.codecov.io/gh/benchopt/benchopt/commit/5c94ae07dbd8587061357d71d3ff960f79d63d1a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=benchopt).

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##             main     #728   +/-   ##
=======================================
  Coverage   80.87%   80.87%           
=======================================
  Files          45       45           
  Lines        3054     3054           
=======================================
  Hits         2470     2470           
  Misses        584      584           
```



</details>

Patch:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 3cf46b0..1250bc3 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -135,7 +135,11 @@ jobs:
               git remote -v
               git fetch origin
               git reset --hard origin/main
-              git clean -xdf
+              mv .git ..
+              mv .gitignore ..
+              rm -rf *
+              mv ../.git .
+              mv ../.gitignore .
               echo "Deploying dev docs for ${CIRCLE_BRANCH}.";
               cp -a /tmp/build/html/* .;
               if [[ -z $(git diff) ]]; then
diff --git a/.github/workflows/expose_link.yml b/.github/workflows/expose_link.yml
index b353b5d..470bd0a 100644
--- a/.github/workflows/expose_link.yml
+++ b/.github/workflows/expose_link.yml
@@ -1,5 +1,10 @@
 name: Expose result artifact
 on: [status]
+# Cancel in-progress workflows when pushing
+# a new commit on the same branch
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
 jobs:
   circleci_result_artifact_redirector_job:
     runs-on: ubuntu-latest
diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
index 738cee4..eadd8d7 100644
--- a/.github/workflows/linting.yml
+++ b/.github/workflows/linting.yml
@@ -1,6 +1,10 @@
 name: linting
-
 on:
+  # Cancel in-progress workflows when pushing
+  # a new commit on the same branch
+  concurrency:
+    group: ${{ github.workflow }}-${{ github.ref }}
+    cancel-in-progress: true
   push:
     branches:
       - main
diff --git a/benchopt/stopping_criterion.py b/benchopt/stopping_criterion.py
index c321058..ff34a4c 100644
--- a/benchopt/stopping_criterion.py
+++ b/benchopt/stopping_criterion.py
@@ -212,7 +212,9 @@ class StoppingCriterion():
         n_eval = len(objective_list) - 1
         objective = objective_list[-1][self.key_to_monitor]
         delta_objective = self._prev_objective - objective
-        delta_objective /= abs(objective_list[0][self.key_to_monitor])
+        first_objective = objective_list[0][self.key_to_monitor]
+        if first_objective != 0:
+            delta_objective /= abs(first_objective)
         self._prev_objective = objective
 
         # default value for is_flat
@@ -430,7 +432,9 @@ class SufficientProgressCriterion(StoppingCriterion):
         # Compute the current objective and update best value
         objective = objective_list[-1][self.key_to_monitor]
         delta_objective = self._best_objective - objective
-        delta_objective /= abs(objective_list[0][self.key_to_monitor])
+        first_objective = objective_list[0][self.key_to_monitor]
+        if first_objective != 0:
+            delta_objective /= abs(first_objective)
         self._best_objective = min(
             objective, self._best_objective
         )


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 946ce56..165168b 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -1,6 +1,10 @@
 name: test
-
 on:
+  # Cancel in-progress workflows when pushing
+  # a new commit on the same branch
+  concurrency:
+    group: ${{ github.workflow }}-${{ github.ref }}
+    cancel-in-progress: true
   push:
     branches:
       - main
diff --git a/benchopt/tests/test_stopping_criterion.py b/benchopt/tests/test_stopping_criterion.py
index b29ec01..bcc952d 100644
--- a/benchopt/tests/test_stopping_criterion.py
+++ b/benchopt/tests/test_stopping_criterion.py
@@ -294,3 +294,40 @@ def test_dual_strategy(no_debug_test):
                 run([str(benchmark.benchmark_dir),
                     *('-s test-solver -d test-dataset --no-plot').split()],
                     standalone_mode=False)
+
+
+def test_objective_equals_zero(no_debug_test):
+
+    objective = """from benchopt import BaseObjective
+
+        class Objective(BaseObjective):
+            name = "test_obj"
+            min_benchopt_version = "0.0.0"
+
+            def set_data(self, X, y): pass
+            def get_one_result(self): pass
+            def evaluate_result(self, beta): return dict(value=0)
+            def get_objective(self): return dict(X=0, y=0)
+    """
+
+    solver = """from benchopt import BaseSolver
+    from benchopt.stopping_criterion import SufficientDescentCriterion
+
+    class Solver(BaseSolver):
+        name = "test-solver"
+        stopping_criterion = SufficientDescentCriterion()
+        def set_objective(self, X, y): pass
+        def run(self, n_iter): pass
+        def get_result(self): return dict(beta=1)
+    """
+
+    with temp_benchmark(
+            objective=objective,
+            solvers=[solver]
+    ) as benchmark:
+        with CaptureRunOutput() as out:
+            run([str(benchmark.benchmark_dir),
+                *('-s test-solver -d test-dataset --no-plot -n 0').split()],
+                standalone_mode=False)
+
+    out.check_output('done', 1)