Repository: uxlfoundation/oneTBB (C++)

Issue:
tbb::global_control does not stop spinning threads
# Summary
Reducing the max. allowed parallelism via `tbb::global_control` does not "stop" threads anymore. The worker threads are spinning in `thread_dispatcher::process()` and are eating CPU power, meaning that the execution is slower. 

The problem gets triggered when executing first e.g. `tbb::parallel_for_each` (to "fire up" threads), then reducing the number of threads to 1 via `tbb::global_control` and then executing a second `tbb::parallel_for_each`. While the second `tbb::parallel_for_each` is executed, only a single core should be active. Using the debugger I also see that the business logic is executed only by a single thread. However, looking in the task manager, I see that all cores are busy. The unnecessary threads cause the code to run slower by ~50% (either because of oversubscription, or because the CPU frequency gets reduced, not sure).

# Version
Version 2021.10.0 was ok. Version 2021.11.0 is broken. The current master (55bf2b33417ebd7f8b9add37709259b5c7ec3a5c) is still broken.

I was able to bisect the problem to commit c45684495599d41ba10893effa0682eceb1a3169 (pull request: #758).

# Environment
* Windows 11
* Intel Core i9-13900
* Microsoft Compiler version 19.40 (`_MSC_FULL_VER=194033811`)
 
# Observed Behavior
100% CPU load (all cores are busy) even though `tbb::global_control` was used to reduce the max. allowed parallelism to 1.

# Expected Behavior
After setting the max. allowed parallelism to 1 via `tbb::global_control`, only a single core should be busy (corresponds to ~3% CPU load in the task manager because of 32 virtual cores of my Intel i9-13900).

# Steps To Reproduce
* Code:
```C++
#include <chrono>
#include <iostream>
#include <numeric>
#include <tbb/global_control.h>
#include <tbb/parallel_for_each.h>
#include <tbb/version.h>
#include <vector>

int main()
{
  std::cout << "Start. TBB: " << TBB_VERSION_STRING << ", TBB_runtime_version=" << TBB_runtime_version()
            << ", TBB_runtime_interface_version=" << TBB_runtime_interface_version() << ", MSVC: " << _MSC_FULL_VER
            << std::endl;

  //------------------------------------------------
  // First call of tbb::parallel_for_each() to 'create' threads
  static constexpr bool TRIGGER_BUG = true;
  if (TRIGGER_BUG) {
    std::cout << "Warmup to trigger bug" << std::endl;
    std::vector<double> args(1024, 42.0);
    tbb::parallel_for_each(args, [](double & arg) {});
    std::cout << "Warmup finished." << std::endl;
  }

  //------------------------------------------------
  // Reduce number of threads
  std::cout << "Running test" << std::endl;
  static constexpr size_t NUM_CORES = 1;
  tbb::global_control tbbControl(tbb::global_control::max_allowed_parallelism, NUM_CORES);

  //------------------------------------------------
  // Second call of tbb::parallel_for_each()
  std::vector<double> args(1024, 42.0);
  auto const startTime = std::chrono::high_resolution_clock::now();
  tbb::parallel_for_each(args, [](double & arg) {
    for (size_t i = 0; i < 1000000; ++i) {
      arg += std::sin(arg);
    }
  });

  double const elapsed
      = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::high_resolution_clock::now() - startTime)
            .count()
        / 1000.0;

  double const result = std::accumulate(args.begin(), args.end(), 0.0);
  std::cout << "Used " << NUM_CORES << " cores. Finished in " << elapsed << "s: " << result << std::endl;
}
```

* Built using cmake: 
```
cmake -DCMAKE_INSTALL_PREFIX="path\to\install\dir" -DTBB_TEST=OFF ..
cmake --build .
cmake --install .
```

* Execute and observe in the task manager that all cores are busy. (That is the problem). Also take note of the execution time. 
* Then either revert to an older TBB version, or set `TRIGGER_BUG=false`. Then run again. Result: Only a single core is busy, and the execution time dropped by ~50%.


Discussion: Hi @Sedeniono, that you for the report. I'm actually surprised this bug was not reported sooner :) 



Fix (patch):
diff --git a/src/tbb/thread_request_serializer.cpp b/src/tbb/thread_request_serializer.cpp
index 6019f732b4..41cf51b0b0 100644
--- a/src/tbb/thread_request_serializer.cpp
+++ b/src/tbb/thread_request_serializer.cpp
@@ -100,13 +100,12 @@ void thread_request_serializer_proxy::set_active_num_workers(int soft_limit) {
 
     if (soft_limit != 0) {
         my_is_mandatory_concurrency_enabled = false;
-        my_serializer.set_active_num_workers(soft_limit);
-    } else {
-        if (my_num_mandatory_requests > 0 && !my_is_mandatory_concurrency_enabled) {
-            my_is_mandatory_concurrency_enabled = true;
-            my_serializer.set_active_num_workers(1);
-        }
+    } else if (my_num_mandatory_requests > 0) {
+        my_is_mandatory_concurrency_enabled = true;
+        soft_limit = 1;
     }
+
+    my_serializer.set_active_num_workers(soft_limit);
 }
 
 int thread_request_serializer_proxy::num_workers_requested() { return my_serializer.num_workers_requested(); }