internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Parsl/parsl

Issue:
Globus data provider
**Describe the bug**
A clear and concise description of what the bug is.
Testing the Globus data_provide whenever I try to get results out of my @python_app on this line https://github.com/Parsl/parsl/blob/master/parsl/data_provider/globus.py#LL97C9-L97C55 it terminates because it gets the following error "TypeError: task_wait() takes 2 positional arguments but 4 were given" 

**To Reproduce**
Steps to reproduce the behavior, for e.g:
1. Setup Parsl 2023.1.23 with Python 3.9.13 on cluster
2. Run a test script [This is my test https://github.com/KiwiFac3/Falcon2Parsl/blob/main/Test/Globus.py ( I had the last line commented since it the one terminating the program)]
3. Wait 5 mins
4. See error

**Expected behavior**
The test should keep running until the transfer of the file is done

**Environment**
 - OS: Ubuntu
 - Python 3.9.13
 - Parsl 2023.1.23
 - globus-sdk: 3.15.1

**Distributed Environment**
- Where are you running the Parsl script from ? Laptop/Workstation or Compute node
- Where do you need the workers to run ? Same as Parsl script



Discussion:
Can you add which version of the globus-sdk package you are using:  `pip list | grep globus-sdk`

globus-sdk: 3.15.1


Resolving patch:
diff --git a/parsl/data_provider/globus.py b/parsl/data_provider/globus.py
--- a/parsl/data_provider/globus.py
+++ b/parsl/data_provider/globus.py
@@ -94,7 +94,7 @@ def transfer_file(cls, src_ep, dst_ep, src_path, dst_path):
         with 60 second timeout limit. If the task is ACTIVE after time runs out 'task_wait' returns False,
         and True otherwise.
         """
-        while not tc.task_wait(task['task_id'], 60, 15):
+        while not tc.task_wait(task['task_id'], timeout=60):
             task = tc.get_task(task['task_id'])
             # Get the last error Globus event
             events = tc.task_event_list(task['task_id'], num_results=1, filter='is_error:1')


Test patch:
Repository: Parsl/parsl

Issue:
parsl resource monitor does not exit properly
**Describe the bug**
I have noticed recently that the parsl resource monitor thread will not exit properly when I ctrl-C the test suite, and ends up filling the provider stderr with stack traces like this:

```
ERROR:root:Exception getting the resource usage. Not sending usage to Hub
Traceback (most recent call last):
  File "/home/benc/parsl/src/parsl/parsl/monitoring/remote.py", line 284, in monitor
    d = accumulate_and_prepare()
        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benc/parsl/src/parsl/parsl/monitoring/remote.py", line 222, in accumulate_and_prepare
    d = {"psutil_process_" + str(k): v for k, v in pm.as_dict().items() if k in simple}
                                                   ^^^^^^^^^^^^
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/__init__.py", line 528, in as_dict
    ret = meth()
          ^^^^^^
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/_common.py", line 487, in wrapper
    raise raise_from(err, None)
          ^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 3, in raise_from
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/_common.py", line 485, in wrapper
    ret = fun(self)
          ^^^^^^^^^
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/__init__.py", line 742, in uids
    return self._proc.uids()
           ^^^^^^^^^^^^^^^^^
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/_pslinux.py", line 1653, in wrapper
    return fun(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/_pslinux.py", line 2257, in uids
    data = self._read_status_file()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/psutil/_pslinux.py", line 1660, in wrapper
    raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: process no longer exists (pid=93728, name='process_worker_')
```

which rapidly fill my home directory.

I think this was probably introduced as part of the daemonisation of the resource monitor code, #2760 



Discussion:

Resolving patch:
diff --git a/parsl/monitoring/remote.py b/parsl/monitoring/remote.py
--- a/parsl/monitoring/remote.py
+++ b/parsl/monitoring/remote.py
@@ -262,7 +262,7 @@ def accumulate_and_prepare() -> Dict[str, Any]:
     next_send = time.time()
     accumulate_dur = 5.0  # TODO: make configurable?
 
-    while not terminate_event.is_set():
+    while not terminate_event.is_set() and pm.is_running():
         logging.debug("start of monitoring loop")
         try:
             d = accumulate_and_prepare()


Test patch:
Repository: Parsl/parsl

Issue:
Missing `ignore_for_cache` docstring in `python_app` and `join_app`
https://github.com/Parsl/parsl/blob/40b969f7a9e16b3fcab2db127753bf85cbcb5375/parsl/app/app.py#L72-L93

The `ignore_for_cache` docstring is missing. Same thing for the `join_app` by the way. Super minor report.


Discussion:

Resolving patch:
diff --git a/parsl/app/app.py b/parsl/app/app.py
--- a/parsl/app/app.py
+++ b/parsl/app/app.py
@@ -34,6 +34,7 @@ def __init__(self, func, data_flow_kernel=None, executors='all', cache=False, ig
                after calling :meth:`parsl.dataflow.dflow.DataFlowKernelLoader.load`.
              - executors (str|list) : Labels of the executors that this app can execute over. Default is 'all'.
              - cache (Bool) : Enable caching of this app ?
+             - ignore_for_cache (list|None): Names of arguments which will be ignored by the caching mechanism.
 
         Returns:
              - App object.
@@ -90,6 +91,8 @@ def python_app(function=None,
         Labels of the executors that this app can execute over. Default is 'all'.
     cache : bool
         Enable caching of the app call. Default is False.
+    ignore_for_cache : (list|None)
+        Names of arguments which will be ignored by the caching mechanism.
     """
     from parsl.app.python import PythonApp
 
@@ -126,6 +129,8 @@ def join_app(function=None,
         be omitted only after calling :meth:`parsl.dataflow.dflow.DataFlowKernelLoader.load`. Default is None.
     cache : bool
         Enable caching of the app call. Default is False.
+    ignore_for_cache : (list|None)
+        Names of arguments which will be ignored by the caching mechanism.
     """
     from parsl.app.python import PythonApp
 
@@ -167,6 +172,8 @@ def bash_app(function=None,
         Labels of the executors that this app can execute over. Default is 'all'.
     cache : bool
         Enable caching of the app call. Default is False.
+    ignore_for_cache : (list|None)
+        Names of arguments which will be ignored by the caching mechanism.
     """
     from parsl.app.bash import BashApp
 


Test patch:
Repository: Parsl/parsl

Issue:
Cannot Instantiate SlurmProvider in parsl-2023.12.11
**Describe the bug**
When I attempt to instantiate a SlurmProvider in parsl-2023.12.11 I get the following error:
`AttributeError: class SlurmProvider uses qos in the constructor, but does not define it as an attribute`

**To Reproduce**
Steps to reproduce the behavior:
1. Install parsl-2023.12.11
2. Attempt to instantiate SlurmProvider
3. See error

**Expected behavior**
The provider should be instantiated

**Environment**
 - OS: Seen on Ubuntu and MacOS github workers and Linux Mint 21.2
 - Python version: 3.8 and 3.11
 - Parsl version: 2023.12.11

**Distributed Environment**
- I'm not actually running a script at all just trying to instantiate the provider/a config with the provider



Discussion:
 Probably introduced by #2986 - maybe @tylern4 can look at this.

Basically for RepresentationMixin, each constructor argument needs to be stored on `self` so that it can be used to create `repr` output later.

This gets checked at instantiation, not statically, so it's more likely to happen for classes we don't test in CI.

Resolving patch:
diff --git a/parsl/providers/slurm/slurm.py b/parsl/providers/slurm/slurm.py
--- a/parsl/providers/slurm/slurm.py
+++ b/parsl/providers/slurm/slurm.py
@@ -132,6 +132,8 @@ def __init__(self,
         self.exclusive = exclusive
         self.move_files = move_files
         self.account = account
+        self.qos = qos
+        self.constraint = constraint
         self.scheduler_options = scheduler_options + '\n'
         if exclusive:
             self.scheduler_options += "#SBATCH --exclusive\n"


Test patch:
Repository: Parsl/parsl

Issue:
`SlurmProvider` is missing a docstring for `init_blocks`
**Describe the bug**
The `SlurmProvider` is missing the docstring for its `init_blocks` keyword argument [here](https://github.com/Parsl/parsl/blob/72df44c105a294b90d1b7f9022ed24309fc41885/parsl/providers/slurm/slurm.py#L37).



Discussion:

Resolving patch:
diff --git a/parsl/providers/slurm/slurm.py b/parsl/providers/slurm/slurm.py
--- a/parsl/providers/slurm/slurm.py
+++ b/parsl/providers/slurm/slurm.py
@@ -65,6 +65,8 @@ class SlurmProvider(ClusterProvider, RepresentationMixin):
     mem_per_node : int
         Specify the real memory to provision per node in GB. If set to None, no
         explicit request to the scheduler will be made. Default is None.
+    init_blocks : int
+        Number of blocks to provision at the start of the run. Default is 1.
     min_blocks : int
         Minimum number of blocks to maintain.
     max_blocks : int


Test patch:
Repository: Parsl/parsl

Issue:
missing process_worker_pool `--available-accelerators` commandline gives awkward error
**Describe the bug**

If an invocation of process_worker_pool is missing an `--available-accelerators` parameter, this exception happens:

```
    benc@parsl-dev-3-12-9646:~/parsl/src/parsl$ parsl/executors/high_throughput/process_worker_pool.py --cert_dir . --task_port 3 --result_port 4 --cpu-affinity none
    Traceback (most recent call last):
      File "/home/benc/parsl/src/parsl/parsl/executors/high_throughput/process_worker_pool.py", line 851, in <module>
        logger.info("Accelerators: {}".format(" ".join(args.available_accelerators)))
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    TypeError: can only join an iterable
```


**To Reproduce**
Run the above command line.

**Expected behavior**

It would be nicer if this was detected closer to command-line handling, for example, requiring it to be mandatory with a better error message or defaulting to a better value.


**Environment**
my laptop, parsl d1b85100f1826d74b0dc650835ea142ecec9a56d


Discussion:
Hi @benclifford ,

I've reviewed the issue you reported about receiving a TypeError when the --available-accelerators parameter isn't included in the process_worker_pool.py invocation. It appears the script expects an iterable for this parameter, which causes an issue when the parameter is omitted. To address this, we could consider a few adjustments: ensuring the argument is compulsory and documenting its necessity, defaulting to an empty list to gracefully handle its absence, or refining our error handling to guide users more effectively when the argument is missing. These enhancements aim to preempt the confusion at the source, making the script more intuitive to use. Do you think these modifications would be suitable for solving the reported issue? Enhancing our command-line interface in such ways could significantly boost usability and error messaging clarity.

Resolving patch:
diff --git a/parsl/executors/high_throughput/process_worker_pool.py b/parsl/executors/high_throughput/process_worker_pool.py
--- a/parsl/executors/high_throughput/process_worker_pool.py
+++ b/parsl/executors/high_throughput/process_worker_pool.py
@@ -858,7 +858,7 @@ def strategyorlist(s: str):
                         required=True,
                         help="Whether/how workers should control CPU affinity.")
     parser.add_argument("--available-accelerators", type=str, nargs="*",
-                        help="Names of available accelerators")
+                        help="Names of available accelerators, if not given assumed to be zero accelerators available", default=[])
     parser.add_argument("--enable_mpi_mode", action='store_true',
                         help="Enable MPI mode")
     parser.add_argument("--mpi-launcher", type=str, choices=VALID_LAUNCHERS,


Test patch:
Repository: Parsl/parsl

Issue:
Remove BlueWaters from userguide
**Describe the bug**

Bluewaters is a supercomputer at the NCSA (UIUC) that is now retired. Our userguide still contains an example configuration for this defunct machine that should be removed.

Here's a link to the userguide section as rendered: https://parsl.readthedocs.io/en/stable/userguide/configuring.html#blue-waters-ncsa


**Expected behavior**

Bluewaters should be removed from our userguide section.

Here's a quick sketch of the work involved:
1. Remove the section on Bluewaters from `docs/userguide/configuring.rst`
2. Remove the example configuration file here `parsl/configs/bluewaters.py`
3. Rebuild the documentation: `cd docs; make clean html; `
4. Check the newly rebuild docs with `cd docs/_build/html; python3 -m http.server 8080` and load `http://localhost:8080` in your browser to load the html pages that was newly rebuilt in step.3.




Discussion:

Resolving patch:
diff --git a/parsl/configs/bluewaters.py b/parsl/configs/bluewaters.py
deleted file mode 100644
--- a/parsl/configs/bluewaters.py
+++ /dev/null
@@ -1,28 +0,0 @@
-from parsl.config import Config
-from parsl.executors import HighThroughputExecutor
-from parsl.launchers import AprunLauncher
-from parsl.providers import TorqueProvider
-
-
-config = Config(
-    executors=[
-        HighThroughputExecutor(
-            label="bw_htex",
-            cores_per_worker=1,
-            worker_debug=False,
-            provider=TorqueProvider(
-                queue='normal',
-                launcher=AprunLauncher(overrides="-b -- bwpy-environ --"),
-                scheduler_options='',  # string to prepend to #SBATCH blocks in the submit script to the scheduler
-                worker_init='',  # command to run before starting a worker, such as 'source activate env'
-                init_blocks=1,
-                max_blocks=1,
-                min_blocks=1,
-                nodes_per_block=2,
-                walltime='00:10:00'
-            ),
-        )
-
-    ],
-
-)


Test patch:
Repository: Parsl/parsl

Issue:
Bad string templating in deprecation message
**Describe the bug**

PR #3117 changed the name of the max_workers parameter, and introduced a deprecation message if someone still tries to use the old max_workers name.

That message prints out like this:

```
python3 -X dev
>>> import parsl as p
>>> p.HighThroughputExecutor(max_workers=8)

/home/benc/parsl/src/parsl/parsl/executors/high_throughput/executor.py:278: DeprecationWarning: max_workers is deprecated and will be removed in a future release. Please use {new} instead.
  self._warn_deprecated("max_workers", "max_workers_per_node")
[... lots of stuff ...]
```
but that is wrong: there is a string template substitution `{new}` that is meant to be happening here (putting in the new name max_workers_per_node) but something is broken with the string substitution.


**Expected behavior**
The message should contain both the old and the new names, and not broken string templates.

**Environment**
my laptop



Discussion:

Resolving patch:
diff --git a/parsl/executors/high_throughput/executor.py b/parsl/executors/high_throughput/executor.py
--- a/parsl/executors/high_throughput/executor.py
+++ b/parsl/executors/high_throughput/executor.py
@@ -328,7 +328,7 @@ def __init__(self,
     def _warn_deprecated(self, old: str, new: str):
         warnings.warn(
             f"{old} is deprecated and will be removed in a future release. "
-            "Please use {new} instead.",
+            f"Please use {new} instead.",
             DeprecationWarning,
             stacklevel=2
         )


Test patch:
Repository: Parsl/parsl

Issue:
parsl.providers.cluster_provider _write_submit_script should return nothing, rather than constant True
**Describe the bug**

_write_submit_script in parsl.providers.cluster_provider indicates failure by raising an exception, and so should not be returning a True (or False) value. Instead it should return None by either `return` on its own, or falling off the end of the method without a return statement.

To tidy this up, change that return handling. Edit the docstring to match.




Discussion:
@benclifford please assign me this issue. I'd very much like to work on it.

ok, go ahead @MundiaNderi 

Resolving patch:
diff --git a/parsl/providers/cluster_provider.py b/parsl/providers/cluster_provider.py
--- a/parsl/providers/cluster_provider.py
+++ b/parsl/providers/cluster_provider.py
@@ -91,7 +91,7 @@ def _write_submit_script(self, template, script_filename, job_name, configs):
               - configs (dict) : configs that get pushed into the template
 
         Returns:
-              - True: on success
+              - None
 
         Raises:
               SchedulerMissingArgs : If template is missing args
@@ -117,8 +117,6 @@ def _write_submit_script(self, template, script_filename, job_name, configs):
             logger.error("Uncategorized error: %s", e)
             raise e
 
-        return True
-
     @abstractmethod
     def _status(self):
         pass


Test patch:
Repository: Parsl/parsl

Issue:
[proposal] remove PARSL_TRACKING environment variable
**Is your feature request related to a problem? Please describe.**

Currently Parsl takes usage tracking configuration from both `Config` (like other configuration) and from the `$PARSL_TRACKING` environment variable.

The interactions between these two parameters, as well as two conflicting defaults (one in `Config` and one in `usage.py`), are quite subtle and hard to understand intuitively and in the code.

PR #3400 makes that configuration more complicated: instead of a `bool`, the usage tracking level becomes an `int` (of restricted range). In reviewing that PR, along with @khk-globus, it's become pretty apparent that fusing two configurations and defaults together is even more complex in this situation.

Other configuration options in Parsl do not have this "specify multiple configuration values, fuse them together" behaviour: configuration comes from the `Config` object (and objects contained within that class).

**Describe the solution you'd like**

I would like to simplify this piece of Parsl by removing `$PARSL_TRACKING`: this removes a bunch of complexity both for user understanding of what usage level they have actually configured, and for programmers trying to understand these code paths.

**Describe alternatives you've considered**
A more generalised fuseable configuration mechanism that can fuse *any* configuration options, not only usage tracking level.

**Additional context**
This is part of a funded deliverable for usage tracking coordinated by @kylechard and @yadudoc, which @NishchayKarle is working on right now.


Discussion:
Makes sense to put it all in the configuration. 

Resolving patch:
diff --git a/parsl/usage_tracking/usage.py b/parsl/usage_tracking/usage.py
--- a/parsl/usage_tracking/usage.py
+++ b/parsl/usage_tracking/usage.py
@@ -1,6 +1,5 @@
 import json
 import logging
-import os
 import platform
 import socket
 import sys
@@ -117,22 +116,12 @@ def __init__(self, dfk, port=50077,
     def check_tracking_enabled(self):
         """Check if tracking is enabled.
 
-        Tracking will be enabled unless either of these is true:
+        Tracking will be enabled unless the following is true:
 
             1. dfk.config.usage_tracking is set to False
-            2. Environment variable PARSL_TRACKING is set to false (case insensitive)
 
         """
-        track = True
-
-        if not self.config.usage_tracking:
-            track = False
-
-        envvar = str(os.environ.get("PARSL_TRACKING", True)).lower()
-        if envvar == "false":
-            track = False
-
-        return track
+        return self.config.usage_tracking
 
     def construct_start_message(self) -> bytes:
         """Collect preliminary run info at the start of the DFK.


Test patch:
Repository: bentoml/BentoML

Issue:
Failing to create and API on Windows
**Describe the bug**
Failing to create an api on windows , 
`bentoml serve {saved_path}`

**To Reproduce**
Steps to reproduce the behavior:
1.  Failing with an error of the  fnctl  was not found 
2. fixed it by using this and creating a fcntl.py 
```
def fcntl(fd, op, arg=0):
    return 0

def ioctl(fd, op, arg=0, mutable_flag=True):
    if mutable_flag:
        return 0
    else:
        return ""

def flock(fd, op):
    return

def lockf(fd, operation, length=0, start=0, whence=0):
    return
```
3. Then the  import pwd started failing , for which I dont have any work around till now .


**Environment:**
 - OS: [windows 10 ]
 - Python/BentoML Version [Python3.6]

**Additional context**
Add any other context about the problem here.



Discussion:

Resolving patch:
diff --git a/bentoml/cli/__init__.py b/bentoml/cli/__init__.py
--- a/bentoml/cli/__init__.py
+++ b/bentoml/cli/__init__.py
@@ -33,7 +33,6 @@
     load_bento_service_metadata,
 )
 from bentoml.server import BentoAPIServer, get_docs
-from bentoml.server.gunicorn_server import GunicornBentoServer
 from bentoml.cli.click_utils import BentoMLCommandGroup, conditional_argument, _echo
 from bentoml.cli.deployment import get_deployment_sub_command
 from bentoml.cli.config import get_configuration_sub_command
@@ -274,6 +273,7 @@ def serve_gunicorn(
 
         track_cli('serve_gunicorn')
 
+        from bentoml.server.gunicorn_server import GunicornBentoServer
         gunicorn_app = GunicornBentoServer(archive_path, port, workers, timeout)
         gunicorn_app.run()
 


Test patch:
Repository: bentoml/BentoML

Issue:
Add support for Alpine based docker image
**Is your feature request related to a problem? Please describe.**

Allow users to use `bentoml/model-server:0.7.8-alpine` as the base image, which is currently defined here:  https://github.com/bentoml/BentoML/blob/master/docker/model-server/Dockerfile-alpine

**Describe the solution you'd like**

Improve the `bentoml_init.sh` script to make sure it works on both debian and alpine based docker image.

**Describe alternatives you've considered**
n/a

**Additional context**

See https://github.com/bentoml/BentoML/issues/693



Discussion:

Resolving patch:
diff --git a/bentoml/saved_bundle/bundler.py b/bentoml/saved_bundle/bundler.py
--- a/bentoml/saved_bundle/bundler.py
+++ b/bentoml/saved_bundle/bundler.py
@@ -135,6 +135,7 @@ def save_to_dir(bento_service, path, version=None, silent=False):
         f.write(MANIFEST_IN_TEMPLATE.format(service_name=bento_service.name))
 
     # write Dockerfile
+    logger.debug("Using Docker Base Image %s", bento_service._env._docker_base_image)
     with open(os.path.join(path, "Dockerfile"), "w") as f:
         f.write(
             MODEL_SERVER_DOCKERFILE_CPU.format(


Test patch: