internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: locustio/locust

Issue:
FastHttpLocust is very slow at returning the response text
Getting resp.text when using FastHttpLocust does a string decode (fasthttp.py:257) and it is crazy slow for large responses (~12s for a 1.5MB payload).

        with self.client.request(method, f"https://{host}{url}", name, catch_response=True) as resp:
            t = resp.text: # this takes ages
            ....
            return resp


Discussion:
Oh, and also it doesnt cache the value, so every call to .text will need to decode the string again.

I'm looking in to it...
Oh, the actual problem is the character encoding detection in apparent_encoding()
I can solve this by using cchardet instead of chardet. Encoding detection time drops from 12 seconds to 0.03s for me.

@heyman ok to add this as a dependency?
Or we could just read the Content-Type: charset header?
> Or we could just read the Content-Type: charset header?

Yeah, maybe that's best. And assume ISO-8859-1 if none is specified. I guess it could cause problems when the server is setting incorrect chaset headers, though it shouldn't be that big of a problem since users of a load testing tool usually have control over both client and server side.
I'll have a go at it

Resolving patch:
diff --git a/locust/contrib/fasthttp.py b/locust/contrib/fasthttp.py
--- a/locust/contrib/fasthttp.py
+++ b/locust/contrib/fasthttp.py
@@ -246,28 +246,20 @@ class FastResponse(CompatResponse):
     
     _response = None
     
+    encoding = None
+    """In some cases setting the encoding explicitly is needed. If so, do it before calling .text"""
+
     @property
     def text(self):
         """
         Returns the text content of the response as a decoded string
         (unicode on python2)
         """
-        try:
-            charset = self.headers.get('content-type', '').partition("charset=")[2]
-            content = unicode(self.content, charset or 'utf-8', errors='replace')
-        except (LookupError, TypeError):
-            # A LookupError is raised if the encoding was not found which could
-            # indicate a misspelling or similar mistake.
-            #
-            # A TypeError can be raised if encoding is None
-            #
-            # Fallback to decode without specifying encoding
-            if self.content is None:
-                content = None
-            else:
-                content = unicode(self.content, errors='replace')
-        return content
-    
+        if self.content is None:
+            return None
+        self.encoding = self.encoding or self.headers.get('content-type', '').partition("charset=")[2] or 'utf-8'
+        return unicode(self.content, self.encoding, errors='replace')
+
     def raise_for_status(self):
         """Raise any connection errors that occured during the request"""
         if hasattr(self, 'error') and self.error:


Test patch:
Repository: locustio/locust

Issue:
Fix simple typo: betwen -> between
# Issue Type

[x] Bug (Typo)

# Steps to Replicate

1. Examine locust/core.py.
2. Search for `betwen`.

# Expected Behaviour

1. Should read `between`.




Discussion:

Resolving patch:
diff --git a/locust/core.py b/locust/core.py
--- a/locust/core.py
+++ b/locust/core.py
@@ -314,7 +314,7 @@ class ForumPage(TaskSet):
     wait_function = None
     """
     Deprecated: Use wait_time instead.
-    Function used to calculate waiting time betwen the execution of locust tasks in milliseconds. 
+    Function used to calculate waiting time between the execution of locust tasks in milliseconds. 
     Can be used to override the wait_function defined in the root Locust class, which will be used
     if not set on the TaskSet.
     """


Test patch:
Repository: locustio/locust

Issue:
We should check limits (ulimit) and warn if they are too low
Occasionally people file tickets/SO issues that seem to relate to this:

https://github.com/locustio/locust/issues/496

https://stackoverflow.com/questions/61633170/locust-error-nodename-nor-servname-provided-or-not-known/61651507?noredirect=1#comment109058148_61651507

We could add something like this:
```
if resource.getrlimit(resource.RLIMIT_NOFILE)[0] < 10000:
    logger.warning("System open file limit setting is not high enough for load testing. See https://docs.locust.io/en/stable/installation.html#increasing-maximum-number-of-open-files-limit for more info.")
```


Discussion:
It appears you can actually change the NOFILE setting from inside the python process (at least it worked for me). Maybe that is even better. Or we could do both (try to set it really high, silently ignore if it fails, but warn if the limit after trying to set it is too low)
> It appears you can actually change the NOFILE setting from inside the python process (at least it worked for me). Maybe that is even better. Or we could do both (try to set it really high, silently ignore if it fails, but warn if the limit after trying to set it is too low)

Interesting! Sounds like a good idea, though I'd like us to test this out on some different platforms (Windows/Linux/Mac) and running as non-admin, to make sure it would behave as expected.

Resolving patch:
diff --git a/locust/main.py b/locust/main.py
--- a/locust/main.py
+++ b/locust/main.py
@@ -6,6 +6,7 @@
 import socket
 import sys
 import time
+import resource 
 
 import gevent
 
@@ -153,6 +154,14 @@ def main():
         # list() call is needed to consume the dict_view object in Python 3
         user_classes = list(user_classes.values())
     
+    try:
+        if resource.getrlimit(resource.RLIMIT_NOFILE)[0] < 10000:
+            # Increasing the limit to 10000 within a running process should work on at least MacOS.
+            # It does not work on all OS:es, but we should be no worse off for trying.
+            resource.setrlimit(resource.RLIMIT_NOFILE, [10000, resource.RLIM_INFINITY])
+    except:
+        logger.warning("System open file limit setting is not high enough for load testing, and the OS wouldnt allow locust to increase it by itself. See https://docs.locust.io/en/stable/installation.html#increasing-maximum-number-of-open-files-limit for more info.")
+
     # create locust Environment
     environment = create_environment(user_classes, options, events=locust.events)
     


Test patch:
Repository: locustio/locust

Issue:
Update flask version
Our minimum required flask version is too old (saw at least one person having an issue https://stackoverflow.com/questions/61969924/typeerror-when-i-run-a-locustfile-py)

https://flask.palletsprojects.com/en/1.1.x/changelog/#version-0-12-5 is a minimum, but we should probably go to 1.x right away.

I can do the PR


Discussion:
Actually, the underlying issue is probably with Werkzeug. But 1.1.0 of Flask pins the version to a reasonably recent version of W, so I think that should be the most appropriate solution (rather than specifying an explicit version of W)
Sounds like a good idea!

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -20,7 +20,7 @@
     version=version,
     install_requires=[
         "gevent>=1.5.0",
-        "flask>=0.10.1", 
+        "flask>=1.1.2", 
         "requests>=2.9.1", 
         "msgpack>=0.6.2", 
         "pyzmq>=16.0.2", 


Test patch:
Repository: locustio/locust

Issue:
Misleading log message in distributed mode
<!-- 
If you have a general question about how to use Locust, please check Stack Overflow first https://stackoverflow.com/questions/tagged/locust

You can also ask new questions on SO, https://stackoverflow.com/questions/ask just remember to tag your question with "locust". Do not immediately post your issue here after posting to SO, wait for an answer there instead.

Use this form only for reporting actual bugs in locust. Be mindful that the developers of locust are unpaid volunteers, so make sure you have tried everything you can think of before filing a bug :) 
-->

### Describe the bug
When running locust in distributed mode you can still get this warning message on the master node:
```
/WARNING/locust.main: System open file limit setting is not high enough for load testing, and the OS didn't allow locust to increase it by itself. See https://github.com/locustio/locust/wiki/Installation#increasing-maximum-number-of-open-files-limit for more info.
```
Since the master won't execute any of the tests this warning is misleading.

<!-- A clear and concise description of what the bug is -->

### Expected behavior
<!-- Tell us what you think should happen -->
No log message about open file limit on master when running distributed

### Actual behavior
Log message about low open file limit on instance running master mode
<!-- Tell us what happens instead. Include screenshots if this an issue with the GUI. -->

### Steps to reproduce
<!-- Please provide a minimal reproducible code example (https://stackoverflow.com/help/minimal-reproducible-example) --> 
Run `locust --master` on a host with low open file limit (e.g 2048).

### Environment

- OS: locust docker image running on Linux host
- Python version: Embedded version in docker image with tag `1.2.3`
- Locust version: 1.2.3
- Locust command line that you ran: `locust --master`
- Locust file contents (anonymized if necessary): N/A


Discussion:

Resolving patch:
diff --git a/locust/main.py b/locust/main.py
--- a/locust/main.py
+++ b/locust/main.py
@@ -168,7 +168,8 @@ def main():
         # list() call is needed to consume the dict_view object in Python 3
         user_classes = list(user_classes.values())
 
-    if os.name != "nt":
+    if os.name != "nt" and not options.master:
+
         try:
             import resource
 


Test patch:
Repository: locustio/locust

Issue:
/swarm web endpoint should not require user_count and spawn_rate when shape_class is used


### Is your feature request related to a problem? Please describe.
/swarm endpoint requires two int values for user_count and spawn_rate, which makes sense only for non-shaped locustfiles

### Describe the solution you'd like
in web.py get user_count and spawn_rate could be used only if necessary



Discussion:
Related to #1659

Resolving patch:
diff --git a/locust/web.py b/locust/web.py
--- a/locust/web.py
+++ b/locust/web.py
@@ -134,8 +134,6 @@ def index():
         @self.auth_required_if_enabled
         def swarm():
             assert request.method == "POST"
-            user_count = int(request.form["user_count"])
-            spawn_rate = float(request.form["spawn_rate"])
 
             if request.form.get("host"):
                 # Replace < > to guard against XSS
@@ -146,6 +144,8 @@ def swarm():
                 return jsonify(
                     {"success": True, "message": "Swarming started using shape class", "host": environment.host}
                 )
+            user_count = int(request.form["user_count"])
+            spawn_rate = float(request.form["spawn_rate"])
 
             environment.runner.start(user_count, spawn_rate)
             return jsonify({"success": True, "message": "Swarming started", "host": environment.host})


Test patch:
Repository: locustio/locust

Issue:
Locust stopped working after Flast 2.0 got released
in setup.py I can see:
` "flask>=1.1.2", `
I guess it should be hardcoded to ==1.1.2 for now.

it crashes with:
```
File "/root/.local/share/virtualenvs/xxxxxxx/lib/python3.6/site-packages/locust/web.py", line 102, in __init__
app.jinja_options["extensions"].append("jinja2.ext.do")
KeyError: 'extensions'
```


Discussion:
I also faced the same error, I tried to google for a solution. 

Based on what chfaber shared in this [StackOverflow post](https://stackoverflow.com/questions/52683386/how-to-use-a-jinja2-extension-direct-from-a-flask-render-template) , the workaround would be to replace `app.jinja_options['extensions'].append('jinja2.ext.do')` with `app.jinja_env.add_extension('jinja2.ext.do')` 

However, I am not familiar with how Flask and Jinja work together, so I can't offer much explanation. Hope this still helps!



Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -19,7 +19,7 @@
     version=version,
     install_requires=[
         "gevent>=20.9.0",
-        "flask>=1.1.2",
+        "flask==1.1.2",
         "Werkzeug>=1.0.1",
         "requests>=2.9.1",
         "msgpack>=0.6.2",


Test patch:
Repository: locustio/locust

Issue:
time display in live charts switches to the local time upon refresh
### Describe the bug

There is a bug in the display of the time in live graphs (that thankfully doesnt seem to appear when you go to “Download Report”), where the hour (time zone?) switches. It will show the time in the server's own timezone... and then at some point in the graph (depending on when you refreshed your browser to load the web UI), the time display will switch to the local timezone of your browser system.

### Expected behavior

The time display should always be consistent. Either take the cue from the server (this is the default, and is fantastic)... or from the browser system.

### Actual behavior

See "Describe the bug" section.

Here is a screenshot of the problem, showing how the hour switches from 11 (the server is GMT) to 19 (my system is GMT+8):
![Screenshot 2021-07-30 at 19 14 22](https://user-images.githubusercontent.com/7270/127743292-f41ad7bd-c559-4576-bf0c-d22d42c53125.png)

### Steps to reproduce

Set up a server in a different time zone from your local system. Load the web UI and start a run. Refresh your browser, and click "Charts". You will notice that the last time as shown in the charts will *always* be in your local timezone, and as more points get added to the graphs, all of those will be in your local time.

This behaviour is repeatable. Refresh your browser again, click on "Charts" again... and you will see again that all of the time values will reset to the server's time zone save for the last one.

### Environment

- OS: Ubuntu 20.04
- Python version: 3.8.10
- Locust version: 1.6.0


Discussion:

Resolving patch:
diff --git a/locust/stats.py b/locust/stats.py
--- a/locust/stats.py
+++ b/locust/stats.py
@@ -791,7 +791,7 @@ def stats_history(runner):
             break
         if runner.state != "stopped":
             r = {
-                "time": datetime.datetime.now().strftime("%H:%M:%S"),
+                "time": datetime.datetime.utcnow().strftime("%H:%M:%S"),
                 "current_rps": stats.total.current_rps or 0,
                 "current_fail_per_sec": stats.total.current_fail_per_sec or 0,
                 "response_time_percentile_95": stats.total.get_current_response_time_percentile(0.95) or 0,


Test patch:
Repository: locustio/locust

Issue:
the report cant show the right time
<!-- 
If you have a general question about how to use Locust, please check Stack Overflow first https://stackoverflow.com/questions/tagged/locust

You can also ask new questions on SO, https://stackoverflow.com/questions/ask just remember to tag your question with "locust". Do not immediately post your issue here after posting to SO, wait for an answer there instead.

Use this form only for reporting actual bugs in Locust. Remember, the developers of Locust are unpaid volunteers, so make sure you have tried everything you can think of before filing a bug :) 

Always make sure you are running an up to date Locust version (pip3 install -U locust)
-->

### Describe the bug
<!-- A clear and concise description of what the bug is -->
![image](https://user-images.githubusercontent.com/47803360/138236838-6d62d47c-aa4d-4de7-b714-d94b7dff639a.png)
![image](https://user-images.githubusercontent.com/47803360/138236861-d318f852-da06-41e9-abcb-fa18d47166c7.png)
this picture is right time  i am in china


### Expected behavior
<!-- Tell us what you think should happen -->
Wrong time zone reported
### Actual behavior
<!-- Tell us what happens instead. Include screenshots if this an issue with the GUI. -->

### Steps to reproduce
<!-- Please provide a minimal reproducible code example (https://stackoverflow.com/help/minimal-reproducible-example) --> 

### Environment

- OS:
- Python version:
- Locust version: (please dont file issues for anything but the most recent release or prerelease builds)
- Locust command line that you ran:
- Locust file contents (anonymized if necessary):


Discussion:

Resolving patch:
diff --git a/locust/html.py b/locust/html.py
--- a/locust/html.py
+++ b/locust/html.py
@@ -20,11 +20,11 @@ def get_html_report(environment, show_download_link=True):
     stats = environment.runner.stats
 
     start_ts = stats.start_time
-    start_time = datetime.datetime.fromtimestamp(start_ts).strftime("%Y-%m-%d %H:%M:%S")
+    start_time = datetime.datetime.utcfromtimestamp(start_ts).strftime("%Y-%m-%d %H:%M:%S")
 
     end_ts = stats.last_request_timestamp
     if end_ts:
-        end_time = datetime.datetime.fromtimestamp(end_ts).strftime("%Y-%m-%d %H:%M:%S")
+        end_time = datetime.datetime.utcfromtimestamp(end_ts).strftime("%Y-%m-%d %H:%M:%S")
     else:
         end_time = start_time
 


Test patch:
Repository: locustio/locust

Issue:
I hope to add a column of 99%ile on the Web UI
<!-- 
If you have a general question about how to use Locust, please check Stack Overflow first https://stackoverflow.com/questions/tagged/locust

You can also ask new questions on SO, https://stackoverflow.com/questions/ask just remember to tag your question with "locust".
-->

### Is your feature request related to a problem? Please describe.
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
I use 99%ile as a benchmark for loading test.
When I test with locust, I always check 99%ile from `/stats/report` , but it is more convenient to see it from the top page.

### Describe the solution you'd like
<!-- A clear and concise description of what you want to happen -->
![スクリーンショット 2021-12-30 17 11 10](https://user-images.githubusercontent.com/14051784/147733508-f1a2ea7c-b077-48d9-8f9f-304177fba845.jpg)


### Additional context
<!-- Add any other context or screenshots about the feature request here -->
If you find this feature useful, please let me try to do.


Discussion:

Resolving patch:
diff --git a/locust/web.py b/locust/web.py
--- a/locust/web.py
+++ b/locust/web.py
@@ -271,6 +271,7 @@ def request_stats():
                         "current_fail_per_sec": s.current_fail_per_sec,
                         "median_response_time": s.median_response_time,
                         "ninetieth_response_time": s.get_response_time_percentile(0.9),
+                        "ninety_ninth_response_time": s.get_response_time_percentile(0.99),
                         "avg_content_length": s.avg_content_length,
                     }
                 )


Test patch:
Repository: locustio/locust

Issue:
locust:2.7.1 exits when clicking "Stop Tests" in the UI
### Describe the bug
When using the 2.7.1 docker image, locust exits with code 0 upon clicking "Stop Tests" in the UI. 
We swapped in 2.7.0 with the same locustfile and config, and the problem was fixed.

### Expected behavior
Locust should continue running (as it does in 2.7.0) after clicking "Stop Tests". 

### Actual behavior
Cannot run multiple tests, as after each test locust exits, therefore we can't compare results from previous runs.

### Steps to reproduce

1. Run locust:2.7.1 docker image
2. Browse to the UI
3. Start tests
4. Stop tests
5. Observe that locust has exited

### Environment

- OS:  locustio:locust:2.7.1 docker image
- Python version: 3.9 (locustio:locust:2.7.1 docker image)
- Locust version: 2.7.1
- Locust command line that you ran: `locust --config=/home/locust/master.conf`
- Locust file contents (anonymized if necessary):

```
import time
import os
from locust import HttpUser, FastHttpUser, task, events
from locust.runners import MasterRunner
from kubernetes import client, config
from kubernetes.stream import stream


API_TOKEN = ""  # currently not used


# Basic requests that sleep - for testing queue latency with Puma
class SleepUser(FastHttpUser):
    network_timeout = 300

    @task(3)
    def sleep_one(self):
        self.client.get("/sleep?time=1", verify=False)

    @task(2)
    def sleep_five(self):
        self.client.get("/sleep?time=5", verify=False)

    @task(5)
    def sleep_half(self):
        self.client.get("/sleep?time=0.5", verify=False)

    @task(1)
    def sleep_thirty(self):
        self.client.get("/sleep?time=30", verify=False)


# Hook that runs when locust starts
@events.test_start.add_listener
def on_test_start(environment, **kwargs):
    print("Starting on_test_start hook")
    generate_api_token_script = # removed
    success, output = execute_on_app_pod(generate_api_token_script)
    global API_TOKEN
    API_TOKEN = [line for line in output.split('\n') if 'PERFAPITOKEN:' in line][1].replace('PERFAPITOKEN:', '')


def execute_on_app_pod(command):
    # removed - function that executes commands on an in-cluster k8s pod
```

locust conf:
```
users = 100
spawn-rate = 10
web-host: *
web-port: 8089
```

env vars:
LOCUST_HOST= # url we're testing
LOCUST_LOCUSTFILE=/home/locust/locustfile.py



Discussion:

Resolving patch:
diff --git a/locust/web.py b/locust/web.py
--- a/locust/web.py
+++ b/locust/web.py
@@ -174,8 +174,6 @@ def stop():
                 self._swarm_greenlet.kill(block=True)
                 self._swarm_greenlet = None
             environment.runner.stop()
-            if self.greenlet is not None:
-                self.greenlet.kill(block=True)
             return jsonify({"success": True, "message": "Test stopped"})
 
         @app.route("/stats/reset")


Test patch:
Repository: locustio/locust

Issue:
2.18 - fasthttp error for no content response
### Prerequisites

- [X] I am using [the latest version of Locust](https://github.com/locustio/locust/releases/)
- [X] I am reporting a bug, not asking a question

### Description

Sending a request that generates a no content response is throwing a error when using fasthttp rest. This isn't happening under 2.17




### Command line

Locust UI

### Locustfile contents

```python3
File "/usr/local/lib/python3.11/contextlib.py", line 137, in __enter__
worker-1  |     return next(self.gen)
worker-1  |            ^^^^^^^^^^^^^^
worker-1  |   File "/opt/venv/lib/python3.11/site-packages/locust/contrib/fasthttp.py", line 382, in rest
worker-1  |     if resp.text is None:
worker-1  |        ^^^^^^^^^
worker-1  |   File "/opt/venv/lib/python3.11/site-packages/locust/contrib/fasthttp.py", line 467, in text
worker-1  |     return str(self.content, self.encoding, errors="replace")
worker-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker-1  | TypeError: str() argument 'encoding' must be str, not None
```


### Python version

3.11

### Locust version

2.18

### Operating system

linux


Discussion:
I think its due to this change https://github.com/locustio/locust/pull/2416/files

it is possible that get_encoding_from_headers function will return None which will throw the above error. Need to return as a empty string as per 2.17
Oh... yea sounds about right. Or maybe we should just not try to decode at all when there is no content? I dont have time to look at this right now...
Could just throw in a condition as a quick fix so are able to still utilise the requests lib

```
self.encoding = get_encoding_from_headers(self.headers)
if self.encoding is None:
     self.encoding = ""
```

```
self.encoding = get_encoding_from_headers(self.headers) or ""
```

Resolving patch:
diff --git a/locust/contrib/fasthttp.py b/locust/contrib/fasthttp.py
--- a/locust/contrib/fasthttp.py
+++ b/locust/contrib/fasthttp.py
@@ -463,7 +463,7 @@ def text(self) -> Optional[str]:
             if self.headers is None:
                 self.encoding = "utf-8"
             else:
-                self.encoding = get_encoding_from_headers(self.headers)
+                self.encoding = get_encoding_from_headers(self.headers) or ""
         return str(self.content, self.encoding, errors="replace")
 
     @property


Test patch: